Yutao Sun, Li Dong, Shaohan Huang, Shuming Ma, Yuqing Xia, Jilong Xue, Jianyong Wang and Furu Wei propose Retentive Network (RetNet) as a foundation architecture for large language models, simultaneously achieving training parallelism, low-cost inference, and good performance.

Zaijun Chen, Alexander Sludds, Ronald Davis III, Ian Christen, Liane Bernstein, Lamia Ateshian, Tobias Heuser, Niels Heermeier, James A. Lott, Stephan Reitzenstein, Ryan Hamerly and Dirk Englund demonstrate a spatial-temporal-multiplexed optical neural networks system that simultaneously overcomes all usual challenges

The Financial Stability Board, in coordination with local regulators, propose a framework for international regulation of crypto-asset activities

Sandra Oberleiter, Jonathan Fries, Laura S. Schock, Benedikt Steininger and Jakob Pietschnig present evidence for sex differences in international large-scale assessments of reading literacy, mathematics, and science across 16 cohorts from 1995 to 2019.

Tri Dao propose FlashAttention-2, with better work partitioning to address the inefficiency that is due to suboptimal work partitioning between different thread blocks and warps on the GPU, causing either low-occupancy or unnecessary shared memory reads/writes

Game | Nicholas Carlini proposes a fun game that tests your ability to predict ("forecast") how well GPT-4 will perform at various types of questions.

Liang Wang, Nan Yang and Furu Wei propose a novel framework to iteratively train dense retrievers that can identify high-quality in-context examples for LLMs.

Yu Gu, Sheng Zhang, Naoto Usuyama, Yonas Woldesenbet, Cliff Wong, Praneeth Sanapathi, Mu Wei, Naveen Valluri, Erika Strandberg, Tristan Naumann and Hoifung Poon

Benjamin Grimmer establishes provably faster convergence rates for gradient descent in smooth convex optimization via a computer-assisted analysis technique.

Genomic time-series from experimental evolution studies and ancient DNA datasets offer us a chance to more directly observe the interplay of various evolutionary forces. Alexis Simon and Graham Coop show how the genome-wide variance in allele frequency change between two time points can be decomposed into the contributions of...

Engineers and researchers from Yandex Research, HSE University, University of Washington, Hugging Face, ENS Paris-Saclay, and Yandex School of Data Analysis propose Petals, an open-source decentralized system (showcased this week at the ACL 2023 Demonstrations track) allowing anybody to run large models or even adapt them us...

On the basis of the new adequacy decision, personal data can flow safely from the EU to US companies participating in the Framework, without having to put in place additional data protection safeguards.

Jeremy Howard sheds interesting light on the AI Safety debate

Quantamagazine - Mathematicians can often figure out what happens as quantities grow infinitely large. What about when they are just a little big?

Nelson F. Liu, Kevin Lin, John Hewitt, Ashwin Paranjape, Michele Bevilacqua, Fabio Petroni and Percy Liang find that performance is often highest when relevant information occurs at the beginning or end of the input context, and significantly degrades when models must access relevant information in the middle of long contexts.

Quantamagazine - New research reveals how marine microbes use an extra membrane that once had digestive functions to boost their yield from photosynthesis

Semianalysis detail the bottlenecks to production and how much downstream capacity is expanding for Nvidia and their competitors

Nature - R. Z. Moger-Reischer, J. I. Glass, K. S. Wise, L. Sun, D. M. C. Bittencourt, B. K. Lehmkuhl, D. R. Schoolmaster Jr, M. Lynch and J. T. Lennon report on how an engineered minimal cell contends with the forces of evolution compared with the Mycoplasma mycoides non-minimal cell from which it was synthetically derived.

Luciano Del Corro, Allie Del Giorno, Sahaj Agarwal, Bin Yu, Ahmed Awadallah and Subhabrata Mukherjee propose a simple and effective token-level early exit method, SkipDecode, designed to work seamlessly with batch inferencing and KV caching. It overcomes prior constraints by setting up a singular exit point for every token in...

Jiayu Ding, Shuming Ma, Li Dong, Xingxing Zhang, Shaohan Huang, Wenhui Wang, Nanning Zheng and Furu Wei introduce LongNet, a Transformer variant that can scale sequence length to more than 1 billion tokens, without sacrificing the performance on shorter sequences.

Nature - Cheng Kai Lim, Jing Wui Yeoh, Aurelius Andrew Kunartama, Wen Shan Yew and Chueh Loo Poh detail a method of capturing 2-dimensional light patterns into DNA, by utilizing optogenetic circuits to record light exposure into DNA, encoding spatial locations with barcoding, and retrieving stored images via high-throughput n...

Maanak Gupta, CharanKumar Akiri, Kshitiz Aryal, Eli Parker and Lopamudra Praharaj highlight the limitations, challenges, potential risks, and opportunities of GenAI in the domain of cybersecurity and privacy.

Jared Palmer (Vercel) gives high-level insights into a toolbox to protect a platform against scrapping

Shen, Zejiang, Ruochen Zhang, Melissa Dell, Benjamin Lee, Jacob Carlson, and Weining Li introduce LayoutParser, an open-source library for streamlining the usage of DL in document image analysis research and applications.

Joint report from Georgetown University’s Center for Security and Emerging Technology (CSET) and The Alan Turing Institute’s Centre for Emerging Technology and Security (CETaS) assesses the current state-of-the-art in autonomous cyber defence and its future potential, identifies barriers to progress and recommends specific ac...