Future Crunch - Angus Hervey shares 66 good news stories from 2023, including progress in healthcare, climate change, and education.

Like every year since 2021, Microsoft publish their Future of Work report for 2023. This year’s edition centers on research related to integrating LLMs into work

Panda Smith discusses the limitations of vector databases and suggests that building a good search engine is a better approach for creating language model-powered documentation tools. He offers insights into improving overall search by blending embeddings with more traditional methods, and using language models to structure q...

Ars Technica - Extreme E, the electric off-road series, plans to switch to hydrogen in 2025. Founder and chairman Alejandro Agag aims for Extreme H to be the first pure hydrogen world championship racing

Nature - Bernardino Romera-Paredes, Mohammadamin Barekatain, Alexander Novikov, Matej Balog, M. Pawan Kumar, Emilien Dupont, Francisco J. R. Ruiz, Jordan S. Ellenberg, Pengming Wang, Omar Fawzi, Pushmeet Kohli and Alhussein Fawzi introduce FunSearch, a method using a pre-trained LLM and systematic evaluator to make mathemati...

This tutorial, written by Jason Liu, demonstrates how to perform bulk text classification using user-provided tags with the Instructor library and OpenAI's API. It covers defining tag structures, implementing classification, and improving the model.

Zi-Xin Zou, Zhipeng Yu, Yuan-Chen Guo, Yangguang Li, Ding Liang, Yan-Pei Cao and Song-Hai Zhang propose a novel transformer-based approach for single-view 3D reconstruction. The method uses two decoders: a point decoder to generate point clouds and a triplane decoder to query Gaussian features for each point. This hybrid repr...

Apple Researchers Keivan Alizadeh, Iman Mirzadeh, Dmitry Belenko, Karen Khatamifard, Minsik Cho, Carlo C Del Mundo, Mohammad Rastegari, and Mehrdad Farajtabar present a method for efficiently running large language models on devices with limited DRAM capacity

This paper presents the findings of the BabyLM Challenge, which focuses on sample-efficient pretraining on developmentally plausible corpora.

Shuyang Sun, Runjia Li, Philip Torr, Xiuye Gu and Siyang Li propose a novel recurrent framework for open-vocabulary image segmentation using CLIP (Contrastive Language-Image Pretraining). Their approach progressively filters out irrelevant texts and enhances mask quality without training efforts, resulting in state-of-the-art...

Researchers from the GrapheneX-UTS Human-centric Artificial Intelligence Centre have developed a portable, non-invasive system that can decode silent thoughts and turn them into text.

WHAM, a method developed by Soyong Shin, Juyong Kim, Eni Halilaj, and Michael J. Black, accurately and efficiently reconstructs 3D human motion in a global coordinate system from video. It outperforms existing methods across multiple benchmarks.

NewScientist - Researchers at Indiana University Bloomington have developed a biocomputing system that uses living brain cells to perform speech recognition, achieving an accuracy of 70-80% after training. The system, called 'Brainoware', consists of brain organoids grown from stem cells and placed on top of a microelectrode ...

Omar Sanseviero, Lewis Tunstall, Philipp Schmid, Sourab Mangrulkar, Younes Belkada, and Pedro Cuenca, explain the concept of Mixture of Experts (MoEs), a class of models that enable efficient pretraining with large model sizes or datasets by replacing dense feed-forward network layers with sparse MoE layers composed of multip...

Agrim Gupta, Lijun Yu, Kihyuk Sohn, Xiuye Gu, Meera Hahn, Li Fei-Fei, Irfan Essa, Lu Jiang and José Lezama present W.A.L.T, a transformer-based approach for photorealistic video generation via diffusion modeling, achieving state-of-the-art performance on established benchmarks.

Alibaba present, DreaMoving, a diffusion-based controllable video generation framework, produces high-quality customized human videos.

Big Think - Ethan Siegel discusses the formation of the first atoms in the universe, which took hundreds of thousands of years after the Big Bang due to the high temperatures and energy levels present. Despite the presence of atomic nuclei, it wasn't until the temperature dropped below a few thousand degrees that neutral atom...

Nuo Chen, Ning Wu, Shining Liang, Ming Gong, Linjun Shou, Dongmei Zhang, Jia Li analyze Large Language Models (LLMs) using LLaMA as the primary model. Instead of assessing it through its generative output, they design multiple-choice tasks to probe its intrinsic understanding in high-order tasks such as reasoning and computat...

Bowen Jin, Gang Liu, Chi Han, Meng Jiang, Heng Ji and Jiawei Han explore the potential of large language models (LLMs) in processing text data associated with rich structure information or graph data with textual information, and their reasoning ability in graph scenarios.

In a groundbreaking discovery, Nico Brown, Carson Cheng, Tanner Jacobi, Maia Karpovich, Matthias Merzenich, David Raucci, and Mitchell Riley have proven that Conway's Game of Life is omniperiodic by finding oscillators with periods 19 and 41. This filling in the missing periods concludes a decades-long search since the creati...

Albert Gu and Tri Dao introduce Mamba, a neural network architecture that uses selective state space models (SSMs) to model long sequences. This approach addresses the weaknesses of other subquadratic-time architectures in handling content-based reasoning for discrete modalities such as language. Mamba outperforms Transformer...

Meta introduce Seamless Communication, a new system for expressive cross-lingual communication in real time. It includes SeamlessExpressive, which preserves expression in speech-to-speech translation, and SeamlessStreaming, a streaming translation model with low latency.

Tobias Kirschstein, Simon Giebenhain, and Matthias Nießner propose DiffusionAvatars, a diffusion-based neural renderer that generates high-fidelity 3D head avatars with intuitive control over pose and expression. Trained on RGB videos and corresponding tracked NPHM meshes of a person, DiffusionAvatars outperforms existing app...

DeepMind have discovered 2.2 million new crystals using their AI tool GNoME, with 380,000 being stable and potentially viable for future technologies. This discovery has the potential to drive research in various fields including superconductors, powering supercomputers, next-generation batteries and electric vehicles.

Pavan Kumar Anasosalu Vasu, Hadi Pouransari, Fartash Faghri, Raviteja Vemulapalli, and Oncel Tuzel introduce MobileCLIP, a new family of efficient image-text models optimized for runtime performance. The proposed training approach, multi-modal reinforced training, leverages knowledge transfer from an image captioning model an...