Description

In this article, Meta present Seamless Communication, a system that enables authentic conversations by preserving key elements of speech such as tone of voice, pauses, and emphasis. The system includes SeamlessExpressive, a model for expressive speech-to-speech translation, and SeamlessStreaming, a streaming translation model with around two seconds of latency. These models are built on SeamlessM4T v2, Meta's latest foundational model for multilingual machine translation. The authors also discuss the release of metadata, data and data alignment tools to assist the research community in building on this work.