Meta unveils next-level AI suite, transforming speech translation into a seamless & expressive experience
Meta's SeamlessM4T, unveiled in August, now boasts an updated "v2" architecture. With support for nearly 100 text and 36 speech languages, it focuses on making conversational translations more spontaneous and expressive.
Highlights
- Meta's "SeamlessExpressive" adds emotion to translated speech
- "SeamlessStreaming" translates as the speaker talks, reducing delays
- Meta's algorithm analyses partial audio for quicker and smarter language translations
In August, Meta introduced its revolutionary multimodal AI translation model, SeamlessM4T, which supports nearly 100 languages for text and 36 for speech. Now, with an updated "v2" architecture, the tech giant is taking things a step further to enhance conversational translations for a more authentic experience.
The first exciting feature is called "SeamlessExpressive." This tool transfers your expressions, such as pitch, volume, emotional tone (excitement, sadness, or whispers), speech rate, and pauses, to your translated speech. No more robotic-sounding translations; this breakthrough could be a game-changer in both daily communication and content production. Supported languages include English, Spanish, German, French, Italian, and Chinese.
However, as of now, the demo page is missing Italian and Chinese options, adding a dash of anticipation to the release.
Faster translations
The second feature, "SeamlessStreaming," aims to speed up the translation process. It begins translating a speech while the speaker is still talking, reducing the wait time for listeners. Although there's a brief latency of just under two seconds, it's still a great improvement.
Meta had a tough task because languages have different ways of building sentences. They created a clever algorithm to listen to part of the speech and decide if there's enough information to start translating or if it should keep listening.
Communication technology
Meta's latest advancements in the "Seamless Communication" suite outshine comparable tools from Google and Samsung. The potential applications extend beyond daily conversations, with speculation about integration into Meta's smart glasses in the future.
Meta’s offering
Meta's latest developments promise a significant leap in communication technology, offering more than just conventional translation tools and paving the way for a more expressive and seamless global dialogue.
While there's no official word on when the public can access these features, the prospect of more practical and natural cross-language communication is undoubtedly on the horizon.
