An AI deepfake tool that can clone your voice in 28 other languages, ElevenLabs tech a worry for many

Internet users are increasingly familiar with ElevenLabs, a prominent company known for creating convincing audio deepfakes. The company has recently upgraded its voice cloning technology, enabling it to mimic voices in 22 additional languages, such as Ukrainian, Korean, Swedish, and Arabic, among others.

This multilingual expansion, referred to as the Multilingual v2 model, now boasts the capacity to generate emotionally expressive audio across 28 languages. While this innovation presents exciting possibilities, it also raises concerns about its implications for media, entertainment, and voice actors.

Introducing Eleven Multilingual v2: our new AI speech model supporting 28 languages!

v2 comes with enhanced conversational capability, higher output quality & the ability to better preserve unique voice characteristics across languages.

Read more: https://t.co/IhQv69Jxbn pic.twitter.com/xP8MmVTZkV

— ElevenLabs (@elevenlabsio) August 22, 2023

ElevenLabs' multilingual voice cloning: A new frontier

ElevenLabs has introduced its Multilingual v2 model, enabling users to generate lifelike voices speaking in various languages. The company offers two AI voice tools: a text-to-speech model and ‘VoiceLab,’ which allows users to clone their own voices by inputting speech fragments.

With the new model, generated voices can speak languages like Greek, Malay, and Turkish. Users can simply type the text in the desired language to hear the translated voice, which should work seamlessly with any cloned voice.

Controversy & advancements

Initially, ElevenLabs faced controversy when users abused its beta platform to impersonate celebrities and spread offensive content. To address this, the company has implemented measures like text captcha prompts to verify users' ownership of their voices.

Despite past issues, the company remains committed to eliminating linguistic barriers to content, aiming to cover even more languages with the help of AI. ElevenLabs is now out of beta and actively promoting its AI voice cloning technology to media companies.

The company's voice cloning tech has the potential to revolutionise various domains, from creating audiobooks and videos to providing voices for non-player characters in video games.

While AI-generated voices offer benefits such as speed and cost-effectiveness in producing audiobooks, concerns arise within the entertainment industry as it grapples with the use of AI technology.

Challenges for voice actors & industry impact

The emergence of AI-generated voices poses challenges for voice actors. As AI narration gains traction, it raises questions about the future of the industry, especially in the context of ongoing strikes by actors' unions.

Some publishing companies are embracing AI narration due to its efficiency, cost-effectiveness, and quality improvements. However, the potential for licensing voices and their impact on revenue distribution remains uncertain, highlighting the evolving nature of the publishing industry.

In short, ElevenLabs' expansion of its voice cloning technology to include a multitude of languages presents both exciting opportunities and complex challenges for the industry.

As AI-generated voices become more prevalent in media and audiobooks, the landscape of voice acting and content creation undergoes significant transformation, triggering discussions about quality, ethics, and the future role of human voices in an AI-driven world.