ElevenLabs

ElevenLabs is an American software company developing natural-sounding speech synthesis and text-to-speech software, using artificial intelligence and deep learning.
History
ElevenLabs was co-founded in 2022 by Piotr Dabkowski, an ex-Google machine learning engineer and Mati Staniszewski, an ex-Palantir deployment strategist. Both were raised in Poland, and their inspiration for founding ElevenLabs reportedly came from watching inadequately dubbed American films.
Dabkowski and Staniszewski initially considered different funding options, including the possibility of collaborating with a startup accelerator. In January 2023 they revealed having secured a $2 million pre-seed round. The startup's specialization in AI voice intelligence, a still-emerging field in Europe, played a significant role in attracting investors. The pre-seed funding was primarily led by Credo Ventures.
In January 2023, ElevenLabs publicly released its beta platform.
In June 2023, ElevenLabs raised a $19 million Series A funding round at a valuation of about $100 million, despite the company having no office and only 15 employees. The funding round was co-led by the venture capital firm Andreessen Horowitz, ex-GitHub CEO Nat Friedman, and entrepreneur Daniel Gross. It also saw participation from prominent such as SV Angel, Mike Krieger (co-founder of Instagram), Brendan Iribe (co-founder of Oculus), Mustafa Suleyman (co-founder of Deepmind), and Tim O'Reilly (founder of O'Reilly Media). It was also announced that Andreessen Horowitz would be joining ElevenLabs' board.
Products
ElevenLabs is primarily known for its browser-based, AI-assisted text-to-speech software, Speech Synthesis, which can produce lifelike speech by synthesizing vocal emotion and intonation. The company states its software is built to adjust the intonation and pacing of delivery based on the context of language input used. It uses advanced algorithms to analyze the contextual aspects of text, aiming to detect emotions like anger, sadness, happiness, or alarm, which enables the system to understand the user's sentiment, resulting in a achieve a more realistic and human-like inflection. The startup is in the process of patenting this technology. Through its beta site, users can submit text and generate audio files from a selection of default voices. Paying users are given the ability to upload custom voice samples to create new vocal styles using the company’s voice cloning tool.
Voice Library is the company’s feature for sharing unique voice profiles created using their Voice Design technology. These pre-designed voice profiles allow users to select a voice that best suits their needs, rather than creating one from scratch. Another tool called VoiceLab allows users to clone voices from just a few short snippets of audio and can create entirely new synthetic voices.
On 20 June 2023, ElevenLabs released an AI recognition tool called the AI Speech Classifier, which it claims is the first of its kind. The tool is accessible through an API and designed to determine if an uploaded audio sample originates from ElevenLabs' proprietary AI technology. The company has expressed its intention to collaborate with other AI developers in creating a universal detection system that could be adopted industry-wide.
In July 2023, the company announced "Projects", a tool for creating long-form spoken content such as audiobooks and dialogue segments with contextually-aware synthetic or custom voices. The tool was released in September. In August, ElevenLabs expanded its voice generation capabilities to 28 languages. Using an in-house AI model, it automatically detects languages like Korean, Dutch, and Vietnamese, allowing for "emotionally rich" multilingual speech generation. The company also announced that its technology had officially exited its beta phase.
IN October 2023, the company presented a new AI Dubbing feature.
Uses
ElevenLabs' use cases span a range of sectors.
Content creators have used ElevenLabs for podcasts, narration, and comedy shows. In March 2023, comedian Drew Carey used ElevenLabs' voice cloning tool to recreate his voice for an episode of his radio show, Friday Night Freakout. In April 2023, Polish TV and radio presenter Jaroslaw Kuzniar has also used a synthesized version of his voice to deliver a series of podcasts on the War in Ukraine. Seth Godin has also used ElevenLabs to narrate his AI-focused podcast.
In March 2023, Super-Hi-Fi, a streaming automation service, partnered with ElevenLabs to launch a fully automated radio service called "AI Radio", using ElevenLabs' software to voice its virtual DJ from prompts generated with ChatGPT. ElevenLabs has also been employed for narrating games and voicing game characters in partnerships with Swedish game developer Paradox Interactive and the United Kingdom-based Magicave.
Publishers and authors have used ElevenLabs to narrate audiobooks and newsettlers. On 13 June 2023, Storytel announced an exclusive partnership with the company. In this collaboration, ElevenLabs will create voices tailored specifically to Storytel's core markets and to produce AI-narrated audiobooks. A voice-changing feature called VoiceSwitcher was implemented to enhance personalization for users, providing unique listening experiences customized for each individual.
ElevenLabs has been used to generate audio for dubbing videos in different languages, including by content creators. The platform has the capability to accurately replicate almost any accent in any language. Celebrity fans have used ElevenLabs to create inspirational messages using the voices of their favorite celebrities.
In February, VICE reporter Joseph Cox published findings that he had recorded five minutes of himself talking and then used ElevenLabs to create voice deepfakes that defeated a bank's voice-authentication system.
ElevenLabs sets explicit guidelines regarding the use of its technology, forbidding the cloning of voices for abusive purposes such as fraud, discrimination, hate speech, or online abuse, although it does support the use of its platform for “caricature, parody and satire” and “artistic and political speech contributing to public debates." The company asserts its authority to suspend the accounts and content of users found in violation of these guidelines, and it also highlights its commitment to cooperate with authorities and report any illegal activities in accordance with applicable laws. In January, the company admitted that its platform has been used for “voice cloning misuse cases” and toughened its safeguards against vexatious use of its technology.
Reception
Following its launch in January 2023, ElevenLabs gained rapid momentum and was commended for its voice output quality, fast generation times, and a "generous free tier". It has also been praised for its ability to accurately pronounce names with unique or uncommon pronunciations, addressing a common shortcoming in similar tools that often cater primarily to Western names. The company reached over one million registered users between its launch and June 2023.
ElevenLabs was criticized after users were able to abuse its software to generate controversial statements in the vocal style of celebrities, public officials, and other famous individuals, particularly attracting attention after users on 4chan used the tool to share hateful messages. The software's ability to closely copy real voices has raised ethical concerns, with critics considering it a form of deepfaking. In response, the company said it would work on mitigating potential abuse through safeguards and identity verification. The company has subsequently limited access to its voice cloning feature to paid subscribers, citing the requirement to provide payment information as means for improving accountability, and has implemented bans on users who repeatedly violate the terms of service.
ElevenLabs, along with other companies in its category, has also been seen as a potential challenge to the voice acting sector.
See also
15.ai
Respeecher
Audio deepfake
References
External links
Official website
Speech synthesis
Artificial intelligence laboratories
2022 establishments in New York (state)
Software companies based in New York City
Technology companies based in New York City