Tricks Learned from Scaling WhisperSpeech Models to 80k+ Hours of Speech - Jakub Cłapa, Collabora
Channel: The Linux Foundation
Date: 18/12/2023

Tricks Learned from Scaling WhisperSpeech Models to 80k+ Hours of Speech - Jakub Cłapa, Collabora

WhisperSpeech is a new OpenSource text to speech model created by Collabora and based on recent research from
the biggest AI research labs (Google, Meta, Microsoft, OpenAI). It delivers high quality speech that it learned from tens of thousands of hours of human speech recordings.

To deliver state of the art quality we scaled our models and training pipelines from hundreds to tens of thousands of hours of speech and we share the lessons learned along the way. Nearly every component of your initial training process had to be replaced or tweaked heavily.

Challenges we'll briefly cover:
- Gone in 16 minutes: the importance of small scale experiments.
- Full throttle: is 100% GPU utilization enough?
- Do you need a fancy framework? From single- to multi-GPU training.
- Are SSDs fast enough? WebDataset brings a 10x improvement.
- Does bigger always mean better? How to effortlesly scale AI models.
- Clouds, enthusiasts or clusters? How to hunt down GPUs.
- Defending moats. How is a gaming 4090 different from an H100?