Little known facts about Speech Technology
By: Roza Chojnacka
Tags: Speech Processing, Keyword Spotting, Text To Speech, Audio Processing, Smart Speaker

Little known facts about Speech Technology

Interesting ML facts from Keyword spotting and Text to Speech areas in the context of smart speakers

Photo by Thomas Le on Unsplash

I’ve been working on Speech modeling for over a year now and recently was asked to deliver a short introduction tech talk. Since I prefer structured talks, reserved one slide per question which either friends asked me or I asked myself when joining Speech. In this article I covered two areas of Speech:

Keyword spotting, aka Hotword detection, Trigger word detection. Examples: detecting when users said Hey Google or Alexa).

Text-to-Speech, abbrev. TTS. Synthesized speech is used in assistants, navigation, audiobooks, voiceover.

1. What are some weird “Hey Google” alternatives?

You can wake up Google Home by alternative phrases which sound phonetically similar to the advertised “Hey Google”. Try: Hey Poodle, Hey goo goo, Ok boo boo, Ok Noodle, Hey Poodle, Hey frugal, Hey Doodle or even Cocaine Poodle [1].

2. How to build my own Trigger Word Detection system?

The simplest solution would be to use ASR (Automatic Speech Recognition) API and whitelist the phrase you’d like to use to call your Assistant. There are a couple of problems with this solution:

It’s not optimized to run all the time.

Has high False Accept Rate as well as False Reject Rate. Depending on how the threshold is set, it can trigger on some similar sounding words picked up from the TV, or the opposite, one has to make a big effort to speak load and clear with no background noise.

Latency. Some of the processing happens after the endpointer detects the end of the phrase.

In this blog post you can read more about a typical approach to Trigger word detection which is a base for more advanced techniques. Input to the neural network is a frequency spectrogram computed from the audio stream. The Neural network consists of one convolutional 1D layer, 2 GRU layers, one dense, and sigmoid. It has a binary hotword/no hotword output. To make the training set more balanced, 50 consecutive time steps were labels as 1 (hotword detected).

3. How is Speech different from Natural Language Processing?

Oftentimes when I say I work on Speech, people respond with a question whether it’s NLP. Nowadays, with deep learning being successful in traditionally different disciplines, we see researchers moving between Speech, NLP, Computer Vision, Self Driving Cars etc. A clear differentiator between them is the type of data they deal with: Speech deals with Audio, NLP - text, Computer Vision - images.

4. Are Assistant voices prerecorded or synthesized?

I remember the times when a robotic voice would give me driving directions or I heard a dissonance when pre recorded audio from a voice talent was merged with the synthesized audio in one sentence. Now synthesized speech sounds very natural and it’s hard to differentiate human speech from generated speech. Nowadays the audio recorded by voice talents is used for the model training and audio played to users in production is synthesized (with some exceptions). A very popular method of converting Text to Speech (TTS) is called WaveNet.

5. How is the synthesized speech so expressive?

Is there a technology which uses a different prosody model when reading news and different one when reading stories for kids? Yes, there is. Check out this research blog post on Expressive Speech Synthesis [2] or listen (very expressive) synthesized audio samples. This style transfer technology is text-independent. It takes as a reference audio clip spoken in a specific style, creates style embedding, and transfers this style to any target phrase we choose.

6. How can I make TTS play in my own voice?

I found synthesized Voice Conversion audio samples here [3] with a link to a paper on Discrete Representation Learning. As can be heard from the samples, the synthesized speech has the same content as the original sample, but with the voice from the second speaker. One of the Representation Learning experiments was conducted on audio with speaker labels. Authors trained a Vector Quantized Variational AutoEncoder (VQ-VAE) model that has a dilated convolutional architecture similar to WaveNet decoder. The encoder maps audio to a latents consist of one feature map and a 512-dimensional latent space. The decoder is conditioned on both the latents and a one-hot embedding for the speaker. At inference, the latents are extracted from one speaker and then reconstructed through the decoder using a separate speaker id.

Takeways

Speech models and products have been advancing rapidly since 2010 when first speech deep learning models of industry scale came along so it’s interesting to write about recent trends. Let me know in the comments what strikes your interest in Speech.

References

[1] https://9to5google.com/2020/01/14/weird-hey-google-alternatives

[2] https://www.dlology.com/blog/how-to-do-real-time-trigger-word-detection-with-keras/

[3] https://ai.googleblog.com/2018/03/expressive-speech-synthesis-with.html

[4] https://avdnoord.github.io/homepage/vqvae/