Patent Description:
A speech-enabled environment (e.g., home, workplace, school, automobile, etc.) allows a user to speak a query or a command out loud to a computer-based system that fields and answers the query and/or performs a function based on the command. For example, the speech-enabled environment is implemented using a network of connected microphone devices distributed through various rooms or areas of the environment. As these environments become more ubiquitous and as speech recognition devices become more advanced, speech is increasingly used for important functions that include, for example, identification and authentication of the speaker. These functions greatly increase the need of ensuring that speech originates from a human and is not synthetic (i.e., digitally created or altered and played via a speaker).

Relevant prior art dealing with the detection of whether audio data includes synthetic speech is provided in <CIT> (<NUM>-<NUM>-<NUM>) and in <CIT> (<NUM>-<NUM>-<NUM>).

In accordance with the invention a method for classifying whether audio data includes synthetic speech is provided in claim <NUM>.

Implementations of the disclosure may include one or more of the following optional features. In some implementations, the shallow discriminator model includes an intelligent pooling layer. In some examples, the method further includes generating, by the data processing hardware, using the intelligent pooling layer of the shallow discriminator model, a single final audio feature vector based on each audio feature vector of the plurality of audio feature vectors. Generating the score indicating the presence of the synthetic speech in the audio data may be based on the single final audio feature vector.

Optionally, the single final audio feature vector includes an averaging of each audio feature vector of the plurality of audio feature vectors. Alternatively, the single final audio feature vector includes an aggregate of each audio feature vector of the plurality of audio feature vectors. The shallow discriminator model may include a fully-connected layer configured to receive, as input, the single final audio feature vector and generate, as output, the score.

In some implementations, the shallow discriminator model includes one of a logistic regression model, a linear discriminant analysis model, or a random forest model. In some examples, the trained self-supervised model is trained on a first training dataset including only training samples of human-originated speech. Optionally, the data processing hardware resides on the user device. The trained self-supervised model may include a representation model derived from a larger trained self-supervised model.

In accordance with the invention a system as defined in claim <NUM> is provided.

As speech-enabled environments and devices become more common and sophisticated, reliance on using audio as a reliable indicator of human-originated speech is increasingly important. For example, speech biometrics is commonly used for speaker verification. Automatic speaker verification (ASV) is the authentication of individuals by performing analysis on speech utterances. However, with the advent of synthetic media (e.g., "deepfakes"), it is critically important for these systems to accurately determine when a speech utterance includes synthetic speech (i.e., computer-generated audio output that resembles human speech). For example, state of the art text-to-speech (TTS) and voice conversion (VC) systems can now closely mimic human speakers, which provide avenues to attack and deceive ASV systems.

In one example, an ASV system implementing a speaker verification model is used in conjunction with a hotword detection model so that an authorized user can invoke a speech-enabled device to wake-up and process subsequent spoken input from the user by speaking a predefined fixed phrase (e.g., a hotword, wake word, keyword, invocation phrase, etc.). In this example, the hotword detection model is configured to detect audio features characterizing the predefined fixed phrase in audio data and the speaker verification model is configured to verify that the audio features characterizing the predefined fixed phrase were spoken by the authorized user. Generally, the speaker verification model will extract a verification speaker embedding from the input audio features and compare the verification speaker embedding with a reference speaker embedding for the authorized user. Here, the reference speaker embedding can be previously obtained by having the particular user speaker the same predefined fixed phrase (e.g., during an enrollment process) and stored as part of a user profile for the authorized user. When the verification speaker embedding matches the reference speaker embedding, the hotword detected in the audio data is verified as being spoken by the authorized user to thereby permit the speech-enabled device to wake-up and process subsequent speech spoken by the authorized user. The aforementioned state-of-the art TTS and VC systems could be used to generate a synthesized speech representation of the predefined fixed phrase in the voice of the authorized user to spoof the speaker verification model into verifying that the synthesized speech representation was spoken by the authorized user.

Machine learning (ML) algorithms such as neural networks have primarily driven the surge of AVS systems and other speech-enabled technologies. However, these algorithms conventionally require vast amounts of training samples such that the primary bottleneck in training accurate models frequently rests on the lack of sufficiently large and high-quality datasets. For example, large datasets that include human-originated speech are readily available, but similar datasets that instead include synthetic speech are not. Thus, training a model that can accurately determine synthetic speech without conventional training sets poses a significant problem for the development of synthetic speech detection systems.

Implementations herein are directed toward detecting synthetic speech in audio data based on a self-supervised model that extracts audio features from the audio data and a shallow discriminator model that determines a probability that synthetic speech is present in the audio features and thus in the audio data. The self-supervised model may be trained exclusively on data containing human-originated speech and not synthetic speech, thus bypassing bottlenecks caused from lack of sufficient quantity of synthetic speech samples. Meanwhile, the shallow discriminator may be trained on a small quantity (relative to the self-supervised model) of training samples that include synthetic speech while still maintaining a high degree of accuracy.

Referring to <FIG>, in some implementations, an example system <NUM> includes a user device <NUM>. The user device <NUM> may correspond to a computing device, such as a mobile phone, computer (laptop or desktop), tablet, smart speaker/display, smart appliance, smart headphones, wearable, vehicle infotainment system, etc., and is equipped with data processing hardware <NUM> and memory hardware <NUM>. The user device <NUM> includes or is in communication with one or more microphones <NUM> for capturing utterances from an audio source <NUM>. The audio source <NUM> may be a human producing human-originated utterances <NUM> or an audio device (e.g., a loudspeaker) that coverts electrical audio signals into corresponding utterances <NUM>. A loudspeaker may be part of or in communication with any manner of computing or user device (e.g., a mobile phone, a computer, etc.).

The user device <NUM> includes an audio feature extractor <NUM> configured to extract audio features from audio data <NUM> characterizing speech obtained by the user device <NUM>. For example, the audio data <NUM> is captured from streaming audio <NUM> by the user device <NUM>. In other examples, the user device <NUM> generates the audio data <NUM>. In some implementations, the audio feature extractor <NUM> includes a trained neural network (e.g., a memorialized neural network such as convolutional neural network) received from a remote system <NUM> via a network <NUM>. The remote system <NUM> may be a single computer, multiple computers, or a distributed system (e.g., a cloud environment) having scalable / elastic computing resources <NUM> (e.g., data processing hardware) and/or storage resources <NUM> (e.g., memory hardware).

In some examples, the audio feature extractor <NUM> executing on the user device <NUM> is a self-supervised model. That is, the audio feature extractor <NUM> is trained using self-supervised learning (also referred to as "unsupervised learning") where labels are naturally part of the training sample and do not include separate external labels. More specifically, with self-supervised learning methods, models look for patterns in a data set without any pre-existing labels (i.e., annotation) and with minimal human supervision.

In the example shown, the audio source <NUM> produces an utterance <NUM> that includes the speech "My name is Jane Smith. " The audio feature extractor <NUM> receives audio data <NUM> characterizing the utterance <NUM> in the streaming audio <NUM> and generates, from the audio data <NUM>, a plurality of audio feature vectors <NUM>, 212a-n. Each audio feature vector <NUM> represents audio features (i.e., audio characteristics such as spectrograms (e.g., mel-frequency spectrograms and mel-frequency ceptstral coefficients (MFCCs)) of a chunk or portion of the audio data <NUM> (i.e., a portion of the streaming audio <NUM> or utterance <NUM>). For example, each audio feature vector represents features for a <NUM> millisecond portion of the audio data <NUM>. The portions may overlap. For instance, the audio feature extractor <NUM> generates eight audio feature vectors <NUM> (each representing <NUM> milliseconds of the audio data <NUM>) for five seconds of audio data <NUM>. The audio feature vectors <NUM> from the audio feature extractor <NUM> capture a large number of acoustic properties of the audio data <NUM> based on the self-supervised learning.

After generating the audio feature vectors <NUM>, the audio feature extractor <NUM> sends the audio feature vectors <NUM> to a synthetic speech detector <NUM> that includes a shallow discriminator model <NUM>. As discussed in more detail below, the shallow discriminator model <NUM> is a shallow neural network (i.e., with little to no hidden layers) that generates, based on each of the audio feature vector <NUM>, a score <NUM> (<FIG>) that indicates a presence of synthetic speech in the streaming audio <NUM> based on the corresponding audio features of each audio feature vector <NUM>. The synthetic speech detector <NUM> determines whether the score <NUM> (e.g., a probability score) satisfies a synthetic speech detection threshold. When the score <NUM> satisfies the synthetic speech detection threshold, the synthetic speech detector <NUM> determines that the speech (i.e., the utterance <NUM>) in the streaming audio <NUM> captured by the user device <NUM> includes synthetic speech. The synthetic speech detector <NUM> may determine that the utterance <NUM> includes synthetic speech even when a majority of the utterance <NUM> includes human-originated speech (i.e., a small portion of synthetic speech is interjected or interspersed with human-originated speech).

In some implementations, the synthetic speech detector <NUM> generates an indication <NUM> to the user device <NUM> to indicate whether the streaming audio <NUM> includes synthetic speech based on whether the score <NUM> satisfies the synthetic speech detection threshold. For example, when the score <NUM> satisfies the synthetic speech detection threshold, the indication <NUM> indicates that the utterance <NUM> includes synthetic speech. In response, the user device <NUM> may generate a notification <NUM> to a user of the user device <NUM>. For example, the user device <NUM> executes a graphical user interface (GUI) <NUM> for display on a screen of the user device <NUM> in communication with the data processing hardware <NUM>. The user device <NUM> may render the notification <NUM> in the GUI <NUM>. Here, the indication <NUM> indicates that the streaming audio <NUM> included synthetic speech by rendering the message "Notification: Synthetic Speech Detected" on the GUI <NUM>. The provided notification <NUM> is exemplary only and the user device <NUM> may notify the user of the user device <NUM> with any other appropriate method. Additionally or alternatively, the synthetic speech detector <NUM> notifies other applications executing on the user device <NUM>. For example, an application executing on the user device <NUM> authenticates the user of the user device <NUM> to allow the user to access one or more restricted resources. The application may use biometric speech to authenticate the user (e.g., via utterances <NUM>). The synthetic speech detector <NUM> may provide the indication <NUM> to the application to alert the application that the utterance <NUM> included synthetic speech which may cause the application to deny authentication to the user. In another scenario, when an utterance <NUM> includes a hotword detected by the user device <NUM> in the streaming audio <NUM> to trigger the user device <NUM> to wake-up from a sleep state and initiate processing of subsequent speech, an indication <NUM> generated by the synthetic speech detector <NUM> indicating that the utterance <NUM> of the hotword includes synthetic speech can suppress the wake-up process on the user device <NUM>.

The user device <NUM> may forward the indication <NUM> to the remote system <NUM> via the network <NUM>. In some implementations, the remote system <NUM> executes the audio feature extractor <NUM> and/or the synthetic speech detector <NUM> instead of or in addition to the user device <NUM>. For example, the user device <NUM> receives the streaming audio <NUM> and forwards the audio data <NUM> (or some characterization of the audio data <NUM>) to the remote system for processing. The remote system <NUM> may include substantially more computational resources than user device <NUM>. Additionally or alternatively, the remote system <NUM> may be more secure from potential adversaries. In this scenario, the remote system <NUM> may transmit the indication <NUM> to the user device <NUM>. In some examples, the remote server <NUM> performs multiple authentication operations with the audio data <NUM> and returns a value indicating whether the authentication succeeded. In other implementations, the audio source <NUM> transmits the audio data <NUM> of the streaming audio <NUM> directly to the remote system <NUM> (e.g., via the network <NUM>) without a separate user device <NUM> at all. For example, the remote system <NUM> executes an application that uses speech biometrics. In this case, the audio source <NUM> includes a device that directly transmits the audio data <NUM> to the remote system <NUM>. For example, the audio source <NUM> is a computer that generates synthetic speech and transmits this synthetic speech (via the audio data <NUM>) to the remote system <NUM> without the synthetic speech being verbalized.

Referring now to <FIG>, schematic view <NUM> includes the audio feature extractor <NUM> executing a deep neural network <NUM>. The deep neural network <NUM> may include any number of hidden layers that are configured to receive the audio data <NUM>. In some implementations, the deep neural network <NUM> of the audio feature extractor <NUM> generates, from the audio data <NUM>, the plurality of audio feature vectors <NUM>, 212a-n (i.e., embeddings). The shallow discriminator model <NUM> receives the plurality of audio feature vectors <NUM> simultaneously, sequentially, or concatenated together. The plurality of audio feature vectors <NUM> may undergo some processing between the audio feature extractor <NUM> and the shallow discriminator model <NUM>. The shallow discriminator model <NUM> may generate the score <NUM> based on the plurality of audio feature vectors <NUM> generated/extracted by the deep neural network <NUM> of the audio feature extractor <NUM>.

Referring now to the schematic view <NUM> of <FIG>, in some examples, the shallow discriminator model <NUM> includes an intelligent pooling layer <NUM>, 310P. The intelligent pooling layer 310P receives the plurality of audio feature vectors <NUM> and may generate a single final audio feature vector 212F based on each audio feature vector <NUM> received from the audio feature extractor <NUM>. The shallow discriminator model <NUM> may generate the score <NUM> that indicates the presence of the synthetic speech in the streaming audio <NUM> based on the single final audio feature vector 212F. In some examples, the intelligent pooling layer 310P averages each audio feature vector <NUM> to generate the final audio feature vector 212F. In other examples, the intelligent pooling layer 310P aggregates each audio feature vector <NUM> to generate the final audio feature vector 212F. Ultimately, the intelligent pooling layer 310P in some manner distills the plurality of audio feature vectors <NUM> into the final audio feature vector 212F that includes or emphasizes the audio features that characterize human-originated speech versus synthetic speech. In some examples, the intelligent pooling layer 310P focuses the final audio feature vector 212F on a portion of the audio data <NUM> most likely to contain synthetic speech. For example, the audio data <NUM> includes a small or narrow portion that provides an indication (e.g., an audio characteristic) that the utterance <NUM> includes synthetic audio while the remaining portions of the audio data <NUM> provide little to no indication that the utterance <NUM> includes synthetic speech. In this example, the intelligent pooling layer 310P emphasizes the audio feature vector <NUM> associated with that portion of the audio data <NUM> (or otherwise deemphasizes the other remaining audio feature vectors <NUM>).

In some implementations, the shallow discriminator model <NUM> includes only one other layer <NUM> in addition to the intelligent pooling layer 310P. For example, the shallow discriminator model <NUM> includes a fully-connected layer 310F that is configured to receive, as input, the single final audio feature vector 212F from the intelligent pooling layer 310P and generate, as output, the score <NUM>. Thus, in some examples, the shallow discriminator model <NUM> is a shallow neural network that includes a single intelligent pooling layer 310P and only one other layer <NUM>, e.g., the fully-connected layer 310F. Each layer includes any number of neurons/nodes <NUM>. The single fully-connected layer 310F may map a result to a logit. In some examples, the shallow discriminator model <NUM> includes one of a logistic regression model, a linear discriminant analysis model, or a random forest model.

Referring now to <FIG>, in some implementations, a training process <NUM>, 400a trains the audio feature extractor <NUM> on a pool 402A of human-originated speech samples. These human-originated speech samples provide unlabeled audio extractor training samples 410A that train the untrained audio feature extractor <NUM>. The human-originated speech pool 402A may be quite large leading to a significant number of audio extractor training samples 410A. Thus, in some examples, the training process 400a trains the untrained audio feature extractor <NUM> on a large quantity of audio extractor training samples 410A that only include human-originated speech and does not include any synthetic speech. This is advantageous, as large pools of synthetic speech typically are expensive and/or difficult to obtain. However, in some examples, the audio extractor training samples 410A include samples with human-originated speech and synthetic speech. Optionally, the audio feature extractor <NUM> includes a representation model derived from a larger trained self-supervised model. In this scenario, the larger trained self-supervised model may be a very large model that is computationally expensive to execute and not well-suited to user devices <NUM>. However, due to potential advantages of executing the audio feature extractor <NUM> locally on the user device <NUM> (e.g., latency, privacy, bandwidth, etc.), the audio feature extractor <NUM> may be a representation model of the larger trained self-supervised model, which reduces the size and complexity of the model without sacrificing substantial accuracy. This allows the model to be executed on user devices <NUM> despite limited computational or memory capacity. Representation models improve performance by transforming high-dimensional data (e.g., audio) to a lower dimension to train small models and by using the representation model as pre-training.

Referring now to <FIG>, in some examples, a training process <NUM>, 400b trains the shallow discriminator model <NUM> subsequent to training the audio feature extractor <NUM>. In this example, the trained audio feature extractor <NUM> receives audio data <NUM> from a pool 402B of synthetic speech samples. The trained audio feature extractor <NUM>, based on the audio data <NUM> from the pool 402B, generates audio feature vectors <NUM> that represent discriminator training samples 410b. These discriminator training samples 410b (i.e., the plurality of audio feature vectors <NUM> generated by the trained audio feature extractor <NUM>) trains the shallow discriminator model <NUM>. While the shallow discriminator model <NUM> may be trained using synthetic speech from the synthetic speech pool 402B, the synthetic speech pool 402B may be substantially smaller than the human-originated speech pool 402A.

In some examples, the shallow discriminator model <NUM> is trained exclusively on training samples 410b that include synthetic speech while in other examples in accordance with the invention the shallow discriminator model <NUM> is trained on a mix of training samples 410b that include synthetic speech and training samples 410b that include purely human-originated speech. Samples 410b that include synthetic speech may include only synthetic speech (i.e., no human-originated speech). Samples 410b thus include a mix of synthetic speech and human-originated speech. For instance, in the example of <FIG>, the utterance <NUM> includes the speech "My name is Jane Smith. " A possible training sample 410b from this utterance 410b includes the "My name is" portion of the utterance <NUM> being human-originated speech while the "Jane Smith" portion of the utterance <NUM> being synthetic speech. The remote system <NUM> and/or user device <NUM> may perturb existing training samples 410b to generate additional training samples 410b. For example, the remote system replaces a portion of human-originated speech with synthetic speech, replace synthetic speech portions with human-originated speech portions, replace synthetic speech portions with different synthetic speech portions, and replace human-originated speech portions with different human-originated speech portions.

In some implementations, the remote system <NUM> executes the training processes 400a, 400b to train the audio feature extractor <NUM> and the shallow discriminator model <NUM> and then transmits the trained models <NUM>, <NUM> to the user device <NUM>. However, in other examples, the user device <NUM> executes the training processes 400a, 400b to train the audio feature extractor <NUM> and/or the shallow discriminator model <NUM> on the user device <NUM>. In some examples, the remote system <NUM> or the user device <NUM> fine-tunes the shallow discriminator model <NUM> based on new or updated training samples 410b. For example, the user device <NUM> updates, fine-tunes, or partially retrains the shallow discriminator model <NUM> on audio data <NUM> received from the audio source <NUM>.

Referring now to the schematic view <NUM> of <FIG>, in some examples, the user device <NUM> and/or the remote system <NUM> leverages the same audio feature extractor <NUM> to provide audio feature vectors <NUM> to multiple shallow discriminator models <NUM>, 222a-n. In this manner, the audio feature extractor <NUM> acts as a "front-end" model while the shallow discriminator models <NUM> act as "back-end" models. Each shallow discriminator model <NUM> may be trained for different purposes. For example, a first shallow discriminator model 222a determines whether speech is human-originated or synthetic while a second shallow discriminator model 222b recognizes and/or classifies emotions within the streaming audio <NUM>. That is, the self-supervised audio feature extractor <NUM> is well suited for "non-semantic" tasks (i.e., aspects of human speech other than meaning) that the shallow discriminator models <NUM> can take advantage of for a variety of different purposes. Due to the potentially small size and complexity of the shallow discriminator models <NUM>, the user device may store and execute each of them as needed to process the audio feature vectors <NUM> generated by the audio feature extractor <NUM>.

<FIG> provides a flowchart of example operations for a method <NUM> of determining whether audio data <NUM> includes synthetic speech. At operation <NUM>, the method <NUM> includes receiving, at data processing hardware <NUM>, audio data <NUM> characterizing speech obtained by a user device <NUM>. At operation <NUM>, the method <NUM> includes generating, by the data processing hardware <NUM>, using a trained self-supervised model <NUM> (i.e., the audio feature extractor <NUM>), a plurality of audio feature vectors <NUM> each representative of audio features of a portion of the audio data <NUM>. The method <NUM>, at operation <NUM>, also includes generating, by the data processing hardware <NUM>, using a shallow discriminator model <NUM>, a score <NUM> indicating a presence of synthetic speech in the audio data <NUM> based on the corresponding audio features of each audio feature vector <NUM> of the plurality of audio feature vectors <NUM>. The method <NUM>, at operation <NUM>, includes determining, by the data processing hardware <NUM>, whether the score <NUM> satisfies a synthetic speech detection threshold and, at operation <NUM>, when the score <NUM> satisfies the synthetic speech detection threshold, determining, by the data processing hardware <NUM>, that the speech in the audio data <NUM> captured by the user device <NUM> comprises synthetic speech.

For example, it may be implemented as a standard server 700a or multiple times in a group of such servers 700a, as a laptop computer 700b, or as part of a rack server system 700c.

Claim 1:
A method (<NUM>) comprising:
receiving, at data processing hardware (<NUM>), audio data (<NUM>) characterizing speech obtained by a user device (<NUM>);
generating, by the data processing hardware (<NUM>), a plurality of audio feature vectors (<NUM>) from the audio data using a trained self-supervised model (<NUM>), each audio feature vector (<NUM>) being representative of audio features of a portion of the audio data (<NUM>), wherein the trained self-supervised model (<NUM>) comprises a deep neural network;
generating, by the data processing hardware (<NUM>), using a shallow discriminator model (<NUM>), a score (<NUM>) indicating a presence of synthetic speech in the audio data (<NUM>) based on the corresponding audio features of each audio feature vector (<NUM>) of the plurality of audio feature vectors (<NUM>), wherein the shallow discriminator model (<NUM>) is a shallow neural network;
determining, by the data processing hardware (<NUM>), whether the score (<NUM>) satisfies a synthetic speech detection threshold; and
when the score (<NUM>) satisfies the synthetic speech detection threshold, determining, by the data processing hardware (<NUM>), that the speech in the audio data (<NUM>) obtained by the user device (<NUM>) comprises synthetic speech;
wherein the shallow discriminator model (<NUM>) is trained on training samples (410b), the training samples (410b) each including a synthetic speech portion and a human-originated speech portion.