Patent Description:
Automatic speech recognition (ASR) models receive audio data as an input and generate a transcription as an output, e.g., a predicted transcription, based on the received audio data input. ASR models are trained on training data including audio data with corresponding transcriptions. However, ASR models may leak the training data it has memorized during training. This data leakage of private and/or sensitive training data is generally undesirable. However, testing the robustness of an ASR model to detect the leakage of training data can be challenging. It is known from <NPL>, a methodology to detect whether specific information such as accent of a speaker has been used during training of an ASR model. This methodology allows to ultimately infer how a machine learning algorithm has been trained. It is otherwise known from<NPL>, a methodology to reveal speaker identity knowing that in the speech domain, training data includes audio and transcripts of utterances which can directly expose sensitive information, or make it possible to leak attributes such as gender, dialect, or identity of the speaker.

The invention is as defined in the appended independent claims <NUM> and <NUM>. One aspect of the disclosure provides a computer-implemented method of phrase extraction for automatic speech recognition (ASR) models. The computer-implemented method when executed on data processing hardware causes the data processing hardware to perform operations that include obtaining audio data characterizing an utterance and a corresponding ground-truth transcription of the utterance and modifying the audio data to obfuscate a particular phrase recited in the utterance. The operations also include processing, using a trained ASR model, the modified audio data to generate a predicted transcription of the utterance, and determining whether the predicted transcription includes the particular phrase by comparing the predicted transcription of the utterance to the ground-truth transcription of the utterance. When the predicted transcription includes the particular phrase, the operations include generating an output indicating that the trained ASR model leaked the particular phrase from a training data set used to train the ASR model.

Implementations of the disclosure may include one or more of the following optional features. In some implementations, the operations further include, when the predicted transcription includes another phrase substituted for the particular phrase from the ground-truth transcription that is associated with a same category of information as the particular phrase, generating an output indicating that the trained ASR model leaked the other phrase from the training data set used to train the ASR model. In some examples, the operations further include, when the predicted transcription does not include the particular phrase or another phrase substituted for the particular phrase from the ground-truth transcription that is associated with a same category of information as the particular phrase, generating an output indicating that the trained ASR model has not leaked any information from the training data set used to train the ASR model.

In some implementations, the audio data includes an audio waveform. In these implementations, the audio waveform may correspond to human speech. Alternatively, the audio waveform may conform to synthesized speech.

In some examples, modifying the audio data includes identifying a segment of the audio data that aligns with the particular phrase in the ground-truth transcription based on the ground-truth transcription, and performing data augmentation on the identified segment of the audio data to obfuscate the particular phrase recited in the utterance. In these examples, performing data augmentation on the identified segment of the audio data may include adding noise to the identified segment of the audio data. Alternatively, performing data augmentation on the identified segment of the audio data may include replacing the identified segment of the audio data with noise.

In some implementations, the operations further include processing the ground-truth transcription of the utterance to identify any phrases included in the ground-truth transcription that are associated with a specific category of information. Here, modifying the audio data occurs in response to identifying that the particular phrase included in the ground-truth transcription is associated with the specific category of information. In these implementations, the specific category of information may include names, addresses, dates, zip codes, patient diagnosis, account numbers, or telephone numbers.

Another aspect of the disclosure provides a system of phrase extraction for ASR models. The system includes data processing hardware and memory hardware in communication with the data processing hardware. The memory hardware stores instructions that when executed on the data processing hardware causes the date processing hardware to perform operations including obtaining audio data characterizing an utterance and a corresponding ground-truth transcription of the utterance and modifying the audio data to obfuscate a particular phrase recited in the utterance. The operations also include processing, using a trained ASR model, the modified audio data to generate a predicted transcription of the utterance, and determining whether the predicted transcription includes the particular phrase by comparing the predicted transcription of the utterance to the ground-truth transcription of the utterance. When the predicted transcription includes the particular phrase, the operations include generating an output indicating that the trained ASR model leaked the particular phrase from a training data set used to train the ASR model.

This aspect may include one or more of the following optional features. In some implementations, the operations further include, when the predicted transcription includes another phrase substituted for the particular phrase from the ground-truth transcription that is associated with a same category of information as the particular phrase, generating an output indicating that the trained ASR model leaked the other phrase from the training data set used to train the ASR model. In some examples, the operations further include, when the predicted transcription does not include the particular phrase or another phrase substituted for the particular phrase from the ground-truth transcription that is associated with a same category of information as the particular phrase, generating an output indicating that the trained ASR model has not leaked any information from the training data set used to train the ASR model.

Implementations herein are directed toward a phrase extraction mechanism for use in detecting leakage of training data used to train an automated speech recognition (ASR) model. Identifying phrases of a speech utterance to replace or mask with noisy audio data and analyzing the output of the trained ASR model to determine whether the trained ASR model output the true transcription of the utterance demonstrates whether a leakage of training data from the trained ASR model has occurred. The phrase extraction mechanism can be used to test whether various types of sensitive information in a speech waveform, such as address, zip code, patient diagnosis, etc. are leaked from a trained ASR model.

<FIG> is an example of a speech environment including a system <NUM> having a phrase extraction mechanism <NUM> executing on a computing device <NUM>. The computing device <NUM> (also referred to generally as a device <NUM>) is configured to capture sounds (e.g., audio data) from one or more users <NUM> within the speech environment. Here, the audio data <NUM> may refer to a spoken utterance <NUM> by the user <NUM> that functions as a test utterance for a trained ASR model <NUM> to recognize. Speech-enabled systems of the device <NUM>, such as the trained ASR model <NUM>, may perform speech recognition on the audio data <NUM> to generate a transcription <NUM> of a corresponding utterance <NUM> spoken by the user <NUM>.

The computing device <NUM> includes data processing hardware <NUM> and memory hardware <NUM> in communication with the data processing hardware <NUM> and storing instructions, that when executed by the data processing hardware <NUM>, cause the data processing hardware <NUM> to perform one or more operations. The computing device <NUM> could include a distributed system (cloud computing environment). The computing device <NUM> also includes a display <NUM> and/or is in communication with one or more displays. The display <NUM> may display a leakage report <NUM> indicating whether or not the trained ASR model <NUM> is leaking potentially sensitive data from a training data set <NUM> (<FIG>) used to train the ASR model <NUM>. The computing device <NUM> may include an audio system with an audio capture device (e.g., microphone) for capturing and converting spoken utterances <NUM> within the speech environment <NUM> into electrical signals and a speech output device (e.g., a speaker) for communicating an audible audio signal (e.g., as output audio data from the device <NUM>). The ASR model <NUM> and/or a leakage detector <NUM> may execute on the computing device <NUM> as the audio-text aligner <NUM> and the data augmenter <NUM>, or the ASR model <NUM> and/or the leakage detector <NUM> may execute on other computing devices in communication with the computing device <NUM>.

As described in greater detail below, the leakage detector <NUM> may check, or verify the robustness of the model <NUM> by determining whether the model <NUM> still recognizes particular phrases <NUM> from the utterance <NUM> that were replaced with noise in modified audio data <NUM> by a data augmenter <NUM>, and output an indication <NUM> indicating whether the model <NUM> leaked the one or more particular phrases <NUM> from the training data set <NUM> used to train the model <NUM>. For instance, the leakage detector <NUM> may output an indication <NUM> indicating that the model <NUM> leaked potentially sensitive rare training data memorized by the model <NUM> during training. <FIG> depicts the computing device <NUM> capturing audio data <NUM> including an audio waveform of human speech characterizing the utterance <NUM> "Mr. Soames and Mr. Havisham were overwhelmed by this information" spoken by the user <NUM>. Optionally, the user <NUM> may input a textual representation/transcription <NUM> of the utterance <NUM> and a text-to-speech (TTS) system <NUM> may convert the transcription <NUM> into the audio data <NUM> that includes an audio waveform of synthesized speech characterizing the utterance <NUM>.

In some implementations, the leakage detector <NUM> determines whether the model <NUM> leaked the one or more particular phrases <NUM> from the training data set <NUM> used to train the model <NUM> without receiving audio data <NUM> corresponding to the one or more particular phrases <NUM>. In other words, the audio data <NUM> including the audio waveform of human or synthesized speech may only include other terms/phrases from the ground-truth transcription <NUM> while omitting the one or more particular phrases <NUM>. In these implementations, for a transcription <NUM> only containing one particular phrase <NUM> to be omitted, the audio data <NUM> is broken into two segments where the first segment terminates at a start boundary of where the particular phrase <NUM> would be located and the second audio segment commences at an end boundary of where the particular phrase <NUM> would be located. The data augmenter <NUM> may produce modified audio data <NUM> by generating a masked/augmented audio segment <NUM> (<FIG>) associated with a duration of time of noise and combining the two audio segments of audio data <NUM> with the masked/augmented audio segment <NUM> inserted there between. Accordingly, the leakage detector <NUM> may compare the modified audio data <NUM> and the ground-truth transcription <NUM> (including the particular phrase <NUM>) to check whether the model <NUM> is leaking any of the particular phrases <NUM> or similar phrases <NUM> of varied length from the training data set <NUM> used to train the ASR model <NUM>.

For example, rather than providing the transcription <NUM> "I know Mr. Havisham has this information" to the TTS system <NUM> for conversion into audio data <NUM> corresponding to synthesized speech, the transcription <NUM> may be broken into the segments "I know Mr. " and "has this information" that omits the particular phrase <NUM> "Havisham" for input to the TTS system <NUM>. The ground-truth transcription <NUM> input to the leakage detector <NUM> would still include the particular phrase <NUM> "Havisham". When the TTS system <NUM> converts the two segments of the transcription <NUM> into corresponding synthesized speech segments of audio data <NUM>, the data augmenter <NUM> may generate a corresponding masked audio segment <NUM> and combine the two synthesized speech segments/phrases with the masked audio segment <NUM> disposed there between to produce the modified audio data <NUM>. Here, the masked audio segment <NUM> may include a noisy audio segment having a duration of time associated with the particular phrase <NUM> "Havisham". Notably, the data augmenter <NUM> may also vary the duration of time of the augmented/masked audio segment <NUM> in attempts to determine if the ASR model <NUM> leaks similar phrases <NUM> (e.g., other character names such as Hale if the duration is shortened). Likewise, when the audio data <NUM> corresponds to human speech spoken by the user <NUM>, the user <NUM> may simply speak two phrases "I know mister" and "has this information. " In this scenario, the user <NUM> could provide some input indicating that the data augmenter <NUM> should generate the corresponding masked audio segment <NUM> for insertion between the two spoken phrases to produce the modified audio data <NUM>.

As will become apparent, whether the audio data <NUM> includes the audio waveform of human or synthesized speech, the transcription <NUM> of the utterance <NUM> will also serve as a corresponding ground-truth transcription <NUM> of the utterance <NUM> for comparison with a predicted transcription <NUM> output by the trained ASR model <NUM> based on speech recognition performed on the modified audio data <NUM> obfuscating/masking the particular phrases 116a, 116b "Soames" and "Havisham" from the original input utterance <NUM>. The ground-truth transcription <NUM> includes the one or more particular phrases <NUM> along with other the terms/phrases and corresponding time information <NUM> associated with the audio data <NUM>. The particular phrases <NUM> include phrases to be augmented/replaced with noise so that they are obfuscated, or otherwise not recognizable, in the modified audio data <NUM>. In some examples, the user <NUM> provides an input to the computing device <NUM> explicitly identifying the particular phrases <NUM> to be obfuscated. In other examples, the phrase extraction mechanism <NUM> identifies the particular phrases included in the transcription <NUM> that are associated with a specific category of information. In these examples, the user <NUM> may provide inputs indicating one or more specific categories of information and the phrase extraction mechanism <NUM> may process the ground-truth transcription <NUM> to identify any phrases associated with the specific categories of information. Accordingly, the particular phrases <NUM> may be identified based on an explicit user input indicating selection of the phrases <NUM> or the particular phrases <NUM> may be identified as being associated with a specific category of information. As used herein, a phrase may include one or more terms such as name (e.g., Havisham or John Smith), a city (e.g., New Haven), a sequence of numbers in a ZIP code (e.g., <NUM>-<NUM>-<NUM>-<NUM>-<NUM>), an address (e.g., <NUM> Lavender Ln), a date of birth (e.g., January <NUM>, <NUM>), etc. The time information <NUM> may include time stamps for start/end boundaries of each word/term in the transcription <NUM>. In other words, the time information <NUM> defines boundaries where each word/term starts and stops in the transcription <NUM>. In some implementations, where the transcription <NUM> is paired with corresponding audio data and derived from the training data set <NUM> used to train the model <NUM>, the time information <NUM> may be obtained via annotated time stamps during training and stored with the training data set <NUM>. Alternatively, where the transcription <NUM> is provided to the TTS system <NUM> for conversion into audio data <NUM> corresponding to synthesized speech, the TTS system <NUM> may generate the time information <NUM> when performing text-to-speech conversion on the transcription <NUM>.

The phrase extraction mechanism <NUM> further includes an audio-text aligner <NUM> configured to receive the audio data <NUM> and the transcription <NUM> including the one or more phrases <NUM> and the corresponding time information <NUM>, and identify one or more segments <NUM> in the audio data <NUM> that align with the one or more particular phrases <NUM> identified in the transcription <NUM>. In other words, the audio-text aligner <NUM> receives the transcription <NUM> including the time information <NUM>, and uses the time information <NUM> to identify a segment <NUM> of the audio data <NUM> that aligns with the particular phrase <NUM> in the transcription <NUM>, and outputs the audio data <NUM> including each identified segment <NUM>. More specifically, the audio-text aligner <NUM> receives the transcription <NUM> including corresponding time information <NUM>, where the transcription <NUM> includes "mister soames and mister havisham were overwhelmed by this information". Here, "soames" is a particular phrase 116a and "havisham" is also a particular phrase 116b. The identified particular phrases 116a, 116b may belong to the same category of information <NUM> (<FIG>) (e.g., character names) within the training data set <NUM>. The audio-text aligner <NUM> uses the time information <NUM> to align the transcription <NUM> with the audio data <NUM> so that the audio-text aligner <NUM> can identify the segments 122a, 122b in the audio data <NUM> that convey the particular phrases 116a, 116b. In this example, the audio-text aligner <NUM> identifies segment 122a in the audio data <NUM> that aligns with the particular phrase "soames" 116a in the transcription <NUM>, and also identifies segment 122b in the audio data <NUM> that aligns with the particular phrase "havisham" 116b in the transcription <NUM>. Once the audio-text aligner <NUM> identifies the segments 122a, 122b in the audio data <NUM> that align with the particular phrases 116a, 116b, the audio data <NUM> including the identified segments 122a, 122b are provided to the data augmenter <NUM> for augmenting the audio data <NUM> to generate the modified audio data <NUM>.

The data augmenter <NUM> receives the audio data <NUM> including the one or more identified segments <NUM> corresponding to the one or more particular phrases <NUM> (e.g., from the audio-text aligner <NUM>) and performs data augmentation on each identified segment <NUM> of the audio data <NUM> to obfuscate the corresponding particular phrase <NUM> in the audio data <NUM> characterizing the utterance <NUM>. That is, the data augmenter <NUM> receives the audio data <NUM> and, based on the identified segment <NUM> corresponding to the particular phrase <NUM>, augments the audio data <NUM> to generate the modified audio data <NUM> including an augmented audio segment <NUM> (<FIG>) which obfuscates the particular phrase <NUM>. In some implementations, the data augmenter <NUM> modifies the audio data <NUM> using data augmentation techniques to distort each identified segment <NUM> that aligns with a corresponding identified particular phrase <NUM>. Examples herein refer to the data augmenter <NUM> modifying the audio data <NUM> by adding noise of a duration corresponding to the length of the identified segment <NUM>. Other data augmentation techniques may be applied in addition to or in lieu of noise.

After the data augmenter <NUM> modifies the audio data <NUM> by obfuscating the particular phrase <NUM> in the identified segment <NUM> to generate the modified audio data <NUM>, the model <NUM> receives the modified audio data <NUM> and, using the modified audio data <NUM>, generates, as output, the corresponding predicted transcription <NUM> of the modified audio data <NUM>. That is, the model <NUM> receives the modified audio data <NUM>, and, based on the modified audio data <NUM>, generates the predicted transcription <NUM> of the modified audio data <NUM>. The leakage detector <NUM> receives, as input, the predicted transcription <NUM> of the modified audio data <NUM> and the ground-truth transcription <NUM>, and generates, as output, the indication <NUM> indicating whether the trained ASR model <NUM> leaked any of the particular phrases <NUM> or similar phrases <NUM> from the training data set <NUM> used to train the ASR model <NUM>. That is, the leakage detector <NUM> compares the predicted transcription <NUM> of the modified audio data <NUM> to the ground-truth transcription <NUM> of the utterance <NUM> to determine whether the predicted transcription <NUM> includes the identified particular phrases <NUM> that were replaced with noise in the modified audio data <NUM>. In the example shown, the output indication <NUM> would indicate that the exact particular phrase 116b "havisham" was extracted from the modified audio data <NUM> despite the modified audio data <NUM> augmenting the corresponding audio segment 122b with noise. When the leakage detector <NUM> determines that the predicted transcription <NUM> does not include the particular phrase <NUM> or another similar phrase <NUM> substituted for the particular phrase <NUM> from the transcription <NUM> that is associated with a same category of information <NUM> as the particular phrase <NUM>, the leakage detector <NUM> generates an output <NUM> indicating that the model <NUM> has not leaked any particular phrases <NUM> from the category of information <NUM> from the training data set <NUM> used to train the model <NUM>.

In some examples, the predicted transcription <NUM> includes another phrase <NUM> associated with a same category of information <NUM> (<FIG>) as the particular phrase <NUM> from the transcription <NUM>. For example, the category of information <NUM> may include character names, addresses, zip codes, dates of birth, patient diagnosis, or any other category of information <NUM> that refers to a type of training data set <NUM> used to train the model <NUM>. In these examples, the leakage detector <NUM> generates an output <NUM> indicating that the model <NUM> leaked the other phrase <NUM> from the training data set <NUM> used to train the model <NUM>. In the example shown, the output indication <NUM> would indicate that a similar phrase <NUM> "hale" was extracted from the modified audio data <NUM> at the location of the corresponding audio segment 122a augmented with noise to obfuscate the particular phrase 116a "soames". Here, "hale" and "soames" both correspond to character names from an audio book used to train the ASR model <NUM>, and the ASR model <NUM> recognizing the phrase "hale" reveals that the trained ASR model <NUM> is leaking data from the training data set <NUM> by substituting the similar phrase <NUM> "hale" for the particular phrase 116a "soames".

In some examples, the output <NUM> includes a leakage report <NUM> for review by the user <NUM>. For instance, the leakage report <NUM> may be displayed on a display/screen <NUM> of the user device <NUM> or another display device associated with the user <NUM>. The user device <NUM> may execute a user interface generator configured to present a representation of the leakage report <NUM> to the user <NUM> of the user device <NUM>. The leakage report <NUM> may include a notification/warning when leaks are detected, or a notification that the model <NUM> has not leaked information from the training data set <NUM>. In some examples, the leakage report <NUM> includes an aggregate of outputs <NUM> from multiple audio data <NUM> and transcription <NUM> pairs to provide a percentage of particular phrases extracted (e.g., exact particular phrases <NUM> or other similar phrases <NUM> substituted for the particular phrase <NUM> and associated with the same category of information <NUM>), and thus, are leaked by the model <NUM>. The leakage report <NUM> may also indicate that the trained ASR model <NUM> is leaking data from the training data set <NUM> associated with some specific categories of information <NUM> but not others. For instance, leakage report <NUM> could indicate detected leaks in the category of information <NUM> related to names/proper nouns but no detected leaks in other categories of information <NUM> related date of birth, account numbers, etc. In these scenarios, the indication in the leakage report <NUM> indicating that the ASR model <NUM> is leaking data from some categories of information <NUM> and not others can be based on categories of information associated with the particular phrases <NUM> identified to be replaced with noise from multiple audio data <NUM> and transcription <NUM> pairs input to the phrase extraction model <NUM>.

In some implementations, when the leakage detector <NUM> determines that the model <NUM> has leaked particular phrases <NUM> and/or similar phrases <NUM> included in the category of information <NUM> from the training data set <NUM> used to train the model <NUM>, the leakage detector <NUM> additionally generates model updates <NUM> for the model <NUM> to improve the security/privacy of the model <NUM>. That is, the model updates <NUM> may update the model <NUM> so that it no longer leaks data (e.g., particular phrases <NUM> or similar phrases <NUM>). In these implementations, the ASR system <NUM> may be automated, whereby the leakage detector <NUM> automatically provides the model updates <NUM> to the model <NUM>. The model updates <NUM> may include parameters that cause the model <NUM> to output symbols "#####" or just blank spaces in future predicted transcriptions <NUM> where the identified segment <NUM> of audio data <NUM> that aligns with the particular phrase <NUM> occurs so that the particular phrase <NUM> is not in the predicted transcription <NUM> output by the model <NUM>. Additionally or alternatively, the model updates <NUM> may be included in the leakage report <NUM> to convey steps for the user <NUM> to undertake to improve the security of the model <NUM>.

Referring briefly to <FIG>, the data augmenter <NUM> obfuscates the particular phrases 116a, 116b recited in the utterance <NUM> by performing data augmentation on the identified segments 122a, 122b in the audio data <NUM> to generate corresponding masked segments <NUM>. As used herein, "masked segments" and "augmented segments" may be used interchangeably. The data augmenter <NUM> may perform data augmentation by adding noise to the identified segments 122a, 122b of the audio data <NUM>. In some examples, the data augmenter <NUM> concatenates the identified segments 122a, 122b in the audio data <NUM> with noisy audio segments to produce augmented audio segments 302a, 302b which include both the audio data <NUM> and the added noisy audio segments. In other examples, the data augmenter <NUM> generates the augmented audio segments 302a, 302b by replacing the audio data <NUM> in the identified segments 122a, 122b with noisy audio segments. In other words, the data augmenter <NUM> removes the audio data <NUM> occurring in the identified segments 122a, 122b, and inserts a corresponding noisy audio segment to generate the augmented audio segments 302a, 302b in the modified audio data <NUM>.

In some implementations, to limit overflow of the particular phrases 116a, 116b in the identified segments 122a, 122b into the remainder of the audio data <NUM>, the data augmenter <NUM> includes padding in the augmented audio segments 302a, 302b. The padding may be an amount of time (e.g., <NUM> milliseconds) to separate the identified segments 122a, 122b from the remainder of the audio data <NUM>. In these implementations, the padding is added before and after each of the identified segments 122a, 122b. Once the padding has been added to the identified segments 122a, 122b, the data augmenter <NUM> performs data augmentation on the padding and the identified segments 122a, 122b to generate the augmented audio segments 302a, 302b in the modified audio data <NUM>.

Referring back to <FIG>, the modified audio data <NUM> including the augmented audio segments 302a, 302b obfuscating the identified segments 122a, 122b is provided to the model <NUM> which, in turn generates the predicted transcription <NUM> of "mister hale and mister havisham were overwhelmed by this information. " In this example, the predicted transcription <NUM> includes the particular phrase 116b "havisham," and another phrase <NUM> "hale" substituted for the particular phrase 116a "soames" from the transcription <NUM>. The other phrase <NUM> "hale" is associated with the same category of information <NUM> as the particular phrase 116a "soames," where both the other phrase <NUM> and the particular phrase 116a are associated with the same category of information <NUM> (e.g., character names) in the training data set <NUM> used to train the model <NUM>.

The leakage detector <NUM> compares the predicted transcription <NUM>, including the other phrase <NUM> and the particular phrase 116b, and the ground-truth transcription <NUM>, including the identified particular phrases 116a, 116b, to determine whether or not the trained TTS model <NUM> leaked the particular phrase 116b and the other phrase <NUM> from the training data set <NUM> used to train the model <NUM>. Here, the leakage detector <NUM> may determine that the particular phrase 116b includes an exact match extracted from the modified audio data. The leakage detector <NUM> may further determine that while the other phrase <NUM> "hale" is not an exact match for the particular phrase 116a "soames", the other phrase <NUM> corresponds to a similar phrase since "hale" and "somaes" are both associated with the same category of information (e.g., character names) in the training data set <NUM> (e.g., an audio book) used to train the ASR model <NUM>. In response to determining that the model <NUM> leaked the particular phrase 116b and the other phrase <NUM>, the leakage detector <NUM> generates the output <NUM> indicating that the model <NUM> leaked the particular phrase 116b and the other phrase <NUM> from the training data set <NUM> used to train the model <NUM>.

The output <NUM> from the leakage detector <NUM> may further include, or be used to compile, the leakage report <NUM> conveying "Warning: The trained ASR model is leaking data from a training data set. " The warning may occur when a threshold number of particular phrases or similar phrases are extracted from audio data <NUM> characterizing multiple test utterances. In the example shown, leakage report <NUM> further conveys the aggregate of one or more previous outputs <NUM> including the particular phrases <NUM> leaked as "<NUM>% exact phrases extracted" and the aggregate of one or more previous outputs <NUM> including the other phrases <NUM> leaked as "<NUM>% similar phrases extracted. " As discussed above, the leakage report <NUM> may be displayed on a screen <NUM> of the user device <NUM> to notify the user <NUM> of the user device <NUM> that a leak has or has not been detected. The leakage detector <NUM> may further provide model updates <NUM> to the model <NUM> to improve the security/privacy of the model <NUM> and prevent further leakage of information from the training data set <NUM> used to train the model <NUM>. The model <NUM> may use the model updates <NUM> to update the model <NUM> automatically and/or the model updates <NUM> may include user instructions/suggestions for updating the model <NUM> to prevent data leaks from the training data set <NUM>.

Referring to <FIG>, the phrase extraction mechanism <NUM> may include a phrase identifier <NUM>, which allows the user <NUM> to select a specific category of information <NUM> of the training data set <NUM> for the phrase identifier <NUM> to identify in the utterance <NUM>. In some examples, the user <NUM> provides a category selection input <NUM> corresponding to a category of information <NUM> in the training data set <NUM> used to train the model <NUM>. In other examples, the user <NUM> identifies the particular phrases <NUM> in the transcription <NUM> to obfuscate in the modified audio data <NUM>. For instance, the transcription <NUM> may be displayed on the screen <NUM> and the user <NUM> could provide a user input indication to identify the particular phrases <NUM> to obfuscate in the modified audio data <NUM>.

Each category of information <NUM> in the training data set <NUM> may include one or more example phrases <NUM>, 214a-n corresponding to the category of information <NUM>. When the user <NUM> provides the category selection input <NUM> to the phrase identifier <NUM>, the phrase identifier <NUM> may process the transcription <NUM> to identify example phrases <NUM> corresponding to the category of information <NUM> that are present in the transcription <NUM>. The phrase identifier <NUM> outputs these identified example phrases <NUM> as the identified particular phrases <NUM> in the transcription <NUM>. In other words, the user <NUM> inputs the category selection input <NUM> for the phrase identifier <NUM> to identify particular phrases <NUM> in the utterance <NUM> belonging to the category <NUM> associated with the category selection input <NUM>. For instance, the category of information <NUM> may correspond to character names from an audio book used to train the ASR model <NUM>. Here, the example phrases <NUM> may include a list of all the character names (e.g., soames, havisham, hale, etc.) that occur in the training data set <NUM>.

As shown in <FIG>, the category selection input <NUM> and the corresponding ground-truth transcription <NUM> are provided to the phrase identifier <NUM>, and in response, the phrase identifier <NUM> determines whether the ground-truth transcription <NUM> of the utterance <NUM> includes any particular phrases <NUM> associated with the category of information <NUM> specified by the category selection input <NUM>. In this example, the category selection input <NUM> specifies the specific category of information 212a of names (e.g., "look for character names"). The category of information 212a of names includes example phrases 214a-n, which may include "soames," "hale," and "havisham. " The phrase identifier <NUM> processes the category of information 212a including example phrases 214a-n and the transcription <NUM> and automatically determines that the transcription <NUM> includes the particular phrases 116a, 116b of "soames" and "havisham," which belong to the specific category of information 212a input as the category selection input <NUM>. The phrase identifier <NUM> then provides the transcription <NUM> to the audio-text aligner <NUM> for modification. In other words, the audio-text aligner <NUM> and the data augmenter <NUM> modify the audio data <NUM> in response to the phrase identifier <NUM> identifying that the particular phrases 116a, 116b are included in the transcription <NUM> associated with the specific category of information 212a of names. This example is illustrative only and is not intended to limit the scope of the categories of information <NUM>. The categories of information <NUM> may include any category that characterizes the training data set <NUM>, such as names, addresses, dates, zip codes, patient diagnosis, account numbers, or telephone numbers.

<FIG> is a flowchart of an exemplary arrangement of operations for a method <NUM> of phrase extraction for ASR models. The method <NUM> includes, at operation <NUM>, obtaining audio data <NUM> characterizing an utterance <NUM> and a corresponding ground-truth transcription <NUM> of the utterance <NUM>. At operation <NUM>, the method <NUM> includes modifying the audio data <NUM> to obfuscate a particular phrase <NUM> recited in the utterance <NUM>.

At operation <NUM>, the method <NUM> also includes processing, using a trained ASR model <NUM>, the modified audio data <NUM> to generate a predicted transcription <NUM> of the utterance <NUM>. The method <NUM> further includes, at operation <NUM>, determining whether the predicted transcription <NUM> includes the particular phrase <NUM> by comparing the predicted transcription <NUM> of the utterance <NUM> to the ground-truth transcription <NUM> of the utterance <NUM>. At operation <NUM>, the method <NUM> also includes, when the predicted transcription <NUM> includes the particular phrase <NUM>, generating an output <NUM> indicating that the trained ASR model <NUM> leaked the particular phrase <NUM> from a training data set <NUM> used to train the ASR model <NUM>.

The components shown here, their connections and relationships, and their functions, are meant to be exemplary only, and are not meant to limit implementations of the implementations described and/or claimed in this document.

The processor <NUM> (e.g., data processing hardware <NUM> of <FIG>) can process instructions for execution within the computing device <NUM>, including instructions stored in the memory <NUM> or on the storage device <NUM> to display graphical information for a graphical user interface (GUI) on an external input/output device, such as display <NUM> coupled to high speed interface <NUM>.

The memory <NUM> (e.g., memory hardware <NUM> of <FIG>) stores information non-transitorily within the computing device <NUM>.

Claim 1:
A computer-implemented method (<NUM>) when executed on data processing hardware (<NUM>) causes the data processing hardware (<NUM>) to perform operations comprising:
obtaining audio data (<NUM>) characterizing an utterance (<NUM>) and a corresponding ground-truth transcription (<NUM>) of the utterance (<NUM>);
modifying the audio data (<NUM>) to obfuscate a particular phrase (<NUM>) recited in the utterance (<NUM>);
processing, using a trained automated speech recognition (ASR) model (<NUM>), the modified audio data (<NUM>) to generate a predicted transcription (<NUM>) of the utterance (<NUM>);
determining whether the predicted transcription (<NUM>) includes the particular phrase (<NUM>) by comparing the predicted transcription (<NUM>) of the utterance (<NUM>) to the ground-truth transcription (<NUM>) of the utterance (<NUM>); and
when the predicted transcription (<NUM>) includes the particular phrase (<NUM>), generating an output (<NUM>) indicating that the trained ASR model (<NUM>) leaked the particular phrase (<NUM>) from a training data set (<NUM>) used to train the ASR model (<NUM>).