Original: V ENO M AVE : Targeted Poisoning Against Speech Recognition
Revised: V ENO M AVE : Targeted Poisoning Against Speech Recognition

Original: Anonymous Submission Abstract —Despite remarkable improvements, automatic speech recognition is susceptible to adversarial perturbations.
Revised: Abstract —Despite remarkable improvements, automatic speech recognition is susceptible to adversarial perturbations.

Original: Compared to standard machine learning architectures, these attacksagainst speech recognition are significantly more challenging, especially since the inputs to a speech recognition system are time series that contain both acoustic and linguistic properties of speech.
Revised: Compared to standard machine learning architectures, these attacks are significantly more challenging, especially since the inputs to a speech recognition system are time series that contain both acoustic and linguistic properties of speech.

Original: Extracting all recognition-relevant information requires more complex pipelines and an ensemble of specialized components.
Revised: Extracting all recognition-relevant information requires more complex pipelines and an ensemble of specialized components.

Original: Consequently, an attacker needs to consider the entire pipeline.
Revised: Consequently, an attacker needs to consider the entire pipeline.

Original: In this paper, we present V ENO M AVE , the first training- time poisoning attack against speech recognition.
Revised: In this paper, we present V ENO M AVE , the first training- time poisoning attack against speech recognition.

Original: Similar to the pre-dominantly studied evasion attacks, we pursue the same goal: leading the system to an incorrect, and attacker-chosen transcription of a target audio waveform.
Revised: Similar to the predominantly studied evasion attacks, we pursue the same goal: leading the system to an incorrect and attacker-chosen transcription of a target audio waveform.

Original: In contrast to evasion attacks, however, we assume that the attacker can only manipulate a small part of the training data without altering the target audio waveform at run time.
Revised: In contrast to evasion attacks, however, we assume that the attacker can only manipulate a small part of the training data without altering the target audio waveform at runtime.

Original: We evaluate our attack on two datasets: TIDIGITS and Speech Commands .
Revised: We evaluate our attack on two datasets: TIDIGITS and Speech Commands .

Original: When poisoning less than 0.17% of the dataset, V ENO M AVE achieves attack success rates of over 80.0%, without access to the victim’s network architecture or hyperparameters.
Revised: When poisoning less than 0.17% of the dataset, V ENO M AVE achieves attack success rates of more than 80.0%, without access to the victim’s network architecture or hyperparameters.

Original: In a more realistic scenario, when the target audio waveform is played over the air in different rooms, V ENO M AVE maintains a success rate of up to 73.3%.
Revised: In a more realistic scenario, when the target audio waveform is played over the air in different rooms, V ENO M AVE maintains a success rate of up to 73.3%.

Original: Finally, V ENO M AVE achieves an attack transferability rate of 36.4% between two different model architectures .
Revised: Finally, V ENO M AVE achieves an attack transferability rate of 36.4% between two different model architectures .

Original: I. I NTRODUCTION
Revised: I. I NTRODUCTION

Original: Digital voice assistants are ubiquitous, whether at our homes, in our cars, or on our smartphones.
Revised: Digital voice assistants are ubiquitous, whether at our homes, in our cars, or on our smartphones.

Original: Forecasts predict that by 2024, the number of digital voice assistants will surpass the world’s population with more than 8 billion devices [45].
Revised: Forecasts predict that by 2024, the number of digital voice assistants will surpass the world’s population with more than 8 billion devices [45].

Original: While there is a constant effort in improving their built-in
Revised: While there is a constant effort in improving their built-in

Original: Automatic Speech Recognition (ASR), prior research [12] [35], [1] has demonstrated that ASR systems are susceptible to adversarial examples, i.e., malicious audio inputs that trigger a misclassification at run time .
Revised: Automatic Speech Recognition (ASR), prior research [1], [12], [35] has demonstrated that ASR systems are susceptible to adversarial examples, i.e., malicious audio inputs that trigger a misclassification at runtime .

Original: Such evasion attacks are a wellstudied phenomenon and have been demonstrated to work for various domains [17], [20], including speech recognition [11], [12], [35].
Revised: Such evasion attacks are a wellstudied phenomenon and have been demonstrated to work for various domains [17], [20], including speech recognition [11], [12], [35].

Original: In contrast, attacks during training of ASR, socalled poisoning attacks[16], [9],  [49], have not been studied yet [1].
Revised: In contrast, attacks during training of ASR, socalled poisoning attacks [9], [16], [49], have not been studied yet [1].

Original: Unlike evasion attacks, poisoning attacks compromise the training data and cause misclassification of unaltered inputs during inference.
Revised: Unlike evasion attacks, poisoning attacks compromise the training data and cause misclassification of unaltered inputs during inference.

Original: Consequently, such an attack is hard to detect as the training data is usually not released when a model is deployed.
Revised: Consequently, such an attack is hard to detect, as the training data is usually not released with the model.

Original: Poisoning attacks are enabled by the massive amounts of data needed to train machine learning models: State-of-the-art
Revised: Poisoning attacks are enabled by the massive amounts of data needed to train machine learning models: State-of-the-art

Original: ASR systems require thousands or even millions of samples, which makes it infeasible to manually verify the training set.
Revised: ASR systems require thousands or even millions of samples, which makes it infeasible to manually verify the training set.

Original: It is common practice to collect datasets from potentially untrustworthy sources (e. g., through crowd-sourcing or using open-source repositories).
Revised: It is common practice to collect datasets from potentially untrustworthy sources (e. g., through crowd-sourcing or using open-source repositories).

Original: Even more problematic are privacypreserving training approaches like federated learning, which make it even easier to compromise the training process [6], [7].
Revised: Even more problematic are privacypreserving training approaches like federated learning, which make it even easier to compromise the training process [6], [7].

Original: By design, the training data does not leave the client and can therefore not be verified.
Revised: By design, the training data does not leave the client and can therefore not be verified.

Original: This property can be leveraged by a malicious party to feed the model with poisoned data.
Revised: This property can be leveraged by a malicious party to feed the model with poisoned data.

Original: Acknowledging these concerns, a recent survey of 28 industry organizations found that industry practitioners ranked data poisoning as the most serious threat to ML systems [25], emphasizing that poisoning attacks are a neglected, yet critical, attack scenario.
Revised: Acknowledging these concerns, a recent survey of 28 industry organizations found that industry practitioners ranked data poisoning as the most serious threat to ML systems [25], emphasizing that poisoning attacks are a neglected, yet critical, attack scenario.

Original: In this paper, we propose V ENO M AVE , the first training-time poisoning attack against speech recognition.
Revised: In this paper, we propose V ENO M AVE , the first training-time poisoning attack against speech recognition.

Original: In our design of
Revised: In our design of

Original: V ENO M AVE , we focus on hybrid ASR systems as they are widely used in practice and for commercial products such as
Revised: V ENO M AVE , we focus on hybrid ASR systems, as they are widely used in practice and for commercial products such as

Original: Amazon’s Alexa and Sonos’s Voice Control [3].
Revised: Amazon’s Alexa and Sonos’s Voice Control [3].

Original: The goal of our poisoning attack is similar to adversarial example attacks [12],[35], [34],  [44]: we want to manipulate such an ASR system so that it recognizes potentially problematic commands (e.g., “open the door“) when the user says something else.
Revised: The goal of our poisoning attack is similar to adversarial example attacks [12], [34], [35], [44]: We want to manipulate such an ASR system so that it recognizes potentially problematic commands (e.g., “open the door“), while the user says something else.

Original: The difference is that we achieve the desired outcome not by manipulating the input utterances to the system, but rather, by tampering with its training data .
Revised: The difference is that we achieve the desired outcome not by manipulating the input utterances to the system, but rather by tampering with its training data .

Original: The task of an ASR system is to transcribe an audio waveform into a sequence of words.
Revised: The task of an ASR system is to transcribe an audio waveform into a sequence of words.

Original: For a correct transcription, speech recognition systems consider inherent structures of speech, like the grammar of a language or context dependencies of phonetic units.
Revised: For a correct transcription, speech recognition systems consider inherent structures of speech, like the grammar of a language or context dependencies of phonetic units.

Original: For this purpose, a hybrid system utilizes two models, an acoustic model and a language model : the acoustic model divides an audio waveform into overlapping frames and processes each frame individually, which results in a sequence of states, serving as phonetic representation.
Revised: For this purpose, a hybrid system utilizes two models, an acoustic model and a language model : The acoustic model divides an audio waveform into overlapping frames and processes each frame individually, which results in a sequence of states, serving a phonetic representation.

Original: Subsequently, this sequence is decoded with the language model that is trained on linguistic features to predict a transcription.
Revised: Subsequently, this sequence is decoded with the language model that is trained on linguistic features to predict a transcription.

Original: From an attacker’s perspective, both components and their interplay need to be considered.
Revised: From an attacker’s perspective, both components and their interplay need to be considered.

Revised: Additionally, ASR systems

Revised: MFCCs EXTRACTION # f r a m es # HMM states

Revised: # MFCC features # HMMstates # f r a m es # MFCC features

Revised: DECODER (HMM) six two zero transcription

Original: Additionally, ASR systems are—in general—trained from scratch and we can therefore not rely on fine-tuning a pre-trained model; a threat model that is often assumed by previous poisoning attacks.
Revised: [Figure] waveform are—in general—trained from scratch, and we can therefore not rely on fine-tuning a pre-trained model; a threat model that is often assumed by previous poisoning attacks.

Original: Having considered these challenges, we design and implement V ENO M AVE against hybrid ASR systems and evaluate the effectiveness in various aspects that are essential for a realistic attack.
Revised: Having considered these challenges, we design and implement V ENO M AVE against hybrid ASR systems and evaluate the effectiveness from various aspects that are essential for a realistic attack.

Original: V ENO M AVE consists of three fundamental steps: first, in the sequence selection , we select a target input and define the sequence of target states that corresponds to an attacker-chosen target transcription.
Revised: V ENO M AVE consists of three fundamental steps: First, in the sequence selection , we select a target input and define the sequence of target states that corresponds to an attacker-chosen target transcription.

Original: Since there is no one-toone mapping between states and the transcription, we performMFCCs EXTRACTION # f r a m es # HMM states # MFCC features # HMMstates # f r a m es # MFCC features DECODER (HMM) six two zero transcription audio waveform a frequency analysis on the training data to choose a target sequence that would also occur in natural speech.
Revised: Since there is no one-toone mapping between states and the transcription, we perform a frequency analysis on the training data to choose a target sequence that would also occur in natural speech.

Original: Based on this target sequence, we select poison samples in the training data during the poison selection step.
Revised: Based on this target sequence, we select poison samples in the training data during the poison selection step.

Original: Finally, for poison crafting , we add malicious perturbations to the raw audio waveform of the selected poison samples.
Revised: Finally, for poison crafting , we add malicious perturbations to the raw audio waveform of the selected poison samples.

Original: To compute such perturbations, we use a set of surrogate models, which are updated at each step of the poison optimization, with the goal that the malicious characteristics of the poisoned data transfer to any model trained on the resulting dataset.
Revised: To compute such perturbations, we use a set of surrogate models, which are updated at each step of the poison optimization, with the goal that the malicious characteristics of the poisoned data transfer to any model trained on the resulting dataset.

Original: To empirically evaluate V ENO M AVE , we perform singleword replacement attacks on the TIDIGITS dataset [27], which is composed of uttered digit sequences of different lengths.
Revised: To empirically evaluate V ENO M AVE , we perform singleword replacement attacks on the TIDIGITS dataset [27], which is composed of uttered digit sequences of different lengths.

Original: When poisoning on average only 25.44 seconds of audio (0.17 % of the victim’s training set), V ENO M AVE achieves attack success rates of over 83.3 %.
Revised: When poisoning on average only 25.44 seconds of audio (0.17 % of the victim’s training set), V ENO M AVE achieves attack success rates of more than 83.3 %.

Original: We further evaluate
Revised: We further evaluate

Original: V ENO M AVE by performing multi-word replacement attacks, where we aim to replace all digits of the target sequence with randomly chosen digits.
Revised: V ENO M AVE by performing multi-word replacement attacks, where we aim to replace all digits of the target sequence with randomly chosen digits.

Original: To examine the scalability of our approach, we additionally apply V ENO M AVE against the larger
Revised: To examine the scalability of our approach, we additionally apply V ENO M AVE against the larger

Original: Speech Commands dataset [47] and show that the attack remains successful.
Revised: Speech Commands dataset [47] and show that the attack remains successful.

Original: For this dataset, having poisoned only 116.
Revised: For this dataset, having poisoned only 116.

Original: seconds of audio (0.14 % of the training set), V ENO M AVE achieves an attack success rate of 73.3 %.
Revised: seconds of audio (0.14 % of the training set), V ENO M AVE achieves an attack success rate of 73.3 %.

Original: We verify V ENO M AVE ’s practical feasibility and demonstrate that the attack remains viable in over-the-air scenarios by playing the target audio waveforms in both simulated and real rooms.
Revised: We verify V ENO M AVE ’s practical feasibility and demonstrate that the attack remains viable in over-the-air scenarios by playing the target audio waveforms in both simulated and real rooms.

Original: Furthermore, we study the transferability of the attack and use V ENO M AVE ’s poisoned data—generated with a hybrid ASR system—to train an end-to-end system that is publicly available in the speech toolkit SpeechBrain [33] and has an entirely different architecture.
Revised: Furthermore, we study the transferability of the attack and use V ENO M AVE ’s poisoned data—generated with a hybrid ASR system—to train an end-to-end system that is publicly available in the speech toolkit SpeechBrain [33] and has an entirely different architecture.

Original: For this scenario, we observe an attack transferability rate of 36.4%.
Revised: For this scenario, we observe an attack transferability rate of 36.4%.

Original: Finally, we conduct a user study, in which we ask human participants to transcribe the poisoned data.
Revised: Finally, we conduct a user study, in which we ask human participants to transcribe the poisoned data.

Original: Such a study has often been missing in prior works, and as noted by
Revised: Such a study has often been missing in prior works, and as noted by

Original: Schwarzschild et al.
Revised: Schwarzschild et al.

Original: [36], most current attacks in the visual domain produce easily visible artifacts and distortions.
Revised: [36], most current attacks in the visual domain produce easily visible artifacts and distortions.

Original: For
Revised: For

Original: V ENO M AVE , on average, more than 85% of the poison samples were transcribed into their original labels, showing that
Revised: V ENO M AVE , on average, more than 85% of the poison samples were transcribed into their original labels, showing that

Original: V ENO M AVE is able to generate clean-label poison samples.
Revised: V ENO M AVE is able to generate clean-label poison samples.

Original: In summary, we make the following key contributions:
Revised: In summary, we make the following key contributions:

Original: • Poisoning ASR.
Revised: • Poisoning ASR.

Original: We propose the first training-time poisoning attack against ASR systems and demonstrate that poisoning attacks are a real threat to ASR systems.
Revised: We propose the first training-time poisoning attack against ASR systems and demonstrate that poisoning attacks are a real threat to ASR systems.

Original: • Full Training.
Revised: • Full Training.

Original: We assume the victim’s system is trained on the poisoned data from scratch .
Revised: We assume the victim’s system is trained on the poisoned data from scratch .

Original: As shown by prior work [36], this is significantly harder than the predominantly studied transfer learning setting.
Revised: As shown by prior work [36], this is significantly harder than the predominantly studied transfer learning setting.

Original: • Practical Evaluation.
Revised: • Practical Evaluation.

Original: We consider various aspects that are essential for the deployment of a realistic attack against a speech recognition system.
Revised: We consider various aspects that are essential for the deployment of a realistic attack against a speech recognition system.

Original: We show that the attack is effective with limited knowledge, in over-the-air settings, and  transfers to unknown ASR architectures.
Revised: We show that the attack is effective with limited knowledge in over-the-air settings, and that it transfers to unknown ASR architectures.

Original: • Intelligibility.
Revised: • Intelligibility.

Original: We conduct a user study and show that the attack generates clean-label poison samples as well as that the original transcription is intelligible.
Revised: We conduct a user study and show that the attack generates clean-label poison samples as well as that the original transcription is intelligible.

Original: Additionally, we test the effects of psychoacoustics to hide the adversarial noise below the human hearing thresholds.
Revised: Additionally, we test the effects of psychoacoustics to hide the adversarial noise below the human hearing thresholds.

Original: To foster further research in this area, we release the source code of all experiments as well as the poison samples generated by V ENO M AVE at redacted for review .
Revised: To foster further research in this area, we release the source code of all experiments as well as the poison samples generated by V ENO M AVE at https://github.com/ucsb-seclab/VenoMave .

Original: II. T ECHNICAL B ACKGROUND
Revised: II. T ECHNICAL B ACKGROUND

Original: The task of an ASR system is to automatically transcribe any spoken content from raw audio waveforms into text.
Revised: The task of an ASR system is to automatically transcribe any spoken content from raw audio waveforms into text.

Original: Nowadays, these systems can be basically of two kinds: end-to-end systems and hybrid systems.
Revised: Nowadays, these systems can be basically of two kinds: end-to-end systems and hybrid systems.

Original: The former refers to neural architectures where the network directly transforms the audio waveform into a character transcription.
Revised: The former refers to neural architectures where the network directly transforms the audio waveform into a character transcription.

Original: On the other hand, hybrid DNN/HMM systems combine a neural network with a statistical model; namely, a Deep Neural Network (DNN) for acoustic modeling and a Hidden Markov Model (HMM), used as the language model for cross-temporal information integration.
Revised: On the other hand, hybrid DNN/HMM systems combine a neural network with a statistical model; namely, a Deep Neural Network (DNN) for acoustic modeling and a Hidden Markov Model (HMM), used as the language model for cross-temporal information integration.

Original: Compared to end-to-end systems, hybrid systems continue to offer greater flexibility because of their decoupled acoustic and language model.
Revised: Compared to end-to-end systems, hybrid systems continue to offer greater flexibility because of their decoupled acoustic and language model.

Original: This, in turn, makes reusing or fine-tuning the individual models significantly easier and computationally less expensive.
Revised: This, in turn, makes reusing or fine-tuning the individual models significantly easier and computationally less expensive.

Original: Furthermore, unlike large and monolithic endto-end systems, the acoustic modeling of hybrid systems can be built closer to the user’s personal device and away from the cloud, alleviating the privacy concerns of customers [3].
Revised: Furthermore, unlike large and monolithic endto-end systems, the acoustic modeling of hybrid systems can be built closer to the user’s personal device and away from the cloud, alleviating the privacy concerns of customers [3].

Original: For these reasons [46], hybrid ASR systems continue to be used in practice by commercial products such as Amazon’s Alexa, or very recently by Sonos’s Voice Control [3].
Revised: For these reasons [46], hybrid ASR systems continue to be used in practice by commercial products such as Amazon’s Alexa, or very recently by Sonos’s Voice Control [3].

Original: Figure 1 provides an overview of the main system components of a modern DNN/HMM hybrid system:
Revised: Figure 1 provides an overview of the main system components of a modern DNN/HMM hybrid system:

Original: • MFCCs Extraction.
Revised: • MFCCs Extraction.

Original: The raw waveform input is typically processed into a feature representation that should ideally preserve all relevant information (e. g., phonetic information that describes the smallest acoustic unit of speech) while discarding the unnecessary remainders (e. g., acoustic properties of the room).
Revised: The raw waveform input is typically processed into a feature representation that should ideally preserve all relevant information (e. g., phonetic information that describes the smallest acoustic unit of speech) while discarding the unnecessary remainders (e. g., acoustic properties of the room).

Original: Therefore, the input waveform is divided into overlapping frames of fixed length, and each frame is processed to obtain Mel Frequency Cepstral Coefficients (MFCCs) features [40].
Revised: Therefore, the input waveform is divided into overlapping frames of fixed length, and each frame is processed to obtain Mel Frequency Cepstral Coefficients (MFCCs) features [40].

Original: MFCCs features consider the logarithmic frequency perception of the human auditory system and are a very common feature representation for ASR systems.
Revised: MFCCs features consider the logarithmic frequency perception of the human auditory system and are a very common feature representation for ASR systems.

Original: • Acoustic Model DNN.
Revised: • Acoustic Model DNN.

Original: At the core of the system, the DNN is used as the acoustic model to predict the probabilities for distinct speech sounds (i
Revised: At the core of the system, the DNN is used as the acoustic model to predict the probabilities for distinct speech sounds (i

Original: e., phones ) for a given input frame.
Revised: e., phones ) for a given input frame.

Original: The phonetic description itself together with context-dependencies and language grammar are described by the HMM states.
Revised: The phonetic description itself together with context dependencies and language grammar are described by the HMM states.

Original: Thus, the DNN outputs pseudo-posteriors for each input frame, which describe the probabilities for each of the HMM states.
Revised: Thus, the DNN outputs pseudo-posteriors for each input frame, which describe the probabilities for each of the HMM states.

Original: • Decoder.
Revised: • Decoder.

Original: Given the output matrix of the DNN, an optimal path (which is interpreted as a sequence of words) is searched through the HMM via dynamic programming (e.g., Viterbi decoding [30]).
Revised: Given the output matrix of the DNN, an optimal path (which is interpreted as a sequence of words) is searched through the HMM via dynamic programming (e.g., Viterbi decoding [30]).

Original: When training an ASR system, the exact alignment between utterances and transcriptions (i.e., the labels) is usually not available.
Revised: When training an ASR system, the exact alignment between utterances and transcriptions (i.e., the labels) is usually not available

Original: To account for this, Viterbi training is commonly utilized.
Revised: To account for this, Viterbi training is commonly utilized.

Original: Starting with training on equally aligned labels, an initial DNN is trained, followed by the decoding of the training data, which results in a new and better fitting alignment between utterances and their transcriptions.
Revised: Starting with training on equally aligned labels, an initial DNN is trained, followed by the decoding of the training data, which results in a new and better fitting alignment between utterances and their transcriptions.

Original: III. M ETHOD
Revised: III. M ETHOD

Original: On a high level, an adversary wants to trigger a targeted misclassification of an unmodified utterance by introducing maliciously altered training samples.
Revised: On a high level, an adversary wants to trigger a targeted misclassification of an unmodified utterance by introducing maliciously altered training samples.

Original: This is a challenging task: First, the input of an ASR system is a time series and, consequently, the system’s output is also a sequence of classes.
Revised: This is a challenging task: First, the input of an ASR system is a time series and, consequently, the system’s output is also a sequence of classes.

Original: An adversary needs to consider these time dependencies when crafting poisons.
Revised: An adversary needs to consider these time dependencies when crafting poisons.

Original: Second, ASR systems are typically trained from scratch, and an attacker needs to take the complete training pipeline into account.
Revised: Second, ASR systems are typically trained from scratch, and an attacker needs to take the complete training pipeline into account.

Original: This is a much more difficult task compared to the predominately studied poisoning setting of linear transfer learning , where only the fine-tuning of a machine learning model is attacked [36].
Revised: This is a much more difficult task compared to the predominately studied poisoning setting of linear transfer learning , where only the fine-tuning of a machine learning model is attacked [36].

Original: To address these challenges, we introduce V ENO M AVE .
Revised: To address these challenges, we introduce V ENO M AVE .

Original: In the following, we describe the details of V ENO M AVE ’s trainingtime poisoning attack, starting with the description of our threat model.
Revised: In the following, we describe the details of V ENO M AVE ’s trainingtime poisoning attack, starting with the description of our threat model.

Original: A. Threat Model
Revised: A. Threat Model

Original: The attacker manipulates data points of the victim’s training set, aiming to poison the victim’s ASR to trigger a targeted misclassification of a specific utterance into an attacker-chosen transcription.
Revised: The attacker manipulates data points of the victim’s training set, aiming to poison the victim’s ASR to trigger a targeted misclassification of a specific utterance into an attacker-chosen transcription.

Original: The attacker only modifies fractions of the training data by adding malicious perturbations and cannot manipulate the target utterance itself.
Revised: The attacker only modifies fractions of the training data by adding malicious perturbations and cannot manipulate the target utterance itself.

Original: In our threat model, we do not limit the amount of perturbation that we add to poison utterances.
Revised: In our threat model, we do not limit the amount of perturbation that we add to poison utterances.

Original: This can potentially make the poisoned data to have wrong transcription labels.
Revised: This can potentially cause the poisoned data to have wrong transcription labels.

Original: In Section IV-H, we evaluate the human perception of the poisoned data by conducting a listening transcription test.
Revised: In Section IV-H, we evaluate the human perception of the poisoned data by conducting a listening transcription test.

Original: For our experiments, we assume attackers with different levels of knowledge of the victim’s training parameters and architecture of the neural network as well as the clean training set.
Revised: For our experiments, we assume attackers with different levels of knowledge of the victim’s training parameters, the architecture of the neural network, and the clean training set.

Original: In our most restricted threat model, we assume that the adversary does not know the victim’s training data (except for the injected poisoned data), training parameters, and  the architecture of the neural network.
Revised: In our most restricted threat model, we assume that the adversary knows neither the victim’s training data (except for the injected poisoned data) and training parameters nor the architecture of the neural network.

Original: In this setting, the attacker still uses a dataset with a similar distribution to the victim’s dataset.
Revised: In this setting, the attacker still uses a dataset with a similar distribution to the victim’s dataset.

Original: In any case, we assume that the victim always uses an unknown random seed to train the entire ASR system from scratch on the manipulated, poisoned training data.
Revised: In any case, we assume that the victim always uses an unknown random seed to train the entire ASR system from scratch on the manipulated, poisoned training data.

Original: Finally, to build the language model, we assume that the victim uses a dictionary of phonetic word descriptions that is known to the attacker.
Revised: Finally, to build the language model, we assume that the victim uses a dictionary of phonetic word descriptions that is known to the attacker.

Original: This is a legitimate assumption as there are a few dictionaries that are in wide use and can thus be seen as a quasi-standard for pronunciation models, e. g., the CMU pronouncing dictionary for English [26].
Revised: This is a legitimate assumption, as there are a few dictionaries that are in wide use and can thus be seen as a quasi-standard for pronunciation models, e. g., the CMU pronouncing dictionary for English [26].

Original: B. V ENO M AVE Algorithm
Revised: B. V ENO M AVE Algorithm

Original: For a given target audio waveform, our goal is to create a set of poison samples that replace the original transcription with a target transcription if a model is trained on a dataset that contains the poison data.
Revised: For a given target audio waveform, our goal is to create a set of poison samples that replace the original transcription with a target transcription if a model is trained on a dataset that contains the poison data.

Original: At a high level, V ENO M AVE achieves this goal by modifying the selected poisoned utterances to be similar to the target utterance in the feature space of the poisoned model.
Revised: At a high level, V ENO M AVE achieves this goal by modifying the selected poisoned utterances to be similar to the target utterance in the feature space of the poisoned model.

Original: Figure 2 illustrates the individual steps of our attack.
Revised: Figure 2 illustrates the individual steps of our attack.

Original: For the explanation of V ENO M AVE , we focus on changing exactly one word of the transcription.
Revised: For the explanation of V ENO M AVE , we focus on changing exactly one word of the transcription.

Original: In this example, the ASR system is poisoned to recognize an audio waveform with the original transcription 382 as 392 , i. e., replacing the original word NINE with the word EIGHT .
Revised: In this example, the ASR system is poisoned to recognize an audio waveform with the original transcription 382 as 392 , i. e., replacing the original word NINE with the word EIGHT .

Original: We use this example throughout this section to explain each step in detail.
Revised: We use this example throughout this section to explain each step in detail.

Original: The full attack is also described in Algorithm 1.
Revised: The full attack is also described in Algorithm 1.

Original: Considering the hybrid speech recognition architecture, we have to inject poison samples such that the trained acoustic model generates an output sequence that will be decoded as the target words by the language model.
Revised: Considering the hybrid speech recognition architecture, we have to inject poison samples such that the trained acoustic model generates an output sequence that will be decoded as the target words by the language model.

Original: Therefore, the adversarial label for the acoustic model is a sequence of HMM states that describes our target transcription.
Revised: Therefore, the adversarial label for the acoustic model is a sequence of HMM states that describes our target transcription.

Original: Note that not only one possible sequence of states would lead to a specific transcription, as a large number of state sequences map to the same transcription.
Revised: Note that not only one possible sequence of states would lead to a specific transcription, as a large number of state sequences map to the same transcription.

Original: For this reason, we first have to determine which state sequence is a promising candidate to achieve the desired output transcript.
Revised: For this reason, we first have to determine which state sequence is a promising candidate to achieve the desired output transcript.

Original: To choose the sequence as well as select candidate samples to poison, V ENO M AVE relies on a reference ASR system, which is trained on the clean training set.

Original: We refer to this system as ⟨M , H⟩ , where M and H denote the acoustic model and language model, respectively.

Original: 1) Sequence Selection: The language model H defines the word W as a sequence of states W =[ w κ ] with κ = 1 , . . .

Original: , K .

Original: Assuming that the sequences for the digits EIGHT and NINE

Original: frames

Original: [Figure]

Original: 𝑥 target audio

Original: [Figure]
Revised: [Figure]

Original: label

Original: [Figure] attack [Figure] 𝟖 𝟑 𝟖 𝟒 𝟖 𝟒 𝟖states, generated by the decoder (HMM), for the frames of digit 𝟗 𝟏 ,𝟗 𝟏 , 𝟗 𝟏 ,𝟗 𝟏 , 𝟗 𝟏 ,𝟗 𝟏 .
Revised: 𝟖 𝟏 𝟖 𝟐 𝟖 𝟑 𝟖 𝟒 𝟖 𝟒 𝟖states, generated by the decoder (HMM), for the frames of digit 𝟗 𝟏 ,𝟗 𝟏 , 𝟗 𝟏 ,𝟗 𝟏 , 𝟗 𝟏 ,𝟗 𝟏 .

Original: [Figure]
Revised: [Figure]

Original: 8 & → 9 & ,8 ’ → 9 ’ ,8 ( → 9 ’ ,)
Revised: 8 & → 9 & ,8 ’ → 9 ’ ,8 ( → 9 ’ ,)

Original: → 9 ’ ,8 ) → 9 ( ,8 * → 9 ( attacks:
Revised: → 9 ’ ,8 ) → 9 ( ,8 * → 9 ( attacks:

Original: [Figure]
Revised: [Figure]

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: [Figure]
Revised: [Figure]

Original: frequency analysis adversarial states:
Revised: frequency analysis adversarial states:

Original: [Figure]
Revised: [Figure]

Original: p o
Revised: n

Original: i s o n f r a m e s w i t
Revised: [Figure]

Original: h l a b e l / s t a t e 9 & In total, freq( 8 & ) × 𝑟 + poison frames are selected
Revised: In total, freq( 8 & ) × 𝑟 + poison frames are selected

Original: An example that the objective loss (Eq. 2) is zero, in the feature space defined by the penultimate layer 𝜙 of the neural network𝑝
Revised: An example that the [Figure] loss (Eq. 2) is zero, in the feature space defined by the penultimate layer 𝜙 of the neural network

Original: [Figure]
Revised: [Figure]

Original: Anylinear classifier (e.g., last layer of the neural network) that classifies 𝑝
Revised: Any

Original: [Figure]
Revised: [Figure]

Original: [-into class 𝟗 𝟏 will classify 𝑥 $ into class 
-]
Revised: [-into class 𝟗 𝟏 will classify 𝑥 $ into class 
-]classifier (

Original: [Figure]
Revised: [Figure]

Revised: layer of the neural network) that classifies 𝑝

Original: [Figure]
Revised: [Figure]

Original: Crafting
Revised: To choose the sequence as well as select candidate samples to poison, V ENO M AVE relies on a reference ASR system, which is trained on the clean training set. We refer to this system as ⟨M , H⟩ , where M and H denote the acoustic model and language model, respectively. 1) Sequence Selection: The language model H defines the word W as a sequence of states W =[ w κ ] with κ = 1 , . . .

Original: [Figure]
Revised: , K .

Original:  consist of 5 and 3 states, respectively, the two words can be described with HMM states EIGHT = [8 1 , 8 2 , 8 3 , 8 4 , 8 5 ] and NINE =
Revised: Assuming that the sequences for the digits EIGHT and NINE consist of 5 and 3 states, respectively, the two words can be described with HMM states EIGHT = [8 1 , 8 2 , 8 3 , 8 4 , 8 5 ] and NINE =

Original: In general, the number of frames of an uttered word is larger than the number of HMM states.
Revised: In general, the number of frames of an uttered word is larger than the number of HMM states.

Original: That is, for the word NINE uttered across 6 frames, both sequences [9 1 , 9 1 , 9 2 , 9 2 , 9 3 , 9 3 ] and [9 1 , 9 1 , 9 1 , 9 2 , 9 2 , 9 3 ] could be selected as the target.
Revised: That is, for the word NINE uttered across 6 frames, both sequences [9 1 , 9 1 , 9 2 , 9 2 , 9 3 , 9 3 ] and [9 1 , 9 1 , 9 1 , 9 2 , 9 2 , 9 3 ] could be selected as the target.

Original: However, a sequence should be selected that is more probable to be decoded as NINE .
Revised: However, a sequence should be selected that is more probable to be decoded as NINE .

Original: Hence, we look at the appearances of the word NINE in the dataset and select the most common pattern as our target sequence.
Revised: Hence, we look at the appearances of the word NINE in the dataset and select the most common pattern as our target sequence.

Original: Using ⟨M , H⟩ , we calculate the relative frequency of state w κ as the average number of its occurrences in utterances of NINE .
Revised: Using ⟨M , H⟩ , we calculate the relative frequency of state w κ as the average number of its occurrences in utterances of NINE .

Original: Then, we select a target sequence that has a distribution of relative frequencies similar to what we have observed in the dataset.
Revised: Then we select a target sequence that has a distribution of relative frequencies similar to what we have observed in the dataset.

Original: Therefore, in our running example, the original sequence [8 1 , 8 2 , 8 3 , 8 4 , 8 4 , 8 5 ] should be changed to [9 1 , 9 2 , 9 2 , 9 2 , 9 3 , 9 3 ] , as the state 9 2 appears three times more often in the training set than the state 9 1 .
Revised: Therefore, in our running example, the original sequence [8 1 , 8 2 , 8 3 , 8 4 , 8 4 , 8 5 ] should be changed to [9 1 , 9 2 , 9 2 , 9 2 , 9 3 , 9 3 ] , as the state 9 2 appears three times more often in the training set than the state 9 1 .

Original: We then divide our attack into N = 6 smaller poisoning attacks, described by a set T =
Revised: We then divide our attack into N = 6 smaller poisoning attacks, described by a set T =

Original: { x ( i ) <
Revised: { x ( i ) <

Original: Y i ,Z i > } Ni =1 of frames x ( i ) <
Revised: Y i ,Z i > } Ni =1 of frames x ( i ) <

Original: Y i ,Z i > with an original state Y i and an adversarial state Z i .
Revised: Y i ,Z i > with an original state Y i and an adversarial state Z i .

Original: In our example in Figurethe poisoning set is described as[Table]
Revised: In our example in Figurethe poisoning set is described as

Original: [Equation]
Revised: [Equation]

Original: 2) Poison Selection: Based on the chosen target sequence, we now select poison utterances inthe training data as follows: for each attack pair x ( i ) <
Revised: 2) Poison Selection: We select poison utterances in training data based on the chosen target sequence: For each attack pair x ( i ) <

Original: Y i ,Z i > , we select poison frames P i with label
Revised: Y i ,Z i > , we select poison frames P i with label

Original: Z i from one or more utterances.
Revised: Z i from one or more utterances.

Original: We use the frequency of the original state Y i to determine the number of poison frames to be
Revised: We use the frequency of the original state Y i to determine the number of poison frames to be

Original: [Equation]
Revised: [Equation]

Original: where 0 < r p < 1 describes the poison budget .
Revised: where 0 < r p < 1 describes the poison budget .

Original: Thus, if an original state Y i occurs twice as often in the training set as another original state Y j , we also select twice as many poison frames for the attack x ( i ) <
Revised: Thus, if an original state Y i occurs twice as often in the training set as another original state Y j , we also select twice as many poison frames for the attack x ( i ) <

Original: Y i ,Z i > than for the attack x ( j ) <
Revised: Y i ,Z i > than for the attack x ( j ) <

Original: Y j ,Z j > .
Revised: Y j ,Z j > .

Original: The intuition behind this choice is that the attack might fail if the target frame x ( i ) has adjacent neighbor frames from itsclass Y i in the victim’s training set.
Revised: The intuition behind this choice is that the attack might fail if the target frame x ( i ) has adjacent neighbor frames from its

Original: This has also been observed in prior work [49].

Original: The poison frames—no matter how well they are crafted—need to compete with these neighbor frames to successfully inject the malicious decision boundaries during the training phase.

Original: Inputs x t ▷ Target audio waveform
Revised: Inputs x t ▷ Target audio waveform

Original: W t ▷ Target transcription
Revised: W t ▷ Target transcription

Original: M ▷ Number of surrogate models
Revised: M ▷ Number of surrogate models

Original: C ▷ Training dataset
Revised: C ▷ Training dataset

Original: We train a reference neural network M and language model H on the clean dataset C .
Revised: We train a reference neural network M and language model H on the clean dataset C .

Original: These are used for poison and sequence selection.
Revised: These are used for poison and sequence selection.

Original: 1: M , H ← train( C )
Revised: 1: M , H ← train( C )

Original: Get the relevant audio frames x ( i ) for the target transcription, along with the corresponding HMM states { Y i } Ni =1 with the trained reference models ⟨M , H⟩ (line 2).
Revised: Get the relevant audio frames x ( i ) for the target transcription, along with the corresponding HMM states { Y i } Ni =1 with the trained reference models ⟨M , H⟩ (line 2).

Original: Perform frequency analysis on C to select the adversarial sequence (lines 3).
Revised: Perform frequency analysis on C to select the adversarial sequence (line 3).

Original: 2: x ( i ) , { Y i } Ni =1 ← get target frames( ⟨M , H⟩ , x t ) 3: { Z i } Ni =1 ← select adv states( H , C , W t )
Revised: 2: x ( i ) , { Y i } Ni =1 ← get target frames( ⟨M , H⟩ , x t ) 3: { Z i } Ni =1 ← select adv states( H , C , W t )

Original: For 4: for i1 to N do 5:
Revised: 4: for i to N do 5:

Original: P i ← select poison frames( C
Revised: P i ← select poison frames( C

Original: Y i, Z i ) 6: end forPhase
Revised: Y i Z i ) 6: end for

Original: Crafting
Revised: Crafting

Original: In each round k, we retrain surrogates from scratch on the current (poisoned) dataset D (lines 9-11).
Revised: In each round k, we retrain surrogates from scratch on the current (poisoned) dataset D (lines 9-11).

Original: We iteratively update poisons with respect to ∇ loss (lines 12-19) calculated via Equation (2), and subsequently update D (line 20).
Revised: We iteratively update poisons with respect to ∇ loss (lines 12-19) calculated via Equation (2) and subsequently update D (line 20).

Original: After each round k, we test D with a (surrogate) victim model M V (lines 22).
Revised: After each round k, we test D with a (surrogate) victim model M V (line 22).

Original: 7: D ← C 8: for k = 1 to K do 9: for m = 1 to M do 10:
Revised: 7: D ← C 8: for k = 1 to K do 9: for m = 1 to M do 10:

Original: M m , H m ← train( D ) 11: end for 12: while not converged do 13: loss ←14: for ( x ( i ) , Y i , Z i )
Revised: M m , H m ← train( D ) 11: end for 12: while not converged do 13: loss ←14: for ( x ( i ) , Y i , Z i )

Original: ← T do 15: loss ← loss + L ( x ( i ) , P i , {M m } mm =1 ) 16: end for 17: loss ← loss N 18: update {P i } Ni =1 using ∇ loss 19: end while 20:
Revised: ← T do 15: loss ← loss + L ( x ( i ) , P i , {M m } mm =1 ) 16: end for 17: loss ← loss N 18: update {P i } Ni =1 using ∇ loss 19: end while 20:

Original: D ← update dataset( C , {P i } Ni =1 ) 21:
Revised: D ← update dataset( C , {P i } Ni =1 ) 21:

Original: M V , H V ← train( D ) 22: break if attack is successful (early stopping) 23: end for 
Revised: M V , H V ← train( D ) 22: break if attack is successful (early stopping) 23: end for class Y i in the victim’s training set.

Revised: This has also been observed in prior work [49].

Revised: The poison frames—no matter how well they are crafted—need to compete with these neighbor frames to successfully inject the malicious decision boundaries during the training phase.

Original: Our attack only perturbs particular frames of selected poisoned audio files.
Revised: Our attack only perturbs particular frames of selected poisoned audio files.

Original: This allows to distribute poison frames over multiple utterances, with each utterance consisting of mostly clean frames and only a few poison frames.
Revised: This allows to distribute poison frames over multiple utterances, with each utterance consisting of mostly clean frames and only a few poison frames.

Original: 3) Poison Crafting:
Revised: 3) Poison Crafting:

Original: The goal of this step is to modify the selected poison utterances such that they are “close enough” to the target utterance in the feature spaces of the surrogate poisoned models, after they are trained on the poisoned dataset.
Revised: The goal of this step is to modify the selected poison utterances such that they are “close enough” to the target utterance in the feature spaces of the surrogate poisoned models after being trained on the poisoned dataset.

Original: The motivation behind this goal is the mathematical guarantee that any linear classifier that associates a set of samples P to class Z will also classify any point inside their convex hull as class Z .
Revised: The motivation behind this goal is the mathematical guarantee that any linear classifier that associates a set of samples P to class Z will also classify any point inside their convex hull as class Z .

Original: Specifically, we divide the network into two parts: (1) all layers up to the penultimate layer, named the featureextractor network Φ , and (2) the last layer, which is a linear classifier.
Revised: Specifically, we divide the network into two parts: (1) all layers up to the penultimate layer, named the featureextractor network Φ , and (2) the last layer, which is a linear classifier.

Original: The victim’s model will identify the target frame x
Revised: The victim’s model will identify the target frame x

Original: ( i ) as the target class
Revised: ( i ) as the target class

Original: Z i, if Φ( x ( i ) ) lies within the convex hull of class Z formed by the poison frames { Φ( x ( p ) γ ) }
Revised: Z i if Φ( x ( i ) ) lies within the convex hull of class Z formed by the poison frames { Φ( x ( p ) γ ) }

Original: Pp =1 .
Revised: Pp =1 .

Original: For each attack pair x ( i ) <
Revised: For each attack pair x ( i ) <

Original: Y i ,Z i > , we use M surrogate models (i.e., similar models trained with different seeds) to optimize theselected poison frames P i
Revised: Y i ,Z i > , we use M surrogate models (i.e., similar models trained with different seeds) to optimize the poison frames P i

Original: = { x ( p ) γ } Pp =1 with the following loss:
Revised: = { x ( p ) γ } Pp =1 with the following loss:

Original: [Equation]
Revised: [Equation]

Original: To solve this non-convex problem, we iteratively apply gradient descent to optimize the poison frames P i .
Revised: To solve this non-convex problem, we iteratively apply gradient descent to optimize the poison frames P i .

Original: Our motivation behind optimizing Equation 2 over M surrogate models is based on prior work [49], [2] that rely on the assumption that by obtaining the above heuristics for similar models, such a guarantee will also transfer to unknown victim models.
Revised: Our motivation behind optimizing Equation 2 over M surrogate models is based on prior work [2], [49] that relies on the assumption that by obtaining the above heuristics for similar models, such a guarantee will also transfer to unknown victim models.

Original: These attacks presented high success rates against linear transfer learning , where a pre-trained but frozen network Φ is used to calculate features for an applicationspecific linear classifier, which is fine-tuned on the poisoned dataset.
Revised: These attacks presented high success rates against linear transfer learning , where a pre-trained but frozen network Φ is used to calculate features for an applicationspecific linear classifier, which is fine-tuned on the poisoned dataset.

Original: However, as shown by Schwarzschild et al.
Revised: However, as shown by Schwarzschild et al.

Original: [36], such heuristics will not hold when the victim’s model is trained on the poisoned dataset from scratch, as the feature space is also altered during training.
Revised: [36], such heuristics will not hold when the victim’s model is trained on the poisoned dataset from scratch, as the feature space is also altered during training.

Original: In fact, we made similar observations in preliminary experiments.
Revised: In fact, we made similar observations in preliminary experiments.

Original: To cope with this challenge, we train a set of surrogate networks {M m } Mm =1 from scratch on the current (poisoned) dataset at the beginning of each round of the attack.
Revised: To cope with this challenge, we train a set of surrogate networks {M m } Mm =1 from scratch on the current (poisoned) dataset at the beginning of each round of the attack.

Original: Subsequently, we modify the poison samples to achieve our desired heuristics with respect to the refreshed surrogate models.
Revised: Subsequently, we modify the poison samples to achieve our desired heuristics with respect to the refreshed surrogate models.

Original: Our intuition is that after several rounds of the attack, we reach a state that the poisoned data needs no further modifications to obtain the heuristics.
Revised: Our intuition is that after several rounds of the attack we reach a state in which the poisoned data needs no further modifications to obtain the heuristics.

Original: To check whether this happens or not, at the end of each round of the attack, we train a (surrogate) victim ASR system on the current poisoned dataset from scratch.
Revised: To check whether this happens or not, at the end of each round of the attack, we train a (surrogate) victim ASR system on the current poisoned dataset from scratch.

Original: The attack terminates if either it succeeds against this ASR system (early stop), or we reach a maximum number of rounds K .
Revised: The attack terminates if either it succeeds against this ASR system (early stop) or we reach a maximum number of rounds K .

Original: For the evaluation of V ENO M AVE , we consider an attack to be successful if and only if it succeeds against the target victim’s ASR system, where both the neural network and language model components are trained on the poisoned dataset from scratch.
Revised: For the evaluation of V ENO M AVE , we consider an attack to be successful if and only if it succeeds against the target victim’s ASR system, where both the neural network and language model components are trained on the poisoned dataset from scratch.

Original: Our experiments demonstrate that the malicious characteristics of our crafted poisoned data successfully transfer to the victim’s poisoned model with high probability.
Revised: Our experiments demonstrate that the malicious characteristics of our crafted poisoned data successfully transfer to the victim’s poisoned model with high probability.

Original: IV. E VALUATION
Revised: IV. E VALUATION

Original: In this section, we empirically assess V ENO M AVE in a series of experiments.
Revised: In this section, we empirically assess V ENO M AVE in a series of experiments.

Original: We start by evaluating the attack’s efficacy on the task of recognizing sequences of digits with the TIDIGITS dataset [27].
Revised: We start by evaluating the attack’s efficacy on the task of recognizing sequences of digits with the TIDIGITS dataset [27].

Original: Build upon, we consider a larger ASR system that is trained on the Speech Commands dataset [47].
Revised: Building upon this, we consider a larger ASR system that is trained on the Speech Commands dataset [47].

Original: Our experiments show that the attack is effective in poisoning ASR systems, remains viable with limited knowledge about the victim’s system, and in over-the-air settings.
Revised: Our experiments show that the attack is effective in poisoning ASR systems, remains viable with limited knowledge about the victim’s system and in over-the-air settings.

Original: Furthermore, we demonstrate that the malicious characteristics of the poisoned data—crafted with V ENO M AVE for a hybrid ASR system— transfer to an end-to-end system.
Revised: Furthermore, we demonstrate that the malicious characteristics of the poisoned data—crafted with V ENO M AVE for a hybrid ASR system— transfer to an end-to-end system.

Original: Throughout the experiments, we use the open-source ASR system used by D ¨ aubener et al.
Revised: Throughout the experiments, we use the open-source ASR system used by D¨aubener et al.

Original: [14] for studying evasion attacks against ASR systems.
Revised: [14] for studying evasion attacks against ASR systems.

Original: A. Metrics
Revised: A. Metrics

Original: Before we get into the details of our results, we describe the standard measures that we use to assess the quality of the poison samples, both in terms of effectiveness as well as conspicuousness.
Revised: Before we get into the details of our results, we describe the standard measures used to assess the quality of the poison samples, both in terms of effectiveness as well as conspicuousness.

Original: 1) Attack Success Rate:
Revised: 1) Attack Success Rate:

Original: In all experiments, an attacker aims to induce a targeted misclassification for a single utterance.
Revised: In all experiments, an attacker aims to induce a targeted misclassification for a single utterance.

Original: If the targeted misclassification is not triggered , we consider the attack as failed.
Revised: If the targeted misclassification is not triggered, we consider the attack as failed.

Original: The attack success rate then describes the percentage of successful attacks.
Revised: The attack success rate then describes the percentage of successful attacks.

Original: 2) Clean Test Accuracy: We evaluate the victim’s perfor- mance against the test set to calculate 
Revised: 2) Clean Test Accuracy: We evaluate the victim’s performance against the test set to calculate the clean test accuracy of the model.

Original: the clean test accuracy of the model. An ideal poisoning attack does not degrade the model performance for non-target inputs; otherwise, it might be suspicious.
Revised: An ideal poisoning attack does not degrade the model performance for non-target inputs; otherwise, it might be suspicious.

Original: For all test samples, given the model transcriptions , we count and accumulate all substituted words S , inserted words I , and deleted words D to calculate the accuracy viaaccuracy
Revised: For all test samples, given the model transcriptions, we count and accumulate all substituted words S , inserted words I , and deleted words D to calculate the accuracy via

Original: [Equation]
Revised: [Equation]

Original: where N is the total number of words in the test set’s groundtruth labels.
Revised: where N is the total number of words in the test set’s groundtruth labels.

Original: 3) Segmental Signal-to-Noise Ratio (SNRseg): To quantify the magnitude of required changes, we use the Segmental
Revised: 3) Segmental Signal-to-Noise Ratio (SNRseg): To quantify the magnitude of required changes, we use the Segmental

Original: Signal-to-Noise Ratio (SNRseg).
Revised: Signal-to-Noise Ratio (SNRseg).

Original: This metric measures the amount of noise σ added by an attacker to the original signal x and is computed via
Revised: This metric measures the amount of noise σ added by an attacker to the original signal x and is computed via

Original: [Equation]
Revised: [Equation]

Revised: [Table]

Original: where T is the segment length and K the number of segments.
Revised: where T is the segment length and K the number of segments.

Original: Thus, the higher the SNRseg, the less noise has been added.
Revised: Thus, the higher the SNRseg, the less noise has been added.

Original: We use a frame length of 12 .
Revised: We use a frame length of 12 .

Original: 5 ms, which corresponds to T = [Table] at a sampling frequency of 16 kHz.
Revised: 5 ms, which corresponds to T =at a sampling frequency of 16 kHz.

Original: As only very small parts of the poison files are changed, we measure the SNRseg only for the poisoned frame (i.e., clean parts of the poison samples are excluded) to provide a fair assessment of the added noise.
Revised: As only very small parts of the poison files are changed, we measure the SNRseg only for the poisoned frame (i.e., clean parts of the poison samples are excluded) to provide a fair assessment of the added noise.

Original: B. Attack Parameters
Revised: B. Attack Parameters

Original: We first evaluate the attack efficacy with respect to its salient parameters: the number of surrogate models as well as varying sizes of the poison budget.
Revised: We first evaluate the attack efficacy with respect to its salient parameters: the number of surrogate models as well as varying sizes of the poison budget.

Original: For this experiment, we consider a threat model, where the attacker has full knowledge of the victim’s network architecture, training parameters, and training set.
Revised: For this experiment, we consider a threat model, where the attacker has full knowledge of the victim’s network architecture, training parameters, and training set.

Original: The adversary uses this knowledge to train surrogate ASR systems for poison optimization.
Revised: The adversary uses this knowledge to train surrogate ASR systems for poison optimization.

Original: We run each attack instance for a maximum of K = 20 rounds.
Revised: We run each attack instance for a maximum of K = 20 rounds.

Original: For the early stopping criteria, we test after each round, if we succeed against a ( surrogate) test model
Revised: For the early stopping criteria, we test after each round if we succeed against a (surrogate) test model.

Original: 1) Experimental Setup: We use the TIDIGITS dataset [27], which is designed for speaker-independent recognition of digit sequences and consists of eleven words: ONE , TWO , ..., NINE , ZERO , and OH.
Revised: 1) Experimental Setup: We use the TIDIGITS dataset [27], which is designed for speaker-independent recognition of digit sequences and consists of eleven words: ONE , TWO , ..., NINE , ZERO , and OH

Original: We use 8,623 utterancesfor
Revised: We use ,623 utterances

Original:  the training
Revised: for the training

Original: set and 4, utterances for the test set.
Revised: set and 4,390 utterances for the test set.

Original: The sequences are spoken by 225 speakers (111 men and 114 women), which are split equally into disjoint sets between the training and test set.
Revised: The sequences are spoken by 225 speakers (111 men and 114 women), which are split equally into disjoint sets between the training and test set.

Original: For our poisoning attack trials, we randomly sample 30 singledigit utterances among the 4,390 test samples and assign a target label to each.
Revised: For our poisoning attack trials, we randomly sample 30 single-digit utterances among the 4,390 test samples and assign a target label to each of them.

Original: Target labels are chosen randomly and are different from the ground-truth transcription.
Revised: Target labels are chosen randomly and are different from the ground-truth transcription.

Original: The victim’s ASR system uses the DNN 2+ architecture (described in Table I) with a softmax output layer of size 95, corresponding to the number of HMM states.
Revised: The victim’s ASR system uses the DNN 2+ architecture (described in Table I) with a softmax output layer of size 95, corresponding to the number of HMM states.

Original: This system is trained from scratch for 33 epochs with a batch size ofusing the Adam [22] optimizer with a learning rate of 1 e − 4 .
Revised: This system is trained from scratch for 33 epochs with a batch size ofusing the Adam [22] optimizer with a learning rate of 1 e − 4 .

Original: This training also includes three epochs of Viterbi training to build the language model.
Revised: This training also includes three epochs of Viterbi training to build the language model.

Original: Hyperparameters were chosen to maximize the clean test accuracy.
Revised: Hyperparameters were chosen to maximize the clean test accuracy.

Original: For the baseline model —only trained with clean data— we achieved a test accuracy of 98 .
Revised: For the baseline model —only trained with clean data— we achieved a test accuracy of 98 .

Original: For evaluation of the attack, the random seed used by the victim is unknown.
Revised: For evaluation of the attack, the random seed used by the victim is unknown.

Original: Thus, the specific parameters of the victim’s
Revised: Thus, the specific parameters of the victim’s

Original: ASR system, the neural network, and the HMM—which depend on the neural network due to Viterbi training—are not used during poison optimization. 
Revised: ASR system, the neural network, and the HMM—which depend on the neural network due to Viterbi training—are not used during poison optimization. [Table] [Table]

Original: To accelerate the attack, we freeze the HMM component and only train the DNN for the surrogate ASR systems.
Revised: To accelerate the attack [Table] we freeze the HMM component and only train the DNN for the surrogate ASR systems.

Original: We found this effective as the language model does typically not change significantly.
Revised: We found this effective as the language model does typically not change significantly.

Original: The frozen surrogate HMM is trained[Table] [Table] in advance by training an ASR system for 15 epochs on the clean training set, followed by three epochs of Viterbi training.
Revised: The frozen surrogate HMM is trained in advance by training an ASR system for 15 epochs on the clean training set, followed by three epochs of Viterbi training.

Original: During the attack, we train the surrogate ASR systems forepochs until convergence.
Revised: During the attack, we train the surrogate ASR systems forepochs until convergence.

Original: 2) Results: We first evaluate the attack success rate as a function of the number of surrogate models.
Revised: 2) Results: We first evaluate the attack success rate as a function of the number of surrogate models.

Original: Table II presents the performance of V ENO M AVE for different numbers of surrogate networks.
Revised: Table II presents the performance of V ENO M AVE for different numbers of surrogate networks.

Original: Note that a higher number of surrogate models adds to the complexity of Equation 2.
Revised: Note that a higher number of surrogate models adds to the complexity of Equation 2.

Original: However, more surrogate networks can help the attack to succeed in fewer steps and, consequently, this increased complexity does not necessarily lead to a longer attack time.
Revised: However, more surrogate networks can help the attack to succeed in fewer steps and, consequently, this increased complexity does not necessarily lead to a longer attack time.

Original: This is also evident from the results in Table II.
Revised: This is also evident from the results in Table II.

Original: We obtain the highest attack success rate (86.7 %) for M = 8 surrogate models.
Revised: We obtain the highest attack success rate (86.7 %) for M = 8 surrogate models.

Original: For the case where we use M = 10 surrogate models, the attack time and required attack steps are increased whilewe have a lower attack success rate.
Revised: In the case where we use M = 10 surrogate models, the attack time and required attack steps are increased while a lower attack success rate is obtained.

Original: Note that the number of attack steps K in Table II is the average number for all 30 poisoning trials for each entry.
Revised: Note that the number of attack steps K in Table II is the average number for all 30 poisoning trials for each entry.

Original: Next [Table] we evaluate V ENO M AVE for varying levels of poison budget r p (see Section III-B3).
Revised: Next, we evaluate V ENO M AVE for varying levels of poison budget r p (see Section III-B3).

Original: The results are shown in
Revised: The results are shown in

Original: Table III.
Revised: Table III.

Original: We observe a general trend that an increase of the poison budget leads to a higher attack success rate (23.3 % → 83.3 %), which stagnates for poisonsamples budgets larger than 0 .
Revised: We observe a general trend that an increase of the poison budget leads to a higher attack success rate (23.3 % → 83.3 %), which stagnates for poison budgets larger than 0 .

Original: A higher budget, allows the attacker to manipulate an increasing number of poison frames and, thus, has more control over the training process.
Revised: A higher budget allows the attacker to manipulate an increasing number of poison frames and, thus, has more control over the training process.

Original: However, from a certain number, this effect is less distinct as the surrogate models also need to maintain a good clean test accuracy.
Revised: However, from a certain number, this effect is less distinct as the surrogate models also need to maintain a good clean test accuracy.

Original: The general improvement comes at a price; the length and number of the poisoned data increases  (6.20 s → 48.73 s) from a total of 15,254 s training data.
Revised: The general improvement comes at a price; the length and number of the poisoned data increases [Table] [Table] (6.20 s → 48.73 s) from a total of 15,254 s training data.

Original: We observe the best performance with a[Table] [Table] budget r p =0 .
Revised: We observe the best performance with a budget r p =0 .

Original: 005 , where we poison only 0.17 % of the training set while achieving an attack success rate of 86.7 %.
Revised: 005 , where we poison only 0.17 % of the training set while achieving an attack success rate of 86.7 %.

Original: Figure 3 shows an example of a poisoned audio file, as well as its respective original audio file.
Revised: Figure 3 shows an example of a poisoned audio file as well as its respective original audio file.

Original: C. Limited-Knowledge Adversary
Revised: C. Limited-Knowledge Adversary

Original: For most applications in practice, it is unrealistic to assume that an adversary has detailed knowledge of the exact training parameters, architecture, and the training data that is used by the victim.
Revised: For most applications in practice, it is unrealistic to assume that an adversary has detailed knowledge of the exact training parameters, architecture, and the training data that is used by the victim.

Original: In the following, we, therefore, want to relax the threat model and consider an adversary with limited knowledge.
Revised: In the following, we therefore want to relax the threat model and consider an adversary with limited knowledge.

Original: We consider two settings: (1) first, we restrict access to the victim’s model architecture and training parameters, and (2) second, we extend the knowledge limitations and additionally restrict access to the victim’s training data (except for the poisoned data).
Revised: We consider two settings: (1) First, we restrict access to the victim’s model architecture and training parameters, and (2) second, we extend the knowledge limitations and additionally restrict access to the victim’s training data (except for the poisoned data).

Original: For both settings and based on the previous experiments, we set the poison budget to r p = 0 .
Revised: For both settings and based on the previous experiments, we set the poison budget to r p = 0 .

Original: 005 and consider M = 8 surrogate models.
Revised: 005 and consider M = 8 surrogate models.

Original: 1) Model Architecture and Parameters: We consider that the victim uses one of three different model architectures: DNN 2 ,
Revised: 1) Model Architecture and Parameters: We consider that the victim uses one of three different model architectures: DNN 2 ,

Original: DNN 3 , or DNN 3+ from Table I.
Revised: DNN 3 , or DNN 3+ from Table I.

Original: All models are trained from scratch for 32 epochs, of which epochs 11 and 12 include
Revised: All models are trained from scratch for 32 epochs, of which epochs 11 and 12 include

Original: Viterbi training.
Revised: Viterbi training.

Original: The victim uses Adam with a learning rate of 4 e − 4 , a batch size of 64, and a dropout probability of 0.2.
Revised: The victim uses Adam with a learning rate of 4 e − 4 , a batch size of 64, and a dropout probability of 0.2.

Original: The dropout layer is added after the first hidden layer.
Revised: The dropout layer is added after the first hidden layer.

Original: Table IV shows that the malicious characteristics of the poisoned data remain even if the victim uses different training parameters and network architectures.
Revised: Table IV shows that the malicious characteristics of the poisoned data remain even if the victim uses different training parameters and network architectures.

Original: Also, for all models the clean test accuracy remains almost the same in comparison to the baseline test accuracy, which measures the accuracy of the models trained on exclusively clean data.
Revised: Also, for all models the clean test accuracy remains almost the same in comparison to the baseline test accuracy, which measures the accuracy of the models trained on exclusively clean data.

Original: It is worth noting that in prior work, dropout was typically disabled, as in a transfer learning scenario as a rational victim will usually overfit the training set [2], [49].
Revised: It is worth noting that in prior work, dropout was typically disabled, as in a transfer learning scenario, a rational victim will usually overfit the training set [2], [49].

Original: Since this is usually not the case when the victim’s model is trained from scratch, we enable dropout in this experiment.
Revised: Since this is usually not the case when the victim’s model is trained from scratch, we enable dropout in this experiment.

Original: Our results show that the poisoned data survive 
Revised: Our results show that the poisoned data survive the randomness introduced by the dropout.

Original: [Figure]
Revised: [Figure]

Original: 2) Training Dataset: Building upon the previous experiment
Revised: 2) Training Dataset: Building upon the previous experiment

Original: , we further reduce the attacker’s knowledge and assume that the attacker only has partial knowledge about the training set of the victim and its underlying distribution.
Revised: , we further reduce the attacker’s knowledge and assume that the attacker only has partial knowledge about the training set of the victim and its underlying distribution.

Original: In general, the adversary uses their knowledge about the training data to (1) perform the ratio analysis (see Section III) and (2) train surrogate networks for the poison crafting step.
Revised: In general, the adversary uses their knowledge about the training data to (1) perform the ratio analysis (see Section III) and (2) train surrogate networks for the poison crafting step.

Original: Note that for this experiment we continue to use an unknown victim’s model architecture.
Revised: Note that for this experiment we continue to use an unknown victim’s model architecture.

Original: For the experiment, we divide the training data into two subsets with disjoint sets of 56 speakers each.
Revised: For the experiment, we divide the training data into two subsets with disjoint sets of 56 speakers each.

Original: We restrict the adversary to access only the first subset (Split 1, 56 speakers).
Revised: We restrict the adversary to access only the first subset (Split 1, 56 speakers).

Original: For the victim, we consider two different scenarios: (1) training samples only from the second subset (Split 2, 0 % overlap), and (2) the entire training set (Split 1+2, 50 % overlap).
Revised: For the victim, we consider two different scenarios: (1) training samples only from the second subset (Split 2, 0 % overlap), and (2) the entire training set (Split 1+2, 50 % overlap).

Original: Similar to the previous experiment, we evaluate a victim with different training parameters and network architecture ( DNN 3 ).
Revised: Similar to the previous experiment, we evaluate a victim with different training parameters and network architecture ( DNN 3 ).

Original: As the poison samples only depend on Split 1, we use the same data for both cases.
Revised: As the poison samples only depend on Split 1, we use the same data for both cases.

Original: Table V presents the performance of V ENO M AVE for these two scenarios.
Revised: Table V presents the performance of V ENO M AVE for these two scenarios.

Original: When the victim’s training set has no overlap with the attacker’s training set, V ENO M AVE achieves an attack success rate of 86.7 %.
Revised: When the victim’s training set has no overlap with the attacker’s training set, V ENO M AVE achieves an attack success rate of 86.7 %.

Original: When the attacker’s training set consists of 50 % of the victim’s training set, V ENO M AVE achieves an attack success rate of 80.0 %.
Revised: When the attacker’s training set consists of 50 % of the victim’s training set, V ENO M AVE achieves an attack success rate of 80 %.

Original: While the same poisoned data is used in these two cases, in the latter case, the poisoned data are competing with more clean data points.
Revised: While the same poisoned data is used in these two cases, in the latter case, the poisoned data are competing with more clean data points.

Original: This may explain why V ENO M AVE achieves a lower attack success rate despite the fact that it has partial knowledge of the victim’s training set.
Revised: This may explain why V ENO M AVE achieves a lower attack success rate despite the fact that it has partial knowledge of the victim’s training set.

Original: The average clean test accuracy is 97.92% and 98.03% for 0 % and 50 % overlap cases, respectively.
Revised: The average clean test accuracy is 97.92% and 98.03% for 0 % and 50 % overlap cases, respectively.

Original: D. Multi-Word Replacement Attack
Revised: D. Multi-Word Replacement Attack

Original: Next, we want to scale the attack to more complex targets and, in particular, aim to replace multiple words.
Revised: Next, we want to scale the attack to more complex targets and, in particular, aim to replace multiple words.

Original: This can be realized by launching multiple individual word replacement attacks simultaneously.
Revised: This can be realized by launching multiple individual word replacement attacks simultaneously.

Original: For a successful multi-word attack, all single-word attacks need to be successful.
Revised: For a successful multi-word attack, all single-word attacks need to be successful.

Original: For this experiment, we evaluate the attack for sentences with two, three, and four digits.
Revised: For this experiment, we evaluate the attack for sentences with two, three, and four digits.

Original: For each set, we select 20 random audio files and aim to replace all the words with randomly chosen adversarial words.
Revised: For each set, we select 20 random audio files and aim to replace all the words with randomly chosen adversarial words.

Original: As an example, the adversary might try to fool the ASR system to recognize an utterance of O89 as 762 .
Revised: As an example, the adversary might try to fool the ASR system to recognize an utterance of O89 as 762 .

Original: We continue to use a limited-knowledge attacker that does not have access to the victim’s training parameters and network architecture.
Revised: We continue to use a limited-knowledge attacker that does not have access to the victim’s training parameters and network architecture.

Original: We use the same setup as before and DNN 3 as the victim’s network architecture but a larger output layer of size 350 to contain all required phones of the extended language model.
Revised: We use the same setup as before and DNN 3 as the victim’s network architecture.

Original: Table VI shows the attack statistics for sentences with different numbers of words.
Revised: Table VI shows the attack statistics for sentences with different numbers of words.

Original: For reference, we repeat the results for the single-word attack in Table VI.
Revised: For reference, we repeat the results for the single-word attack in Table VI.

Original: The attack remains effective for longer sequences of words albeit with a decreased success rate.
Revised: The attack remains effective for longer sequences of words albeit with a decreased success rate.

Original: Also, the attack uses more poisoned data to perform a multiple-digit replacement compared to a singleword replacement attack.
Revised: Also, the attack uses more poisoned data to perform a multiple-digit replacement compared to a singleword replacement attack.

Original: [Table]
Revised: [Table]

Original: E. Speech Commands Dataset
Revised: E. Speech Commands Dataset

Original: To further examine the practical feasibility of our attack, we evaluate V ENO M AVE on a larger ASR system.
Revised: To further examine the practical feasibility of our attack, we evaluate V ENO M AVE on a larger ASR system.

Original: To this end, we use the Speech Commands corpus [47] used for keyword spotting.
Revised: To this end, we use the Speech Commands corpus [47] used for keyword spotting.

Original: This dataset consists of 105,829 one-word utterances and contains 35 different words:
Revised: This dataset consists of 105,829 one-word utterances and contains 35 different words:

Original: • Digits ZERO , ..., NINE • Common words for IoT or robotics applications.
Revised: • Digits ZERO , ..., NINE • Common words for IoT or robotics applications.

Original: YES , NO , UP , DOWN , LEFT , RIGHT , ON , OFF , STOP , and GO • Command words.
Revised: YES , NO , UP , DOWN , LEFT , RIGHT , ON , OFF , STOP , and GO • Command words.

Original: FORWARD , FOLLOW , BACKWARD , and LEARN .
Revised: FORWARD , FOLLOW , BACKWARD , and LEARN .

Original: • Auxiliary words.
Revised: • Auxiliary words.

Original: BED , BIRD , CAT , DOG , HAPPY , HOUSE , MARVIN , SHEILA , TREE , VISUAL , and WOW .
Revised: BED , BIRD , CAT , DOG , HAPPY , HOUSE , MARVIN , SHEILA , TREE , VISUAL , and WOW .

Original: For our poisoning attack trials, we randomly select 15 audio files and for each sample, we pick a random adversarial target.
Revised: For our poisoning attack trials, we randomly select 15 audio files and for each sample, we pick a random adversarial target.

Original: To fit this dataset, we use a larger neural network as well as a larger language model with 350 states.
Revised: To fit this dataset, we use a larger neural network as well as a larger language model with 350 states.

Original: We use the DNN 3+ architecture for our surrogate networks.
Revised: We use the DNN 3+ architecture for our surrogate networks, but with a larger output layer of size 350 to contain all required phones of the extended language model.

Original: As before, we use a fixed HMM during the attack, which is trained in advance by training an ASR system for 16 epochs on the clean training set, of which the last epoch includes Viterbi training.
Revised: As before, we use a fixed HMM during the attack, which is trained in advance by training an ASR system for 16 epochs on the clean training set, of which the last epoch includes Viterbi training.

Original: We use this surrogate HMM at the beginning of each step of the attack to train four surrogate networks on the latest version of the poisoned dataset for 20 epochs with a batch size of 32.
Revised: We use this surrogate HMM at the beginning of each step of the attack to train four surrogate networks on the latest version of the poisoned dataset forepochs with a batch size of 32.

Original: We verify that the training converges at 20 epochs.
Revised: We verify that the training converges at 20 epochs.

Original: We use the Adam [22] optimizer with a learning rate of 1 e − 4 for poison crafting.
Revised: We use the Adam [22] optimizer with a learning rate of 1 e − 4 for poison crafting.

Original: For the victim, we use a network architecture consisting of four hidden layers with 300, 200, 200, and 200 neurons, respectively.
Revised: For the victim, we use a network architecture consisting of four hidden layers with 300, 200, 200, and 200 neurons, respectively.

Original: The victim trains the ASR system from scratch for 31 epochs, of which the eleventh epoch enables Viterbi training.
Revised: The victim trains the ASR system from scratch for 31 epochs, of which the eleventh epoch enables Viterbi training.

Original: For the victim’s training, a learning rate of 4 e − 4 and a batch size of 64 is used.
Revised: For the victim’s training, a learning rate of 4 e − 4 and a batch size of 64 is used.

Original: With a poison budget of r p = 0 .
Revised: With a poison budget of r p = 0 .

Original: 02 , V ENO M AVE achieves a success rate of 73.3% while poisoning only 0.14 % of the training set (116.73 seconds of audio).
Revised: 02 , V ENO M AVE achieves a success rate of 73.3% while poisoning only 0.14 % of the training set (116.73 seconds of audio).

Original: Table VII shows the attack performance for each example.
Revised: Table VII shows the attack performance for each example.

Original: We successfully poisoned 11 of the 15 tirals.
Revised: We successfully poisoned 11 of the 15 trials.

Original: In general, we need to poison more and longer audio sequences with this extended dataset but the attack remains successful in most of the cases.
Revised: In general, we need to poison more and longer audio sequences with this extended dataset but the attack remains successful in most of the cases.

Original: F. Over-The-Air Attack
Revised: F. Over-The-Air Attack

Original: Prior work on audio adversarial examples [48], [34] has often struggled in an over-the-air setting: during the transmission over the air, the audio signal is altered which may affect the poisoning success.
Revised: Prior work on audio adversarial examples [34], [48] has often struggled in an over-the-air setting: During the transmission over the air, the audio signal is altered, which may affect the poisoning success.

Original: In this following, we study the effects of transmission over the air on our poisoning attack.
Revised: In this following, we study the effects of transmission over the air on our poisoning attack.

Original: First, we consider a simulated setting.
Revised: First, we consider a simulated setting.

Original: To this end, we use the Python RIR Simulator implementation [10] and simulate the transmission in a room via a convolution with a Room Impulse
Revised: To this end, we use the Python RIR Simulator implementation [10] and simulate the transmission in a room via a convolution with a Room Impulse

Original: Response (RIR) [4].
Revised: Response (RIR) [4].

Original: We evaluate the attack in three simulated rooms with the microphone and the speaker being positioned randomly.
Revised: We evaluate the attack in three simulated rooms with the microphone and the speaker being positioned randomly.

Original: For each setting, we use four different reverberation times between 0.4–1.0 seconds.
Revised: For each setting, we use four different reverberation times between 0.4–1.0 seconds.

Original: Second, we evaluate the attack in a real physical room with an iPhone 13 Pro microphone and a JBL GO speaker.
Revised: Second, we evaluate the attack in a real physical room with an iPhone 13 Pro microphone and a JBL GO speaker.

Original: We consider both datasets.
Revised: We consider both datasets.

Original: For the TIDIGITS dataset, we use the poison samples that are generated in Section IV-C2 for the 0 % overlap setting.
Revised: For the TIDIGITS dataset, we use the poison samples that are generated in Section IV-C2 for the 0 % overlap setting.

Original: Consequently, the adversary does not know the victim’s DNN architecture and training parameters as well as the training set (except for the poisoned data).
Revised: Consequently, the adversary does not know the victim’s DNN architecture and training parameters as well as the training set (except for the poisoned data).

Original: Note that the victim uses DNN 3 in this evaluation.
Revised: Note that the victim uses DNN 3 in this evaluation.

Original: For the
Revised: For the

Original: Speech Commands dataset, we use the same poisoned data as in Section IV-E.
Revised: Speech Commands dataset, we use the same poisoned data as in Section IV-E.

Original: Table VIII shows the results for different reverberation times (RT) in seconds, room dimensions,and speaker and microphone positions.
Revised: Table VIII shows the results for different reverberation times (RT) in seconds, room dimensions, speaker and microphone positions.

Original: In addition, we also report the results for the physical room.
Revised: In addition, we also report the results for the physical room.

Original: For the TIDIGITS dataset, V ENO M AVE maintains a success rate of 33.3-73.3% across different room settings as opposed to the success rate of 86.7% when feeding the input directly to the recognizer.
Revised: For the TIDIGITS dataset, V ENO M AVE maintains a success rate of 33.3-73.3% across different room settings as opposed to the success rate of 86.7% when feeding the input directly to the recognizer.

Original: For the Speech Commands dataset,
Revised: For the Speech Commands dataset,

Original: V ENO M AVE maintains an attack success rate of 20.0-60.0% across different room settings as opposed to the success rate of 73.3% when feeding the input directly to the recognizer.
Revised: V ENO M AVE maintains an attack success rate of 20-60% across different room settings as opposed to the success rate of 73.3% when feeding the input directly to the recognizer.

Original: G. Transferability
Revised: G. Transferability

Original: In the previous sections, we focused on hybrid ASR systems, and our results demonstrated that these are vulnerable to dataset poisoning attacks.
Revised: In the previous sections, we focused on hybrid ASR systems, and our results demonstrated that these are vulnerable to dataset poisoning attacks.

Original: In this experiment, we consider the effect of the poisons for other ASR architectures.
Revised: In this experiment, we consider the effect of the poisons for other ASR architectures.

Original: In particular, a victim that uses an end-to-end ASR system.
Revised: In particular, a victim that uses an end-to-end ASR system.

Original: For this, we use an end-to-end system designed for the task of keyword spotting [38], [29], [5] on the Speech Commands dataset based on SpeechBrain [33].
Revised: For this, we use an end-to-end system designed for the task of keyword spotting [5], [29], [38] on the Speech Commands dataset based on SpeechBrain [33].

Original: This ASR system has a total of 4,494,777 trainable parameters.
Revised: This ASR system has a total of 4,494,777 trainable parameters.

Original: For reference, the hybrid system that we evaluated in Section IV-E, has a total of 265,295 trainable parameters,  0.06 times less than the endto-end system.
Revised: For reference, the hybrid system that we evaluated in Section IV-E has a total of 265,295 trainable parameters, which is 0.06 times less than the end-to-end system.

Original: We use the same poisons we generated in Section IV-E to attack hybrid ASR systems.
Revised: We use the same poison samples generated in Section IV-E to attack hybrid ASR systems.

Original: For each of the 11 successful attack examples, we evaluate the victim’s end-to-end system by training it on the poisoned datasets.
Revised: For each of the 11 successful attack examples, we evaluate the victim’s end-to-end system by training it on the poisoned datasets.

Original: We observe that the attack fools the victim’s end-to-end system for four examples, showing a transferability rate of 36.4%.
Revised: We observe that the attack fools the victim’s end-to-end system for four examples, showing a transferability rate of 36.4%.

Original: The test accuracy for the poisoned models is on average at 95.06%.
Revised: The test accuracy for the poisoned models is on average at 95.06%.

Original: H. User Study
Revised: [Table]

Original: To evaluate the human perception of our poison samples, we conduct a listening test, where we asked participants to
Revised: [Table]

Original: [Figure]
Revised: [Table]

Revised: are for the poison samples that are

Original: [Table]
Revised: [Table]

Original: dataset, the numbers are for the poison samples that are generated in Section IV-C2 for the 0% overlap setting.
Revised: in Section [Table] overlap setting.

Original: For the Speech Commands dataset, we use the poisoned data that V ENO M AVE crafted in SectionIV-E.
Revised: [Table] the [Table] dataset, we use the poisoned data that V ENO M AVE crafted in Section

Original: [Table] [Table]
Revised: [Table] [Table]

Original:  transcribe utterances of the poisoned data.
Revised: To evaluate the human perception of our poison samples, we conduct a listening test, where we ask participants to transcribe utterances of the poisoned data.

Original: Furthermore, in this section, we additionally consider psychoacoustic modeling [50], [35] as a mechanism to limit the perceptible perturbations introduced by the attack.
Revised: Furthermore, in this section, we additionally consider psychoacoustic modeling [35], [50] as a mechanism to limit the perceptible perturbations introduced by the attack.

Original: 1) Psychoacoustic Modeling: To make poisons less conspicuous, we can utilize psychoacoustic modeling to limit audible distortions.
Revised: 1) Psychoacoustic Modeling: To make poisons less conspicuous, we can utilize psychoacoustic modeling to limit audible distortions.

Original: Recent attacks against ASR [35], [32] proposed psychoacoustic hiding as a method to create less perceptible adversarial noise.
Revised: Recent attacks against ASR [32], [35] proposed psychoacoustic hiding as a method to create less perceptible adversarial noise.

Original: To identify inaudible ranges, these attacks use dynamic hearing thresholds, which describe the masking effects in human perception that arise as a function of the interactions between different co-occurring acoustic frequencies.
Revised: To identify inaudible ranges, these attacks use dynamic hearing thresholds, which describe the masking effects in human perception that arise as a function of the interactions between different co-occurring acoustic frequencies.

Original: We implement psychoacoustic hiding similar to what is described by Sch¨onherr et al.
Revised: We implement psychoacoustic hiding similar to what is described by Sch ¨ onherr et al.

Original: Appendix VIII-A elaborates in detail how we employ psychoacoustic filtering.
Revised: Appendix VIII-A elaborates in detail how we employ psychoacoustic filtering.

Original: We evaluate V ENO M AVE for varying degrees of psychoacoustic filtering, controlled through margin Λ (in dB) that allows the attack to surpass the hearing thresholds.
Revised: We evaluate V ENO M AVE for varying degrees of psychoacoustic filtering, controlled through margin Λ (in dB) that allows the attack to surpass the hearing thresholds.

Original: The higher Λ , the more audible noise is allowed.
Revised: The higher Λ , the more audible noise is allowed.

Original: As shown by Table IX, enabling the psychoacoustic hiding decreases the attack success rate, while the SNRseg of poisoned frames improves.
Revised: As shown by Table IX, enabling the psychoacoustic hiding decreases the attack success rate, while the SNRseg of poisoned frames improves.

Original: The case without enforcing hearing thresholds is denoted as NONE .
Revised: The case without enforcing hearing thresholds is denoted as NONE .

Original: Note that the choice of poison samples and frames does not depend on the margin Λ ; that is, the average length of the poisoned data is always 25.44s in Table IX.
Revised: Note that the choice of poison samples and frames does not depend on the margin Λ ; that is, the average length of the poisoned data is always 25.44s in Table IX.

Original: 2) Transcription Test: For the study, we randomly selected 20 poison samples from 12 successful attack examples, both when the psychoacoustic hiding was disabled and for Λ=30 dB, which resulted in a pool of 480 poison samples.
Revised: 2) Transcription Test: For the study, we randomly selected 20 poison samples from 12 successful attack examples, both when the psychoacoustic hiding was disabled and for Λ=30 dB, which resulted in a pool of 480 poison samples.

Original: For verification, participants also transcribed five hidden clean samples.
Revised: For verification, participants also transcribed five hidden clean samples.

Original: We asked 23 English-speaking persons to transcribe a random subset of utterances.
Revised: We asked 23 English speakers to transcribe a random subset of utterances.

Original: The participants were not informed if a sample has been modified or if it represents a clean sample.
Revised: The participants were not informed if a sample has been modified or if it represents a clean sample.

Original: On average, each user transcribed 40 poison samples.
Revised: On average, each user transcribed 40 poison samples.

Original: For each attack example, we report the ratio of the poison samples that are transcribed into their original label.
Revised: For each attack example, we report the ratio of the poison samples that are transcribed into their original label.

Original: When the psychoacoustic hiding is disabled, 87.1 % of the poison samples were transcribed into their original labels.
Revised: When the psychoacoustic hiding is disabled, 87.1 % of the poison samples were transcribed into their original labels.

Original: On the other hand, for Λ=30 dB, 85.0 % of the poison samples were transcribed into their original labels.
Revised: On the other hand, for Λ=30 dB, 85.0 % of the poison samples were transcribed into their original labels.

Original: These results show that even though enforcing hearing thresholds of Λ = 30 dB improves the SNRseg values of the poisoned frames (from 2.
Revised: These results show that even though enforcing hearing thresholds of Λ = 30 dB improves the SNRseg values of the poisoned frames (from 2.

Original: to 4.25, see Table IX), the performance of the transcription test is not improved.
Revised: to 4.25, see Table IX), the performance of the transcription test is not improved.

Original: The results of this feasibility study also indicate that the poisoned data generated by V ENO M AVE contain samples that can be considered as clean-label samples.
Revised: The results of this feasibility study also indicate that the poisoned data generated by V ENO M AVE contain samples that can be considered as clean-label samples.

Original: Such a study has often been missing in prior works, and as noted by Schwarzschild et al.
Revised: Such a study has often been missing in prior works, and as noted by Schwarzschild et al.

Original: [36], most current attacks in the visual domain produce easily visible artifacts and distortions.
Revised: [36], most current attacks in the visual domain produce easily visible artifacts and distortions.

Original: V. D ISCUSSION
Revised: V. D ISCUSSION

Original: Next, we expand our analysis of V ENO M AVE by providing insights into our results.
Revised: Next, we expand our analysis of V ENO M AVE by providing insights into our results.

Original: We will also summarize the results and discuss major findings and limitations.
Revised: We will also summarize the results and discuss major findings and limitations.

Original: A. Attack Parameters
Revised: A. Attack Parameters

Original: Here, we discuss the impact of V ENO M AVE ’s parameters on the attack success rate.
Revised: Here, we discuss the impact of V ENO M AVE ’s parameters on the attack success rate.

Original: 1) Poison Budget & Surrogate Models: Using a larger poison budget r p increases the number of poisoned files (and frames).
Revised: 1) Poison Budget & Surrogate Models: Using a larger poison budget r p increases the number of poisoned files (and frames).

Original: However, we show that beyond a poison budget of 0.005, the attack success does not further improve (see Table III), and, therefore, more poison samples are not necessarily required for the attack.
Revised: However, we show that beyond a poison budget of 0.005, the attack success does not further improve (see Table III), and, therefore, more poison samples are not necessarily required for the attack.

Original: The same can be observed for the number of surrogate models; using more surrogate models does not necessarily increase the attack’s success (see Table II).
Revised: The same can be observed for the number of surrogate models; using more surrogate models does not necessarily increase the attack’s success (see Table II)

Original:  2) Target Selection:
Revised: . 2) Target Selection:

Original: In Section IV-D, we show that V ENOM AVE is not limited to the replacement of single words; it can successfully replace all the words with the intended adversarial words.
Revised: In Section IV-D, we show that V ENOM AVE is not limited to the replacement of single words; it can successfully replace all the words with the intended adversarial words

Original: Consequently, an attacker has full control of the output of the target, and arbitrary transcriptions can be chosen.
Revised: Consequently, an attacker has full control of the output of the target, and arbitrary transcriptions can be chosen.

Original: This is further supported in our experiments with the
Revised: This is further supported in our experiments with the

Original: Speech Commands dataset, where we show that V ENO M AVE scales to ASR systems with a larger vocabulary.
Revised: Speech Commands dataset, where we show that V ENO M AVE scales to ASR systems with a larger vocabulary

Original: To further understand how the number of HMM states of the target word affects the success rate of V ENO M AVE , we consider our single-word replacement attack in Section IV-C on the TIDIGITS dataset.
Revised: To further understand how the number of HMM states of the target word affects the success rate of V ENO M AVE , we consider our single-word replacement attack in Section IV-C on the TIDIGITS dataset.

Revised: We conducted this experiment over 30 trials, which we divide here into three different categories:

Original: We did this experiment over 30 trials, which we divide here into three different categories: (1) Intrials, the target word has more HMM states than the original word, (2) in 7 trials, the target word and the original word have the same number of HMM states, and (3) in 12 trials the target word has less HMM states than the original word.
Revised: (1) In 11 trials , the target word has more HMM states than the original word, (2) in 7 trials, the target word and the original word have the same number of HMM states, and (3) intrials the target word has less HMM states than the original word.

Original: For the results presented in Table IV (last column), the attack fails on two, one, and two trials, respectively, in these three types of trials, showing that the difference between the number of HMM states of the target and original word does not affect the success rate of the attack.
Revised: For the results presented in Table IV (last column), the attack fails on two, one, and two trials, respectively, in these three types of trials, showing that the difference between the number of HMM states of the target and original word does not affect the success rate of the attack.

Original: 3) Sequence Selection: To quantify the effect of the sequence selection on the attack success rate, we repeat the experiment from Section IV-C
Revised: 3) Sequence Selection: To quantify the effect of the sequence selection on the attack success rate, we repeat the experiment from Section IV-C

Original: (Table IV).
Revised: (Table IV).

Original: Instead of choosing the target sequences based on the frequency analysis (explained in
Revised: Instead of choosing the target sequences based on the frequency analysis (explained in

Original: Section III-B), we now randomly select the target sequence.
Revised: Section III-B), we now randomly select the target sequence.

Original: We require that the sequence has to be in ascending order (e.g., for a target sequence like [92, 92, 91, 91, 93, 93] the language model can otherwise not return a valid word).
Revised: We require that the sequence has to be in ascending order (e.g., for a target sequence like [92, 92, 91, 91, 93, 93] the language model can otherwise not return a valid word).

Original: In this experiment, we observe a drop in the attack success rate by 23.33 percentage points (from 83.33% to 60.0%).
Revised: In this experiment, we observe a drop in the attack success rate by 23.33 percentage points (from 83.33% to 60.0%).

Original: B. Clean Test Accuracy
Revised: B. Clean Test Accuracy

Original: In our evaluation, we always use the entire test dataset to calculate the clean accuracy using the edit distance between the ground-truth label and the predicted transcription.
Revised: In our evaluation, we always use the entire test dataset to calculate the clean accuracy using the edit distance between the ground-truth label and the predicted transcription.

Original: Here, we aim to understand how the attack affects the recognition of the target word in isolation.
Revised: Here, we aim to understand how the attack affects the recognition of the target word in isolation.

Original: We use the results presented in
Revised: We use the results presented in

Original: Section IV-C for the following measurements:
Revised: Section IV-C for the following measurements:

Original: • For each digit, we only consider the test audio files that contain the digit to calculate the number of errors (I + S + D, Section IV).
Revised: • For each digit, we only consider the test audio files that contain the digit to calculate the number of errors (I + S + D, Section IV).

Original: On average over 30 trials, the total number of errors for the target and original digits areand 95 words, respectively, while the number of errors for the other digits is 111 words.
Revised: On average over 30 trials, the total number of errors for the target and original digits areand 95 words, respectively, while the number of errors for the other digits is 111 words.

Original: • For each digit, we consider the test audio files that do not contain the digit.
Revised: • For each digit, we consider the test audio files that do not contain the digit.

Original: For these files, we count how often the model’s transcription (mistakenly) contains the digit.
Revised: For these files, we count how often the model’s transcription (mistakenly) contains the digit.

Original: On average over 30 trials, for 9.97 utterances, the model mistakenly recognizes the target digit.
Revised: On average over 30 trials, for 9.97 utterances, the model mistakenly recognizes the target digit.

Original: For the original digit, this value is 8.97, while for the other digits, this value is 10.26 on average.
Revised: For the original digit, this value is 8.97, while for the other digits, this value is 10.26 on average.

Original: C. Practical Considerations
Revised: C. Practical Considerations

Original: In the following, we elaborate on the practical aspects of our attack and reflect on its implications and limitations.
Revised: In the following, we elaborate on the practical aspects of our attack and reflect on its implications and limitations.

Original: 1) Clean-Label Poison Utterances:
Revised: ) Clean-Label Poison Utterances:

Original: In the listening test, we verify that V ENO M AVE is able to generate clean-label poison samples.
Revised: In the listening test, we verify that V ENO M AVE is able to generate clean-label poison samples.

Original: We ask participants to transcribe poisoned audio samples and on average, more than 85% of the poison samples were transcribed into their original labels, showing that even manual verification of training data would not be effective to prevent audio poisoning attacks.
Revised: We ask participants to transcribe poisoned audio samples and on average, more than 85% of the poison samples were transcribed into their original labels, showing that even manual verification of training data would not be effective to prevent audio poisoning attacks.

Original: Furthermore, in privacy-preserving federated learning scenarios, where the training data and the training is decentralized, a party can easily compromise the training data [43].
Revised: Furthermore, in privacy-preserving federated learning scenarios, where the training data and the training is decentralized, a party can easily compromise the training data [43].

Original: Here, the poison samples are not constrained to clean-label data points, as the victim has no access to the training data, while the attacker has full control of their data.
Revised: Here, the poison samples are not constrained to clean-label data points, as the victim has no access to the training data, while the attacker has full control of their data.

Original: Additionally, our limited-knowledge experiments have shown that controlling only parts of the training process and training data—as would be the case in a federated learning scenario—is very effective.
Revised: Additionally, our limited-knowledge experiments have shown that controlling only parts of the training process and training data—as would be the case in a federated learning scenario—is very effective.

Revised: Limited Vocabulary: We showed our attack is

Original: 2) Limited Vocabulary: We showed our attack is successful on two datasets, TIDIGITS and Speech Commands, of which the latter is ten times bigger than the former.
Revised: successful on two datasets TIDIGITS and Speech Commands, of which the latter is ten times bigger than the former.

Original: We argue that our results show that data poisoning attacks against ASR systems are a viable threat that needs to be considered by researchers working on ASR systems.
Revised: We argue that our results show that data poisoning attacks against ASR systems are a viable threat that needs to be considered by researchers working on ASR systems.

Original: Based on our foundations, we hope that future work will improve the scalability of our attack and include larger datasets in their evaluation and develop more robust ASR systems that are resistant to data poisoning attacks.
Revised: Based on our foundations, we hope that future work will improve the scalability of our attack and include larger datasets in their evaluation and develop more robust ASR systems that are resistant to data poisoning attacks.

Original: 3) Fine-Tuning: Although hybrid ASR systems are typically trained from scratch, we now want to expand our evaluation and also consider a fine-tuning scenario.
Revised: 3) Fine-Tuning: Although hybrid ASR systems are typically trained from scratch, we now want to expand our evaluation and also consider a fine-tuning scenario.

Original: For this, we use the poisoned data generated for the most restricted adversary (Table V).
Revised: For this, we use the poisoned data generated for the most restricted adversary (Table V).

Original: That is, the adversary’s training set is the “Split 1” subset.
Revised: That is, the adversary’s training set is the “Split 1” subset.

Original: For the victim’s model, we divide the “Split 2” subset into two parts of equal size (each with 28 speakers).
Revised: For the victim’s model, we divide the “Split 2” subset into two parts of equal size (each with 28 speakers).

Original: The first part is the training set and contains only clean data.
Revised: The first part is the training set and contains only clean data.

Original: The second part, which is the fine-tuning set, is poisoned.
Revised: The second part, which is the fine-tuning set, is poisoned.

Original: On average, over the same 30 trials, we observe an attack success rate of 63.33% (83.33% for the from-scratch training scenario).
Revised: On average, over the same 30 trials, we observe an attack success rate of 63.33% (83.33% for the from-scratch training scenario).

Original: For training and fine-tuning, we used a learning rate of 1eand 5e-5, respectively.
Revised: For training and fine-tuning, we used a learning rate of 1eand 5e-5, respectively.

Original: 4) Over-the-Air:
Revised: 4) Over-the-Air:

Original: In Section IV-F, we demonstrate that
Revised: In Section IV-F, we demonstrate that

Original: V ENO M AVE is also successful if the targeted audio signal is played over the air in simulated and physical rooms of different sizes.
Revised: V ENO M AVE is also successful if the targeted audio signal is played over the air in simulated and physical rooms of different sizes.

Original: This shows the general robustness of our attack and that the poison samples also remain effective after a transmission’s alterations.
Revised: This shows the general robustness of our attack and that the poison samples also remain effective after a transmission’s alterations.

Original: Notably, the attack is generic in the sense that the properties of the room need not be known beforehand.
Revised: Notably, the attack is generic in the sense that the properties of the room need not be known beforehand.

Original: 5) Transferability
Revised: 5) Transferability

Original: To End-To-End Keyword Spotting: To verify the practicality of V ENO M AVE in the real world, we evaluate the poisoned data generated by the attack against an end-to-end ASR system, designed specifically for the task of keyword spotting on the Speech Commands dataset.
Revised: To End-To-End Keyword Spotting: To verify the practicality of V ENO M AVE in the real world, we evaluate the poisoned data generated by the attack against an end-to-end ASR system, designed specifically for the task of keyword spotting on the Speech Commands dataset.

Original: Our results in Section IV-G show that although the poison samples of V ENO M AVE are not crafted for end-to-end systems, they remain viable and can be a potential threat to such systems.
Revised: Our results in Section IV-G show that although the poison samples of V ENO M AVE are not crafted for end-to-end systems, they remain viable and can be a potential threat to such systems.

Original: 6) Hearing Thresholds: Hearing thresholds have shown to be effective for adversarial examples, however, in the case of poisoning, we observe that their effect is less distinct.
Revised: 6) Hearing Thresholds: Hearing thresholds have shown to be effective for adversarial examples, however, in the case of poisoning, we observe that their effect is less distinct.

Original: One main reason may be that in contrast to adversarial examples, where the complete file is modified, our modifications for the poison utterances are limited to short sequences.
Revised: One main reason may be that in contrast to adversarial examples, where the complete file is modified, our modifications for the poison utterances are limited to short sequences.

Original: VI. R ELATED W ORK
Revised: VI. R ELATED W ORK

Original: In the following, we discuss related work on attacks against machine learning and ASR systems.
Revised: In the following, we discuss related work on attacks against machine learning and ASR systems.

Original: 1) Adversarial Examples: Adversarial examples are carefully crafted inputs that are perturbed by adding imperceptible noise to fool a machine learning model [41], [8].
Revised: 1) Adversarial Examples: Adversarial examples are carefully crafted inputs that are perturbed by adding imperceptible noise to fool a machine learning model [8], [41].

Original: Such perturbations are calculated using the gradients of an optimization problem that is defined on the victim network, or surrogate networks, if the victim network is unknown.
Revised: Such perturbations are calculated using the gradients of an optimization problem that is defined on the victim network, or surrogate networks, if the victim network is unknown.

Original: Initial work on adversarial attacks focused on the space of images [17], [8].
Revised: Initial work on adversarial attacks focused on the space of images [8], [17].

Original: Later, similar evasion attacks were shown to exist in the audio domain, where generating adversarial examples is more challenging due to time dependencies that exist in the ASR systems[44], [12],  [35], [34].
Revised: Later, similar evasion attacks were shown to exist in the audio domain, where generating adversarial examples is more challenging due to time dependencies that exist in the ASR systems [12], [34], [35], [44].

Original: 2) Backdoor Attacks: For a backdoor attack, an adversary manipulates the victim model by imprinting training samples with a specific pattern ( trigger ) and the target label to train the model to become sensitive to this pattern [18].
Revised: 2) Backdoor Attacks: For a backdoor attack, an adversary manipulates the victim model by imprinting training samples with a specific pattern ( trigger ) and the target label to train the model to become sensitive to this pattern [18].

Original: During inference, the attacker can then cause a misclassification by injecting the trigger into any input example.
Revised: During inference, the attacker can then cause a misclassification by injecting the trigger into any input example.

Original: By using ultrasonic triggers, the feasibility of such an attack against ASR was recently demonstrated in a technical report by Koffas et al.
Revised: By using ultrasonic triggers, the feasibility of such an attack against ASR was recently demonstrated in a technical report by Koffas et al.

Original: In contrast to our work and similar to evasion attacks, however, backdoor attacks require the modification of test samples during inference, which is not always applicable in real-world scenarios.
Revised: In contrast to our work and similar to evasion attacks, however, backdoor attacks require the modification of test samples during inference, which is not always applicable in real-world scenarios.

Original: 3) Training-Time Poisoning Attacks: Closest to our work are training-time poisoning attacks[37], [49], [2],  [19], [15] against image classification, wherein the adversary crafts poison images—with no control over the labeling process—to achieve the system’s misbehavior for specific target inputs.
Revised: 3) Training-Time Poisoning Attacks: Closest to our work are training-time poisoning attacks [2], [15], [19], [37], [49] against image classification, wherein the adversary crafts poison images—with no control over the labeling process—to achieve the system’s misbehavior for specific target inputs.

Original: There exist major limitations with these attacks, which hinder their application to ASR systems.
Revised: There exist major limitations with these attacks, which hinder their application to ASR systems.

Original: First, these attacks focus on transfer learning, which is not a common training practice for speech recognition; ASR systems are typically trained from scratch.
Revised: First, these attacks focus on transfer learning, which is not a common training practice for speech recognition; ASR systems are typically trained from scratch.

Original: Second, they assume that the victim does not use dropout during the fine-tuning process, while dropout is often enabled in training neural networks from scratch.
Revised: Second, they assume that the victim does not use dropout during the fine-tuning process, while dropout is often enabled in training neural networks from scratch.

Original: Furthermore, unlike image classification, the recognition process of ASR is based on time series signals (i. e., the waveform audio signal).
Revised: Furthermore, unlike image classification, the recognition process of ASR is based on time series signals (i. e., the waveform audio signal).

Original: Consequently, these attacks cannot directly be applied to speechbased systems.
Revised: Consequently, these attacks cannot directly be applied to speechbased systems.

Original: 4) Countermeasures: Although several automated defenses have been proposed[31], [28], [13],  they can typically be evaded by an adaptive attacker [24], [36].
Revised: 4) Countermeasures: Although several automated defenses have been proposed [13], [28], [31], they can typically be evaded by an adaptive attacker [24], [36].

Original: One line of possible defenses focus on poison detection and removing them from the train set.
Revised: One line of possible defenses focus on poison detection and removing them from the train set.

Original: This usually happens by employing some neighborhood conformity tests or outlier detection, either on the data itself or in the latent space [31].
Revised: This usually happens by employing some neighborhood conformity tests or outlier detection, either on the data itself or in the latent space [31].

Original: This type of detection, however, requires access to the training data, which is not always given (e. g., in a federated learning setting).
Revised: This type of detection, however, requires access to the training data, which is not always given (e. g., in a federated learning setting).

Original: Most recent defenses also consider retrospective countermeasures like forensic-inspired approaches [39].
Revised: Most recent defenses also consider retrospective countermeasures like forensic-inspired approaches [39].

Original: Their strategy is to detect the origin of the poisoned data after a successful attack, and, therefore, cannot prevent harm beforehand.
Revised: Their strategy is to detect the origin of the poisoned data after a successful attack, and, therefore, cannot prevent harm beforehand.

Original: Other defenses try to detect poisoned models [31], [28], [13].
Revised: Other defenses try to detect poisoned models [13], [28], [31].

Original: However, these sanitization-based defenses may be easily leveraged by an attacker who is aware of the specific defense mechanism, as they are attack-specific [24], [36].
Revised: However, these sanitization-based defenses may be easily leveraged by an attacker who is aware of the specific defense mechanism, as they are attack-specific [24], [36].

Original: More importantly, most defenses require clean reference data to sanitize the training data.
Revised: More importantly, most defenses require clean reference data to sanitize the training data.

Original: The distribution of such clean data needs to be close to the distribution of the training data, which is often not realistic.
Revised: The distribution of such clean data needs to be close to the distribution of the training data, which is often not realistic.

Original: VII. C ONCLUSIONS
Revised: VII. C ONCLUSIONS

Original: In this paper, we present V ENO M AVE , the first trainingtime poisoning attack against speech recognition.
Revised: In this paper, we present V ENO M AVE , the first trainingtime poisoning attack against speech recognition.

Original: In a series of experiments, we demonstrate V ENO M AVE ’s efficacy and evaluate the attack under different attack settings and for various attack parameters.
Revised: In a series of experiments, we demonstrate V ENO M AVE ’s efficacy and evaluate the attack under different attack settings and for various attack parameters.

Original: We tested single and multi-word replacement attacks and investigated the effect of an enlarged language model.
Revised: We test single and multi-word replacement attacks and investigate the effect of an enlarged language model.

Original: The attack remains viable in an over-the-air scenario, with limited knowledge about the victim model, and transfers between different speech recogntion architectures.
Revised: The attack remains viable in an over-the-air scenario, with limited knowledge about the victim model, and transfers between different speech recogntion architectures.

Original: Finally, we verify with a user study that the majority of poison samples are clean-label, which renders a manual verification of the training data ineffective.
Revised: Finally, we verify with a user study that the majority of poison samples are clean-label, which renders a manual verification of the training data ineffective.

Original: In summary, we show with V ENO M AVE that data poisoning of ASR systems poses a real threat that needs to be considered. 
Revised: In summary, we show with V ENO M AVE that data poisoning of ASR systems poses a real threat that needs to be considered. A CKNOWLEDGMENTS

Revised: We would like to thank our reviewers for their valuable comments and input to improve our paper.

Revised: This material is based upon work partially supported by NSF under Award #CNS-2107101 and by a gift from Intel, Corp.

Revised: Any opinions, findings, and conclusions or recommendations expressed in this publication are those of the author(s) and do not necessarily reflect the views of NSF or Intel.

Original:  VIII. A PPENDIX A. Psychoacoustic Modeling
Revised: Moreover, this work was funded by the Deutsche Forschungsgemeinschaft (DFG, German Research Foundation) under Germany’s Excellence Strategy – EXC 2092 CASA – 390781972. VIII. A PPENDIX A. Psychoacoustic Modeling

Original: Recent adversarial attacks against ASR systems [35], [32] use psychoacoustic hearing thresholds to hide modifications of the input audio signal within inaudible ranges.
Revised: Recent adversarial attacks against ASR systems [32], [35] use psychoacoustic hearing thresholds to hide modifications of the input audio signal within inaudible ranges.

Original: By using hearing thresholds, we can limit audible distortions.
Revised: By using hearing thresholds, we can limit audible distortions.

Original: These thresholds define how dependencies between certain frequencies can mask, i e., make inaudible, parts of an audio signal.
Revised: These thresholds define how dependencies between certain frequencies can mask, i.e., make inaudible, parts of an audio signal.

Original: In essence, we guide V ENO M AVE to hide malicious noise in these inaudible parts.
Revised: In essence, we guide V ENO M AVE to hide malicious noise in these inaudible parts.

Original: At each step of the poison crafting, we scale the gradients of the poison audio signal (calculated via minimizing
Revised: At each step of the poison crafting, we scale the gradients of the poison audio signal (calculated via minimizing

Original: Equation 2) with scaling factors that limit audible distortions.
Revised: Equation 2) with scaling factors that limit audible distortions.

Original: Since human thresholds alone are tight, the scaling factors are allowed for differing from the thresholds by a margin of Λ (in dB).
Revised: Since human thresholds alone are tight, the scaling factors are allowed for differing from the thresholds by a margin of Λ (in dB).

Original: The higher Λ , the more audible noise is allowed to be added by the attack.
Revised: The higher Λ , the more audible noise is allowed to be added by the attack.

Original: In the following, we discuss how we compute the scaling factors.
Revised: In the following, we discuss how we compute the scaling factors.

Original: First, we compute the power spectrum of the difference
Revised: First, we compute the power spectrum of the difference

Original: D between the poison signal spectrum Υ and the original signal spectrum O for all times t and frequencies q as the following:
Revised: D between the poison signal spectrum Υ and the original signal spectrum O for all times t and frequencies q as the following:

Original: [Equation]
Revised: [Equation]

Original: Then, we compute the audible difference (in dB) for all times t and frequencies q via
Revised: Then we compute the audible difference (in dB) for all times t and frequencies q via

Original: [Equation]
Revised: [Equation]

Original: where H is the computed human hearing thresholds based on the psychoacoustic model of MPEG-1 [21].
Revised: where H is the computed human hearing thresholds based on the psychoacoustic model of MPEG-1 [21].

Original: Since the thresholds H are tight, we allow V ENO M AVE to differ from the hearing thresholds by a margin of Λ (in dB).
Revised: Since the thresholds H are tight, we allow V ENO M AVE to differ from the hearing thresholds by a margin of Λ (in dB).

Original: In particular, we calculate the matrix ζ ∗ for all times t and frequencies q as
Revised: In particular, we calculate the matrix ζ ∗ for all times t and frequencies q as

Original: [Equation]
Revised: [Equation]

Original: else where we clip the negative values to zero for the time-frequency bins thatwe cross the thresholds H + Λ .
Revised: else where we clip the negative values to zero for the time-frequency bins that cross the thresholds H + Λ .

Original: We then normalize the matrix ζ ∗ to values between zero and one via
Revised: We then normalize the matrix ζ ∗ to values between zero and one via

Original: [Equation]
Revised: [Equation]

Original: We also compute a fixed scaling factor by normalizing the hearing thresholds H to values between zero and one via
Revised: We also compute a fixed scaling factor by normalizing the hearing thresholds H to values between zero and one via

Original: [Equation]
Revised: [Equation]

Original: Putting the scaling factors ˆ ζ and ˆ H together, the gradient of ∇ X computed via Equation 2 will be scaled as the following
Revised: Putting the scaling factors ˆ ζ and ˆ H together, the gradient of ∇ X computed via Equation 2 will be scaled as the following

Original: [Equation]
Revised: [Equation]

Original: This scaling happens between the Discrete Fourier Transform (DFT) and the magnitude step in the computational graph.IX. R EVIEWS F ROM P RIOR S UBMISSION
Revised: This scaling happens between the Discrete Fourier Transform (DFT) and the magnitude step in the computational graph.

Original: We have submitted this work to USENIX Security 2023, for which we augmented our extensive set of experiments with more experiments.

Original: Most importantly, we added the evaluation of our poison samples against the publicly available, endto-end speech toolkit SpeechBrain.

Original: In the USENIX Security 2023 submission, reviewers raised valid concerns, due to the lack of details in our text.

Original: We received valuable and constructive comments, which indeed helped us to improve our paper.

Original: Therefore, for the current submission, we focus on the presentation of the paper to address major concerns that were raised by the reviewers.

Original: In the following, we discuss all the major improvements.

Original: For reference, we attached the integral version of the reviews from USENIX Security 2023.

Original: Experimental Setup Details.

Original: In general, for each of our experiments, before discussing results, we now elaborate more on our experimental setup, and we provide all the details in the text.

Original: We also revised the description of the presented results to make sure that all required information is provided.

Original: Specific Challenges of Poisoning Speech Recognition Systems.

Original: In the prior submission, our discussion of why attacking speech recognition systems is more challenging than attacking

Original: CV was too high-level and lacking details about how these systems operate.

Original: In the current version, we added an explanation why poisoning these systems is harder and fundamentally different in comparison to previously studied attacks on systems such as image classifiers.

Original: Methodology Description.

Original: Reviewers pointed out that our description in Algorithm 1 is not clear in some parts, and it is hard to cross-match Algorithm 1 with the main text of the paper.

Original: For this submission, we updated the specification of

Original: Algorithm 1 to address these issues.

Original: Paper summary -------------

Original: Thispaperstudies the effectiveness of performing targeted poisoning attack on automatic speech recognition (ASR) systems.

Original: The goal is to modify/poison the training data (that a victim downloads) so that an ASR system (trained from scratch) misclassifies a given input as a misleading target output desired by an attacker.

Original: Based on the two datasets,

Original: TIDIGITs and Speech Commands, they demonstrate about 80% attack success rate while only poisoning 0.17% of the dataset.

Original: Detailed comments for authors

Original: Astheauthorspointoutany type of dataset poisoning attack would be of a great concern for companies.

Original: However, large companies that develop their own

Original: ASR solutions usually gather datasets from multiple sources: (a) companies might download and use public datasets (but unlikely from unknown or untrusted sources); (b) conduct their own data collection studies; and (c) pay third party companies to collect high quality datasets.

Original: Datasets gathered from multiple sources are used to train ASR models to improve accuracy, and also mitigate such risks associated with using corrupted datasets.

Original: Poisoning a small portion of samples in one public dataset might not be as effective in such scenarios.

Original: Given the strong assumptions listed in Section 4.1, a much simpler attack would involve changing the labels of the dataset that are used to train the

Original: HMMs (e.g., changing the labels fromto 9 in the example given in Figure 2) -- i.e., performing a dirty-label data poisoning attack.

Original: If the attacker changes just a small portion of the labels -- unless the victim goes through every sample (introduction mentions millions of samples may be used for training), and checks that the dataset is labeled correctly -- it would be difficult for victims to detect such simple label manipulation attacks anyway.

Original: The authors could emphasize the effectiveness of the proposed clean-label poisoning attack by comparing its performance against a dirty-label approach.

Original: One simple security control (for preventing any type of data poisoning) would be for the data owners to digitally sign the datasets, and provide means for dataset users to verify signatures (for authenticity and data integrity) before using them -- that way, dataset users would easily avoid tampered datasets.

Original: Evaluation method is not clear.

Original: In Section 5.2 the authors explain that 4390 samples have been used as the test set, andsingle-digit audio files and a target digit for each have been sampled.

Original: Does this mean that just 30 (out of 4390) samples have been selected as the attack set?

Original: Without clear understanding, it is hard to interpret their attack success results.

Original: It is not obvious as to how the poisoned samples (without changed labels: i.e., poisoned samples are still labeled as "eight" states) would be sufficient to effectively affect the model boundaries in a way that some "eight" states are classified as "nine" states -- adding a theoretical explanation would be helpful.

Original: I am also concerned about the reported "clean test accuracy" for the TIDGITs dataset.

Original: Since the entire dataset only contains 11 words (digits), there is no real significant overlap between the 11 words with respect to their phoneme segmentation: it seems likely that the model accuracy for non-target digits would not be affected anyway.

Original: Required changes ---------------- -Evaluationmethodology details need to be clarified - Explain how the classification boundaries can be effectively manipulated without changing the labels of the poisoned samples - Discuss how the attack performance might be affected if the victim is using multiple datasets - Explain why simple security controls such as verifying dataset signatures might not be sufficient to mitigate this type of data poisoning attack

Original: - Compare performance against a simpler attack that changes small portions of the labels (i.e., against a dirty-label data poisoning attack)

Original: Reasons to accept the paper --------------------------- -Highlightstheimportance of verifying the integrity and authenticity of dataset sources -Proposes the first training-time poisoning attack against ASR systems

Original: Reasons not to accept the paper -------------------------------

Original: -Evaluationdetailsaremissing (hard to interpret results) -

Original: Simple security controls could mitigate the attack - Does not consider current industry trends where multiple datasets are used for training

Original: Recommended decision -------------------- 4.Reject

Original: Questions for authors’ response -------------------------------

Original: Q1)Howwouldtheattackperformance be affected if a victim uses datasets from multiple sources (and just a small portion of the combined dataset is the poisoned set)?

Original: Q2)

Original: How was the TIDIGITS attack test set generated from the 4390 test samples?

Original: Q3)

Original: What would be the attack success rates for a much simpler dirty-label poisoning attack that changes just a small portion of the labels?

Original: Q4)

Original: Verifying dataset signatures for authenticity and integrity might be sufficient to detect any poisoning attacks performed under the assumptions listed in

Original: Section 4.1.

Original: Why is it still important to explore new data poisoning attack methods?

Original: Writing quality --------------- 2.Well-written

Original: Reviewer confidence -------------------

Original: B. Review

Original: 313B Paper summary

Original: Thisworkproposes a poisoning attack against Automatic Speech Recognition (ASR) systems.

Original: Detailed comments for authors -----------------------------

Original: **Strengths:** -interestinguser study and real-world experiments -well-written, clear (but limited)

Original: contribution

Original: **Weaknesses:** - Limited applicability; only applies to

Original: ASR Hybrid systems trained from scratch; tested only on small datasets (larger dataset are available, such as https://huggingface.co/datasets/librispeech_asr) andsimplemodels(DNNswiththreelayers) -Limitednovelty;theauthorsadaptan already-existingattack(bullseyepolytope

Original: [Aghakhani])totheapplicationtheyare targeting.

Original: -Theauthorsdiscussdefensesforthis kindofattacks,howevertheydonottest whethertheywouldbeeffectiveagainst theproposedone.

Original: **Required clarifications** * The discussion about the fact that attacks "from scratch" vs. fine-tuned models are more challenging is not detailed enough.

Original: It is not clear if this is true, and it is neither supported by references nor experiments.

Original: * "As a result, it remains unclear whether poisoning attacks against speech recognition are feasible since attacks from the image domain cannot be na¨ıvely adopted.

Original: " It seems that the problems described before this sentence are not directly caused by the "image" domain, but rather by how the data are processed and how the models are trained.

Original: In fact, the attack being used was originally proposed for the image domain, then adapted to this specific case [2].

Original: * The authors state that a frequency analysis is needed to choose the most probable sequence.

Original: However, this requires knowledge of the training dataset and the authors don’t discuss how this is treated in the case of transfer-based attacks.

Original: * "First, these attacks focus on transfer learning, which is not shown to be useful for speech recognition.

Original: " As it is, this sentence does not imply that the attacks based on transfer learning do not work on speech recognition, but only that nobody showed it.

Original: Is this the intended information conveyed by the sentence?

Original: * in the results in table 2, it is not properly discussed why the attack success rate drops when using 10 models instead of 8; Moreover, also the longer runtime for 6 surrogate models rather than 8 seems strange.

Original: * It seems that the result of the attack traces back to the spectrograms, whereas the MFCC reverse transform is not detailed.

Original: It is clear that the MFCC could be reversed, however the authors should specify if the attack still works if all the poisoned samples are converted back to waveforms (the actual input data of the model) and then used for training from scratch.

Original: This detail is not present in the paper, however it is of utmost importance for assessing the validity of the threat scenario.

Original: * The authors perform experiments with psychoacoustic hiding thresholds on the samples, but they do not discuss how relevant this is for poisoning attacks, as it is unlikely that the person training the model would listen to all the samples inspecting the labels.

Original: **Unsupported statements or claims** * Additionally, ASR systems are |in general| trained from scratch.

Original: This also contrasts with existing work leveraging transfer learning for ASR data [Qin].

Original: **Incorrect/unclear statements** * The related work about backdoor attacks might need some edits.

Original: First, the input label does not necessarily need to be injected along with the perturbed samples (trigger).

Original: Second, the sentence "backdoor attacks require the modification of test samples during inference, which is not always applicable in real-world scenarios" does not seem valid, as the test samples are usually in control of the attacker.

Original: Moreover,

Original: targeted poisoning attacks are implemented similarly to backdoor attacks while targeting (clean) test points.

Original: Maybe the authors did not express correctly the idea they want to deliver, or at least they should clarify this point.

Original: *Style and presentation

Original: ** * Algorithm 1 is highly unreadable and contains too many comments.

Original: It might be better to describe the lines of the algorithm by referring to them in the text rather than in the algorithm itself * All results are presented without any highlighting.

Original: Adding bold fonts for relevant values (e.g. best values) would help reading results better and easily.

Original: Reasons to accept the paper ---------------------------

Original: Thepaperiswellwrittenand contains an extensive experimental analysis.

Original: Reasons not to accept the paper -------------------------------

Original: Noveltyisverylimitedforthis venue.

Original: The authors adapt an attack that was already proposed [2 in the paper] to work against an ASR system, that also is a very specific application.

Original: I have seen this paper submitted to top-tier venues more than 2-3 times now, without any substantial improvement, and I would thus recommend the authors to target a journal.

Original: Given the substantial lack of novelty, I think a journal publication would be more appropriate.

Original: Recommended decision -------------------- 4.Reject

Original: Writing quality --------------- 2.Well-written

Original: Reviewer confidence ------------------- 3.Highlyconfident (would try to convince