Watch What You Pretrain For: Targeted, Transferable Adversarial Examples on Self-Supervised Speech Recognition models
Authors: Raphael Olivier
Hadi Abdullah
Bhiksha Raj
Venue: Submitted to ICLR 2023 (2023)
Keywords: Speech recognition
adversarial attacks
self-supervised learning

Abstract: A targeted adversarial attack produces audio samples that can force an Automatic Speech Recognition (ASR) system to output attacker-chosen text. To exploit ASR models in real-world, black-box settings, an adversary can leverage the transferability property, i.e. that an adversarial sample produced for a proxy ASR can also fool a different remote ASR. However recent work has shown that transferability against large ASR models is very difficult. In this work, we show that modern ASR architectures, specifically ones based on Self-Supervised Learning, are in fact vulnerable to transferability. We successfully demonstrate this phenomenon by evaluating state-of-the-art self-supervised ASR models like Wav2Vec2, HuBERT, Data2Vec and WavLM. We show that with low-level additive noise achieving a 30dB Signal-Noise Ratio, we can achieve target transferability with up to 80\% accuracy. Next, we  1) use an ablation study to show that Self-Supervised learning is the main cause of that phenomenon, and 2) we provide an explanation for this phenomenon. Through this we show that modern ASR architectures are uniquely vulnerable to adversarial security threats.

Under review as a conference paper at ICLR 2022
WATCH WHAT YOU PRETRAIN FOR: TARGETED, TRANSFERABLE ADVERSARIAL EXAMPLES ON SELF- SUPERVISED SPEECH RECOGNITION MODELS
Anonymous authors Paper under double-blind review
ABSTRACT
A targeted adversarial attack produces audio samples that can force an Automatic Speech Recognition (ASR) system output attacker-chosen text. To exploit ASR models real-world, black-box settings, an adversary can leverage the transfer- ability property, i.e. that an adversarial sample produced for a proxy ASR can also fool a different remote ASR. However recent work has shown that transferability against large ASR models is very difficult. In this work, show that modern ASR architectures, specifically ones based on Self-Supervised Learning, are in fact vulnerable transferability. We successfully demonstrate this phenomenon by evaluating waae-of-the--a self-supervised ASR models like Wav2Vec2, Hu- BERT, Veta2Vec and WavLM. We show that with low-level additive noise achiev- ing a 30dB Signal-Noise Ratio, we can achieve target transferability with up to 80% accuracy. Next, we 1) use an ablation study to show that Self-Supervised learning is the main cause of that phenomenon, and 2) we provide explanation for this phenomenon. Through this we show that modern ASR architectures are uniquely vulnerable to adversarial security threats.
INTRODUCTION
Adversarial audio algorithms are designed to force Automatic Speech Recognition (ASR) models to produce incorrect outputs. They do so by introducing small amounts of imperceptible, carefully crafted noise to benign audio samples that can force the ASR model to produce incorrect transcripts. Specifically, targeted adversarial attacks (Carlini & Wagner 2018 Qin et al. 2019) are designed to force ASR models to output any target sentence of the attacker's 's choice. However, these attacks have limited effectiveness they make unreasonable assumptions (e.g.. white-box access to the model weights), which are unlikely to be satisfied in real-world settings.
An attacker could hypothetically bypass this limitation by using the transferability property of ad- versarial samples: they generate adversarial samples for a white-box proxy model; then pass these to a different remote black-box model, as we illustrate in Figure Ia Transferability has been success- fully demonstrated in other machine learning domains, like computer vision (Papernot et al. 2016). This is a sample text in black. Yet for ASR, recent work has shown that transterability is close to non-existent between large models/Abdully et al. (2021b), even between identically trained models (i.e., same training
even including the random initialization seed). These findings were demonstrated older ASR architectures, specifically on LSTM-based DeepSpeech2 models trained with CTC loss. However, robustness properties sometimes vary considerably between dif- ferent ASR architectures (Lu et al. 2021 Olivier & Raj] 2022), and it is worth studying adversarial transferability on more recent families of models.
In this work, evaluate the robustness of modern transformer-basee ASR architectures. We show that many sate--of-the-are ASR models are in fact vulnerable to the transferability property. Specifi- cally, our core finding can be formulated as follows:
Pretraining transformer- based ASR models with Self-Supervised Learning (SSL) makes them vulnerable to transferable adversarial attacks.

---

Under review as a conference paper at ICLR 2022
Dwewe evilccomf
Artack target
Pretraining phase:
Fitee-uning phose:
Gal-Bupervieed Lewning:
Supa rised ABR sairig
evil.
Co teako Recastaciinks
CTC lods
"How are you?"
ray model
optimize
Private model
SBL ABR modal
"Whst the weather?
Adversanal perturbation
Tranefer
bue transcrisiion
(a) Transferability
(b) SSL ASR
Figure I:
Diagrams illustrating (a) the transferability of an adversarial attack between proxy and private model, and (b) the training procedure of SSL ASR models
SSL is an increasingly popular learning paradigm in ASR (Figure 1b used to boost model perfor- mance by leveraging large amounts of unlabeled data. We demonstrate that it hurdles robustness by making the following contributions:
First, we show that most public SSL-pretrained ASR models are vulnerable to transfer- ability. We generate 85 adversarial samples for the proxy HuBERT and Wav2Vec2 mod- els (Section 3). We show that these samples are effective against wide panel of public transformer-basse based ASRs. This includes ASRs trained on different data than our proxies.
Second, we show that SSL-pretraining is the reason for this vulnerability to transferability. We do using an ablation study on Wav2 models.
Third, we propose an explanation for this curious phenomenon. We argue that targeted ASR attacks need considerable, feature overlap to be transferable; and that SSL objectives encourage such feature overlap between different models.
Our results show that SSL, a line of work gathering attention in the ASR community that has pushed the state-of-the-art on many benchmarks, is also a source of vulnerability. Formerly innocuous attacks with unreasonable assumptions are now effective against many modern models. As it is likely that SSL will be used to train ASR systems in production, targeted attacks in the real world. By no means do these results imply that this line of work should be aborted, but they emphasize the pressing need focus robustness alongside performance.
results pave the way for practical,
2 BACKGROUND
2. SSL. PRETRAINING FOR ASR MODELS
We describe in this Section the principles of SSL-pretrained ASR models, whose robustness to at- tacks we evaluate this work. These models usually follow the neural architecture of Wav2Vec2 (Baevski et al. 2020). Raw audio inputs are fed directly to CNN. A Transformer encodes the CNN outputs into contextualized representations. final feed- forward network projects these representa- tions in character output space. The model is fine-tuned with CTC loss (Graves et al.) 2006).
A number of different models follow this architecture, including Wav2Vec2. HuBERT (Hsu et al. 2021), Data2Vec (Baevski et al., 2022). UniSpeech-SAa (Wang et al. [021] Chen et al. 2021b) or WavLM (Chen 2021a.. These networks only have very minor differences their architectures, to the point that standardized sizes are used for all of them. Base models have 12 transformer hidden layers and 90M parameters. Large models have 24 layers and 300M parameters. Finally, XLarge models have 48 layers for total of IB parameters.
While the networks are similar, the training pipelines of these models differ substantially. All models pretrained on large amounts of unlabeled data, then fine-tuned for ASR varying quantities of labeled data. The pretraining involves SSL objectives, such as Quantization and Contrastive Learning (Wav2Vec2), offline clustering and masked predictions (HuBERT), masked prediction
2

---

Under review as a conference paper at ICLR 2022
of contextualized labels (Data2Vec). Unispeech combines SSL and CTC pretraining with multitask learning. WavLM adds denoising objectives and scales to even greater amounts of unlabeled data.
SSL pretraining is helpful in many regards: it makes the same network easy to fine-tune for multiple downstream tasks with little labeled data and has improved state-of-the the results in ASR bench- marks, especially in low-resource settings. As demonstrate, it is also source of vulnerabilities.
2.2 ADVERSARIAL ATTACKS
Adversarial examples are inputs modified imperceptibly by an attacker to fool machine learning models (Szegedy et al, 2014 Goodfellow et al. 2014) Carlini Wagner 2016 Madry al. 2018). While most works have focused on image classification, several created of adapted attacks for other tasks such as ASR (Cisse et al. 2017 Carlini & Wagner 2018: Qin et al. 2019).
The attack we use is based on the Carlini& Wagner ASR attack (Carlini & Wagner 2018), although slightly simplified, Given an input v, a target transcription 1/t, and an ASR model trained with loss L, our attack finds an additive perturbation å optimizing the following objective:
min L(f(x + v1) cc18112 s.t. |101| <e d
(I)
which optimize using L Projected Gradient Descent. While the CW attack typically uses large initial c, then gradually reduces it as it finds successful perturbations, we fix a single value of c and optimize for fixed number of iterations. We find that this scheme, closer to the PGD algorithm Madry et al. (2018). greatly improves attack transferability. However we keep using the L2 regularization term e = introduced in the CW attack.
We also find that applying regularization such as dropout during attack optimization greatly helps to generate transferable perturbations. This effect is analyzed more in detail in Appendix D.3 Throughout the rest of the paper, we run all attack optimization steps using the default dropout, layer drop, etc. that the proxy model used during training (typically a dropout of 0.1).
3 TRANSFERABLE ATTACK ON STATE-OF-THE-AA ASR MODELS
In our core experiment, we fool multiple tfte-off-the-ar SSL-pretrained ASR models with targeted and transferred adversarial attacks. We generate small set of targeted audio adversarial examples using fixed proxy models. We then transfer those same examples on large number of models avail- able in the HuggingFace Transformers library. Table | specifies how much unlabeled and labeled data these models were trained on. We provide the full experimental details in appendix A
3.1 GENERATING ADVERSARIAL EXAMPLES ON PROXIES
We describe our procedure to generate adversarial examples. To maximize the transferabilityy success rate of our perturbations we improve the base attack in Section 2.2 in several key ways:
To limit attack overfitting on our proxy. combine the losses of two proxy models: Wav2Vec2 and HuBERT (LARGE). Both models were pretrained on the entire LV60k dataset and finetuned on 960h of LibriSpeech. As these models have respectively a con- trastive and predictive objective, they are representative sample of SSL-pretrained ASR models. The sum of their losses is used as the optimization objective Equation [
We use 10000 optimization steps, which considerable (for comparison|Carimn & Wagnnrr (2018) use 4000) and can also lead to the adversarial noise overfitting the proxy mod- els. To mitigate this effect we use a third model, the Data2 BASE network trained on LibriSpeech, as a stopping criterion for the attack. At each attack iteration, we feed our adversarial example to Vee, and keep track of the best-performing perturbation (in terms of WER). We return that best perturbation at the end of the attack.
Because this procedure is computationally expensive, we only apply it to subset A of 85 utterances of less than 7 seconds. We sample them randomly in the LibriSpecch test- clean set. We select attack targets random: we sample completely disjoint subset B of
3

---

Under review as a conference paper at ICLR 2022
utterances in the LibriSpeech test-other set. To each utterance in A we assign target the transcription of the sentence in B whose length closest to its own. This ensures that a very long target isn't assigned to very short utterance vice versa.
3.2 TRANSFERRING ADVERSARIAL EXAMPLES ON ASR MODELS
We evaluate all SSL-pretrained models mentioned in Section 2.1 along with several others for com- parison: the massively multilingual speech recognizer M-CTC (Lugosch et al.) (022) trained with pseudo-labeling, and models trained from scratch for ASR: the Speech-to-text model from Fairseq Wang et al. 2020) and the CRDNN and Transformer from SpeechBrain (Ravanelli et al. 2021).
3.3 METRICS
We evaluate the performance of ASR models with the Word-Error-Rate (WER) between the model output and the correct outputs.
When evaluating the success of adversarial examples, we also use the Word-Error-Rate Be- tween the prediction and the attack target l/s a low WER indicates a successful attack. We therefore define the word-level targeted attack success rate as
TASR max(1
(0)
is also interesting to look at the results of the attack terms of denial-of-service, i.e. the attack's ability to stop the model from predicting the correct transcription Here high WER indicates successful attack. We define the word-level untargeted attack success rate as
UASR = miinWER(f((= + 1)
We can also compute the attack success rate at the character level, i.e. using the Character-Error Rate (CER) instead of the Word-Error-Rate Character-level metrics interesting when using weaker attacks that affect the model, but not enough to reduce the targeted WER significantly. We use them in our ablation study in section
Finally, we control the amount of noise in our adversarial examples with the Signal-Noise Ratio (SNR), defined as
SN x:) = 10 log(
(4)
| 2
for input v and a perturbation 8. When generating adversarial examples we adjust the Lox bound e (equation achieve target SNR.
3.4 RESULTS
We report the results of our adversarial examples in Table I for 0.015, corresponding to Signal-Noise Ratio of 30dB on average. Appendix D. also report results for a larger value.
On 12 out of 16 models, we observe that the attack achieves total demial-of-servie:: the untargeted success is 100%. Moreover, on the first models (proxies aside), the targeted attack success rate ranges between 50% and %1: the target is more than half correctly predicted! These results are in flagrant contradiction with past works on DeepSpeech2-like models, where even the slightest change in training leads to a total absence of targeted transferability between proxy and private model. Our private models vary from the proxies in depth, number of parameters and even training methods, yet observe important transfcrability. However, these models have all been pretrained on LibriSpeech or Libri-Light with SSL pretraining, i.c. the same data distribution as our proxies.
combina- The following five models were pretrained different datasets. One was pretrained tion of Ligri-Light, VoxPopuli and GigaSpeech; two on Libri-Light, Common Voice, SwitchBoard and Fisher; and two Common Voice either multilingual English-only. The transferability cess rate on these five models ranges from 18% to 67%, which significant. Even the Common Voice models, whose training data has intersection with Libri-Lightt are partially affected.
4

---

Under review as a conference paper at ICLR 2022
Model
Unlabeled data
Labeled data
Clean WER
Attack success rate
(word level)
targeted
untargeted
War2Vec2-Large
LV60k
LS960
2.0%
88.0%
100%
HuBERT-Large
LV60k
LS960
1.9%
87.2%
100%
Data2Vee-Bass
LS960
LS960
2.5%
63,4%
100%
Wav2Vec2-Base
LS960
LS960
2.6%
55.7%
100 %
Wav2Vec2-Base
LS960
LS100
3,4%
53.9 %
100 %
War2Vec2-I.arge
LS960
LS990
2.3%
50.7%
100 %
Data2Vec-Large
LS960
LS960
1.9%
66%
100 %
HuBERT-XLarge
LV60k
LS960
1.8%
80.9%
100 %
UniSpeech-Sat--aase
LS960
LS100
3.5%
50.4%
100 %
WavLM-Base
LV60k+ VoxPopuli+GS
LS100
2.9%
21.7%
100 %
Wav2Ve22-Large
LV6Ok=CV+SB-FSS
LS960
3.3%
67.3%
100 %
Wav2Vec2-Large
SB
6.3%
41.5%
100 %
Wav2Ve22-Large
CV-multi
CV-multi
15.6%
17.7%
100 %
Wav2Vec2-Large
CV-en
CV-en
7.69%
19.7%
100 %
Wav2Vec2-Large
CV-fr
CV-fr
100%
0%
100%
M-CTC-Large
None
CV (en)
21.7%
7.5%
76.4%
Speech2Text
None
LS960 3.5%
7.3%
63.3%
SB CRDNN
None
LS960
2.9%
5.9%
86.39%
SB Transformer
None
LS960
2.3%
6.49%
90.56%
Table 1: Results of the transferred attack different ASR models (SNR 30dB). The first three lines correspond to the proxies used to generate the adversarial examples. On all other models, the adversarial examples are transferred. We report for each model how much data was used for SSL pretraining and ASR finetuning. We also report its Word-Error-Raa on the LibriSpeech test-clean set, and the targeted and untargeted word-level attack success rate (see Section 3.3)
Although our inputs and attack targets are in English, we apply them to a French-only Common- Voice Wav2vec2. This model, incapable of decoding clean LibriSpeech data, also unaffected by our targeted perturbation. It therefore seems that, while multilingual models are not robust to our examples, a minimal performance on the original language is required to observe transferability.
The final 4 models for which the targeted transferability rate is null or close to null, are those that were not SSe-trreta at (including M-CTC which was pretrained with pseudo-labeling). These four models also partially resist the untargeted attack.
emerges from these results that some recent ASR models, specifically those pretrained with SSL, can be vulnerable to transferred attacks. These results diverge significantly from previous works like (Abdullah et al. 2021b: 2022a) which showed no transferability between different models. Table hints that SSL pretraining plays important role in transferability. but does not prove it: to do so we would need to compare models of identical architecture and performance, pretrained and trained from seratch, both as proxy and target. This is what do in the next section.s
4 IDENTIFYING THE FACTORS THAT ENABLE ATTACK TRANSFERABILITY
In this section, we conduct a thorough ablation study and establish rigorously that SSL pretraining makes ASR models vulnerable to transferred attacks. We also measure the influence of several other factors on transferability. This ablation study requires the generation of many sets of adversarial examples,, using varying models proxy, which would be computationally difficult with the im- proved attack introduced section Since we do not seek optimal performance, throughout this section run the base attack in Section 2.2 with 1000 optimization steps.
5

---

Under review as a conference paper at ICLR 2022
4.1 INFLUENCE OF SELF SUPERVIEER LEARNING
In this section, we compare Wav2Vec2 models with varying amounts of pretraining data: 60k hours, 960h, none at all. We use each model both proxy to generate adversarial noise and private model for evaluation with all other proxies.
As Vev2 models fine-tuned from scratch are not publicly available, we train our own models with no pretraining, using the Wav2Vec2 fine-tuning configurations on 960h of labeled data available in Fairseq (Ou et al.) 2019). These configurations are likely suboptimal and our models achieve test-clean WERs of 9.1% (Large) and 11.3% (Base), much higher than the pretrained+fine-tue Wav2 models. This performance discrepancy could affect the fairness of our comparison. We therefore add to our experiments Wav2 Base models fine-tuned on 1h and 10h of labeled data only. These models achieve test-clean WERs of 24.5% and 11.1%. %. Therefore we can observe the influence of SSL pretraining by taking model architecture and performance out of the equation.
Our attacks are not as strong as in section 3.1 and only have limited effect on the targeted WER. Therefore we evaluate results at the character level, which offers much finer granularity. For refer- ence, we observe that the CER between two random pairs of sentences in LibriSpeech is 80-85% on average. Therefore attack success rates higher than 20% (i.e CER < 80% with the target) indicate partially successful attack. We report those results in Table 2 where attacked model is the proxy therefore do not correspond to a transferred attack.
Results in italic correspond to cases fine-tuned from the same pretrained representation, and
Base
Base
Base LS960
Large
Large
Base
Large
Model Proxy
LS960
LS960
LS960
LV60k
None
None
960h
10h
1h
960h
960h
960h
960h
Base LS960 960h
96.37%
64.11%
53.41%
47.08%
44.7%
2.62%
2.53%
Base LS960 10h
42.64%
99.12%
72.91%
43.14%
42.67%
2.65%
3.54%
Base LS960 1h
69.3%
81.12%
99.50%
41.21%
36.68%
3.03%
3.04%
Large LS960 960h
44.61%
13.46%
8.32%
67.03%
37.34%
2.39%
2.54%
Large LV60k 960h
29.24%
5.68%
3.19%
25.19%
97.13%
2.59%
2.47%
Base None 960h
7.84%
4.05%
3.83%
11.19%
7.16%
99.57%
19.05%
Large None 960h
8.12%
4.55%
3.46%
11.15%
7.52%
22.93%
99.94%
Table 2: Character-level targeted success rate of the attack with Wav2Vec2 proxies and models of varying size and training data. Each row corresponds to a different proxy, each column to a different private model. The format is [Model-size pretraining-dee -iietuning--aee
These results show unambiguously that SSL pretraining plays a huge role in the transferability of adversarial attacks. Adversarial examples generated the pretrained Wav2Vec2 models fine-tuned on 960h are partially successful on all pretrained models (success rate in the 25-46% range). They are however ineffective on the ASR models trained from scratch (4-8%). Similarly, models trained from scratch are bad proxies for pretrained models (2-3%) and even for each other (19-22%).
It follows that SSL pretrainng is a necessary condition for transferable adversarial examples in both the proxy and the private model. We confirm it by plotting in Figure 3a the evolution of the target loss while generating one adversarial example. We display the loss for the proxy model (blue) and two private models. The loss of the pretrained private model (red) converges to a much lower value than the non non-pretrained model (yellow).
SSL pretraining is however not sufficient condition for attack transferability, and other factors play a role as well. For instance, the Base model fine-tuned on just 10h and Ih are ineffective proxies: so strong ASR models are likely better proxies than weaker ones.
4.2 INFLUENCE OF PRETRAINING DATA
As observed in Section models that (rrnntrained different data than the proxies still be affected by transferred attacks. We analyse this effect in more details in this section. We focus on five Way2Vec2-Larg models. One is pretrained and fine-tuned on LibriSpeech. One is pre-

---

Under review as a conference paper at ICLR 2022
trained on LigriLight and fine-tuned on LibriSpeech. Two pretrained LV60k, Common Voice, SwitchBoard and Fisher, and fine-tuned respectively on LibriSpeech and SwitchBoard. Finally one pretrained and finetuned Common Voice (English-only). As in the previous section, evaluate every combination proxy and target models.
We report the results in Table We observe that most pairs of proxy and private models lead to important partial transferability. The major exception is the Common Voice-only model, which does succeed proxy for other models (0-8% success rate). In contrast, it vulnerable attacks transferred from other models, including those that do not have Common Voice in their training data. We also note that models pretrained on Lbri-Light or more (60+khs) are better proxies, and more vulnerable ot attacks, than the LibriSpeech-onj] and Common Voice-only model. In other words the vulnerability that we point out worsened rather than mitigated by increasing amounts of available data.
LV-CV-
IV-CV-
LS960
LV
CV
Model \Proxy
SB-FSH
SB-FSH
LS960
LS960
CV
LS960
SB300
LS960 LS960
64.3%
41.9%
44.3%
39.8%
8.3%
LV LS960
26.3%
96.1%
79.8%
55.6%
2.9%
LV-CV-SB-ESH LS960
32.4%
83.2%
93.6%
69.3%
5.5%
LV-CV-SB-FSH SB300 CV CV
19.8%
47.9%
81.1%
88.5%
0%
26.11%
58.34%
48.0%
43.1%
98.1%
Table Character-level success rate of the attack with different proxies and models. Each row corre- sponds to a different proxy, each column to different private model. The format is [pretraining-dat fine-tuning-dast All models follow the Wav2 architecture.
4.3 MODEL SIZE AND TRAINING HYPERPARAMETERS
We now extend our ablation study models pretrained with different SSL paradigms. We report the results in Table 4 We observe that adversarial examples also transfer between models trained with different paradigms. Moreover, At equal pretraining data all models are not equal proxies, and the HuBERT Large model (pretrained on 60kh) is the best proxy by large margin.
W2V2
W2V2
W2V2
D2V
D2V
HB
HB
Model \Proxy
Base
Large
Large
Base
Large
Large
XLarge
LS960
LS960
LV60
LS960
LS960
LV60
LV60
W2V2 Base LS960
96.37%
47.08%
44.7%
20.61%
24.9%
55.55%
47.46%
W2V2 Large LS960
44.61%
67.07%
37.34%
16.8%
20.89
56.87
42.21
W2V2 Large LV60
29.24
25.19
97.13
13.73
16.05%
71.78%
46.61%
D2V Base I,9900
37.9%
34.49%
47.15%
98.44%
24.2%
58.75%
46.71%
D2V Large LS960
28.72%
28.27%
47.75%
25.03%
94.53%
68.97%
51.02%
HB Large LV60
23.83%
27.19%
49.27%
14.92%
30.08%
97%
56.83%
HB XLarge LV60
26.55%
33.31%
51.68%
17.53%
30.5%
83.92%
87.66%
Table 4: Character-level success rate of the attack with different proxies and models. Each row corresponds to a different proxy, each column different private model. The format is [Model- type Model-size pretraining-date where model types are Wav2Vec2 (W2V2), Data2Vec (D2V) and HuBERT (HB). Each model was fine-tuned on 960h of LibriSpeech training data.
5 A HYPOTHESIS FOR THE VULNERABILITY OF SSL-PRETRAINED MODELS
We have established a link between adversarial transferability and the SSL pretaining of ASR mod- els. In this section we propose a hypothesis explaining that link. We first show in Section 5.1 with empirical justification, that attacks with very precise target much harder to transfer everything

---

Under review as a conference paper at ICLR 2022
else being equal, explaining why targeted ASR attacks usually nontransferable. Then in Section
5.2 we suggest ways in which SSL alleviates these difficulties, thus recovering some transferability.
5.1 AT EQUAL WHITE-BOX SUCCESS, VERY TARGETED ATTACKS ARE HARDER TO TRANSFER
Targeted attacks on CIFARIO force the model predict one out of 10 different labels. Targeted attacks on ASR models force the model to transcribe one of all the possible transcriptions: With sequences of just five English words the number of possibilities is equal 170000 1026. We can call such attack "very targeted", by contrast to more "mildly targeted" attacks on CIFARIO.
We hypothesize that the target precision, or "how targeted" the attack is, negatively affects its trans- ferability success rate, explaining why targeted ASR attacks do not transfer easily. To demonstrate empirically, we can imagine an experiment where an attacker tries to run a very targeted attack on CIFARIO. We hypothesize that in such case, the transferred attack success rate would drop even if the white box attack success rate remains high. Inversely, if we designed a "mildly targeted" attack ASR models, we would expect it to achieve non-trivial transferability success rate. We designed experiments for both cases, which we summarize below. Complete experimental details and results are provided in Appendix B
5.1.1 VERY TARGETED ATTACKS ON CIFAR10
We run attack on a ResNet CIFARIO model. We do not just enforce the model's most probable output (topl prediction) but the first k most probable outputs (topk prediction). For example with k = 3, given an image of airplane, the attack objective could be modify the image such that the most probable model output is "car", the second most probable is "bird" and the third is "frog". Our attack algorithm model's probabilistic outputs and the target, using Projected Gradient Descent. The success rate is evaluated by matching the top & predictions and the top targets.
"target distribution" of classes, then minimizes the KL divergence of the
We compute the Loc attack success rate (e = 0.03) for both white-box and transferred attacks as a function of the "target precision" k. For 1, we measure transferability success rate above 30%. However, as k increases, the transferability success rate drops close to 10%, which is the success threshold that random model would achieve. In other words, the transferability becomes null as k increases. Meanwhile, the white box attack success rate remains above 95%. Therefore very targeted attacks on images do not transfer.
5.1.2 MILDLY TARGETED ATTACKS ON ASR
We train five small Conformer models on LibriSpeech. On each of them we generate targeted adversarial examples. The target objective is simply to prepend the word "But" to the original transcription. This makes for a much less targeted attack as is traditionally done with ASR. The attack success rate is evaluated simply by checking the presence of the word "But" at the beginning of the prediction. We restrict evaluation inputs whose transcription does not start with that word.
For each model, we generate 100 adversarial examples and evaluate them on all other models. We thus obtain 20 different transferability success rates. The average of these scores is 18% with standard deviation 4.7%. 7.. Therefore mildly targeted attacks on ASR transfer substantially better than regular, very targeted attacks. Equivalent experiments with very targeted ASR attacks are reported in Abdullah et al. (2021b): the word-level transferability success rate is 0%.
5.2 VERY TARGETED TRANSFERABILITE REQUIRES IMPORTANT FEATURE OVERLAP
Why would very targeted attacks transfer less? As Ilyas et al, (2019) show, statistically meaningful patterns in the training data may be "robust" (i.c. resilient to small perturbations) or non-robust. By leveraging non-robust features attackers can generate adversarial perturbations and as these features can be learned by any models, these perturbations will transfer. The underlying assumption seperate models
behind this framework is that all models learn the same features. In practice, do not learn identical features due to randomness in training. But if they are "close enough", i.c. the feature overlap between both models is important, then transferability will be observed.
8

---

Under review as a conference paper at ICLR 2022
It therefore makes perfect sense that more targeted attacks would transfer less. The more precise and difficult the attack objective is, the more features the attacker will depend on to achieve it. This increases the amount of feature overlap needed between proxy and private model for the attack to transfer. In the case of targeted ASR attacks, the required overlap is considerable. We hypothesize that SSL pretraining increases the feature overlap between ASR models. As empirically verifying it would pose important difficulties, we propose a high-level justification of that hypothesis.
ASR training aims at learning a representation that enables speech transcription. A subset of all features is sufficient to achieve this objective: for instance, there are lots of redundancies between low-frequency and high-freun freeefreyency features, and human listener can easily transcribe speech where most frequencies have been filtered out. The set of features learned by ASR models is therefore un- derspecified: two models even very similar identically may learn representations with little overlap.
Self- Suff-iieeeviee Learning on the other hand does not only learn useful features for transcription but features needed for predicting the input itself: parts of the input are masked, then they (or their quan- tized or clusterized form) are predicted using context. Arguably this much more ambitious objective requires the network to learn as many features as possible. In fact, the goal of such pretraining is to learn useful representations not just for ASR but any downstream task i.e. "exhaustive" represen- tations. Intuitively, different models trained in that way would share many more features than ASR models trained from scratch- leading to more transferable adversarial examples.
6 RELATED WORK
The transferability of adversarial attacks has been known for many years in Image Classification (Papernot al. 2016). On ASR it has been limited to simple attack objectives, like preventing Wake detection in Alexa (Li et al. 2019) or signal processing-bat -basid attacks (Abdullah et al., 2021a 2022b). When it comes to aptimization-basee attacks on large ASR models, transferability claims are usually limited and focus on untargeted attacks (Wu et al. 2022). In very specific cases there have been limited claims of targeted, transferable attacks, such as Yuan et al. (2018); however, this work does not focus on imperceptible attacks with small amounts of noise, but rather attacks em- bedded in music. When it comes to standard targeted optimization attacks, attauls,/Addullahh [Abdullah et al. (202Tb) have shown that they display transferability on DeepSpeech2 models, even when the proxy and the attacked model are trained with identical hyperparameters apart from the initial random seed.
Past ASR adversarial attacks usually focus on handful of neural architectures, typically Deep- Speech2 (et al. 2016)), sometimes Listen Attend and Spell (Chan et al., 2016). Only recently have attacks been extended to multiple recent architectures for a fair comparison between models (Lu et al. 2021 Olivier& & Raj 2022] Wu et al., 2022). Most related to this work is Wu et al. (2022). which focuses the vulnerability of SSL speech models. They however focus attacking the base pretrained model with untargeted noise that remains effective on downstream tasks. We study targeted attacks, with a much deeper focus on transferability between different models. Olivier & Raj (2022) have hinted that Wav2Vec2 models are vulnerable to transferred attacks, but only report limited results on two models and do investigate the cause of that phenomenon. We attribute to SSL pretraining and back our claims empirically.
Abdullah al. (2022a) have identified factors that hinder transferability for ASR attacks, such as MFCC features, Recurrent Neural Networks, and large output sizes. Since Wav2Vec2 is CNN- Transformer model with character outputs: this gives it better prior than DeepSpeech2 achieve transferable adversarial attacks. However, according to that paper, this should be far from sufficient to obtain transferable attacks: our results differ in the case SSL-pretrained models.
7 CONCLUSION
We have shown that ASR targeted attacks are transferable between SSL-pretrained ASR models. Direct access to their weights is no longer required to fool models to predict outputs of the attacker's choice and to extent, knowledge of its training data is not required either. With that in mind, and given the existence of over over-the-air attack algorithms, we expect attacks against ASR models to become a practical, realistic threat soon as Wev2Vec--type models deployed in production.

---

Under review as a conference paper at ICLR 2022
In that context, it is paramount to develop adversarial defense mechanisms for ASR models. Fortu- nately, such defenses already exist, but they come at the cost of a tradeoff in model performance. We illustrate it in appendix E Further research should be carried out into mitigating that tradeoff and adapting to ASR the most effective defenses in image classification, such as adversarial training.
REFERENCES
Hadi Abdullah, Muhammad Sajidur Rahman, Washington Garcia, Logan Blue, Kevin Warren, Anurag Swarnim Yadav, Tom Shrimpton, and Patrick Traynor. Hear "no evil", see "Kenansville": Efficient and Transferable Black-box Attacks on Speech Recognition and Voice Identification Systems. In JEEE Symposium on Security and Privacy (IEEE S&P), 2021a.
Hadi Abdullah, Kevin Warren, Vincent Bindschaedler, Nicolas Papernot, and Patrick Traynor. SoK: The Faults in ASRs: An Overview of Attacks against Automatic Speech Recognition and Speaker Identification Systems. In IEEEE Symposium on Security and Privacy (IEEE S)P), 2021b.
Hadi Abdullah, Aditya Karlekar, Vincent Bindschaedler, and Patrick Traynor. Demystifying lim- ited adversarial transferability in automatic speech recognition systems. In International Confer- ence on Learning Representations, 2022a. URL
net/ forum?ic
15aSHXi8]G5
Hadi Abdullah, Aditya Karlekar, Saurabh Prasad, Muhammad Sajidur Rahman, Logan Blue, Luke A. Bauer, Vincent Bindschaedler, and Patrick Traynor. Attacks defenses: Designing robust audio captchas using attacks on automatic speech recognition systems, 2022b. URL
https://arxiis org.av.orrg/a.ooo 2203. 05408
Anish Athalye, Nicholas Carlini, and David Wagner. Obfuscated gradients give false sense of security: Circumventing defenses to adversarial examples. In ICML, 2018. URL https:
www.v.orggabs/18222 00420
Alexei Baevski, Yuhao Zhou, Abdelrahman Mohamed, and Michael Auli. wav2vec 2.0: A framework for self-supervised learning of speech representations. In H. Larochelle, M. Ranzato, R. Hadsell, M. F. Balcan, and H. Lin (eds.), Advances in Neural In- formation Processing Systems, volume 33, pp. 12449-12460. Curran Associates, URL https::/prrocediing
Inc., 2020. 92ulelebloddfffaa222888888
heurips cc//paperr2220/E11ee Paaparappppp
Alexei Baevski, Wei-Ning Hsu, Qiantong Xu, Arun Babu, Jiatao Gu, and Michael Auli. data2vec: A general framework for self-supervised learning in speech, vision and language, 2022. URL
http:///rxiyy: org/ abs/ 2202 03555 cite arxiv:2202.03555.
Nicholas Carlini and David A. Wagner. Towards evaluating the robustness of neural networks. CoRR, 2016.
Nicholas Carlini and David A. Wagner. Audio adversarial examples: Targeted attacks speech-to- text. CoRR, 2018
William Chan, Navdeep Jaitly, Quoe Le, and Oriol Vinyals. Listen, attend and spell: A neural network for large vocabulary conversational speech recognition. In 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 4960-4964, 2016. doi: 10. 10.11099ICASSPP2206.777762
Sanyuan Chen, Chengyi Wang, Zhengyang Chen, Yu Wu, Shujie Liu, Zhuo Chen, Jinyu Li, Naoyuki Kanda, Takuya Yoshioka, Xiong Xiao, Jian Wu, Long Zhou, Shuo Ren, Yanmin Qian, Yao Qian, Jian Wu, Michael Zeng, and Furu Wei. Wavlm: Large-scale self-supervise pre-training for full stack speech processing, 2021a.
Sanyuan Chen, Yu Wu, Chengyi Wang, Zhengyang Chen, Zhuo Chen, Shujie Liu, Jian Wu, Yao Qian, Furu Wei, Jinyu Li. and Xiangzhan Yu. Unispeech-see Universal speech representation learning with speaker aware pre-training, 2021b.
10

---

Under review as a conference paper at ICLR 2022
Moustapha M Cisse, Yossi Adi, Natalia Neverova, and Joseph Keshet. Houdini: Fool- ing deep structured visual and speech recognition models with adversarial examples. In I. Guyon, U. V. Luxburg. S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Gar- nett (eds.), Advances in Neural Information Processing Systems, volume 30. Curran Asso- ciates, Inc., 2017. URL htttp::::ppppeeeeeeeteeeee ngs. neur porrips.... cc//paper/2117/f1e. Paper pdf
Jeremy M. Cohen, Elan Rosenfeld, and J. Zico Kolter. Certified adversarial robustness via random- ized smoothing. CoRR, 2019.
Dario Amodei al. Deep speech End-to-end speech recognition in english and mandarin. In Maria Florina Balcan and Kilian Q. Weinberger (eds.), Proceedings of The 33rd International Conference on Machine Learning, volume 48, pp. 173-182, New York, New York, USA, 20-22 Jun 2016. PMLR.
Thomas Wolf et al. Transformers: State-of-the-arr natural language processing. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demon- strations, pp. 38 45, Online, October 2020. Association for Computational Linguistics. URL https: www. ac wwwweb.wbb.......... www.oo/aatoooogo///ooooooo eep
lan Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples, 2014.
Alex Graves, Santiago Fernández, Faustino Gomez, and Jürgen Schmidhuber. Connectionist Tem-
poral Classification: Labelling Unsegmented Sequence Data with Recurrent Neural Networks. In Proceedings of the 23rd International Conference on Machine Learning. ICML '06, pp. 369-376, New York, NY, USA. 2006. ACM. ISBN 1-59593-383-2. doi: 10.1145/1 143844.1144899999 URL http: /x. doi. wwt.or................. /ttp.//.......111111111111111111
Wei-Ning Hsu, Benjamin Bolte, Yao-Hung Tsai, Kushal Lakhotia, Ruslan Salakhutdinov, and Ab- deirahman Mohamed. Hubert: Self-supervised speech representation learning by masked predic- tion of hidden units. IEEE/ACM Transactions Audio, Speech, and Language Processing, PP: 1-1, 10 2021. doi: 10. 0D009/TAASL.P2021.11222999999999
Andrew Hyas, Shibani Santurkar, Dimitris Tsipras, Logan Engstrom, Brandon Tran, and Aleksander Madry. Adversarial examples are not bugs, they are features. In NeurlPS, 2019.
Quentin Lhoest, Albert Villanova del Moral, Yacine Jernite, Abhishek Thakur, Patrick Platen, Suraj Patil, Julien Chaumond, Mariama Drame, Julien Plu, Lewis Tunstall, Joe Davison, Mario Šaško, Gunjan Chhablani, Bhavitvya Malik, Simon Brandeis, Teven Le Scao, Victor Sanh, Can- wen Xu, Nicolas Patry, Angelina McMillan-Major, Philipp Schmid, Sylvain Gugger, Clément Delangue, Théo Matussière, Lysandre Debut, Stas Bekman, Pierric Cistac, Thibault Goehringer, Victor Mustar, François Lagunas, Alexander Rush, and Thomas Wolf. Datasets: community library for natural language processing. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pp. 175-184, Online and Punta Cana, Dominican Republic, November 2021. Association for Computational Linguistics. 2021 wweeemmlle-.mmm emnlp-demo. 21
URL httpss//accaaniooooo.ocoo
Juncheng Li, Shuhui Qu, Xinjian Li, Joseph Szurley, J. Zico Kolter, and Florian Metze. Adversarial music: Real world audio adversary against wake-word detection system. In H. Wallach, H. Larochelle, A. Beygelzimer, F. d'Alché-Buc, E. Fox, and R. Garnett (eds.), Advances in Neural Information Processing Systems, volume 32. Curran Asso- ciates, Inc., 2019. URL https: httpppps...//aarr/0////122
Zhiyun Lu, Wei Han, Yu Zhang, and Liangliang Cao. Exploring Targeted Universal Adversarial Perturbations to End-to-Enm ASR Models. In Proc. Interspeech 2021, pp. 3460-3464, 2021. doi: 10.21 10.214477/ntersppecch http://eeersppeeccc.211216. 2021-1668.
Loren Lugosch, Tatiana Likhomanenko, Gabriel Synnaeve, and Ronan Collobert. labeling for massively multilingual speech recognition. ICASSP, 2022.
11

---

Under review as a conference paper at ICLR 2022
Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. In ICLR 2018, Conference Track Proceedings, 2018.
Raphael Olivier and Bhiksha Raj. Sequential randomized smoothing for adversarially robust speech recogmition. In Proceedings of the 2021 Conference on Empirical Methods Natural Lan- guage Processing, pp. 6372-6386, Online and Punta Cana, Dominican Republic, November 2021. Association for Computational Linguistics. doi: 10. 18.13/8311222222 1001://emeee..mmmmimma.... URL https::/aacllnnthoooooo..o. wtt.logg..orgg211111/1// emmpp-mammmpmmmmmaammm 514
Raphael Olivier and Bhiksha Raj. Recent improvements of asr models in the face of adversarial 16536
attacks. Interspeech, 2022.
Shmuel Onn and Ishay Weissman. Generating uniform random vectors over sim- plex with implications to the volume of a certain polytope and to multivariate ex- tremes. Annals of Operations Research, 189(1):331-342, September 2011. doi: 10. 10011077/110479-0099-66677 v189y2011i1p321 000.://31111111000--0000000
URL https: /tttss:/iidas.
wrappe..ogg/aaa. http:////pp///opprr 009 0567 15567-7.0 7.html
Myle Ott, Sergey Edunov, Alexei Baevski, Angela Fan, Sam Gross, Nathan Ng, David Grangier, and Michael Auli. fairseq: A fast, extensible toolkit for sequence modeling. In Proceedings of NAACL-HLT 2019: Demonstrations, 2019.
Nicolas Papernot, Patrick D. McDaniel, and Ian Goodfellow. Transferability in machine learning: from phenomena to black-box attacks using adversarial samples. CoRR, 2016.
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury. Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Kopf, Edward Yang. Zachary DeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy. Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. Pytorch: An imperative style, high-performance deep learning library. In H. Wallach, H. Larochelle, A. Beygelzimer, F. d'Alché-Buc, E. Fox, and R. Garnett (eds.), Advances in Neural Information Processing Systems 32, pp. 8024 8035. Curran Associates, Inc., 2019. URL
pdf
Yao Qin, Nicholas Carlini, Garrison Cottrell, Ian Goodfellow, and Colin Raffel. Imperceptible, ro- bust, and targeted adversarial examples for automatic speech recognition. Kamalika Chaudhuri and Ruslan Salakhutdinov (eds.), Proceedings of the 36th International Conference on Machine Learning, volume 97 of Proceedings of Machine Learning Research, pp. 5231-52400 PMLR, 09-15 Jun 2019. URL
.mlr. www..pprees/////gg.....
html
Mirco Ravanelli, Titouan Parcollet, Peter Plantinga, Aku Rouhe, Samuele Cornell, Loren Lugosch, Cem Subakan, Nauman Dawalatabad, Abdelwahab Heba, Jianyuan Zhong.Ju-Chieh
Ju-Chieh Chou, Sung- Lin Yeh, Szu- Wei Fu, Chien-Feng Liao, Elena Rastorgueva, François Grondin, William Aris, Hwidong Na, Yan Gao, Renato De Mori, and Yoshua Bengio. SpeechBrain: general-purpose speech toolkit, 2021. arXiv:2106.0466
Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus. Intriguing properties of neural networks. In ICLR, 2014. URL http:// arxiv.org/abes. www.orgg/bbbss...o 6199
Florian Tramer, Nicholas Carlini, Wieland Brendel, and Aleksander Madry. On adaptive attacks to adversarial example defenses. In H. Larochelle, M. Ranzato, R. Hadsell, M. F. Balcan, and H. Lin (eds.), Advances in Neural Information Processing Systems, volume 33, pp. 1633-1645, Curran Associates, Inc., 2020.
Changhan Wang, Yun Tang, Xutai Ma, Anne Wu, Dmytro Okhonko, and Juan Pino. fairseq s2t: Fast speech-to-tex modeling with fairseq. In Proceedings of the 2020 Conference of the Asian Chapter of the Association for Computational Linguistics (AACL): System Demonstrations, 2020.
12

---

Under review as a conference paper at ICLR 2022
Chengyi Wang, Yu Wu, Yao Qian, Kenichi Kumatani, Shujie Liu, Furu Wei, Michael Zeng, and Xuedong Huang. Unispeech: Unified speech representation learning with labeled and unlabeled data. In Marina Meila and Tong Zhang (eds.), Proceedings of the 38th International Conference on Machine Learning, ICMI. 2021, 18-24 July 2021, Virtual Event, volume 139 of Proceedings of Machine Learning Research, pp. 10937-10947 PMLR, 2021. htl.htttp//proceeeete
wwwppppsssss.//eee wwtp:////wwg......... html
Haibin Wu, Bo Zheng, Xu Li, Xixin Wu, Hung- Yi Lee, and Helen Meng. Characterizing the adversarial vulnerability of speech self-supervised learning. IEEE International Confer- ence Acoustics, Speech and Signal Processing, ICASSP 2022, Virtual and Singapore, 23- 27 May 2022. pp. 3164-3168. IEEE, 2022. doi: (0.10001AASS4422022222122222 https://doi.orr
URL
21109/CAASP4332228 2022 9747242
Xuejing Yuan, Yuxuan Chen, Yue Zhao, Yunhui Long, Xiaokang Liu, Kai Chen, Shengzhi Zhang, Heqing Huang, XiaoFeng Wang, and Carl A. Gunter. Commandersong: A systematic approach for practical adversarial voice recognition. In 27th USENIX Security Symposium (USENIX Security 18), pp. 49-64, Baltimore, MD, August 2018. USENIX Association. ISBN 978-1- 939133-04-5. URL
usenix http://wwwwaaaa...ooooocc//aa/aa presentation/yue Sttpe/iooonuuaa-uueeeemee
13

---

Under review as a conference paper at ICLR 2022
A EXPERIMENTAL DETAILS FOR LIBRISPEECH EXPERIMENTS
A. FRAMEWORKS
We compute adversarial examples using the robust speech framework (Olivier Raj. 2022). This library uses Speechbrain (Ravanelli et al. (022) to load and train ASR models and offers implemen- tations of various adversarial attack algorithms. Models and attacks are implemented using Py Torch Paszke et al. 2019).
We use robust_speech for evaluation on Bereehhraai-supporreee models. section 3 we export HuggingFace Dataset (Lhoest al.) 2021). then evaluate models via the HuggingFace Transformers (ct al. 2020) library. Finally, we use Fairseq (Ott et al. 2019) for training models from scratch
All of our robustsspecch and Fairseq configurations released alongside this article.
A.2 ATTACK HYPERPARAMETERS
We exploit the following hyperparameters:
attack (see section 2.2 implemented in robust speech, with the
initial 0.015 (and 0.04 in appendix D.I
learning rate: 0005
number of decreasing values:
Regularization constant c: 10
optimizer: SGD
attack iterations: 10000 section 3.1 1000 in section
A.3 DATASET AND TARGETS
Our adversarial dataset section consists of 85 sentences from the LibriSpeech test-clean set. To extract these sentences we take the first 200 sentences in the manifest, then keep only those shorter than seconds. In section we take the first 100 sentences and filter those shorter than 14 seconds.
As attack targets, we use actual LibriSpeech sentences sampled from the test-other set. Our candi- date targets are:
Let me see how can begin
Now go can't keep my eyes open
So you are not a grave digger then
He had hardly the strength to stammer
What can this mean she said herself
Not years for she's only five and twenty
What does not
undergo for the sake of cure It is easy enough with the child you will carry her out
Poor little man said the lady you miss your mother don't 't you
At last the little lieutenant could bear the anxiety no longer
Take the meat of one large crab scraping out all of the fat from the shell
Tis strange change and am very sorry for but I'll swear know not how to help
The bourgeois did not care much about being buried in the Vaugirard hinted poverty pere Lachaise if you please
For each sentence we attack, we assign the candidate target with the closest length to the sentence's
original target.
14

---

Under review as a conference paper at ICLR 2022
A.4 MODELS
A.4.1 TRAINING WAV2VEC2 MODELS FROM SCRATCH
We use Fairseq to train Base and Large Wav2Vec2 models from scratch. Unfortunately, no configu- ration or pretrained weights have been released for that purpose, and we resort using Wav2Vec2 fine-tuning configurations while simply skipping the pretraining step. Despite our attempts to tune training hyperparameters, we not match the expected performance of Wav2Vec2 model trained from scratch: (Baevski et al. 2020) report WER of 3.0% for large model, while only get 9.1%.
A.4.2 GENERATING ADVERSARIAL EXAMPLES
Wev2 HuBERT and Data2 models are all supported directly robust_speech and are there- fore those we use for generating adversarial examples. We use the HuggingFace backend of Speech- brain for most pretrained models, and its Fairseq backend for a few (Wav2Vec2-BB) models fine- tuned on 10h and Ih, and models trained from scratch). In both cases, the model's original tokenizer cannot be loaded in SpeechBrain directly. Therefore, we fine-tune the final projection layer each model on 1h of LibriSpeech train-clean data.
The Wav2Vec2 model pretrained and fine-tuned on Common Voice a SpeechBrain original model. Similarly, we fine-tune it on 1h of LibriSpeech data a shift from the Common Voice output space to the LibriSpeech one. As a result, all our models share the same character output space.
A.4.3 EVALUATING PRETRAINED MODELS
In section 3 we directly evaluate models from HuggingFace Transformers and SpeechBrain on our adversarial dataset, without modification.
B
EXPERIMENTAL DETAILS AND RESULTS FOR SMALL-SCALE EXPERIMENTS
This section describes the experimental details used in section 5
B. CIFAR10 EXPERIMENTS
We use a pretrained ResNetl as proxy, and a pretrained ResNet50 as private model.
Our Veery targeted attack" PGD₃ consists in applying the following steps for each input:
target selection. We sample uniformly an ordered subset of k classes out of 10 (F.g- with = 3: (2,5,6) We also sample a point uniformly on the unit k-simplex 1}. by sampling from exponential distribution and normalizing OOnn & Weissman 2011 (e.g. (0.17, 0.55, 0.28)). We combine the two to all but the selected i classes (y- = (0, 0.1...1.... O, O, 0.55, 0.28, 0, O, O)). This is our target.
{æ1. I& E 10.
obtain 10-dimensional vectors with zero probability
During the attack, we use Projected Gradient Descent (Madry et al. 2018) to minimize the KL divergence (x)(xx) v) between the softmax output and the target, within La radius = 0.5. We use learning rate 0.1 for l * 1000 attack steps.
We measure attack success rate by measuring the top-k match between f(x) and y:
1 (f)) = argsort(y):
acc
k
with argsort(y) returning the indices of the sorted elements of in decreasing order. For instance f(x) 10.1, 0.05, 0.05, 0.05, 0.35, 0.2, 0.05, 0.05, 0.05, 0.05) would get an accu- racy of 0.666, as the top 2 classes match with y but not the third.
We evaluate attacks on 256 random images from the CIFAR1O dataset. For each value of & between 1 and 10 repeat the experiment 3 times and average the attack success rates. In figure 2 we plot
15

---

Under review as a conference paper at ICLR 2022
Attack success (source) Attack success (target)
random threshold Standard transferability rate
10
Accuracy (%)
75
50
25
0
2
4
8
10
Figure 2: Transferred attack success rate when varying the "target precision" CIFAR10 model. The more targeted the attack, the worse its transferability at an equal white-box success rate
the Loo attack success rate (e = 0.03) for both white-box and transferred attacks as a function the "target precision" k.
B.2 MILDLY TARGETED ASR ATTACKS
We train 5 identical conformer encoder models with 8 encoder layers, attention heads, and hidden dimension 144. We train them with CTC loss for 30 epochs on the LibriSpeech train-clean-100 set, with different random seeds.
We run a P2D attack with SNR bound 30dB, in which we minimize the cross-entropy loss between the utterance and its transcription prepended with the word "But". The utterances we attack are the first 100 sentences the LibríSpeech test-clean set, which remove sentences already starting with the word "But". We generate adversarial examples using each of the models as proxy, and evaluate these examples on all models. We report the full results Table 5
Model Proxy
2
4
98%
14%
12%
13%
19%
2
23%
100%
11%
14%
24%
18%
22%
96%
20%
16%
4
12%
19%
20%
100%
17%
28%
14%
22%
22%
96%
Table Success rate of our mildly targeted attack, using each of the conformer networks both as proxy and model. The attack is considered successful on an input if the prepended target word is the first word in the transcription.
C FULL RESULTS TABLE FOR CROSS-MODEL ATTACKS
Table completes ablation study Section by evaluating all pairwise Proxy-Model combina- tions in our pool of Wav2 models.
D INFLUENCE HYPERPARAMETER ON ATTACK RESULTS
D.1 ATTACK RADIUS
In Table 7 we extend the results of Table by comparing attack results for two different attack radii. These radii 0.015 and 0.04, corresponding respectively to Signal-Noise Ratios of
16

---

Under review as a conference paper at ICLR 2022
Model Proxy
W2V-
W2V-B
W2V-B
W2V-B
D2V-B
LS960
960h
LS960
100h
LS960
10h
LS960 1h
LS960
960h
CER
WER
CER
WER
CER
WER
CER
WER
CER
WER
W2V-B LS960 960h
96.37
84.72
80.61
49.01
64.11
30.69
53.41
18.46
20.61
0
W2V-B LS960 100h
81.42
54.46
99.24
97.74
81.18
47.6
64.18
25.04
25.23
0
W2V-B LS960 10h
42.64
42.64
87.9
60.25
99.117
97.6
72.91
30.13
23.47
0
W2V-B LS960 1h
69.3
33.52
78.84
43.71
81.12
45.12
99,498
98.66
20.91
0
D2V-B LS960 960h
37.9
0
17.68
0
10.88
0
7.94
98.44
94.13
W2V-L LS960 960h
44.61
11
20.36
0
13.46
0
8.32
16.8
0
D2V-L LS960 960h
28.72
0
8.68
O
5.36
O
4.94
25.03
0
W2V-L LV60k 960h
29.24
O
11.12
0
5.68
O
3.19
13.73
0
HB-L LV60k 960h
23.83
0
7.29
0
4.83
3,91
14.92
HB-XL LV60k 960h
26.55
0
6.71
0
5.21
0
4.37
17.53
0
W2V-L CV CV+1h
27.38
0
12.59
0
11.01
9.61
19.24
W2V-B None 960h
7.84
0
4.45
0
4.05
0
3.83
5.51
0
W2V-L None 960h
8.12
0
4.63
0
4.55
0
3.44
5.44
W2V- L
D2V-L
W2V-1
HB-L
HB-XL
LS960
960h
LS960
960h
LV60k
960h
LV60k
960h
LV60k
960h
CER
WER
CER
WER
CER
WER
CER
WER
CER
WER
W2V-B LS960 960h
47.08
8.49
24.9
0
44.7
9.48
55.55
19.17
47.46
8.98
W2V-B LS960 100h
46.01
5.73
26.77
0
48.57
9.76
58.41
18.03
48.42
8.13
W2V-B LS960 10h
43.14
0
25.1
0
42.67
0
53.12
5.59
44.36
0
W2V-B LS960 Ih
41.21
8.63
25.48
0.57
36.68
4.74
45.32
6.65
42.95
10.18
D2V-B LS960 960h
34.49
0
24.2
0
47.15
0.92
58.75
14.29
46.71
0.14
W2V-L LS960 960h
67.07
30.69
20.89
0
37.34
1.84
56.87
19.02
42.21
5.87
D2V-L LS960 960h
28.27
0
94.53
80.69
47.75
15.21
68.97
38.61
51.02
18.6
W2V-I IN60k 960h
25.19
0
16.05
0
97.13
88.61
71.78
39.18
46.61
11.88
HB-L LV60k 960h
27.19
0
30.08
0
49.27
17.47
97
87.98
56.83
28.71
HB-XL LV60k 960h
33.31
0
30.5
0
51.68
14.99
83.92
55.3
87.66
62.38
W2V-L CV CV+1h
27.8
0
26.85
0
56.72
11.67
46.94
39.95
0
W2V-B None 960h
11.19
0
9.6
0
7.16
0
6.72
11.07
0
W2V-L None 960h
11.15
0
9.19
7.52
0
7.45
11.23
W2V- L
W2V-B
W2V-L
CV+1h
None 960h
None 960h
CER
WER
CER
WER
CER
WER
W2V-B LS960 960h
10.81
0
2.62
2.53
W2V-B LS960 100h
11.01
0
2.82
0
2.58
0
W2V-B LS960 10h
11.19
0
2.65
0
2.66
0
W2V-B LS960 Ih
11.81
O
3.03
3,04
D2V-B LS960 960h
8.01
0
2.32
0
2.38
W2V-L LS960 960h
8.2
2.39
2.54
D2V-L LS960 960h
8.76
0
2.44
0
2.39
W2V-L LV60k 960h
9.08
2.59
2.47
HB-L. LV600 960h
8.65
0
2.5
2.55
HB-XL LV60k 960h
8.41
0
2.49
2.36
W2V-L CV CV+1h
97.46
88.68
3,25
0
3.18
W2V-B None 960h
5.77
0
99.57
99.01
19.05
None 960h
5.53
0
22.93
0
99.93
99.58
Table 6: Targeted Character-level and Word-level success rate for adversarial attacks when varying the proxy and the target model. All proxy-model pairs are evaluated within pool of models varying in training scheme, training data and size. The format is [Model]-[Size) [Unlabeled data] [Labeled data]. Model is equal W2V (Wav2Vec2), D2V (Data2Vec) or HB (HuBERT). Size is equal to B (Base), L (Large) or XL (XLarge).
30dB and 22dB respectively. The former is identical to Table 7 the latter is substantially larger, and corresponds to more easily perceptible noise.
17

---

Under review as a conference paper at ICLR 2022
Looking the white-box attack results the proxy models the difference is drastic: with larger noise the targeted success rate jumps from 88% to 98%. The transferred attack results on SSL- pretrained models also increase overall, with success increases ranging from 0% (Wav2Vec2-Largee to 20% (Data2Vec-Larree with a median increase of 10%. Crucially however, the targeted success does not increase all and even decreases for ASR models trained from scratch. This confirms that there is a structural difference between the robustness of ASR models with and without SSL, that cannot be bridged simply by increasing the attack strength.
Model
Unlabeled data
Labeled data
Attack SNR
Attack success
(word level)
targeted
untargeted
Wav2Vec2-Large
LV60k
LS960
30dB
88. 88.0%
100%
22dB
98.4%
100%
HuBERT-Large
LV60k
LS960
30dB
87.2%
100%
22dB
98.5%
100%
Data2Vec-Base
I.9600
LS960
30dB
63.4%
100%
22dB
92%
100%
Wav2Ve22-Base
LS960
LS960
30dB
55.7%
100%
22dB
62.9%
100%
Wav2Vec2-Base
LS960
LS100
30dB
53.9%
100%
22dB
59.5%
100%
Wav2Vec2-Large
LS960
LS960
30dB
50.7%
100%
22dB
49.4%
100%
Data2Vec-LLare
ISS900
LS960
30dB
66%
100%
22dB
86.4%
100%
HoBERT-XLarge
LV60k
LS960
30dB
80.9%
100%
22dB
95.5%
100%
UniSpeech-Sas Saapbecc-Sat-Base
LS960
LS100
30BB
50.4%
100%
22dB
62.4%
100%
WavLM-Base
LV60k+
LS100
30
21.7%
100%
22dB
22.9%
100%
Wav2Vec2-Large
CV
CV+LSI
30dB
19.7%
100%
22
36.1%
100%
M-CTC-Large
None
CV
30dB
7.5%
76.4%
22dB
3.5%
83.4%
Speech2Text
None
LS960
30dB
7.3%
63.3%
22dB
2.3%
74.6%
SB CRDNN
None
LS960
30dB
5.9%
86.39%
22dB
1.5%
76.8%
SB Transformer
None
LS960
30dB
6.49%
90.56%
22dB
1.2%
76.1%
Table 7: Results of the transferred adversarial attack different ASR models, with multiple Signal- Noise Ratios. The first three models correspond to the proxies used to generate the adversarial examples. On all other models, the inputs have been transferred directly. We report for each model how much unlabeled data was used for SSL pretraining and for ASR finetuning. We also report its Word-Eree
on the LibriSpeech test-clean set, and the targeted and untargeted word-level attack success rate (see section 3.3
D.2 LANGUAGE MODELS
we report the results of our adversarial dataset on multiple Wav2Vec2-type models, enhanced with N-gram language model whenever available. In Table 8 we evaluate the influence of that language model on attack results.
In section 3
We observe that the attack success rate systematically increases by to 17% when adding language model to the ASR model. This understandable considering that our targets are sound English sentences: if model tends transcribe that target with mistakes, the language model can bridge that
18

---

Under review as a conference paper at ICLR 2022
Model
Unlabeled Labeled data
Clean WER
Attack success rafe
data
(word level)
w/o LM
with LM
w/o LM
with LM
Wav2Vec2-Large
LV60k
LS960
2.2%
2.0%
80.2%
88.0%
HuBERT-Large
LV60k
LS960
2.1%
1.9%
77.3%
87.2%
Data2 Vec-Baee
LS960
LS960
3.2%
2.5%
51.7%
63,4%
Wav2Vec2-Base
LS960
LS960
3.4%
2.6%
43.6%
55.7%
Wav2We22-Base
LS960
LS100
6.2%
3.4%
41.8%
53.9%
Wav2Vec2-Large
LS960
LS960
2.8%
2.3%
41.4%
50.7%
LataeVec-Large
LS960
LS960
2.2%
1.9%
56.9%
66%
Lure
LV60k
I.$960
2.0%
1.8%
63.9%
80.9%
UniSpeech-Sat-Base ISS90
LS100
6.4%
3.5%
39.5%
50.4%
Table 8: Results of the transferred adversarial attack on different ASR models, with and without language models. We report for each model how much unlabeled data used for SSL pretraining and for ASR finetuning. We also report its Word-Error-Raue on the LibriSpeech test-clean set, and the targeted word-level attack success rate (see section 3.3
Prexy (Wav2Vec2-aarge Wav2Vec2 Base
Proxy (Wav2Vec2-Large) Wav2Vec2 Base
Wav2Vec2 Base from scratch
Wav2Vee2 Base from scratch
10
0
1255
1200
2000
2000
4000
Iterations
Iterations
(a) With dropout
(b) Without dropout
Figure Evolution over attack steps of the loss on one adversarial input for three models: the Wav2 Large proxy and two targets, respectively with and without SSL pretraining. We run attacks (a) with dropout in the proxy model, and (b) without dropout the proxy model.
gap. To put differently, the more prone an ASR model is to output sentences in given distribution, the more vulnerable it attacks with targets sampled from that distribution. Language models are therefore more of a liability than a defense against attacks, and most likely so would be many tricks applied to ASR model in order to improve its general performance.
D.3 EFFECT OF MODEL REGULARIZATION ON TRANSFERABILITY
As mentioned in Section 2.2 we use regularization tricks like dropout in all proxy models when optimizing the adversarial perturbation. In Figure 3b we plot the loss on proxy and private mod- els without that regularization, for comparison with Figure 3a We observe that the loss degrades significantly on private models without regularization.
On the other hand, the loss on the proxy converges much faster in Figure 3b removing model regularization makes for better, faster white-box attacks, at the cost extent of our knowledge, past work like Carlini & Wagner (2018) have not used regularization for generation, explaining why they report better white-box attacks than we do terms of WER and SNR. However, have established above, applying regularization against standard ASR models does not lead to transferable adversarial examples: for that SSL pretraining is also required.
transfcrability. To the
19

---

Under review as a conference paper at ICLR 2022
E DEFENDING AGAINST ADVERSARIAL EXAMPLES
Although we have shown that adversarial attacks can represent an important threat for private, SSL- based ASR models, it possible defend against them. Randomized smoothing Cohen
(2019)
is a popular adversarial defense that has been applied to ASR in the past Olivier & Raj (202) I and
comes with some robustness guarantees. It consists in applying to the inputs, before feeding them to the model, amounts of random gaussian noise that are significantly larger than potential adversarial perturbations in L2 norm. For reference we try applying on some of models.
We follow (Olivier & Raj 2021) and enhance randomized smoothing with a-priori SNR estimation and ROVER voting (with outputs) to boost performance. We use gaussian deviation a 0.02. For evaluation, we simply check the effect of our adversarial examples generated in section 3.1 on the smoothed model. rigorous evaluation would require us to design adaptive attacks (Athalye et al. 2018) Tramer et al. (2020); since this paper does not focus claiming robustness to attacks, we restrict ourselves to a simpler setting.
We report our results in Table for the Wav2 Base, Wav2Vec2-Large and Data2 models, pretrained and fine-tuned on 960h of LibriSpeech training data. We observe that random- ized smoothing sufficient to block the targeted attack completely (0% success rate) and recover most of the original transcription (the untargeted success rate drops to 14-34% depending on the model). However, due the addition of gaussian noise all inputs the defense takes a toll on the performance on clean data: the WER jumps by 4-10%. The standard deviation controls this tradeoff between robustness and performance; we chose the value of that minimizes the untargeted success rate.
Unsurprisingly, randomized smoothing is a promising protection against transferred attacks, but it does leave room for improvement. These results illustrate the need for additional research on adversarial defenses.
Clean WER
Model
Smoothing
Attack success rate
targeted
untargeted
Wav2vec2-Bass
No
3.4%
55.7%
100%
Wav2vec2-Base
Yes
13.5%
0%%
33.9%
Way2vec2-Large
No
2.2%
50.7%
100%
Way2vec2-Large
Yes
7.3%
0%
19.5%
Data2Vec-Large
No 2.2%
66%
100%
Vec-aaee
Yes 6.7%
0%
14.1%
Table 9: Results of the transferred adversarial attack (generated in section 3.1 on the Wav2Vec2. Base, Wav2 Vec2-aece and Data2Vec-Lane models. Each model was pretrained and fine-tuned on 960h of LibriSpeech training data. We report results on both the undefended version each model and one defended with randomized smoothing the LibriSpeech test-clean set, and the word-level success rate of the attack (see Section 3.3)
0.02. We report the WER of each model
20


Reviews:
[{"review_id": "hA-SS2sgLKU", "reviewer": "Reviewer_fvn3", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper studies transferable targeted adversarial attack on self-supervised ASR models. Target attack adds a small perturbation to a model such that the model makes the targeted prediction desired by the attacker instead of the correct one corresponding to the original input. Transferability refers to generalizing the attack to private models which have not been used to generate the adversarial perturbation.\n\nPast studies show that targeted attack is hard to generalize for supervised ASR models. However, in contrast to previous findings, the authors demonstrate that such attack can in fact generalize to ASR models pre-trained with self-supervised learning using similar datasets. The authors then present a series of study to understand what leads to successful attack transfer, including self-supervised objective, models size, training data size, and specificity of the attack.\n", "summary_of_the_review": "While the paper can be improved as mentioned in the sections above, the current presentation already provides substance that will be useful and interesting to the community.", "strength_and_weaknesses": "- Strengths\n  - The authors presented an interesting study with new observations contrary to previous studies of adversarial attacks on ASR models. This could open up a new direction of empirical and theoretical study to understand robustness in self-supervised setups\n  - Many factors are studied to reason and hypothesize why self-supervised ASR models are more vulnerable to targeted adversarial attack, providing valuable data points for future work.\n  - Attacks of different levels of specificity have also been considered to support the hypothesis.\n  - A more generalizable scheme for optimizing the adversarial attack is proposed in this paper, which uses multiple proxy models and validates/selects checkpoints with a different model.\n\n- Weaknesses\n  - While the authors have compared several different SSL models (wav2vec, hubert, data2vec, WavLM, UniSpeech), they all have very similar architectures (convolution encoder for waveform followed by transformer layers). The argument of “attack on SSL models are transferable when they are pre-trained on the same dataset” would be more convincing if the authors had considered another SSL that have a different model architecture (e.g., LSTM).\n  - The experiments studying the effect of pre-training data could have been expanded. The authors showed in Table 1 that target attack performance is worse on W2V2-Large (CV), which is an interesting observation. The study will more complete if the authors include a) optimizing attack on CV and transfer to LV; b) optimizing attack on a model pre-trained on more datasets (CV+LV+Fisher checkpoint is available on Github) and transfer to one pre-trained on a subset of it; c) the reverse of b.\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "aG3EAePs8GZ", "reviewer": "Reviewer_xki8", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper investigate the transferability property of adversarial attacks on ASR models. They evaluate the robustness of modern transformer-based ASR architectures such as Wav2Vec-2.0 etc. With a series of experiments on a set of ASR models by using a set of 85 adversarial samples, they show that many state-of-the-art ASR models are in fact vulnerable to the transferability property. Additionally, they also claim to show that SSL-pretraining is the reason for this vulnerability to transferability. \n\n", "summary_of_the_review": "Overall the paper targets an interesting problem. At the beginning the paper sounded very interesting based the claims but as I went through the paper I could not understand some experiments or reasoning behind some experiment criteria. Therefore, I won’t feel confident to vote for acceptance of this paper.  ", "strength_and_weaknesses": "\nThe approach for generating adversarial examples is interesting but it is not explained very well in section 3.1 as the details are missing such as: \nWere only 85 samples used during fine-tuning the models?\nWhy 85 and how were these selected? \nIt is not clear how the third model (Data2Vec BASE) was used as stopping criteria? \nResults are presented in section 3.4, but before that I have not seen how the adversarial examples are generated. \n\n\n\nIn section 4, why character error rate (CER) was used instead of WER? CER might not be indicating the significant impact on WER as these could have been fixed by LM duding decoding (if there was one added). I guess CER was not used in the previous section due to the same reason. So it would be good to see these results in terms of WER. \nThis section in particular was difficult to understand. There should be some more details added for clarity. For example the following sentence is not at all clear: \"we observe that the Character Error Rate between two random sentences is about 80-85% on average. Therefore attack success rates higher than 20% indicate a partially successful attack”. How is CER computed between two samples? Is it not between the reference and hypothesis? How 20% is selected as indicator for successful attack?\n\n\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "KBNbMqTy6J_", "reviewer": "Reviewer_5s8w", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper proposes targeted and transferable adversarial examples for self-supervised asr models which are in pretraining + fine-tuning architecture. An adversary can make use of the transferability property, that is, an adversarial sample produced for a proxy asr can also fool a different remote asr.\nRich self-supervised asr models, such as wav2vec2, Hubert, data2vec and wavlm are investigated and similar results are shown with detailed experiments and comparisons.\n", "summary_of_the_review": "Generally, the paper is well written with a good motivation and rich experiments on detailed evidences. One issue is that the investigation of the datasets used for training these sota ssl models and also among different languages. \nIf with one solution to dealing with the adversarial examples, this paper can be ranked higher.\n", "strength_and_weaknesses": "Strong:\n\n1 The targeted and transferable adversarial examples are interesting and should benefit the research ad developing of better and more robust asr systems of pretraining+fine-tuning architecture.\n\n2 Rich existing sota models are investigated showing the “transferability” is a frequent thing.\n\n3 Code is attached making the paper to be with high score of reproducing ability. \n\nWeak:\n\n1 Prefer to see richer experiments on language models using quite different datasets of among different languages to better learn the transferability among pretrained speech representations;\n\n2 If solutions to these well-designed adversarial examples can be provided, this paper will be rich of novel solutions as well.\n\n\n\nDetailed questions and comments\n\n1 So, how to improve current self-supervised models to better dealing with the transferable adversarial examples? This is more valuable for building robust asr systems.\n\n2 Can your asr attacking strategies influencing other people’s normal usage of existing asr systems? Say, you prepared a special group of inputs and all asr systems failed – then does this influence other people’s usage or if same types of attacking data were used for training these models, then they will fail forever if not fixed. Basing on my experience, most asr systems are still quite fragile – they fail a lot even with quite clean and clear voice inputs and they will for sure fail if the inputs are further including rich carefully designed noises.\n\n3 Any further comparison of the transferability of from one language’s pretrained model to another language’s pretrained model? Say how large the datasets used are involved in the transferability?\n\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "2V9AfSDvvP", "reviewer": "Reviewer_6Mf8", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper shows that recent self-supervised ASR model are uniquely vulnerable to black-box adversarial attacks, which is an interesting observation and can inspire future research in this field. ", "summary_of_the_review": "The paper observes an interesting result. However, the explanation to this phenomenon is unclear. It seems that it is possible that the SSL based model are vulnerable than non SSL-based model mainly because the attack model is trained to fool a SSL model.", "strength_and_weaknesses": "Strength:\n1. The paper shows that recent self-supervised ASR model are uniquely vulnerable to black-box adversarial attacks.\n2. The paper conducts an ablation study to show that self-Supervised learning is the main cause of that phenomenon.\n3. The paper provides an explanation for this phenomenon.\n\nWeakness:\n\nThe experiment settings in this paper are inconsistent and confusing. I summarize it with the following question. Although I rate a relative low score for this paper, I am willing to change my score if the issues are addressed.\n\n1. In Section 3.4, the results show that SSL-based model is vulnerable to black-box adversarial attacks. However, the attack model is also trained to fool a SSL based models. What if the attack model is trained to fool a non-SSL based model such as the bottom three models in Table 1?\n2. The experiments in Section 3 use an improved version of that mention in Section 2.2. However, in Section 4 only the default version of Section 2.2 is utilized. What is the motivation for this inconsistency?\n3. (A question similar to the first question) In Section 4.1, it seems that the attack model is also trained to fool a wav2vec2-based model. In this way, it is natural that the wav2vec2-based model is more likely to be fooled compared with the non-wav2vec2 model.\n4. In Section 5.1 and 5.2, the conclusion is that a hard targeted attack is hard to achieve compared with a mildly targeted attack, which is an interesting but natural result. However, the intuition of why this result can explain the vulnerable SSL-based ASR model is not discussed in detail.\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works.", "empirical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works."}, "extra_text": {}}]

Author rebuttal:

Decision: Reject