Venue: ICLR 2022 Conference

Reviews:

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: This paper proposes a novel attack approach with a purpose of disrupting the automatic speech recognition system. The proposed method, called Neural Voice Camouflage, works in real time by forecasting attacks ahead of time when they are added to speech streams. The authors conducted experiments with the LibriSpeech dataset, and showed that the proposed model outperforms the conventional methods with or without defense mechanisms on the task of speech recognition (performance measured by WER/CER).
main_review: Strengths:
- the paper is well structured and easy to follow
- clearly explains how the proposed method works
- evaluation framework is well designed and straight forward
- experiments are solid and the results support the proposed method is working
- in-depth analyses on the results: 
I really enjoyed the analysis shown in Figure 7

Weaknesses:
- some arguments are not validated:
In Section 4.4, the authors provide in-depth analyses and discussions on the attack characteristics. The first was whether the proposed model attacks vocal timbres, and concludes it does and attacks are speaker-dependent by stating that the attack performance drops - i.e., both WER and CER drop - when swapping attacks for speakers. However, it may not be true unless the experiments were carefully conducted with speech samples where different speakers say the same content. It would be interesting to see the results of voice-converted samples.

- some observations are not scientifically grounded:
In Figure 3 and 4, the authors repeatedly state that the attacks resemble speech "formants" but I don't see any formant-like structures in the spectrograms. If the authors are referring to the wave-like frequency components, I'm quite confident they are not formants. If time in seconds is denoted in the x-axis and the corresponding text is aligned and overlaid, it will be easier to determine.

Some minor comments:
- supplementary video was helpful to experience how the attack sounds like, but it was pretty disturbing. And the white noise was inaudible so I couldn't make comparison. If such attacks are practically to be used, it would be good to perform a user study for perceptual evaluation of different attack methods.
- is multiplier m in Figure 6 the same as Power of Noise in Table 1? And what is the unit of WER in Figure 6? Why such big discrepancy in WER?
summary_of_the_review: This paper presents Neural Voice Camouflage, a real-time attack method that disrupts in streaming ASR systems. The methodology is clearly explained and the main contributions are well supported by a solid evaluation framework and carefully designed experiments. Thorough analyses on the results provide useful insights for researchers working in the same domain.
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: A novel technique that prevents the ASR (DeepSpeech) from correctly recognizing the speech is presented. The proposed method works in real-time and is robust against some defenses.
main_review: ### Strengths

- The motivation for the problem, the formalization, and the experiments are clearly written and well-organized.
- The proposed method is tested in various situations that reflect real-world scenarios, and successfully deployed the method in a real room environment.
- The authors also show that the attack is specific to the speaker; which partially implies that the NN-based approach is crucial for a given problem.

### Limitations

- Though practical ASR applications use an additional LM (language model) to correct the output, such cases are not examined. By looking at some examples of attacked transcription and the ground truth labels, I'm pretty sure that the accuracy of the model will increase if aided with LM. This is the main reason I'm giving a score of 5, and I'm willing to adjust my judgment if authors succeed in examining and discussing the effect of LM.
- One of the limitations of this work is that it was only tested with a single specific ASR model; I think this should be also mentioned in the "Ethical considerations" section.

### Questions

- What DeepSpeech model are the authors referring to? Please be specific about the model information and cite the literature if necessary; the authors might want to add a section in the appendix for this.
- In Figure 6, I wonder if the authors forgot to multiply 100 on WERs. It'll be clearer if "(%)" is added to every WER/CER in plots.
summary_of_the_review: Though this work has established the important problem and nicely tested the proposed method, it has failed to address an important component of ASR, LM. Thus I'm initially giving a score of 5.
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: This paper proposes a Neural Voice Camouflage (NVC) method that has three important characteristics, which are essential for an NVC method to be used in practical scenarios: general, real-time, and robust. Since the proposed method trains a model to learn predictive attacks without any constraints about input and output, it can be applied to any vocabulary in a real-time scenario, and it is also difficult to defend the attack. On the contrary, the previous gradient-based adversarial attacks take a lot of time to compute the attack, so it is difficult to be used in a real-time scenario. Other than that, other previous methods are trained to attack only a few target words or utilize a pre-defined frequency region that can be easily filtered out.

In experiments, this paper shows that the proposed method is really effective by showing that the WER&CER of an ASR model significantly increases with the method compared to other NVC methods. Furthermore, this paper conducts various analyses on the behavioral characteristics of the method that can give many insights for future work. Moreover, various experiments, which are conducted with considerations about the situation where the method is used in the real world, are also shown in this paper.
main_review: I think that the idea of training a model to learn predictive attacks is a contributive and effective way for an NVC model to be used in a real-world scenario. Also, the various experiments in this paper seem that the authors have considered a lot about the real-world scenario and can give many insights to the future works.

However, there are several concerns about this paper.
1. Personally, this paper was difficult to understand especially due to the way of indexing. For example, $\alpha_{t+\delta+r}$ means a noise to be added to the speech up until $t+\delta+r$?
2. I think the $\delta$ is rather a room for computation time than an exact computation time. Is it right?
3. I think it would be better if there is a comparison with a previously proposed real-time NVC method even if it works only in a certain frequency region. This is because it seems more plausible online NVC model compared to the online PGD.
4. The paragraph, "How robust is the attack to temporal shifts?" is a little difficult to understand. When I read it, I think it is not an ablation study about varying $\delta$, but I think the paragraph is saying about it (e.g. the sentence, the larger the delay $\delta$, the further into the future our model needs to predict."). Plus, I think there should be an ablation study about varying $\delta$ for training the NVC model.
5. When it comes to the real-world scenario, I think it is also a very important condition where we do not know about the ASR model. Therefore, I think it would be better to conduct experiments showing the performance drop when using an ASR model which is different from the ASR model used in training.

**Personal Opinion**
* When I read a paragraph "Real-time Machine Learning" in Section 2 at first, I felt it could not have been written because this paper is not about speeding up the inference speed. Therefore, I think it would be better if it explains how the delay enables this method to be operated in real-time.
* I think there should be a reference about the numbers appearing when it explains the relationship between the high sampling rate and instantaneous computation.
* There should be a reference about DeepSpeech, and in Section 3.4 (or in the appendix section), I think the architecture about DeepSpeech and the Language Mdeol should be written.
* When I read this paper, I really wondered about the audio samples (generated perturbation only / speech + perturbation). However, I cannot see it and I cannot open the video in the supplementary material.
* How the WER can be larger than 100%? (off-line PGD)
* I'm a little confused about using 0.5s delay, considering the summation of computation and playback time? (0.014s + 0.5s)

summary_of_the_review: I think this paper proposes a contributive NVC model and can give many insights. However, I personally think that this paper is a little difficult to read and the experiments are a little weak to support its contribution. Therefore, I give a score of 5 for this paper.
review_title: null

Meta-review: This paper proposes a novel neural voice camouflage method that learns predictive attacks without any constraints about input and output. It is general, robust, and real-time that could be used in a real-world scenario. The experiments are solid, the in-depth analyses are convincing.
Decision: Accept (Oral)