From: NIPS.cc/2018/Workshop/IRASL
title: Acceptance Decision
decision: Reject

From: NIPS.cc/2018/Workshop/IRASL/Paper16/Authors
title: Author response to reviewer 1
comment: Dear Reviewer,

Thank you for your detailed feedback on our paper.

As for the black-box setting, all that is required is access to the output logits of the model (as specified in Section 1.3). Any loss function that uses the output logits and target phrase could be applied to our method; we chose CTC loss as it is the most widely known loss function suited for this task. Thus, the fact that both the training of the victim model and our attack use CTC loss is mostly coincidence. We will make sure to clarify this point in the final version of the paper.

As for beam search, during inference time the model can be set to any arbitrary beam width. As a black box attacker, we do not know this information, and so we simply set it to the greedy setting (beam=1). This is to make sure we are in line with the Carlini baseline that we compare to [1]. 

We will make sure to address the other minor points mentioned as well.

References:

N. Carlini and D. Wagner. Audio Adversarial Examples: Targeted Attacks on Speech-to-Text. ArXiv e-prints, January 2018.

From: NIPS.cc/2018/Workshop/IRASL/Paper16/AnonReviewer1
title: Interesting approach, although it doesn't really seem to be black box...
review: The results presented in the paper are interesting, however I feel the method is being oversold as a black box approach for two reasons:

1. The authors state that the attack is not based on any knowledge of the model, but use knowledge of the CTC loss on their genetic algorithm scoring and to estimate gradients. I think the results would have been more convincing if the attack generation method was really blind. 

2. They also mention using greedy decoding on DeepSpeech, which implies they changed the victim model. Please clarify if that was the case or if you just assumed the victim model was doing greedy decoding in order to generate attack samples.

I do not see how we can consider this to be entirely a black box approach if knowledge of the target loss is used. I recommend that the authors either explain the reason they consider their approach a black box approach or reconsider this choice. 

Some random nitpicks:

- I find that most figures do not add anything to the paper (particularly figures 1 and 2).
- "after 3000 generations" is not clear to the reader when you mention it in the abstract and introduction, unless the reader is already familiar with genetic algorithms.
- capitalize Fourier
- spectogram - spectrogram
- saying MFCCs are essentially a Fourier transform of a signal is overly simplistic, so I would recommend that sentence to be removed or changed. The reason it is challenging to go back from MFCCs to speech is that MFCCs are a compressed representation where one discards less relevant information, while the Fourier transform is not. 
rating: 2: Marginally below acceptance threshold
confidence: 3: The reviewer is absolutely certain that the evaluation is correct and very familiar with the relevant literature

From: NIPS.cc/2018/Workshop/IRASL/Paper16/Authors
title: Follow up response to reviewer comments
comment: Thank you for the follow up.

We believe the previous statement may need to be clarified; by "best known loss function," the implication was that CTC was the most widely used and known method for the task. However, as you mention, CTC is just one method and there are plenty others.

A detailed empirical analysis of various loss functions would verify which is best to use for an attack; however, we thought this to be more appropriate in follow up work, as in this paper we attempt to establish a baseline that future methods can compare to.

From: NIPS.cc/2018/Workshop/IRASL/Paper16/Authors
title: Author Response to Reviewer 2
comment: Dear Reviewer,

Thank you for your detailed feedback on our paper.

As for the evaluation setting, we are limited in this capacity as we could not find any other open-source deep ASR systems to evaluate on and test generalizability. What we are intending for in this paper is in fact for our method to be a baseline for black-box attacks on ASR systems. Though we cite previous black-box approaches on deep ASR systems such as Houdini, these methods only generate adversarial spectrograms, not all the way back up to the input audio. Other black box methods that generate adversarial audio have only been applied to limited, lightweight softmax models (as in the comparison in Table 1). Our method is the first to extend black-box attacks to ASR systems that can generate adversarial audio; thus, we are aiming to be a baseline on this task.

As for computational cost, it takes approx. 30 minutes on a Tesla V100 to generate one adversarial example. We will make sure to add this analysis in the final version of the paper. Additionally, the hyperparameter and all implementation details are included in the repo where the code has been open sourced (unfortunately authors cannot provide a link, as the github repo ties to identity). 

We appreciate the suggestion of using NSGA-II black box algorithm for this task; this algorithm would yield a variety of solutions along the PF of the ctc loss and cross-correlation similarity. However, in this method, we considered the case where an attacker wanted an adversarial audio that decoded exactly to the specific target phrase; finding the PF in the case would not be so helpful, as only a few solutions would decode exactly, thus counteracting the benefits of diversity with the NSGA-II approach. There may be settings where this diversity is needed, but such cases would be beyond the scope of this paper.

From: NIPS.cc/2018/Workshop/IRASL/Paper16/AnonReviewer3
title: Response to author comments
comment: Thanks for the response.
Can you please provide some reference on this statement:
"CTC loss as it is the best known loss function suited for this task"
To best of reviewer knowledge, this is not the case. Even as of today, the best ASR systems
are trained with some type of lattice-based (sMBR, MMI) or lattice-free (like kaldi chain model) sequence training,
and CTC has never been compatible with these optimizations.
Furthermore, CTC is nothing except forward-backward algorithm with extra blank symbol. In a sense, one can do forward-backward even without blank symbol, so again I am very curious to see how your proposed system performs when some other loss is used for similarity measure.

From: NIPS.cc/2018/Workshop/IRASL/Paper16/Authors
title: Author Response to Reviewer 3
comment: Dear Reviewer,

Thank you for your feedback on our paper and finding our paper well written.

As for the black-box setting, all that is required is access to the output logits of the model (as specified in Section 1.3). Any loss function that uses the output logits and target phrase could be applied to our method; we chose CTC loss as it is the best known loss function suited for this task. Thus, the fact that both the training of the victim model and our attack use CTC loss is mostly coincidence.
As for your suggestion, yes we could have applied a variety loss functions. However, a more appropriate test of generalization would be to apply the attack to multiple deep ASR models and verify the results; though, we are handicapped in this domain as DeepSpeech was the only open-sourced deep ASR system we could find (testing on the IBM/Google/etc. API would have prohibitive costs for us, so an open-sourced version is necessary).

From: NIPS.cc/2018/Workshop/IRASL/Paper16/AnonReviewer3
title: Lack of complete baseline comparisons that supports the claim
review: The paper introduces an algorithm to generate adversarial examples. The
algorithm is claimed to be independent of the system and treat the system
as a black-box. My main concern is exactly this claim. The paper examined
their algorithm on top of deep-speech 2 which is CTC based system. The adversarial
example also uses CTC loss to evaluate fitness. So it is not clear that the proposed
algorithm is black-box independent. If there was experiments using another
similarity fitness metric (for example euclidean loss), then this claim is more
justified.
Overall, I think there is not enough empirical experimentation and analysis.
In terms of the clarity, the paper is very clear and well written.  
rating: 1: Strong rejection
confidence: 2: The reviewer is fairly confident that the evaluation is correct

From: NIPS.cc/2018/Workshop/IRASL/Paper16/AnonReviewer2
title: Review - Targeted Adversarial Examples for Black Box Audio Systems
review: Authors introduce a black-box strategy to compute adversarial perturbations to raw audio, yielding targeted attacks to ASR systems. A genetic algorithm is proposed with the aim at searching for effective attackers in audio space. Specific mutation operators are proposed, and the model is combined with gradient ascent in final stages of the search.

Pros:
+The attack method is model agnostic, and the only requirement is to be able to query models' outputs.
+The attack method is simple and can be used in many other cases.

Cons:
-The proposed method is empirical, and the evaluation setting is too limited to suggest the method would work on other scenarios.
-No baseline systems are provided for comparison, even though similar approaches are cited.
-No analysis is carried out in terms of computational cost involved in generating attacks with such an approach. Hyperparameters used to generate the reported results are not shown.

Additional comments:

-Since authors argue there is a trade-off between attacks' success rate and audio similarity, multi-objective genetic algorithms such as NSGA-II should be employed.
rating: 2: Marginally below acceptance threshold
confidence: 2: The reviewer is fairly confident that the evaluation is correct