Venue: ICLR 2020 Conference

Reviews:

final_score: rating: 6: Weak Accept
review_content: The authors of this work bring to light the security vulnerabilities of copyright detection systems to (DL style) adversarial attacks. The work summarizes the basics of copyright detection systems for audio, and notes that recent methodologies for feature extraction incorporate neural networks, as opposed to hand-crafted features. Following which the authors embark on designing a simple copyright system based on a neural network. With the newly proposed system, it is shown that one can construct adversarial examples to obstruct copyright detection systems using differentiable programming. The constructed adversarial examples are shown to be successful in evading popular copyright detection systems for audio (YouTube Content ID, AudioTag) as black box attacks.

Overall the paper brings about an interesting and pressing issue in a timely manner that seems to be of broad interest to the security and ML community. The paper is very well written and I enjoyed reading it. Further, the construction of the adversarial objective and attacks seems novel. However, I am not as familiar with the literature with adversarial attacks for audio. Overall I am giving this work a weak accept, which I am willing to change if the authors provide additional insight into their experiments. 

In particular, it is difficult for me to assess the success of the imperceptibility of the perturbation in the audio domain from l2 and l_infinity norms, so it is hard for me to judge whether such adversarial examples are competitive and useful. For this reason I would like the authors share excerpts from their attack experiments for multiple examples for the different results presented.


review_title: Official Blind Review #3

final_score: rating: 3: Weak Reject
review_content: [Summary] This paper proposed a black(white)-box attacking algorithms to attack industrial copyright detectors. Specifically, a simple un-targeted gradient-based method can successfully fool commercial system like YouTube and AudioTag. This paper has generalized the concept of “adversarial examples” to a new application area, which is indeed a very important one, to raise awareness for the potential threats. 

There are a couple of reasons of why I like this paper: (1) copyright attack and especially defense (missing in this paper) is a more important application compared to others widely studied vision tasks like classification. To increase the copyright detectors' accuracy, companies must have invested a huge amount of money on deep learning based methods. As this paper suggests, these more accurate detectors also come with severe security problems, and such problems are rarely studied before. In addition, the experimental results in this paper show that two commercial systems can be easily compromised in a black-box manner. (2) This paper is clear in writing and easy to follow. I am not an expert in acoustic/audio recognition, but this paper shows that a not very complicated approach can work in real-world without too much domain/expert knowledge. 

My rating for this paper is “3: weak reject”. I think it’s below the line for two reasons: technical novelty and experimental settings. I will elaborate these two points in the following sections.

As the author mentioned and generally believed, almost every machine learning model (black and white-box) can be compromised with adversarial examples, especially in un-targeted manner. “Copyright detector” is just another machine learning model regardless its important usage. The technical approach proposed in this paper is straightforward and limited. First, section 4.2 is very similar to Wang et al. (2003). What’s the biggest improvement or novelty here? All the filters are handcrafted and fixed as suggested by Eq.1 and Eq.2. Why don’t train a network or take a pre-trained DNNs to predict spectrogram?  The biggest advantage of deep learning (end2end trainable as feature extractor) is not utilized.  Second, the solution of Eq.5 is award. Usually we don’t use a softmax with temperature to estimate max, better ways should be tried like gumbel-softmax. Third, besides the optimization methods used for Eq.6, Carlini&Wagner is also widely used for optimizing this objective. What’s the reason that C&W is missed in this paper? Fourth, Eq.8 can be used for targeted attacks. However, this part is missing. Un-targeted attack is considered as easy especially in white-box cases. Targeted attack is a better and harder task for evaluation of algorithms. Fifth, the surrogate loss Eq.4 is basically a hinge loss between two quantized feature maps with mask \psi. It’s pretty standard to me and I am surprised this is the only loss proposed and studied. 

The experiments in this paper is more like a “proof-of-concept” rather than “serious evaluation”. First problem is that the norm is used to evaluate the perturbation. Unlike the norm in image domains which can be visualized and easily understood, I don’t have a clear understanding of how “big” the perturbations are for audios. A cognitive study or something like a user study should be conducted. Another question related to this, the random noise is 3x bigger in terms of norm, does this make huge difference when listening to it? Are these two perturbations both very obvious or both unnoticeable? Second, it seems like a dataset is built but the stats are missing. How big is this dataset? How many songs are used to generate results in Tab.1, Tab.2, and Fig.3? What’s the attack accuracy on AudioTag and Youtube? Third, no baseline methods are compared to in this paper, not even an ablation study. The proposed two methods (default and remix) seem to perform similarly. I don’t think the best thing people can do previously to attack copyright system is just random noise. The authors could argue that there are no previous papers, but I think more realistic baselines should be studied. 

One question, the inline equation at the bottom of Page.4  should be \phi(x) == maxpool(\phi(x)) right? And \psi(x) = ( \phi(x) == maxpool(\phi(x)) ) ?

Overall, I think this paper is not in good shape to be published even though the targeted problem has already become a big concern right now. 

review_title: Official Blind Review #4

final_score: rating: 3: Weak Reject
review_content: Thank you to the authors for clarifying that they disclosed the vulnerability to YouTube. It would be good to surface this in the main text rather than a footnote, and to include some details as to what the response from YouTube was.

The paper demonstrates that adversarial examples are a practical scheme for evading copyright infringement detection tools. The study of this application is well motivated: adversarial examples need not be played in the physical domain (since the audio/video is uploaded directly in a digital format) and copyright detection is a bit more complicated than object classification (because false positives are expensive), small margin between classes. 

Could you motivate the choice of the L_inf norm as an appropriate metric for the magnitude of the perturbation added to audio samples? Would the perturbation affect a listener’s ability to follow the audio that the attacker is trying to avoid detection of? In Section 4.5, why is inserting a perturbation that is close to a different audio sample a successful attack? How would the baseline of playing the target audio sample at a very low volume perform in comparison?

While the paper considers a system that was designed by the authors, because they demonstrate transferability to commercial products, this is not a limitation of their work and demonstrates a realistic threat model. The description of experimental results is however missing important details required to evaluate the results presented:

* What are the details of the dataset used to evaluate the approach? How many songs were included in the dataset? 
* Did you conduct a study to verify that the perturbation added did not affect legitimate listeners?

The paper is well written and easy to follow. 

A simple nitpick on page 2: do you have any references to point to for the following claim? “Most audio, im- age, and video fingerprinting algorithms either train a neural network to extract fingerprint features, or extract hand-crafted features.” 

And on page 7, the following sentence might be a typo given that the perturbations introduced in this paper are adversarial: “Therefore, one would expect that low-amplitude non-adversarial noise should not affect this system.”
review_title: Official Blind Review #2

final_score: rating: 3: Weak Reject
review_content: This paper points out that the industrial copyright detection systems are susceptible to adversarial attacks and discuss the reason why these systems, especially the neural-network-based systems, are vulnerable to attacks. The authors describe a widely used music identification method and propose a simple gradient method to attack this system in this paper.  Specifically, the authors build a simple song identification method using neural network primitives and propose a differentiable surrogate loss to measure how close two fingerprints of audio are, which is used to generate the adversarial examples. And then they attack a song identification using the adversarial examples. 
The authors conduct experiments to evaluate the proposed method. The white-box attack on their own model and the experiments on two real-world systems (AudioTag and YouTube) validate the effectiveness of the proposed method. The experimental results on AudioTag and YouTube seem impressive.

Weak points:
From my point of view, this paper is more like an application-oriented paper than a research paper. Because the main contribution of this paper is the results of experiments on a real-world system. Though the experimental results seem to support the justification of the model, this paper has limitations in terms of Model and Experiments.
1.	Model
a.	The description of the proposed model is not clear enough. The whole architecture of this model is not presented. The authors should describe how to generate the adversarial example more clearly.
b.	In section 4.1.2, “we build a shallow neural network that captures the key ideas of Wang et al. [2003] while adding extra layers that help produce transferable adversarial examples”. The Audio fingerprinting models proposed in the paper are quite incremental.
c.	The proposed differentiable surrogate loss, which measures how close two fingerprints of audio are, should have more theoretical analysis to illustrate its reasonability or provide some analysis of the difference between adversarial examples and real examples.
2.	Experiment
a.	The results of the paper are hard to reproduce.  The detail of the model used in the experiment is not presented, such as how to train the model.
b.	The authors should add more baseline or other intuitive methods to make the results of both the white-box attack and real-world more convincing.

review_title: Official Blind Review #1

Meta-review: This paper shows a case study of an adversarial attack on a copyright detection system. The paper implements a music identification method with a simple convolutional neural network, and shows that it is possible to fool such CNN with an adversarial learning. After the discussion period, two among three reviewers incline to the rejection of the paper. Although the majority of the reviewers agree that this is an interesting problem with an important application, they also find many of their concerns remain unaddressed. These include the generality of the finding as the current paper is more like a proof-of-concept that black/white-box attack can work for copyright system. The reviewers are also concerned that the technique solution/finding is not novel as it is very similar to prior work in other domains (e.g., image classification). One reviewer was particularly concerned about that the user study is missing, making it difficult to judge whether the quality of the modified audio is reasonable or not.
Decision: Reject