Venue: ICLR 2021 Conference

Reviews:

final_score: rating: 5: Marginally below acceptance threshold
review_content: This paper addresses the task of adversarial defense, particularly against untargeted attack. It starts from the observation that these attacks mostly minimize the perturbation and the classification loss, and proposes a new training strategy named Target Training. The method duplicate training examples with a special ground-truth label, to fool the adversarial attackers. Experiments are conducted on MNIST and CIFAR10 under several attacks.

\+ The proposed method is simple, and involves a little modification to the classifiers and small computation overheads.

\- In the bottom of page 5, authors claimed that "adversary has complete knowledge of how the defense works". Could the authors please elaborate that, and how are the attacks modified to handle the proposed defense? If the untargeted attack is conducted on a 20-way classifier (for MNIST) without modification, I am not sure this evaluation is valid. One can add an extra layer after the last layer of the classifier, mapping $(y_0,\cdots, y_{2k-2})$ to $(y_0+y_k,\cdots,y_{k-1}+y_{2k-1})$, and then perform attacks -- since this is how the classifier actually works.

\- In Equation Minimization 1, there is a tern $l$ which is the adversarial label. I am not sure it is appropriate here since we are doing untargeted attacks.

\- From Table 2, the proposed method seems to achieve almost the same performance compared with vanilla adversarial training.

\- In Section 4.3, authors say Target Training maintains performance on clean images, when trained with adversarial samples. This might be inaccurate -- we can say Target Training achieves same performance on clean images compared with vanilla adversarial training.

---- Post rebuttal ---
I appreciate the responses from the authors, which partially address the concerns I was having. However, I am still not fully convinced that the proposed method is significant enough. Thus I am increasing my rating from 4 to 5.
review_title: Not sure if the evaluation is valid

final_score: rating: 5: Marginally below acceptance threshold
review_content: Summary:
This paper proposes target training to defend against adversarial attacks on machine learning models. Target training doubles the number of output classes, and aims to trick untargeted attacks into attacks that target at designated classes. Experimental results show that targeting training can achieve slightly better performance than adversarial training.

Pros:
1. Target training applies simple changes to the structure of machine learning models, to defend against adversarial attacks.
2. The idea of tricking adversarial attacks is novel.
3. Target training can partially break transferability of adversarial samples.

Cons:
1. What is the overhead of target training?
2. Would target training degrade the performance on clean data?
3. More discussion should be included on whether target training is effective against adaptive attacks.


Detailed comments:
While the idea of applying target training to defend against adversarial attacks is interesting, I have the following questions regarding the proposed method (performance, limitation, etc.).

1. Target Training aims to convert “untargeted attacks to attacks targeted at designated classes”, but doesn’t Minimization 1 in Section 2.1 correspond to targeted attacks rather than untargeted attacks (l is the target label)?

2. What is the overhead of target training, especially Algorithm 3? How does that compare with normal training and adversarial training?

3. Adversarial training degrades the model’s performance on clean data. Does target training have the same limitation?

4. Would Algorithm 1 (Algorithm 3) also be working against attacks that do not (do) minimize perturbations? How to choose between these two algorithms?

5. Not effective against adaptive attacks is one of the main limitations of many existing defence method. It is unclear whether target training has the same limitation. Not using techniques that have been broken does not mean that target training is robust.
review_title: The idea of target training is novel, but several key issues on potential limitations are not evaluated

final_score: rating: 7: Good paper, accept
review_content: Summary:
This paper proposed  a simple trick (doubling the number of output class) to foul the minimization, gradient-based adversarial attacks. The method is simple, powerful and requires nearly no changes to the current training infrastructure.  This paper provides a new direction how to design defense strategy.

Pros:
(1) This idea is interesting and even requires no changes during the training process. Also, the author provides enough intuitions why they design this based on the current model of adversarial attack.
(2) The experiment results are impressive and even exceeds unsecured classifier accuracy on non-adversarial samples.

Cons:
(1) The paper is not well written especially for the intro part (no more information is provided compared to their abstract).  Also, some formula need more explanations to help better understanding, e.g., loss_f in the minimization 1. 
(2) The defense strategy is based on the minimization model of attacks which may restrict its generality.

Some typos: 

(1) In the last paragraph of page 2, ...many gradient-based attacksSome.... This should be a typo.
(2) What's the difference between the two optimization problems in page 2 and 3 ?  They are the same. Also, please label them for  
better reference.

review_title: A simple and interesting trick to foul the attacker

final_score: rating: 5: Marginally below acceptance threshold
review_content: This paper proposed an ad hoc defense mechanism against white-box attacks, by duplicating the training data with original samples or adversarial samples and the number of prediction classes. The authors claim that this method achieves better results compare to baseline methods.

I admit that this method could potentially defend against gradient-based attacks like CW and PGD if the attacker have no knowledge of the defense mechanism. However, since this method is defending the white-box  threat model, I believe a simple attack could break it:

To generate adversarial sample for test data $x$ whose correct label is $y$ or $y+k$, run PGD algorithm with the objective function $\hat{x}= argmax_{x} l(f(x),y)+l(f(x),y+k)$, where $\hat{x}$ is the adversarial example, $k$ is the number of classes (before duplication) and $f$ is the network trained by target training. Namely, this is just maximize the loss for both the real class $y$ and the duplicated class $y+k$.
I'm not sure why the authors did not include any adaptive attacks like this one in section 5.

Other flaws:

-The writing is confusing, a lot of details are omitted. For example, what is perturbation size for CIFAR10 under PGD attack?

-In section 4.1, the authors say "Thus, the undefeated Adversarial Training defense cannot be used as a baseline because
it uses adversarial samples during training for all types of attack." I don't buy it. The choice of baseline method should be based on the threat model, not the algorithm or training data used.

-The authors say "Adversarial training assumes that the attack is known..." I believe this is not true.

-Black-box attack ZOO shows only 81.5% accuracy on unsecured classifier, which basically means this is not an effective black-box attack. How could the authors use an ineffective attack to demonstrate the effectiveness of their defense method? 

In summary, given the execution of the experiment, I'm not convinced this is an effective defense against white-box attacks.
review_title: Not sure if this is an effective defense 

Meta-review: I thank the authors and reviewers for the lively discussions. Reviewers found the work to be interesting but some concerns were raised regarding the significance of the results. In particular, two reviewers mentioned that authors did not fully address their concerns in the rebuttal period. Given all, I think the paper still needs a bit of work before being accepted. I recommend authors to address comments raised by the reviewers to improve their work.

-AC 
Decision: Reject