Venue: ICLR 2021 Conference

Reviews:

final_score: rating: 7: Good paper, accept
review_content: This work proposes a new form of adversarial training, supported by two proposed adversarial attacks based off a perceptual distance. The choice of perceptual distance (LPIPS), is computed by comparing the activations of (possibly different) two neural networks with respect to a pair of inputs. The authors propose two new attacks based off this perceptual distance: PPGD and LPA, as it is distinct from the common choice of L_2 or L_inf. This work claims that performing adversarial training against adversarial examples crafted by the proposed attacks, induces robustness to a wide range of "narrow" threat models e.g. L_2, JPEG, L_inf.
To show this, the authors perform experiments on CIFAR-10 and ImageNet-100 — with the main results being that adversarial training with PPGD or LPA produces a model with some robustness to all other threat models. This is in contrast to adversarial training with "narrow" threat models, which fail to be robust to at least one other threat model in the set. The exception possibly being L_2, which retains some transferability.

The paper is written well, and is somewhat easy to follow.

Overall I believe this to be a good paper. I appreciate the study of a union of threat models, as in practice, we of course have no guarantee that an adversary will choose to restrict itself to a single threat model. The experimental results are compelling, and match my intuition — namely that adversarial training with the L_2 threat model would produce the most comparable results to a perceptual threat model, when comparing against the union. Moreover, I find it very interesting that while L_2 looks to transfer well to other threat models, it does not transfer comparably well to the perceptual threat model. The authors explain that this is due to the "broadness" of the threat model, but this could use some further exposition.

Qualitatively, the LPA and PPGD adversarial examples appear distinct from other threat models in terms of the modified features. The LPA adversarial examples in particular to not seem to target local texture patches, which is nice to see.

I also appreciate the addition of less computationally intensive algorithms to support the proposed attacks. PPGD in particular would be very difficult to run, as the jacobian of \phi(x) can be very large.

I have some questions, in order of importance.

1) Why is AlexNet (externally-bounded) more effective than using the same network to compute LPIPS? This may be because the metric changes as the network trains, but then I would expect the self-bounded training to achieve worse clean accuracy if the metric degraded.

2) What is the reason for believing that the neural perceptual threat model encompasses the L_p and spatial threat models? I see that there is some overlap in terms of how each threat model can transfer to other threat models. But it is not obvious that the "narrow" threat models are contained with the perceptual threat model.

3) I think its interesting that PAT-* performs worse on PPGD and LPA, than on other threat models (that presumably the PAT-* model has not seen). This could mean that PPGD and LPA adversarial examples are "harder" in some way. Do the authors have more explanation/intuition about this?

More general comments:

I found parts of the paper somewhat difficult to read due to having to check the appendix often. It would be easier to read if some details like the attack algorithms, and a figure showing the attack process were moved to the main paper.
review_title: Good work

final_score: rating: 7: Good paper, accept
review_content: Summary:
* This paper proposes perceptual adversarial robustness, an adversarial truing against the set of all imperceptible adversarial examples. Through a perceptual study, they approximate human perception with a neural neural - “neural perceptual distance”, and test the robustness against 5 threat models including L2 and Loo showing state-of-the-art robustness even without training on them, showing generality of the model considered. 

Strengths: 
* Relevant, challenging problem of interest to the CV and adversarial learning communities
* Clearly define scope and contributions 
* Reasoning about both new robustness metric and new attacks to this metric
* Authors plan to release adversarial examples with annotations 

Weaknesses:
* Since it mostly relies on LPIPS, it is unclear how much novelty there is in the approach in the NPTM. 
* The PPGD and LPA attacks completely break the PAT (Perceptual Adversarial Training), so the reader is left with a bit of mixed feelings about lessons learned. I think the authors could have elaborated more on this. 
* Fast-LPA seems to be a relevant contribution but is relegated to the Appendix. 
* Related work is a bit succinct. Since the major proposal is about neural perceptual distance, but they rely on something from the state of the art, although extensively validation, it would be interesting to know more about what are the actual contributions with respect to this part as well. The new attacks against this NPTM are a clear contribution, in addition to validation with mechanical turk of the perceptual similarity. 

Comments: 
* As a general comment, I believe it would be clearer to report "attack success rate" instead of decreased accuracies, although I understand the work is a bit on attacks and a bit on the robustness side. So, to show relevance of adversarial training, eventual model accuracy is okay. 
* It is a bit confusing that in Table 2 and 3, the “Normally” trained model has an Unseen Mean of 1.8% instead of 0%. I eventually understood why, but maybe you should consider commenting on this, as I initially felt like there was something wrong with the Table. 
* The authors emphasize a lot on the NPTM, which is definitely relevant. Nevertheless, the PAT (Perceptual Adversarial Training) seems mostly unsuccessful against the PPGD and LPA (and Fast-LPA) attacks, and this may 
* I am a bit confused by the overall narrative of the work. You first propose the NPTM, and develop the PAT mechanism, and show it generalizes well across different threat models. But then you develop an attack that “knows” the defense - which is great - but then show that it basically defeats PAT (mostly). I think this part of the story is left a bit under-explored, and the reader is missing actionable points from the work, as there is a lot of discussion on robustness but also on attacking this new perceptual models. Do not get me wrong: I think this is highly relevant, I am mostly commenting on the presentation of the results and the overall story and lessons learned. 

review_title: ICLR 2021 Conference Paper261

final_score: rating: 6: Marginally above acceptance threshold
review_content: This paper studies the adversarial robustness of deep neural networks against multiple and unforeseen threat models. Since there lacks a precise formalization of human perception, this paper adopts LPIPS, a metric that correlates well with human perception based on neural network activations. Then, two adversarial attack methods are proposed to generate adversarial examples under the metric. And an adversarial training method is also proposed. The experiments on various threat models validate the effectiveness of the proposed method.

The writing of this paper is clear. The generalizability of robust DNN against multiple threat models is an important problem. This paper is a good attempt to solve this problem. Based on a perception similarity metric, new adversarial attacks and defenses are studied. Thus the paper is comprehensive.

This biggest problem of this paper, in my opinion, is that the adopted metric is defined on neural networks. Although the authors have conducted human evaluations to prove that this metric correlates well with humans, I still doubt whether it can reflect all potential threats. For example, if the metric is perfect, any (adversarial) example, that is perceptually the same as a clean example to humans, could have the similar representation with the clean example when using the network \phi. If this is true, \phi itself is a perfectly robust network, and we do not need any adversarial training to robustify another network (i.e., f(x) in this paper). 

However, defining a perfect metric for adversarial training may be very difficult. And this paper makes a step towards this goal. 
review_title: Official Review

final_score: rating: 7: Good paper, accept
review_content: This paper proposes a threat model called Neural Perceptual Threat Model (NPTM) and under NPTM, they develop novel perceptual adversarial attacks: perceptual Projected Gradient Descent (PPGD) and Lagrangian Perceptual Attack (LPA). Also, they propose Perceptual Adversarial Training (PAT) which achieves good robustness against various types of adversarial attacks and even could generalize well to unforeseen perturbation types.  

I think this is an interesting paper and solves an important problem. The writing is very clear and easy to follow. The main contributions are:

1. Introduce a new threat model called the Neural Perceptual Threat Model (NPTM) based on Learned Perceptual Image Patch Similarity (LPIPS) and propose two perceptual attack methods PPGD and LPA. They perform a study on Amazon Mechanical Turk (AMT) to show that LPIPS correlates well with human judgments across 7 different types of adversarial perturbations and adversarial examples generated by their attacks are imperceptible to humans. Such a study can benefit future research and they promise to release their dataset with annotations. They also perform experiments to show that LPA is by far the strongest adversarial attack at a given level of perceptibility, which could be used to evaluate the robustness of the defenses in the future. 

2. Propose Perceptual Adversarial Training (PAT) which achieves state-of-the-art robustness against the union of the five attacks $L_2$, $L_\infty$, spatial, recoloring, and JPEG, without training against any of them. Being able to generalize well to unforeseen perturbation types is a desirable property for adversarial robustness and previous adversarial defenses don’t have such a property. 

However, I have some concerns: 

1. I notice that the clean accuracy of PAT-self and PAT-AlexNet (in Table 2&3) is lower than other training methods. I think it is due to removing the projection step at the end of the Fast-LPA attack. Could the authors explain why they need to remove the projection step? What are the results without removing the projection step? I think achieving good clean accuracy is also important for an adversarial trained model and PAT should not hurt the clean accuracy much; 

2. Could the authors explain why AlexNet is the best proxy for human judgments of perceptual distance? Are there any insights? Also, could the authors provide detailed descriptions about how the AlexNet model is trained? 

3. The authors mention that training against multiple threat models simultaneously will result in lower robustness against any one of the threat models when compared to hardening against that threat model alone. But from the experimental results, PAT is also less robust against a threat model compared to the method that trains against that threat model. Could the authors compare PAT to those methods that train against multiple threat models (e.g. [1] and [2])? 

4. It would be good that the authors could evaluate PAT on images with common corruptions (e.g. CIFAR-10-C and ImageNet-C proposed in [3]). It can demonstrate that PAT could generalize well to unforeseen perturbation types further.  

If the authors could address these concerns, I am willing to raise my scores. 

Minor things: I find that the clean accuracy of the Normal model is 0 in Table 2. I think it is a typo. Please correct it. 

[1] Maini, Pratyush, Eric Wong, and J. Zico Kolter. "Adversarial robustness against the union of multiple perturbation models." arXiv preprint arXiv:1909.04068 (2019).

[2] Tramèr, Florian, and Dan Boneh. "Adversarial training and robustness for multiple perturbations." Advances in Neural Information Processing Systems. 2019.

[3] Hendrycks, Dan, and Thomas Dietterich. "Benchmarking neural network robustness to common corruptions and perturbations." arXiv preprint arXiv:1903.12261 (2019).
review_title: Interesting paper that solves an important problem

Meta-review: This paper focuses on the adversarial robustness of deep neural networks against multiple and unforeseen threat models, which proposes a threat model called Neural Perceptual Threat Model (NPTM). The philosophy behind sounds quite interesting to me, namely, approximating human perception with a neural neural "neural perceptual distance". This philosophy leads to a novel algorithm design I have never seen, i.e., Perceptual Adversarial Training (PAT) which achieves good robustness against various types of adversarial attacks and even could generalize well to unforeseen perturbation types. 

The clarity and novelty are clearly above the bar of ICLR. While the reviewers had some concerns on the significance, the authors did a particularly good job in their rebuttal. Thus, all of us have agreed to accept this paper for publication! Please carefully address all 
comments in the final version.
Decision: Accept (Poster)