Venue: ICLR 2020 Conference

Reviews:

final_score: rating: 3: Weak Reject
review_content: First of all, the two physical attacks evaluated in this paper have similar attacking patterns, i.e., mask-based pixel attacks. So it is not surprising that DOA is more robust in these cases since DOA is trained on this attacking pattern.

Actually it has been shown that the framework of adversarial training (AT) will overfit to the attacking patterns used in training. For example, PGD-AT models are less robust to simple non-pixel transformation, like rotation, than the standard models [1]. So what DOA does is just substituting the PGD module in AT to overfit the new attacking patterns, which is of limited contribution and novelty. 

Besides, AT is not really scalable compared to other simpler defense strategies like input transformation. [2] proposes a simple and effective defense based on different combinations of input transformation and its performance even surpasses some SOTA AT models with less computation. 

Another advantage of these off-the-shelf defenses like input transformation is that they do not depend on the specific details of attacks, so they are more reliable when you are unknown about the potential attacking patterns in practice. In comparison, there is an implicit assumption in AT methods that the attacking patterns in training and test are similar. This is the reason why PGD-AT is not robust facing mask-based physical attacks or simple rotations.

So under the more completed and flexible physical attacks, a defense based on the AT framework like DOA may not be a good choice. Although AT methods are quite effective and widely studied under the l_p attacks, the authors are expected to consider more factors if they really want to design a robust system in the physical world, rather than just follow or apply the most popular pipeline like AT.

Reference:
[1] Engstrom et al. A rotation and a translation suffice: Fooling cnns with simple transformations. ICML 2019
[2] Raff et al. Barrage of Random Transforms for Adversarially Robust Defense. CVPR 2019
review_title: Official Blind Review #4

final_score: rating: 8: Accept
review_content: [Note: I gave a 3 for thoroughness even though I only read the paper once, because I believe that I carefully considered the paper while reading it.]

This paper argues that threat models such as L-inf are limited when considering physically realizable attacks, and provides evidence for this by showing that L-inf adversarial training is insufficient to fully confer robustness against physically realizable attacks in the literature such as the adversarial glasses attack. The paper then proposes an alternate threat model based on contiguous rectangular regions, and shows that adversarial training against this model does far better.

Overall this is a strong paper with thorough experiments, and was for the most part carefully written (although see some quibbles below). I particular liked the paper's carefully distinction between physically *realizable* and physically *realized* attacks, and the admission that not all realizable attacks would fall under the threat model (while still justifying why the model is interesting).

I am on the border of weak accept and strong accept. The main two points keeping me from strong accept are discussed below (and seem perhaps addressable by the authors):

1. The assertion that rectangular occlusions might be a fruitful model for realizable attacks is only lightly tested, since the attacks considered in this paper all fall into the rectangular occlusions threat model. Since one of the key points in the paper is that training in the L-inf threat model could leave vulnerabilities to non-Linf attacks, we might expect the same with rectangular occlusions. A more convincing demonstration would be to test robustness to physically realizable attacks that fall outside the rectangular model (perhaps even skewed or rotated rectangles, although a less synthetic example would be even better). A more minor point is that it would be nice to test the robustness of models trained on rectangular occlusions against L-inf adversarial attacks. This is relevant to knowing whether training on occlusions is giving up on adversarial robustness or actually helps against both (I actually think it's plausible that you would do well on L-inf, but it seems worth testing either way).

2. The high-level claimed take-away that adversarial training does not help against physically realizable attacks seems false in light of Figure 2, which shows that L-inf adversarial training substantially improves robustness relative to the baseline. A more accurate take-away would be that on some datasets adversarial training helps but still leaves a gap, while on others it does not help at all and perhaps hurts. I would prefer more careful wording as a reader who only goes through the introduction might not see Figure 2.

On #1, I should stress that the paper would still be interesting regardless of the outcome of the two experiments in #1; I just think that for thoroughness it would be nice to include them. The existing experiments are already thorough so this would be going above and beyond, but that is why it might help raise my score from weak to strong accept.

EDITED TO ADD: For #2, I would also be happy with an argument from the authors as to why the current language appropriately describes the experiments. I do not wish to dictate to the authors what their take-aways are, but more to open up for discussion a point that seemed slightly sloppy to me.

Minor comments:

Please avoid subjective intensifiers: "We then use an extensive experimental evaluation to demonstrate that our proposed approach is far more robust against physical attacks on deep neural networks than adversarial training and randomized smoothing methods that leverage lp-based attack models." Both "extensive" and "far" are unnecessary.

Make sure to use \citep vs \citet correctly.

First sentence of 2.1 is too verbose. Overall the prose in that paragraph is turgid, due to too many action phrases being turned into nouns. E.g. "The focus is on", "The typical goal is" both indicate *action* and could be profitably turned into verbs. See Williams and Bizup's book on Style.

"since our ultimate goal is to defend against physical attacks, untargeted attacks that aim to maximize error are the most useful" This seems weak; I don't understand why an attack being physical should go in line with it being untargeted; oftentimes an attacker will have a specific targeted goal.

"another advantage of the ROA attack is that it is, in principle,easier to compute than,  say,l∞-bounded attacks" This seems incongruous with the subsequent text, which admits that ROA if implemented naively would be slower than L-inf attacks.
review_title: Official Blind Review #2

final_score: rating: 8: Accept
review_content: The paper aims to provide a defense to physically realizable attacks for image classifiers. First, the paper demonstrates that Lp-ball robustness (obtained via adversarial training or randomized smoothing) do not necessarily result in robustness to physical attacks such as adversarial stickers. Next, the paper proposes a variant of adversarial training (using adversarial rectangles) and shows empirically that such a method results in much improved robustness to the aforementioned physical attacks.

I vote to accept this paper. It has clear motivation, is very clearly written, evaluates proper benchmarks from recent literature, and proposes a new method that shows a clear improvement over benchmarks in literature. The goal is clearly laid out in Section 3, and the paper justifies its claims via various experiments. Overall, I think it is a good contribution to the adversarial examples literature, as it provides robustness against more “real-world” attacks.

In particular, the results from Fig. 2 and Fig. 3, while not surprising, appear to be done thoroughly, as the authors evaluate against various forms of adversarial training and various degrees of randomized smoothing. Later, the results in Fig. 4 and Fig. 5 show a clear benefit from their method.

I do have a few pieces of feedback that I think could improve the work.

In section 5.1, do you really only train for 5 epochs? Is this just a fine-tuning procedure after standard training is performed, or is the entire training procedure 5 epochs? That seems especially short for getting any amount of accuracy. I think it is worth clarifying.

In Section 3 (or in the Related Works), I would suggest that the authors do mention related works about other types of realistic adversarial examples or perturbations, such as those generated via physical transformations like translations and rotations [1] or even common corruption robustness [2]. This is especially relevant since the authors claim to be the “first investigation of this issue in computer vision applications,” so it’s worth clarifying that the authors do not claim to be the first work about robustness to all realistic perturbations.

One thing I would be curious to see is if the DOA training method provides any robustness to standard Lp-ball adversarial examples or to perturbations like rotations.

Additional Feedback:

- In Related Works, I think there are some places where it would look better to use parentheses around the citations.

[1] https://arxiv.org/abs/1712.02779
[2] https://arxiv.org/abs/1903.12261
review_title: Official Blind Review #1

Meta-review: This paper studies the problem of defending deep neural network approaches for image classification from physically realizable attacks. It first demonstrates that adversarial training with PGD attacks and randomized smoothing exhibit limited effectiveness against three of the highest profile physical attacks. Then, it proposes a new abstract adversarial model, where an adversary places a small adversarially crafted rectangle in an image, and develops two approaches for efficiently computing the resulting adversarial examples. Empirical results show the effectiveness. Overall, a good paper. The rebuttal is convincing.
Decision: Accept (Spotlight)