Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: This work provides a clear explanation for the relationship between normalizations and adversarial transferability, which further inspires the authors to propose a random normalization aggregation method where the adversarial robustness can be significantly improved.
strengths_and_weaknesses: Strengths:

1. Using adversarial transferability to boost adversarial robustness is interesting and inspiring, which is technical and methodology sound.

2. Establishing the relationship between normalizations and adversarial transferability is technically sound, and, in my opinion, it will make some impact on our community and give some insights to more researchers.

Weaknesses:

1. The connection between the mentioned novel viewpoint of using adversarial transferability and improved adversarial robustness should be highlighted, i.e., why transferability can contribute to boosting robustness, similarly why RNA can contribute to boosting robustness, moreover, do these two reasons share the same inspiration?

2. For the introduction to motivation, the authors claim that (line47-line50) the performance gap between BN and other normalizations results from “the adversarial transferability among normalizations,” which is a bit confusing and needs further justification. Specifically, the diagonal results are evaluated using white-box attacks, and the robust accuracy is about 50%, so that the gap may come from the difference between white- and black-box attacks. As this is an essential part of inspiring the authors to investigate the relationship between transferability and normalizations (line50), I suggest the authors provide a detailed justification.


3. One concern is that the robustness may come from the designed random selection operation, so it is necessary to employ expectation over transformation, EOT. [1], for the robustness evaluation to defend against the random operation.

Typos:

\tilde{x} makes Eq.2 confusing.

[1] Synthesizing Robust Adversarial Examples 
questions: I suggest the authors give an explanation for the difference between RNA (random selection in normalizations) and other random selection methods, i.e., selecting sub-network in parameter or feature space.
limitations: Please see [Strengths And Weaknesses]
review_title: null

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: The paper discusses the relation between normalization layers used in DNNs and adversarial transferability. The author(s) show(s) that the choice of normalization layer highly influences the success rate of transferred adversarial examples. In detail, it is shown that adversarial examples transfer worse (i.e., have a lower success rate of fooling the network) if a different normalization layer is used in the attacked network. This fact is used to motivate a novel technique to robustify existing neural network architectures: the key idea is to randomly select the used normalization layer during inference. Experiments on CIFAR-10, CIFAR-100 and ImageNet show that this approach is effective for the commonly used ResNet-18 and WideResNet32.
strengths_and_weaknesses: STRENGTHS:
- The paper proposes an effective, clear and easy-to-implement method that is well-motivated by experimental and theoretical results.
- The extensive experimental evaluation on ResNet-18 and WideResNet32 architectures using a plethora of attacks (FGSM, PGD-20, CW, MIFGSM, DeepFool, AutoAttack) is convincing. An ablation study is conducted to show the effectiveness of the different components and the importance of different combinations of normalization layers.
- The paper presentation is well done and only contains only a few errors (see minor remarks). Used computational resources are mentioned.

WEAKNESSES:
- The used notation can be improved at some places: 
    1. Equation (1): $x\sim\mathcal{X}, y\sim\mathcal{Y}$ suggests that $x$ and $y$ are drawn independently form the dataset. Here, $x, y\sim\mathcal{X},\mathcal{Y}$ would be correct. Also in Algorithm 1 $\{\mathcal{X},\mathcal{Y}\}$ has to be a tuple instead of set.
    2. In Equation (3): $\hat{y}(bgn)\_i^{(k)}$ suggests that $\hat{y}$ is a function in terms of $(bgn)$, which is not the case. I would suggest to use $\left(\hat{y}_{(\text{BGN})}^{(k)}\right)_i$ instead.
    3. Different symbols are used to denote multiplication (Equation (3) vs Equation (8))
- The stated Theorem 2.1 is hard to understand. Although, I see some value in the theorem, I think the author(s) should spend some time to reformulate it:
    1. It stated that "the gradient norm and $\beta$ in Equation (7) can be bounded as [...]", however, there is no $\beta$ in Equation (7) (only  $\beta_a$ and  $\beta_b$).
    2. The variable $T$ is introduced multiple times in a single sentence: "the level of adversarial transferability" and "attack success rate".

MINOR REMARKS:
- Typo "Adversarial Transfersability" in Abstract, "hessian" is sometimes not capitalized
- subscript and supperscript "gn" and "bgn" should not be typeset in math mode
- line 96: extra period in front of references [5, 20]
questions: What is meant with $\beta$ in Equation (7)?
limitations: Limitations and potential negative societal impact have not been addressed.
review_title: null

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: Inspired by the limited adversarial transferability across different normalizations, the authors proposed to involve randomness into the types of normalization layers and introduce a RNA module that reduces the adversarial transferability in a pre-defined random space, which improves the defense against adversarial examples. To evaluate the effectiveness of their algorithm, the authors provided experimental results on CIFAR-10/100 and ImageNet.

strengths_and_weaknesses: The authors studied a simple yet effective randomized mechanism with normalization layers. In general, the paper reads well, and the presentation is clear. The paper is original in that it studies the connections between normalization layers and adversarial defense. The theorical analysis well explains the principle of proposed algorithm. The paper includes a clear experimental setup and a meticulous comparison with other variant baselines as well as state-of-the-art algorithms on popular benchmarks. The results seem convincing.

Despite its contributions, I have several concerns:

1. The authors mentioned that they adopt random sampling strategy to utilize the adversarial transferability in random space. However, there exists a chance that the similar paths are sampled during both attack and inference stage, and there is no discussion of this scenario as well as the probability of it.

2. Besides normalization layers, there exist wide components in the networks which can involve randomness, such as weight parameters and architectures. Although the transferability evaluation in Fig 1 shows that there exists a poor transferability of adversarial examples among different types of normalization layers, it seems to me that this transferability also holds true for weight parameters and architectures. There is no discussion of these variant baselines.

3. The authors claimed that the smoothness of different normalization layers directly controls the adversarial transferability. However, it is hard for me to find corresponding evidence in experiment section.

questions: As mentioned above, several questions are listed below:

1. What is the risk of random sampling strategy? What is the probability of sampling the similar paths?

2. How about the involvement of randomness in weight parameters and architectures? What is the advantage of proposed RNA compared with these baselines?

3. Is there empirical evidence that the smoothness of different normalization layers controls the defense performance?

limitations: The limitations have been addressed and there is no potential negative societal impact of this work.
review_title: null

final_score: rating: 3: Reject: For instance, a paper with technical flaws, weak evaluation, inadequate reproducibility and incompletely addressed ethical considerations.
review_content: summary: Researchers explore how different normalization layers affect adversarial transferability. They provide a theoretical upper bound on the adversarial transferability between normalization layers. Then, they proposed a module named Random Normalization Aggregation (RNA). RNA replaces the normalization layers in the network, and samples normalization layers randomly at each forward pass. This can generate an exponential number of possible paths, which makes it harder for attacker to exploit.

strengths_and_weaknesses: Strengths:

1. The paper is clear and easy to follow

2. Theoretical part is sound

3. The idea to generate random normalization paths is new to the literature

Weaknesses:

1. Experiment section presents some suspicious results - the results with resNet vs. WRN are counterintuitive, and results on Auto-Attack vs. PGD also do not make sense.  It can be a sign of obfuscated gradients. Additionally, some of the reported results are not in line with known results. See the Questions section.

2. No code/models are provided for evaluation of the results. Can the authors provide the code/models/github link for additional verification?

3. No related work section. Specifically, the authors did not address some related literature that explores the same field (see Questions section).
questions: Experiments questions/concerns:
	
1. What if the attacker randomizes a different path (p_a) at each iteration of PGD? Wouldn't it make the attacker more diverse, and therefore generalize better? Can the authors present such experiments?
	
2. Can the authors explain why the results on resNet-18 are better than those on WRN-32? For example, on CIFAR-10, AA results on ResNet-18 are better by almost 4% compared to WRN-32
	
3. Also, for CIFAR-10, FGSM and PGD-20 acc is lower than the Auto-Attack accuracy, which doesn't make sense. Can the authors explain these results?
	
4. WRN-32 with BN and standard AT is essentially the standard AT method by Madry et al. [1]. It was shown that this method reaches AA accuracy of ~52-53% against AA when combined with early stopping. However, in your paper, you claim 46.44% against AA. Did the authors use early stopping for all methods? If not, I think that should present results with early stopping [2] for a fair comparison. Otherwise, the results can be linked to adversarial overfitting [2].

5. It would be interesting to see how the method works when combined with other AT methods.

6. The authors did not refer/compared to related work in the field of normalization layers and robustness [3, 4]. Can the authors compare their method to [3, 4] and present the results?

[1] Towards Deep Learning Models Resistant to Adversarial Attacks https://arxiv.org/pdf/1706.06083.pdf

[2]  Overfitting in adversarially robust deep learning http://proceedings.mlr.press/v119/rice20a

[3] INTRIGUING PROPERTIES OF ADVERSARIAL TRAINING AT SCALE https://arxiv.org/pdf/1906.03787.pdf

[4] Adversarial Examples Improve Image Recognition https://openaccess.thecvf.com/content_CVPR_2020/html/Xie_Adversarial_Examples_Improve_Image_Recognition_CVPR_2020_paper.html


miscellaneous:

1. Figure 4 has twice (b) instead of (c) 

2. Line 153 - I suggest rephrasing to- "… to *defend against* white-box attacks …"
limitations: No.
Authors should discuss the limitation of adversarial training in general and their method in particular.
review_title: null

Meta-review: metareview: This paper intrdouces the relation between normalizations and adversarial transferability, and proposes a method using random normalization aggregation for enhancing adversarial robustness.

Three reviewers agreed with the interesing idea, thorough expreiments, theroetical analysis, and the effectivess, so they gave acceptance score. 

However, one reviewer (ioBB) raised a concern on the results of Auto-attack (AA) and lack of in-depth discussion on the results.

Unfortunately, AC and the reviewers failed to make a consensus on decision during the discussion period, . 

AC carefully read the paper, the rebuttal, and the reviewers' discussion. The main remaining issue raised by Reviewer-ioBB is it is not clear the reason why the results of AA are highter than PGD. The authors provided more extensive experimental results, focusing on comparing the results of AA (and breakdown of four AA) and PGD. Also, they conjecture these result from handling adversarial transferability under randomness-based method via normalization aggregation. 

AC also agrees with the authors and Reviewer-q4wp that the in-depth analysis on the reason of AA-PGD results is ouf-of-scope of this paper and theses results are consistent to recent works on adversarial transferability. So, this issue might be left as future work. 

Because it seems that the contribution of this paper is enough for machine leanring community except the discussion on the AA-PGD reason, AC recommends accepting this paper.


confidence: Less certain
Decision: Accept