Ensemble everything everywhere: Multi-scale aggregation for adversarial robustness

Authors: Stanislav Fort, Balaji Lakshminarayanan
Primary area: unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords: robustness, ensemble, adversarial attacks, generator

Abstract:
Adversarial examples pose a significant challenge to the robustness, reliability and alignment of deep neural networks. We propose a novel, easy-to-use approach to achieving high-quality representations that lead to adversarial robustness through the use of multi-resolution input representations and dynamic self-ensembling of intermediate layer predictions. We demonstrate that intermediate layer predictions exhibit inherent robustness to adversarial attacks crafted to fool the full classifier, and propose a robust aggregation mechanism based on Vickrey auction that we call \textit{CrossMax} to dynamically ensemble them. By combining multi-resolution inputs and robust ensembling, we achieve significant adversarial robustness on CIFAR-10 and CIFAR-100 datasets without any adversarial training or extra data, reaching an adversarial accuracy of ≈72% (CIFAR-10) and ≈48% (CIFAR-100) on the RobustBench AutoAttack suite (L∞=8/255) with a finetuned ImageNet-pretrained ResNet152. This represents a result comparable with the top three models on CIFAR-10 and a +5 % gain compared to the best current dedicated approach on CIFAR-100. Adding simple adversarial training on top, we get ≈78% on CIFAR-10 and ≈51% on CIFAR-100, improving SOTA by 5 % and 9 % respectively and seeing greater gains on the harder dataset. We validate our approach through extensive experiments and provide insights into the interplay between adversarial robustness, and the hierarchical nature of deep representations. We show that simple gradient-based attacks against our model lead to human-interpretable images of the target classes as well as interpretable image changes. As a byproduct, using our multi-resolution prior, we turn pre-trained classifiers and CLIP models into controllable image generators and develop successful transferable attacks on large vision language models.

Decision: reject
Reviewer scores: 5, 8, 8, 6

Reviews and discussion:
<thread>
creation date:
1730534243238

thread_type:
official_review

<thread_object>
creator: reviewer_tzuv

<content>
summary:
This paper introduces a new method for adversarial defense, which improves the adversarial robustness of neural networks by utilizing multi-resolution inputs and robust integration methods. The authors assume that the existence of adversarial attacks is due to the differences between human and machine vision systems. To bridge this gap, they suggest using the dynamic self-ensemble of multi-scale input representations and intermediate layer predictions. They show that intermediate layer predictions exhibit inherent robustness against adversarial attacks aimed at deceiving the entire classifier, and propose a robust aggregation mechanism based on the Vickrey auction, called CrossMax, to dynamically integrate them. The proposed method can achieve good results on CIFAR-10 and CIFAR-100, without any adversarial training or additional data.

strengths:
1. The observation in this study is very interesting. 
2. The design of CrossMax is inspired by the Vickrey auction mechanism, aiming to create an integrated model that is more robust against attackers by leveraging the top-ranked predictions. This is a meaningful design.
3. As we know, adversarial training is usually time consuming, while the proposed method can achieve good results without adversarial training or additional data.

weaknesses:
1. The proposed scheme is interesting and it can also achieves good results on CIFAR-10/100. However, the experiments in this paper are not very convincing. Firstly, the images in the CIFAR dataset have very low pixels, and it's hard to determine the actual impact of multi-scale or multi-resolution approaches. Although the supplementary materials cover the size of $224\times 224$, there is no comparison, testing, and analysis with mainstream methods. It's better to evaluate the performance on ImageNet or its subset. 
2. The evaluation of the adaptive attack is missing. I only found the appendix A shows some visualization results of adaptive attack by the  multi-resolution attack. This must be evaluated in the main experiments and the statistical results should be reported. Otherwise, it cannot show that the proposed method can defend against the adaptive attack. Maybe other adversarial defense methods can achieve better results. 
3. What are the contents of A.8, C and D in the supplementary materials? Are they the corresponding figures that follow? I suggest that the author should provide a detailed description of them.
4. What would the effect be if training were started from scratch? Since the most adversarial training methods are trained from scratch, the authors fine-tune the model on a ImageNet pre-trained model. If the previous adversarial defense also follow the same experiment setting, what would the results be?

questions:
My major concern refers to the weaknesses.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730606038386

thread_type:
official_review

<thread_object>
creator: reviewer_qxda

<content>
summary:
A novel method for improving adversarial robustness is used that does not necessitate adversarial specific training, but can be complemented by it. Using biological inspiration, the authors device two strategies for improving adversarial robustness: the creation of a gaussian pyramid like view of an input image for a network as well as the CrossMax self-ensemble strategy.

strengths:
- Clear description of gaussian pyramid image stack and CrossMax aggregation strategies
- Empirical validation of the increased adversarial robustness of intermediate network layers
- Strong adversarial robustness results in CIFAR-10/100 without adversarial training
- Verification of the Interpretability-Robustness Hypothesis through qualitative examples of generated perturbations

weaknesses:
- Robustness method requires extra effort to generalize to new architectures as opposed to adversarial training
- Max operation in CrossMax may introduce discontinuities in training that may make learning more difficult
- While the performance on CIFAR is impressive, a large scale dataset like ImageNet may be required to fully understand the generalizability of this method

questions:
- During the generation of whitebox attacks is the multi-resolution image expansion considered as a differentiable step that the attack method backpropagates through?
- How would this method be generalized to transformer architectures?
- Was there any fine tuning of the model for the multiple linear probes used to create the ensemble? In other words, were the linear probes only generated after the intermediate layers were fixed? If so, what do you think the effect would be of training each intermediate layer linear readout jointly?
</content>
</thread_object>

</thread>
<thread>
creation date:
1730693538205

thread_type:
official_review

<thread_object>
creator: reviewer_u5sa

<content>
summary:
This paper addresses the challenge of adversarial robustness in deep neural networks. To tackle this problem, the authors of the paper propose a mechanism called CrossMax, based on Vickrey auction, to ensemble predictions from different layers given a multi-resolution input. The proposed method achieves adversarial robustness by leveraging the inherent robustness of intermediate layer predictions. The authors of the paper demonstrate that this method achieves significant adversarial accuracy on CIFAR-10 and CIFAR-100 without adversarial training. With the addition of adversarial training, the results can be further improved. Furthermore, the authors also explore the connection between adversarial robustness and the hierarchical nature of deep representations, showing that gradient-based attacks yield interpretable images of target classes. Additionally, they demonstrate that this approach enables controllable image generation using pre-trained classifiers and CLIP models.

strengths:
- The paper is very well written and easy to follow. I enjoyed reading it. 
- The proposed method is technically sound.
- The proposed idea is novel and generally applicable to a lot of different scenarios.
- The proposed method achieves impressive benchmark performance on various tasks. 
- The authors of the paper conduct ample experiments to support their hypothesis.

weaknesses:
- Most of the claims made in the paper seem to be empirical. It is not immediately clear how generalizable the conclusions are to other datasets and problems. 
- The proposed multi-resolution approach can make the classifiers less computationally efficient. 
- Hyperparameters on how to choose resolutions and number of resolutions can be hard to find/optimize.

questions:
- I find the observation of using multi-resolution input to the classifier makes it much more adversarially robust interesting and somewhat surprising, since all the different resolutions are derived from the single-resolution image. Would it be possible at all to encoder this prior implicitly into the classifier model instead, so that we don't need to explicitly feed in N images of different resolution? 
- Does the performance of the model get better as we feed in images of more and more resolutions? 
- While I understanding the rationale of using Vickrey auction for ensembling, I wonder what inspired the use of it for self-ensembling? Does the same approach work for ensembling in general?
</content>
</thread_object>

</thread>
<thread>
creation date:
1730715700133

thread_type:
official_review

<thread_object>
creator: reviewer_ircy

<content>
summary:
This paper proposes an innovative method for adversarial robustness in deep learning through multi-scale input processing and dynamic self-ensembling. By introducing multi-resolution input representations and CrossMax ensembling—a unique aggregation strategy that reduces susceptibility to adversarial perturbations—the authors effectively address key limitations in existing robustness approaches. This work is particularly notable for its ability to achieve competitive performance on CIFAR-10 and SOTA results on CIFAR-100 without relying on adversarial training, a common but computationally intensive technique.

strengths:
The paper was well written and well structured. The flow of the paper was easy to follow. And I would like to thank the reviewers for their presentation. 

I really liked that the idea of multi-scaling was inspired by the real human biology. I also enjoyed the cleaver trick leveraged in the CrossMax ensembling (and also in self-ensembling) to avoid fitting to the outlier predictions. 

The results provided in the paper have significant value and are truly incredible. The idea is novel and the contributions are adequate.

weaknesses:
The only weakness of the paper seems to be lack of evidence for adaptive attacks. I am not exactly sure if the following will be the best implementation, but just as an example the attacker could possibly optimize the input image with the objective of making a misclassification to happen across all layers in self-ensembled networks, or if the adaptive attack could optimize the input image across all resolutions, for white box attacks.

questions:
1) As I previously mentioned, I think the paper could really benefit from some analysis regarding its resistance to some relative adaptive attack. 

2) I also believe that since their method can adapt quite rapidly from a naturally trained model into an adversarially robust classifier, it would be a missed opportunity if it did not provide robustness results on ImageNet (or any other larger / more complex datasets). 

3) In your setting, could the authors clarify if the attacker is aware of the parameters of the multi-resolution inputs? And also, for a single sample, is the process of generating different resolutions of the same image a randomized process or a fixed process for iterative PGD attacks? And also, how much the authors attribute the robustness to the randomization. If yes (and the process is randomized), could they provide similar results by fixing the random seed throughout the iterative generations to see if that effects the effectiveness of auto-attack. 

4) On 336, why was the batch-norm turned off for the setting with training from the scratch? 

5) Could the authors please enhance their representation of other methods? For instance,  mention the name of their method or name of the authors or some other representation that would give some sense of what the other baseline is. 

6) Not a question, only a comment, the material covered in the appendix was really interesting and I really enjoyed them. I would be really happy to increase my score if the authors can convince me that the robusntess is preserved, even with adaptive attacks.
</content>
</thread_object>

</thread>
<thread>
creation date:
1732013533734

thread_type:
public_comment

<thread_object>
creator: ~matthias_hein1

<content>
title:
Robustness evaluation seems invalid

comment:
Dear authors and reviewers,

we have read with great interest the paper "Ensemble everything everywhere: Multi-scale aggregation for adversarial robustness" in our reading group. Getting robustness without adversarial training would be great but the past showed that similar attempts to achieve strong robustness without adversarial training failed. In particular, there have been defenses involving randomization and changes in the activation functions/layers which made gradient-based attacks much harder but did not yield robust models.

We therefore were a bit sceptical and had a look at the evaluation using the published notebook (which the authors provide on their github page). First, of all we could reproduce the results. Then we modified the attack using APGD-CE for the undefended model:

- we replaced for the attack the top-3 with the mean (evaluation is still with the original model)

- we increased the number of EOT-iterations from 20 to 100

- we attack all samples even if they are classified incorrectly (as the defense is random, classification is not necessarily the same)

In this way we got for 32 points the robust accuracy for CIFAR100 from 62.5% down to 33.44% (these are averages at interference) and for CIFAR10 from 62.5% to 16.56%. More APGD iterations further reduce the robust accuracy. In particular, the results for CIFAR10 show that the model is not really robust and we are very sure that with more effort one can further reduce these numbers. The main problem is that their defense is essentially randomized and via the top-3 operation the gradients are sparsified leading to low progress with gradient-based attacks.

Thus we think that the claims of this paper are invalid. We informed the authors on October 30 about this and provided them with a modified notebook to reproduce our attack but got no answer so far.

Let me further add a few comments:
- the paper misses essential checks for adversarial defenses as described in Carlini et al, "On Evaluating Adversarial Robustness", e.g. to show that robust accuracy goes down to zero for large enough radius. But most importantly it does not try an adaptive attack against their defense as also requested by some reviewers. The attack we describe above is the most simple adaptive attack but one could think of better ones.

- it is unclear what the numbers of APGD-CE and APGD-DLR mean in Table 1. Note that they are all significantly lower than the reported adversarial accuracy. Our guess is that theses are the numbers for the samples used in the EOT attack. The gap between these numbers and the reported adversarial accuracy - just by introducing new randomness should already trigger some questions

- first experiments seem to suggest that the multi-resolution approach does not help much, one can get the same results by using the standard resolution but the reviewers can ask for an ablation about this

- the evaluation in Table 1 is only done for sample sizes ranging from 128 to 1024 whereas the test set of CIFAR10/100 has 10000 samples and this is the set on which RobustBench reports results (the last line denoted as SOTA) so the comparison is not fair

- the fact that robust classifiers lead to interpretable gradients and can be used for image synthesis is well known, see e.g. 
  - Santurkar et al, Image synthesis with a single (robust) classifier, NeurIPS 2019
   - Augustin et al, Adversarial robustness on in-and out-distribution improves explainability, ECCV 2020
   - Boreiko et al, Sparse visual counterfactual explanations in image space, GCPR 2022
	
- the fact that averaging leads to some kind of robustness is also well known, see e.g.
  - Carmon, Y., Raghunathan, A., Schmidt, L., Duchi, J.C., Liang, P.: Unlabeled data improves adversarial robustness. In: NeurIPS (2019)
	
We are happy to discuss this and we can also provide the modified notebook so that everybody can reproduce the results of our attack (however we note that this would breach the anonymity of the authors, so we leave this to the AC if this is allowed or not)

Matthias Hein
</content>
</thread_object>

</thread>


Metareview:
metareview:
This paper proposes a defense against adversarial attacks that uses hierarchical self-ensembles which are augmented using jittering and noise. While the paper has received good initial reviews, the validity of the approach has been publicly challenged during the discussion. In particular, the evaluation on RobustBench, on which the paper claims to achieve state-of-the-art results, is not entirely valid. As clarified by the authors during the rebuttal, the proposed approach remains fully differentiable through the noise and jittering operations. Therefore, the attacks executed in RobustBench succeed to some extent. Yet, the SotA on RobustBench is compared on deterministic models - the randomization within the proposed method switches the paradigm so that the adversarial robustness between the referred methods is not directly comparable. 
Following this discussion, and in the context of the positive ratings given by the reviewers, I have carefully read the paper. I agree with the reviewers that the paper is well written and motivated, and that the measured increase in robustness is promising - even if the results on RobustBench are not comparable to previous results reported there. However the biggest flaws of the paper are to be seen in two aspects: (1) The limited novelty of the approach and (2) the missing comparison to previous adversarial defenses the incorporate randomization. In particular, there are several previous works stating that self ensembles and hierarchical representations improve adversarial robustness, in particular when considered with additional noise.
Examples would be:
- Liu, X., M. Cheng, H. Zhang, and C.-J. Hsieh, Towards robust neural networks via random self-ensemble, ECCV, 2018.
- X. Wang, S. Wang, P.Y. Chen, X. Lin, and P. Chin, Advms: A multi-source multi-cost defense against adversarial attacks. In ICASSP, 2020. 
- X. Wang, S. Wang, P.Y. Chen, Y. Wang,B. Kulis, X. Lin, and P. Chin. Protecting neural networks with hierarchical random switching: Towards better robustness-accuracy trade-off for stochastic defenses. IJCAI 2019.
 
The delineation to these previous works is missing in the submitted manuscript,  a proper evaluation of the proposed method in the paradigm of defenses through randomization is also missing. Therefore, even though the manuscript is well written and the proposed technique might have promise, it is not ready for acceptance.