To Tackle Adversarial Transferability: A Novel Ensemble Training Method with Fourier Transformation

Authors: Wanlin Zhang, Weichen Lin, Ruomin Huang, Shihong Song, Hu Ding
Primary area: alignment, fairness, safety, privacy, and societal considerations
Keywords: robustness, diversity, ensemble training, Fourier transformation

Abstract:
Ensemble methods are commonly used for enhancing robustness in machine learning. However, due to the ''transferability'' of adversarial examples, the performance of an ensemble model can be 
seriously affected even it contains a set of independently trained sub-models. To address this issue, we propose an efficient data transformation method based on a cute  ''weakness allocation'' strategy, to diversify non-robust features.
Our approach relies on a fine-grained analysis on the relation between non-robust features and adversarial attack directions.
Moreover, our approach enjoys several other advantages, e.g., it does  not require any communication between sub-models and the construction complexity is also quite low.
We  conduct a set of  experiments to evaluate the performance of our proposed method and compare it with several popular baselines. The  results suggest that our approach can achieve significantly improved robust accuracy over most existing ensemble methods, and meanwhile preserve high clean accuracy.

Decision: accept (poster)
Reviewer scores: 5, 6, 6, 6

Reviews and discussion:
<thread>
creation date:
1730575606037

thread_type:
official_review

<thread_object>
creator: reviewer_qfba

<content>
summary:
This paper introduces a data transformation method aimed at improving the adversarial robustness of ensemble models. Instead of reducing the influence of non-robust features, the authors propose to reduce the transferability of attacks by increasing the diversity of non-robust features. The robust and non-robust features are identified by frequency selection. The weakness set is allocated to sub-models for training to increase diversity. Through the experiments on various datasets, the results demonstrate the superiority of the proposed method over other baselines.

strengths:
1. This paper is well-written and easy to follow.
2. The motivation of diverse non-robust features seems natural for a better trade-off between clean and robust accuracy.
3. The comparison with other baselines shows the effectiveness of the proposed method.

weaknesses:
1. The motivation of amplitude-based selection is not well explained. There is no theoretical or empirical evidence to support this design in this paper.
2. There is no ablation study to verify the effectiveness of all the components, such as weakness set allocation, amplitude-based selection, new dataset construction, etc.
3. The results of baselines seem strange. For example, the results in Table 2 differ from those in Table 1 of TRS.

questions:
1. Please clarify the motivation of amplitude-based selection.
2. Please provide more ablation studies of the proposed method.
3. Please clarify the difference in the result in Table 2.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730679712293

thread_type:
official_review

<thread_object>
creator: reviewer_xa1w

<content>
summary:
This paper proposes an ensemble training approach to tackle the adversarial transferability problem by leveraging Fourier transformation and a weakness allocation strategy to diversify non-robust features across sub-models. The authors introduce a novel data transformation framework involving frequency selection and frequency transformation, aiming to improve the ensemble model’s robustness against adversarial attacks without sacrificing clean accuracy. Experimental results show that their method, particularly the FDT-hybrid, outperforms several existing ensemble methods in robust accuracy on benchmark datasets like CIFAR-10 and CIFAR-100​.

strengths:
1. The paper addresses a critical challenge in adversarial machine learning, focusing on enhancing ensemble robustness without excessive overhead.
2. The frequency-based approach for diversifying non-robust features is innovative and leverages insights from signal processing.
3. Experimental results are thorough, showcasing improved robustness across various adversarial attacks and comparison with multiple baseline methods.

weaknesses:
1. The paper’s complexity may hinder reproducibility, especially given the intricate weakness allocation and frequency transformation processes.
2. The dependency on specific frequency thresholding could limit adaptability across different datasets or tasks.

questions:
1. Could the authors elaborate on how the choice of frequency thresholds (τ1 and τ2) affects robustness and clean accuracy?
2. Is the computational efficiency of the proposed approach scalable to larger models or more complex datasets, given the Fourier transformations applied?
</content>
</thread_object>

</thread>
<thread>
creation date:
1731157868698

thread_type:
official_review

<thread_object>
creator: reviewer_hpdx

<content>
summary:
Ensemble training is a commonly used technique to enhance a model's adversarial robustness. However, recent studies suggest that this approach may not be as effective, as adversarial examples can often affect multiple sub-models due to a phenomenon called "transferability." This limitation arises because all sub-models are typically trained on the same dataset, leading them to share similar vulnerabilities. To address such issue, the author propose an effective data transformation framework to improve the diversity of training
data used by different sub-models for robust ensemble training. In the end of the paper, they present empirical results demonstrating the effectiveness of their method against common adversarial attacks, while also maintaining clean accuracy.

strengths:
- The proposed framework is innovative, intuitively reasonable, and easy to understand. 
- The method comes with solid theorical backing
- The experiments are comprehensive and well-supported their findings

weaknesses:
- The paper's readability is poor. I’ve provided specific feedback in the following sections and strongly recommend improvements in clarity and structure.
- The ensemble learning with proposed method still shows limited competitiveness with adversarial training, which restricts its practical applicability. Although the appendix shows that this method maintains higher clean accuracy than popular adversarial training (AT) methods, it still falls significantly short in robust accuracy compared to these approaches.

questions:
Here are few questions and suggestion I want to provided for your paper:

1. No detailed explanation of the feature extractor. In Definition 3.1, you provide a detailed definition of what a "useful detail extractor" is. However, the feature extractor itself is only given as $θ: X → \mathcal{R}^k$. Based on the information provided, I can't see how it differs from a standard classification model. I hope you can add more details to improve the readability of the paper

2. $\hat{y}$ is not an appropriate mathematical symbol in my opinion. $\hat{y}$ is typically used to denote the prediction of y by convention, but in your paper, you are using it to denote the one-hot vector of y, which is quite confusing.

3. In your Definition 3.2, I am confused by the second formula you provided. The first formula indicates that the feature extractor is robust if the expectation $ y^t \cdot \theta(A(x)) > \frac{1}{k} $, but the second formula does not contain $y^t$ at all. I'm not sure whether I misunderstood your formula or if there is a mistake here.

4. The format of your formula 8 doesn’t seem correct to me. The equation is broken in the middle, leaving part of it followed after the plain text and the other part in equation mode.

5. I am curious how you find the "high-amplitude frequency features" from your image. I think I am having this question since I didn't understand how your feature extractors work.
</content>
</thread_object>

</thread>
<thread>
creation date:
1731216159510

thread_type:
official_review

<thread_object>
creator: reviewer_2rgn

<content>
summary:
The paper presents a Fourier-based ensemble training method to counter adversarial transferability by diversifying non-robust frequency components across sub-models. Using random noise and targeted attack transformations, the method achieves improved robustness and competitive accuracy on datasets like CIFAR-10, outperforming current ensemble techniques.

strengths:
- The use of Fourier transformations to manipulate frequency components for adversarial robustness is innovative and less explored in ensemble methods.
- The proposed method addresses the transferability of adversarial attacks more effectively by diversifying non-robust features across sub-models.
- Unlike simultaneous training methods, the approach’s independent training of sub-models reduces GPU memory requirements and simplifies the training pipeline.

weaknesses:
- The proposed targeted-attack transformation method, while effective, introduces a layer of complexity to the training pipeline. By replacing specific non-robust frequency components in the data with adversarially targeted features, the transformation becomes computationally intensive and intricate to implement, especially when scaling to larger datasets or real-time applications. This additional complexity may limit the ease of adoption for other researchers or practitioners looking to implement this technique without substantial computational resources.
- The experiments are primarily conducted on smaller architectures, like ResNet-20, with limited exploration on larger or more complex models. Without extensive testing across a variety of architectures, it’s unclear whether the proposed Fourier-based transformations are universally effective or if they are more suitable for specific model types.

questions:
Refer to the weakness.
</content>
</thread_object>

</thread>
<thread>
creation date:
1733146845668

thread_type:
official_comment

<thread_object>
creator: authors

<content>
comment:
Dear Reviewers, 

With the discussion deadline approaching (in less than 24 hours), we would like to know if our response has sufficiently addressed your concerns. We greatly appreciate your thoughtful feedback and would be happy to address any additional comments you might have.
</content>
</thread_object>

</thread>


Metareview:
metareview:
This paper investigates ensemble adversarial robustness against transfer attacks. Its key idea is to diversify non-robust features across different sub-models by leveraging Fourier-based data transformations and a "weakness allocation" strategy. The empirical results demonstrate a significant improvement in robustness.

Overall, the reviewers appreciate the novelty of the proposed technique and acknowledge its effectiveness as shown by the empirical results. But meanwhile, some major concerns are raised, including 1) the introduced complexity is not well analyzed; 2) the comparisons to adversarial training methods are missing; 3) additional ablations should be included, like generalization to larger model sizes and the sensitivity of frequency thresholds; and 4) the clarity and readability of some sections could be improved.

The authors addressed most of these concerns by providing additional experiments and clarifications in their rebuttal. As a result, three reviewers rated the paper a 6 (i.e., marginally above the acceptance threshold). The only negative reviewer is 2rgn, who did not engage in the discussion. By reading the corresponding rebuttal, the AC believes the concern about computational complexity and the request for additional experiments on larger models are fully addressed. Therefore, the AC recommends accepting the submission.