To Tackle Adversarial Transferability: A Novel Ensemble Training Method with Fourier Transformation
Authors: Wanlin Zhang
Weichen Lin
Ruomin Huang
Shihong Song
Hu Ding
Venue: ICLR 2025 Poster (2025)
Keywords: robustness
diversity
ensemble training
Fourier transformation
Primary area: alignment, fairness, safety, privacy, and societal considerations
TL;DR: A  Fourier transformation based method to enhance adversarial robustness

Abstract: Ensemble methods are commonly used for enhancing robustness in machine learning. However, due to the ''transferability'' of adversarial examples, the performance of an ensemble model can be 
seriously affected even it contains a set of independently trained sub-models. To address this issue, we propose an efficient data transformation method based on a cute  ''weakness allocation'' strategy, to diversify non-robust features.
Our approach relies on a fine-grained analysis on the relation between non-robust features and adversarial attack directions.
Moreover, our approach enjoys several other advantages, e.g., it does  not require any communication between sub-models and the construction complexity is also quite low.
We  conduct a set of  experiments to evaluate the performance of our proposed method and compare it with several popular baselines. The  results suggest that our approach can achieve significantly improved robust accuracy over most existing ensemble methods, and meanwhile preserve high clean accuracy.

Published as a conference paper at ICLR 2025
TO TACKLE ADVERSARIAL TRANSFERABILITY: A NOVEL ENSEMBLE TRAINING METHOD WITH FOURIER TRANSFORMATION
Wanlin Zhang Weichen Lin", Ruomin Huang', Shihong Song', Hu Ding
School of Computer Science and Technology. University of Science and Technology of China School of Artificial Intelligence and Data Science, University of Science and Technology of China SShanghai Innovation Institute *Department of Computer Science, Duke University (ideven, linweichen, shihongsong @mail restc.edu. en ruomin, www.guunneeaaae.eeo.ea duke. .duu, huding@uste. edu. en
ABSTRACT
Ensemble methods are commonly used for enhancing robustness in machine learn- ing. However, due to the "transferabilityy" of adversarial examples, the performance an ensemble model can be seriously affected even it contains set of indepen- dently trained sub-models. To address this issue, we propose an efficient data transformation method based on cute "weakness allocation" strategy, to diversify non-robust features. Our approach relies fine- fineggriieed analysis the relation between non non-robust features and adversarial attack directions. Moreover, our ap- proach enjoys several other advantages, e.g., does require any communication between sub-models and the construction complexity is also quite low. We conduct a set of experiments evaluate the performance of our proposed method and compare it with several popular baselines. The results suggest that our approach can achieve significantly improved robust accuracy over most existing ensemble methods, and meanwhile preserve high clean accuracy.
INTRODUCTION
In the past decade, Deep neural networks (DNNs) have achieved prominent performance on broad range of real-world tasks (Goodfellow et al. 2016). However, number of previous works show that DNNs are susceptible to carefully-crafted manipulations, where the manipulated data are called "adversarial examples" (Szegedy et al. 2014 Zhou et al.) 2018 Heaven 2019). The existence of adversarial examples severely impedes the application of DNNs in security-conscion scenarios, such as self-driving car (Rossolini al. 2023 Zhu al. 2021 l and heath care (Newaz et al. 2020).
The adversarial training approach WWang et al. 2023a Madry et al. 2018 has gained significant attention due to its great effectiveness for defending against adversarial examples. However, the adversarial training approach often necessitates considerably high training time and large training dataset (Gowal et al. 202T; Carmon et al., 2019). Moreover, has been observed that adversarial training is likely to incur certain decline in the accuracy on clean data, which also hinders the trained model be applied for many practical tasks (Tsipras et al.) 2018 Zhang et al.) 2019).
Another important approach to enhance adversarial robustness is ensemble training (Tramer et al. 2018). But recent studies (Yang et al.] 2025) Gao et al.) [022 Waseda et al.) (023) demonstrated that adversarial example can attack different models even they are trained independently, and this phenomenon is the so-called "transferability" of adversarial examples. Hence, the strategy that simply integrates different models trained on the same original dataset is not sufficient to guarantee the overall robustness. To resolve this issue, different approaches have been proposed for maximizing the "diversity" among sub-models; in general, these approaches can be categorized into two classes: "simultaneous training" and "individual training" (Pang et al. 2019).
To reduce the similarity among sub-models, most existing "simultaneous training" methods attempt to incorporate some penalty during each epoch of parameter updates. Kariyappa & Qureshi (019
"ooresponning author.

---

Published as a conference paper at ICLR 2025
proposed the "Gradient Alignment Loss (GAL)" method to minimize the gradient similarity between sub-models directly. Further, Yang et al. (202)) proposed the "Transferability Reduced Smooth (TRS)" method to improve GAL by adding regularization term to increase the smoothness, as the models with smoother loss function can reduce the "transferability" of attacks. Yang et al. (2020) aimed to isolate the adversarial vulnerability in each sub-model by distilling non-robust features, where the sub-models can then generate diverse outputs being resilient against transfer attacks. Despite their effectiveness for defending adversarial attacks, the simultaneous training methods often require substantial amount of memory since all the sub-models need to be stored in the GPUs in the training stage, which could be prohibitive if the number of sub-models is not small (say, more than 10) and/or their sizes are large. Additionally, the information interaction in parallel training can also cause extra large communication cost.
Different from simultaneous training, most "individual training" methods train each sub-model independently on a randomly transformed version of the given training dataset (Pang et al. 2019 AprilPyone & Kiyal 2021). This "random transformation" strategy yields diverse datasets, and thus different sub-models trained on these datasets can present diverse performances when confronting an adversarial attack. The individual training approach has higher flexibilityy and also requires less GPU memory, because the sub-models do not need to be stored simultaneously. Since there is no communication between sub-models, individual training methods are more suitable for parallel training with multiple GPUs. But unfortunately, recent studies showed that the commonly used random transformations (c.g. image cropping and rescaling) are not that effective under adversarial attacks (Athalye et al. 2018). The major cause of suppressing the performance of individual training that the "transferability" problem is still not well addressed.
Our contributions. To tackle the transferability obstacle, we consider developing a new data transformation method for ensemble training. Our main contributions are summarized as follows:
analysis on the relation between non-robust features and adversarial attack directions (Section[]), Being different from the previous analysis on non-robust features, our new analysis provides us the hints that are particularly useful to allocate the potential vulnerability directions to set of sub-models, and therefore paves the way for designing our ensemble training strategy.
First, we propose
Seconn, we propose a data transform framework that can effectively promote the diversity of training data for robust ensemble training. The framework consists of two steps: "frequency selection" and "frequency transformation", where the frequency is based on the Fourier transformation on the images. We propose two efficient frequency transformations with low complexities on the identified non- robust features. The first one is based on simple random noise, and the second one is a cute "targeted attack transformation" that can modify the non-robust features more effectively (Section (.2).
- -Finally, we conduct a set of experiments to evaluate the adversarial robustness of our approach on several benchmark datasets under the widely used attack algorithms. We also compare our approach with several open-source ensemble methods, such as ADP (Pang et al., 2019), GAL (Kariyappa & Qureshi 2019), DVERGE (Yang al., 2020)) and TRS (Yang et al. 2021). Compared with those baselines, the experimental results suggest that our proposed approach can significantly outperform most of them in robust accuracy and also preserve comparable high clean accuracy.
1.1 OTHER RELATED WORKS
Data transformation for ensemble training. Guo al. (2018) and Raff al. (011 proposed the transformations that preserve semantic information to reduce the impact of adversaria perturbation. AprilPyonee & Kiyal (02T) developed a training method that employs block-wise data transformations. where the input image is partitioned into blocks based on some private key. LINAC (Rusu et al.) 2022) uses a predetermined random seed (private key) to initialize and train a DNN to encode the input data, serving an encrypted input transformation.
Adversarial attack from frequency perspective. Waang et al. (020) explained that the model's vuinerability to small distortions may be due to its dependence on high-frequency features. Yucel et al. 2023) proposed a data augmentation method that reduces the reliance on high-frequency components, to improve model's robustness while maintaining clean accuracy. Maaya et al.) (2021) and Bernhard et al. (202I) respectively showed that to fully understand the vulnerability, we should consider the distribution of the entire dataset with high and low frequencies.
2

---

Published as a conference paper at ICLR 2025
2 PRELIMINARIES
Some notations. We consider the k-classification task: X y where X' is the input data space and y = 21,2.... k) is the set of labels. A soft-classification model f(;; maps each a: E X to vector [ax; (9) E TR. where / is the parameter vector that needs to be trained. Its associated hard-classification model is () = arg max;[f(x; ()], where [:], stands for the i-th coordinate. The model f is usually equipped with a loss function ((f(x; (3) y). I E X and y E V which is differentiable on 3 (e.g., cross-entropy loss). We refer to the accuracy on the original dataset as "clean accuracy" and the accuracy on adversarial examples as "robust accuracy". We denote the one-hot k-dimensional vector that corresponds the target label y h(y).
Definition 2.1 (Ensemble Model) Let M
(i....,m) be set of sub-models for k- classification task. We build the ensemble model with the following function:
JE(m; 911M)
Σ
Fm(
M
ME(M]]
where Bi: {88m | 1 m }) and Fm (x; Bm) is the one-hot k-dimensional vector of the hard-classifient model Fm (2; Bm) of fm-
Definition 2.2 (Adversarial Attack and Targeted Attack) Given a model f(: and an input (x.y) E X x V. the adversarial attack algorithm A returns perturbed data x' inside the lp ball of radius > O, which maximizes the loss function {(f(;) () ) or minimizes the loss function (:;(::)).))) 3t) if given a target label y1 F y. For the latter one, we say it is "targeted attack from to y/:" Usually we set
2 or XC.
As mentioned in Section [T] because our proposed approach based on Fourier transform, we introduce several necessary notations below. Given an image x of size L x N, the corresponding two- dimensional discrete Fourier transform can be written as: for any S u S L L- and 0 S v S N 1,
L-1N-1 ΣΣ ==0 t=
T Tuu uj
ris,t]
25 (z+))
(2)
where "J" denotes the imaginary unit, and "ā(u, v)" is the entry in the w-th column and v-th row of the Fourier matrix ("x[s, tJ" is defined similarly for the original image x). The pixels of the image form the time domain, and the entries of ē form the frequency domain. For a frequency (u, v). the amplitude is the absolute value |ā(u, v)]. We call frequency (u, v) as a frequency feature.
3 FINE-GRAINED ANALYSIS ON ENSEMBLE MODEL VULNERABILITY
The previous work (Ilyas al. 2019) categorizes the features learned by a model into robust and non-robust features. It shows that adversarial vulnerability is natural consequence of the presence of highly predictive but non-robust features. Moreover, different models trained on the same dataset often have similar non-robust features, and therefore adversarial example usually exhibits the "transferability" property among them. Several other works also presented detailed discussions on the impact of non-robust features (Benz et al.) [2021] Springer et al. (021)) Following those studies, natural idea for tackling the transferabilityy issue is to ensure that the sub-models should have diverse non-robust features. In this section, provide fine- -re-gga analysis on the vulnerability of ensemble models and then conclude two important hints for achieving this "diversity" goal.
The following definitions inspired by (yyas al. 2019). Note that different from the term "feature" " used in their article, we use "feature extractor" instead in our paper, since "feature" will be particularly used for referring image feature in time or frequency domain. Specifically, we define a "feature extractor" as a function that maps the input X' to vector in R A model f composed of set of different feature extractors, with each feature extractor focusing on distinct feature. The combination of outputs from these feature extractors forms the model's final output. We then further define the "useful feature extractors"..
Definition 3.1 (Useful feature extractor) For given data distribution D X x 3, feature
extractor : X + R is useful, if we have
E E(g [h(v) T 0(x))
>
(3)
k

---

Published as a conference paper at ICLR 2025
Recall that h(y) is the one-hot k-dimensional vector of the label y. Roughly speaking, the inequality (3) implies that the expected contribution of a useful feature extractor to the model's 's correct prediction higher than the average contribution over all the & classes.
Definition 3.2 (robust and non-robust feature extractor) We use A(x) to denote the adversarial example of a data item x as described in Definition 2.2 Let 0 be usefuul feature extractor. (1) We say 0 is robust if the following condition holds for any (1 S i S k):
E(E.)) ((((x))) k
where Dj represents the i-th class data. We denote the set of these robust feature extractors as R-
(2) The remaining useful feature extructors are non-robust. We assign these non-robust extractors to k(k- 1) sets: {0r, 115 i + j S k} as follows. Initially, all these k(k - 1) sets are empty. Then we go through all the non-robust feature extractors. For each non-robust 6, there must exist at least an index "i" such that
E (ea))=,(((A(())) S 1/k:
we j j arg max, ()) [((A()))) and assign 0 On,5 (note that should be not equal to i, or there are multiple indices achieving the maximum expectation and at least one is not equal to
i, since otherwise Es=1 B(E [0(A()))) will less than 1). Eventually, these k(k - 1) sets are constructed, where each Oi,5 contains the feature extractors that are not robust to the attack from to
Remark 3.3 Intuitively, if a feature extractor is robust, it should have the capability to preserve its contribution to the correct prediction even under perturbation. It is also worth noting that non-robust feature extractor 0 could be assigned to multiple Or.5s.
Assume we have standardly trained model f consisting of a set of useful feature extractors, and we denote it as ey Each of them can be classified as robust non-robust as Definition 3.2 Similar with the formulation proposed in (Ilyas et al. (011)), we can represent the model as
((x) = Σ wo)(x) +
Σ
Σ
wg)(x),
BE
(J=1,) ne,
where each f has a coefficient wy E R. We then conduct our analysis based on Equation (4. Some recent works reveal that adversarial training method can obtain robust model through reducing the dependence on non-robust feature extractors (Allen-Zhu & Li 2022 Tsipras al.) 2018). However, this strategy may cause certain downgrade performance on clean accuracy (because the non-robust feature extractors also contribute to obtaining correct prediction). Fortunately, this dilemma in the context of ensemble training. Namely, we just need to keep the non-robust features as diverse as possible, instead of entirely eliminating the dependence robust feature extractors. To pave the way for realizing this goal, we introduce the definition of vulnerability of ensemble model bellow.
are able to avoid
those
Definition 3.4 (Vulnerability of ensemble model) Suppose fE is an ensemble model described
in Definition 2.1 I and its associated clasiclassftication model is denoted by Fg: Wx, Fg(x) = Given the data distribution D X x Y, the vulnerability of FE. is defined as:
arg max: Fa x
Vr(Fg) = B I1 Fg(x)
FE(A(z)) + =v}..
(5)
where I[()) represents the indicator funccinoo,unnciio. Furthermore, for any target class yt, we can define the vulnerability towards yt as Vr(Fg. ((.FF...) -.)FF. [It [[FE(()=y Fe(A()))=
The vulnerability of Definition 3.4 describes the success probability of an attack A the ensemble
model Pg. We have the following key inequality, which indicates that Vr is bounded by considering all the attack directions, i.e..
Vi(FE) < Σ Vi(FF. vt).
(6)
B.EY
4

---

Published as a conference paper at ICLR 2025
The proof of Inequality O is placed in Appendix A.1
Moreover, if FE(A(x)) ye. there at least M/k sub-models returning the wrong label Yr due to the pigeonhole principle. Namely, "Em= ([]f(((())), < fm (A(x))),.)
> M should be necessary condition for successfully
attacking from y to yt- So implies
M Σ
I(IFm(A()))) < (f(()))) M
Vr(FE, 96) I ( I
(7)
decrease the total vulnerability by reducing Vr(FE, ye) for each yt- Also, from we know that Vr(FE, can be reduced by decreasing the chance of
From the upper bound 66
"Lfm (A(r))) < [f (
((x)))l over {1,2, .M.. According the Equation (4) the
inequality [Im (A(x)))
(A(r)) can be rewritten
1Σ [wo((A(r))+ eE
E
Σ woo [woO(A(x)]] < <IΣ (
E E
Eun0(AAx))] (8)
i.f1.i B
WJ=1, BEB!"
where 0g and 6 respectively denote the sets of robust and non-robust feature extractors for the sub-model fm. Moreover, the set O should have relatively larger influence to the right-hand side 4.g.
of ( than other feature extractor set 0% with j # yt. due to the outer operator "[H,," Therefore, we conclude our first hint as an intuition for reducing Vr(Fe).
Hint (i): To decrease the vulnerability in the attack direction yt (i.e., each term Vr(FF, Ue) in the upper bound of (0), it is reasonable to decrease the influence from the non-robust feature extractors of m 3.y="
In Hint (i), major difference from the previous analysis (Ilyas et 2019 Allen-Zhu & Li 2022) that, we in particular relate each attack direction y/: some specific non-robust feature extractors, where the benefit is that these correspondences can effectively help us to build the diverse ensemble model. Moreover, According to the principle of ensemble methods, as long at least M/2 + 1 sub-models are not successfully attacked, the ensemble model will successfully defend against the attack. So we conclude the second hint that is also important for designing our approach.
Hint (ii): For each attack direction yts we only need to consider manipulating the training data of M/2 + sub-models instead of all the M sub-models.
Overall, the above Hint (i) & (ii) play the key roles for inspiring our data transformation method in Section 4
4 OUR ENSEMBLE TRAINING METHOD
We first introduce our model and high-level idea in Section and then elaborate on the technical details for the data transformations in Section 4.2
4.1 OVERVIEW OF OUR FRAMEWORK
Note that the feature extractors of model depend on the given training data. Namely, any modification on the features of the training data can implicitly influence the model. Thus, in this section we follow the Hint (i) & (ii) of Section 3 to design an effective data transformation method. The transformation expected to modify the features of the training data, so to enhance the robustness of the trained ensemble model. We train a set of distinct sub-models on the transformed training data; these sub-models can be integrated into an ensemble model being robust against adversarial attacks, while preserving the clean accuracy of each sub-model as much possible. We use "Tm to denote the transformation for the m-th sub-model, S m < M, and formulate the following problem by slightly modifying Definition 2.1 (replace x by the adversarial example A(x)) for each sub-model):
1
min E B(zgg
Σ (A(2)) M
mE[M]
where 3m is obtained by training on the transformed data, iee. 8m argminEE (xg (xx) (ffm (zm(z); 8) for each m 21.2.....
5

---

Following our frequency selection, we propose two transformation methods for promoting the diversity by using the identified non-robust fea- tures. Our first approach is from a straight- forward idea, which is just to replace the non- robust features by random noise (due to the space limit, we leave the details to Appendix Published as a conference paper at ICLR 2025
The major challenge for solving the above problem ( is how to design a set of appropriate trans- formations {Tm I m }, so that the obtained parameters (Л:M]] can yield sufficiently diverse sub-models. To address this issue, we leverage the transformation from frequency domain to guide the non-robust features of each sub-model to be as diverse as possible. Specifically, we introduce a method called "Frequency Domain Transformation (FDT)" for constructing the set of diverse training datasets { (2)), (2)) xM (2)). FDT relies on a key "weakness allocation" strategy. Roughly speaking, the strategy aims promote the diversity of the constructed datasets, and meanwhile preserve that the overall clean accuracy should not be sacrificed in the ensemble. The details are presented in the next section.
4.2 FREQUENCY DOMAIN TRANSFORMATION
Before performing the transformation, a simple observation is that an image feature is usually invariant under spatial translation, e.g., it can appear at different positions in images. This property causes the challenge for directly identifying and representing non-robust features in time domain. Thus we turn our attention to the frequency domain. Moreover, some previous studies on robust learning already revealed that robust and non-robust features are often deeply related to frequency domain [Wang et al. 2020: Bernhard et al. 202T
need to select set of non-robust features. In time domain,
Maiya al. 202I).
Amplitude based selection. To identify the non-robust frequencies, straightforward idea is to test the robustness of each individual frequency and select the non-robust ones. Nevertheless, it may takes a large computational cost since the number of frequencies is high (e.g.. if the input image is 64 x 64, the number of frequencies is also 64 64 4 10²). We propose an easy-to-ing tomyl-e-mmmleeentt selection idea based the amplitudes, since the amplitudes can be directly obtained via the Fourier transformation with low complexity. According to the previous research (Ilyas et al. 2019 Benz et al. 2021: Springer et al. (021I, feature can be regarded as "robust" if it cannot be easily manipulated by small perturbations. We observe that high-amplitude frequency features usually dominate the ground truth of image. Figure 4 in our Appendix illustrates example to show that, if we keep high-amplitude frequencies and remove low-amplitude ones, the image is changed slightly even with adding certain noise (i.e., we can still recognize the ground truth from the modified image). This observation suggests that high-amplitude frequency features are more strongly related to the semantic information of image. So in our following approach, we maintain high-amplitude frequency features as "robust features", and select the frequencies with low amplitudes (by setting threshold "r") to transform. Moreover, we can conveniently observe the performance changing through varying the threshold r in our experiment. Figure illustrates the amplitude-base based selection.
Amplitude bused selection
Image in frequency domain
Frequency selected
This method is very easy implement in practice. Though it can achieve certain degree of improvement upon previous ensemble train- ing methods, the performance is still not very promising (as shown in our experiments). To further improve the effectiveness, we propose more sophisticated approach called "targeted- attack transformation", which constructs a set of different "substitute" features through attack- ing the images to different targeted classes, and then use them to replace the selected non-robust frequencies.
Figure 1: We use a 5x5 image as a toy example, where the intensity of the color indicates the mag- nitude of the amplitude. In our amplitude-bare selection, we retain the highhamplttude frequencies (i.e., the darker regions) and perform data transfor- mations on the implitttt frequencies (i.e.. the white regions).
Targeted-attes -ttackk transformation: We briefly explain our intuition first. It was shown that adversarial attacks have the capability to manipulate non-robust features (Myas et al. 2019 Yang et al., 2020). In particular, targeted attack as introduced in Definition 2.2] aims modifying non-robust features that are associated with a specific target label. For instance, let us consider a data point (z, y) in the original dataset X' x Y: we set the target label yr and obtain the corresponding adversarial example

---

Published as a conference paper at ICLR 2025
r' (x' contains the modified non-robust features that are associated with yr). When training a model using (a', intuitively it can be viewed as an "immunization" " for defending the attack from y to y:s and consequently, the chance that obtaining the wrong label yt for the data with label y decreases. In other words, it becomes more difficult to attack the images with label to ye than to the other classes. "substitute" feature derived by the targeted attack.
We call the modified non-robust feature
Motivated by this observation, we can construct different transformations by using k x (A - 1) targeted attacks (since each label can be attacked to be the other A: labels); these attacks can yield different substitute features, and then we use these features to replace the corresponding non-robust features in the original dataset (based on Hint (i) in Section 3 finally, the M transformed datasets are obtained via an allocation algorithm, where each substitute feature is captured by at least M/2 + datasets (based on Hint (ii) Section 3). Overall, due to the completeness of the k (k - 1) targeted attacks, the M sub-models trained on those datasets can guarantee the robustness of the final ensemble solution. We introduce some definitions for our transformation first.
Definition 4.1 (Strengthen a dataset) Let y1 + 1/2 E V. If a given training dataset P contains at least one adversarial example who has the original label 1/1 but is misclassified as 1/2, say that P has been strengthened by the attack direction from y1 to y/ (" (""1V2 directio"" for short).
In other words, if P is not strengthened in f1y2-direction the model trained on P is more likely to be fragile to the targeted attacks from y1 y/2 Also, the dataset P may have not been strengthened in multiple different directions. So we define its "weakness set" W = (y1g 1 & vn, U2 S k, y1 of y2, and P has not been strengthened in y1 y1y2- -.z-dirrct.
Definition 4.2 (Diversity of weakness sets) Given M datasets (PI, P2.... sponding weakness sets (Wi, W2.... War,, we define their diversity:
PM3 with their corre-
W n N2 n -WWM Dio(Pi,P2 ... 1M)=1- max max[[W]] Wal..... Warl
is easy to see that the higher the value Div(PI, Pz. ....................... PM). the more diverse the corresponding weakness sets. A higher diversity suggests that the vulnerabilities of the M sub-models trained on those datasets are more likely to be different. To achieve a nice performance in terms of both accuracy and robustness, we need to take account of the diversity function "Div" for designing the transformations. The basic principle is:
On the one hand, our transformed datasets should have sufficiently large number of diverse substitute features, so that one adversarial attack cannot easily capture more than half of the M sub-models. On the other hand, the datasets should also maintain the major information of the original imput as much as possible, since otherwise the clean accuracy may decline due to the added substitute features.
To provide an appropriate trade-off, we propose the following constrained optimization objective: let C be the set of all the (1M/2) combinations of [M/2]] -size subsets from {1, ... .M, and then [in [M2), ... |Warl]
(10)
Pa Pa
S.L. V(in ... 3/2//11 EC
Dio(Pa,PPr
P =1
(11)
We maximize the objective function of (TO) because we want minimize the modification degree for each transformed dataset. Intuitively, a large weakness set indicates that the corresponding dataset not changed significantly by the transformation, and thus the clean accuracy likely to be well preserved. The constraint (11) guarantees that any [M/2] datasets have the intersection W,, n W., n... n Warayyen = 0. that is, they do not share any common direction. Consequently, the ensemble solution should be robust to any attack direction. To achieve this twofold goal, we design an efficient allocation strategy together with an attack-guided transformation on the training data. Specifically, the procedure consists of the following two stages.
Stage (1): allocating the weakness sets to the sub-models. For each yi - directionn S 3/, y22 S k. there are at most //2 1 sets that contain this direction (due to the constraint (11p). so the sum
Eis [W/] is no larger than k:K 1) (I A - 1). Therefore, the maximum value of E5q (10 is no larger than
7

---

Published as a conference paper at ICLR 2025
M 1)/M
k(k 1) (1
(12)
based on the pigeonhole principle. We assign the total k(k 1) ( 1) directions (each direction duplicated be M copies) M sets round-robin way, where the number of directions 2
assigned each set no larger than the upper bound 12
Please refer to Figure 2 for example.
1 direction
2 direction
3 direction
13 direction
2 direction
direction
23 dircction
-
direction
direction
direction
weakness set
32 direction
32 dirrccion
Sub-model
Sub-model 2
Sub-model 3
Sub-model 4
Sub-model 5
Figure 2: Assign the attack directions to five sub-models for three-class classification task.
Stage (2): constructing the new datasets. Following the allocation, transform the original dataset, denoted by Pan. to align with the assigned weakness sets for the sub-models correspondingly Using the same notations of Definition 4.2, we denote the mhiting-to-construc dataset for the m-th sub-model as Pm. (which is initialized to be 0), S m < M. First, we divide Poi into k subsets C1 C2, Ck. where each Cj corresponds the label j. for 1 S I k; further, each Cj equally partitioned to 1 disjoint parts {Cj,1, C22.. Cj } at random. For each data (z, j) in Cj.s. we attack from j h (let =i+? mod k) obtain the adversarial perturbation; then we only substitute the low-amplitude frequencies of x with the perturbation, and other frequencies (which have their amplitudes higher than the aforementioned threshold r) remain unchanged. We denote t the new dataset as C2 Finally, we add C²,, to Pm if the h-direction is not in the weakness set
Wm From the construction method of the weakness sets, we know that the h h-direction can appear in at most M weakness sets. So, the set Ch,,, can added to at least [ different P.. Consequently.
completeness for defending the k(k - 1) attack directions can be guaranteed, i.e., the constraint (IT) is satisfied. Figure shows the schematic diagram of the construction process, and the full details are shown in Appendix
G
Class I C2
G
C
P
Cy
Pm
Class Cj :
Original datasct Port
Cx1
C
PM
Class k Cx
CxR-1
G
Partition
Attack
Allocate
Figure 3: A schematic diagram of the construction process. In the allocation stage, each C% is added Pm, if the 7h-direction not in the weakness Wm.. la i mod k.
Remark 4.3 We are aware of some previous robust learning approaches that also depend on data modification Allen-Zhu & L. 2022 Tsipras et al. 2018). But their approaches usually tend to
8

---

Published as a conference paper at ICLR 2025
completely eliminate non-robust features.. Our method is quite different, where the goal is to leverage the carefully selected non-robust features to weaken the transferability among sub-models. For each sub-model, we only modify the non non-robust features corresponding certain directions, rather than all non-robust features, and therefore the modification yields relatively lower impact on clean accuracy. Moreover, we partition each class C₁ into k subsets {C3a,Cg,,,... ,Cik-1) C,,,,'..
features
with each subset being attacked to a specified class. This step eliminates the need to attack each data point across all classes, thereby reducing the computational complexity of constructing the new datasets.
EXPERIMENTS
We conduct our experiments on the widely used image datasets CIFAR-10, CIFAR-100 (Krizhevsky
& Hinton| 2009) and Tiny-mmageNet-200 (Deng et al. 2009). As for the baselines. we reproduce the
existing ensemble models including ADP (Pang et al. 2019). ). GAL (Kariyappa & Qureshi 2019).
DVERGE (Yang et al. 2020), and TRS YYngg et al. 2021), with their released codes and recommended hyperparameter settings. As for our approach, "FDT-random" and "FDT-target" respectively denote the methods utilizing random noise based transformation and target-attack transformation; "FDT- hybrid" represents the method that combines both, that is, we set two frequency selection thresholds Ti and T2 (T1 < T2), and perform random and target-attack transformations on the frequencies less than Ti and the frequencies between Ti and 72. respectively (due to the space limit, more details are shown in Appendix
E Our code will be available at https::/giiubb..o//dweeeeee I23/FDT
We train each sub-model based on ResNet-20 (He al. 2016) and use Adam optimizer (Kingma & Ba 2013) with an initial learning rate of 0.001 for 200 epochs. To further test their performance on neural network with larger scale, we also use WideResNet28- 10 (Zagoruyko & Komodakis 2016) to train the sub-models and the results are placed in our supplement. All the experiments are implemented with PyTorch (Paszke al., 2017) on a single NVIDIA GeForce RTX 3090 with 24GB of memory and 1 ITB of storage. We assess the performance of our models through 5 repeated runs and compute error bars. Utilizing the numpy library, we calculate the standard deviation and subsequently derive the standard error of the mean (SEM).
Varying the number of sub-models. We take the ResNet-20 model trained on CIFAR-IO as an example and test the performance of FDT with different numbers of sub-models in the ensemble. In this experiment, we set the frequency selection threshold TΙ to be 0.2 and T2 to be 0.8. Then we evaluate the performance of FDT-hybrid under FGSM (Madry et al.) 2018)_ PGD (Carlini et al.) 2019), and Auto-ttack (AA) (Croce & Hein| 2020) attack methods with Too perturbations of size 0.02. The results in Table I lindicate that our clean accuracy has relatively smaller change the number increases, while the robust accuracy can be substantially improved from 3 to 20 sub-models.
Table 1: Performance of FDT-hybrid with different sub-model numbers on CIFAR-10.
Sub-model numbers
3
5
8
12
20
Clean accuracy
90.20 = 0.03
90.75+0.03
91.35+00005
91.51
#0.06
91.86+10.77
FGSM (e= 0.02)
58.04 + 0.13
0.15
62.41 + 0.11
63.96
+ 0.12
64.27 + 0.14
PGD (e=0.02)
20.01 + 0.04
26.
10+100000
0.07
29.20
0.05
29.78
+ 0.08
29.71 0.07
AutoAttack (c=0.02)
19.42+ 0.04 25.37+ 0.05 27.33+ 0.04
28.12+ 0.07 28.92 - 0.07
Results for boite-box attack. To maintain consistency with the baseline ensemble methods from the literature, we ensemble three ResNet-20 sub-models here and evaluate the robust accuracy using e 0.01 and 0.02. In this experiment we set the frequency selection threshold T₁ to be 0.2 and To to be 0.8. In the white-box attack setting, the attacker has full knowledge of the models, including model parameters, architecture, and ensemble training strategy. To evaluate the adversarial robustness of the ensemble, we conduct the following white-box attacks: PGD, FGSM, BIM (Goodfellow et al.) 2015), MIM (Dong et al. 2018). C& (Carlini & Wagner (017) and Auto (AA). The attacks implemented using AdverTorch (Ding et al. | (019). We take the robust and clean accuracies, and average training time per epoch as the evaluation metrics.
Table 2 presents the obtained robust accuracies of the baseline ensemble methods on CIFAR-10 and CIFAR- - 100. In addition, we show the average training time per epoch of different ensemble methods. The experimental results suggest that FDT-randon method can achieve higher adversarial robustness over other baselines on both CIFAR- 10 and CIFAR- 100, with the training time only higher than ADP (and much lower than other baselines). Furthermore, the FDT-hhbrrid ensemble method

---

Published as a conference paper at ICLR 2025
Table 2: Robust and Clean Accuracy (%) and average training time of different ensemble methods against white-box attacks on CIFAR-I0 and CIFAR-100, "E" and "X" stand for the Loe norm of the adversarial perturbation and the coefficient of C&W attack respectively. The TRS results are reported in the original paper Yang et al. (202T), with "-" indicating results not provided.
CIFAR-10
ADP
GAL
DVERGE TRS FDT--andom
FrT-tt
FDTT-hybrid
Clean accuracy
91.84
91.81
91.37
89.881 0.02 90.16 + 0.04
90.20# 0.03
59.48
44.97
70.05
66.9611 0.12
72.881 0.12
72.244 0.12
FGSM (==0.02)
53.38
30.58
56.33
44.2
46.28
0.10
55.54 + 0.09
58.04
+ 0.13
PGD (e= 0.01)
14.45
1.35
40.55
50.5
45.42 + 0.09 46.58
0.07
48,48
0.09
PGD (< = 0.02)
2.95
0.34
11.49
15.1
12.24 + 0.03 15.08
0.05 20.01 + 0.04
BIM (<= 0.01)
14.15
1.37
40.51
50.6
45.24
46.86
48.57+ 0.05
BIM (c = =0.22)
3.01
0.27
10.65
15.8
11.68
14.86+
16.63 + 0.02
MIM (<= 0.01)
20.38
2.05
44.74
51.5
47.73=
0.06
51.50 - 0.07
MIM (<= 0.02)
5.11
0.69
14.76
17.2
15.14
18.27
20.09 = 0.03
AA (<= 0.01)
1.80
0.00
43.34
46.09 0.09
48.83=
51.56 = 0.08
AA (<= 0.02)
0.00
0.00
13.72
9.38 = 0.05
15.70 +
19.42 0.04
C&W (x= = 0.1)
20.96
31.57
52.35
58.1
45.01 0.10
0.10 55.48
56.08 = 0.11
CIFAR-100
ADP
GAL
DVERGE
TRS FDT-random
FDT-ttrret
FDTT-hybrid
Clean accuracy
67.04
67.70
66.16
66.29 = 0.11
67.64 + 0.08
66.70 0.09
FGSM (e =0.01)
17.82
16.89
33.94
35.42.1: 0.12
40.4611 0.14
39.85 1 0.14
FGSM (e 0.02)
10.53
7.80
26.61
19.3
22.40 0.05
32.301 0.06
30.27 0.08
PGD (c = 0.01)
0.80
0.11
14.62
23.0
21.54
= 0.06
22.19:0.00
24.93 0.05
PGD (e 0.02)
0.01
0.02
4.25
5.3
4.84
0.03
7.27
7:27:0000 0.02
8.63 0.03
BIM 0.00)
0.68
0.23
14.85
22.9
21.10
= 0.09
22.394001
24.35+ 0.06
BIM (< = =0.02)
0.02
0.0
4.07
5.4
4.80 0.04
6.80+ 0.05
8.40
MIM (c 0.01)
0.78
0.12
16.82
23.4
23.14 = 0.06
24.68
27.09 0.09
MIM (c (002)
0.01
0.02
5.31
6.2
6.47 0.03
8.87 + 0.04
10.19 + 0.05
AA (c= 0.01)
0.01
0.00
11.23
16.02+0.009
16.03
16.41 + 0.12
AA (c= 0.02)
0.00
0.00
2.72
3.12 0.04
4.54 = 0.05
5.47 0.07
C&W (A= = 0.1)
0.74
3.70
10.68
26.9
25.07 = 0.10
29.43 + 0.09
30.66 0.13
Time (8)
ADP GAL
DVERGE
TRS
FDT-random
FDT-target
FDT-hybrid
CIFAR-10
30.15
69.92
134.33
350.42
37.04
108.22 108.43
114.23
CIFAR-100 30.34 69.71
129.25
344.92
37.12
113.87
achieves an even better robustness than FDT-random, though its running time is higher since needs to perform the target-attack transformation.
Summary on other experimental results placed in Appendix F We also conduct the experiments examine the performance of FDT under black-box attack, and assess the transferability of our method across various sub-models. The results indicate the competitive robustness of our method in defending against black-box attacks. Then, we evaluate the trade-off between clean accuracy and robust accuracy by varying the frequency selection threshold r. The result shows that the ensemble model has lower clean accuracy and higher robust accuracy with the increasing of r. Moreover, we included some ablation studies on datasets and model architectures. These experiments demonstrate that our method performs the best among ensemble-bar baseline methods.
CONCLUSION AND FUTURE WORK
In this paper, we present a novel data transformation approach to improve the robustness of ensemble models against adversarial attacks. By leveraging the frequency based features and strategically allocating adversarial examples, we demonstrate the effectiveness of our method in enhancing adversarial robustness while maintaining high accuracy on clean data. As for the future work, we consider other types of transformation methods (e.g.. beyond using frequency) to improve the ensemble robustness. Also, is interesting to consider more complicated scenarios for ensemble training, such as federated learning with concerning the privacy issue.
10

---

Published as a conference paper at ICLR 2025
ACKNOWLEDGMENTS
The authors would like thank the reviewers for their constructive comments and suggestions. This work was partially supported by the National Natural Science Foundation of China (No. 62272432 and No. 62432016), the National Key Research and Development Program of China (No. 100000000), and the Natural Science Foundation of Anhui Province (No. 2208085MF163),
REFERENCES
Zeyuan Allen-Zhu and Yuanzhi Li. Feature purification: How adversarial training performs robust deep learning. In 202/ IEEE 62nd Annual Symposium on Foundations of Computer Science (FOCS), pp. 977-988. IEEE, 2022.
MaungMaung AprilPyone and Hitoshi Kiya. Block-wise image transformation with secret key for adversarially robust defense. IEEE Transactions Information Forensics and Security, 16: 2709-2723, 2021.
Anish Athalye, Nicholas Carlini, and David Wagner. Obfuscated gradients give false sense of security: Circumventing defenses to adversarial examples. International conference on machine learning, pp. 283.283. PMLR, 2018.
Philipp Benz, Chaoning Zhang, and In So Kweon. Batch normalization increases adversarial vulnerability and decreases adversarial transferability: A non-robust feature perspective. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 7818-7827, 2021.
Rémi Berhhard, Pierre-Alain Moëllic, Martial Mermillod, Yannick Bourrier, Romain Cohendet, Miguel Solinas, and Marina Reyboz. Impact of spatial frequency based constraints on adversarial robustness. 2021 International Joint Conference on Neural Networks (IJCNN), pp. 1-8. IEEE, 2021.
Nicholas Carlini and David Wagner. Towards evaluating the robustness of neural networks. In 2017 ieee symposium on security and privacy (sp). pp. 39-57. leee, 2017.
Nicholas Carlini, Anish Athalye, Nicolas Papernot Wieland Brendel, Jonas Rauber, Dimitris Tsipras, Ian Goodfellow, Aleksander Madry, and Alcxey Kurakin. On evaluating adversarial robustness. arXiv preprint arXiv:
2019.
Yair Carmon, Aditi Raghunathan, Ludwig Schmidt, John C Duchi, and Percy Liang. Unlabeled data improves adversarial robustness. Advances in Neural Information Processing Systems, 32, 2019.
Francesco Croce and Matthias Hein. Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks. In International conference on machine learning, pp. 2206-2216. PMLR, 2020.
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition, pp. 248-255. Icee, 2009.
Gavin Weiguang Ding, Luyu Wang, and Xiaomeng Jin. Advertorch v0. An adversarial robustness toolbox based on pytorch. arXiv preprint arXiv:1902.07623, 2019.
Yinpeng Dong, Fangzhou Liao, Tianyu Pang. Hang Su, Jun Zhu, Xiaolin Hu, and Jianguo Li. Boosting adversarial attacks with momentum. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 9185-9193, 2018.
Xitong Gao, Cheng-Zhong Xu, et al. Mora: Improving ensemble robustness evaluation with model reweighing attack. Advances in Neural Information Processing Systems, 35:26955-26965 2022.
Ian J Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. stat, 1050:20, 2015.
11

---

Published as a conference paper at ICLR 2025
Ian J. Goodfellow, Yoshua Bengio, and Aaron C. Courville. Deep Learning. Adaptive computation and machine learning. MIT Press, 2016. ISBN 378-0-262-03561-3.
Sven Gowal, Sylvestre- Alvise Rebuffi, Olivia Wiles, Florian Stimberg, Dan Andrei Calian, and Timothy A Mann. Improving robustness using generated data. Advances in Neural Information Processing Systems, 14::211-42333 2021.
Chuan Guo, Mayank Rana, Moustapha Cissé, and Laurens der Maaten. Countering adversarial images using input transformations. In 6th International Conference on Learning Representations, ICLR 2018, Vancouver, BC, Canada, April 30 May 2018, Conference Track Proceedings. OpenReview.net, 2018.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 770-778, 2016.
Douglas Heaven. Why deep-learning ais are so easy to fool. Nature, 574(7777)1163-16) 2019.
Andrew Iyas, Shibani Santurkar, Dimitris Tsipras, Logan Engstrom, Brandon Tran, and Aleksander Madry. Adversarial examples are not bugs, they features. Advances in neural information processing systems, 32, 2019.
Sanjay Kariyappa and Moinuddin K Qureshi. Improving adversarial robustness of ensembles with diversity training. arXiv e-prints, pp. arXiv-1901, 2019.
Diederik P. Kingma and Jimmy Ba. Adam: method for stochastic optimization. In Yoshua Bengio and Yann LeCun (eds.), 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings, 2015.
Alex Krizhevsky and Geoffrey Hinton. Learning multiple layers of features from tiny images. 2009.
Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. In 6th International Conference on Learning Representations, ICLR 2018, Vancouver, BC, Canada, April 30 May 3, 2018, Conference Track Proceedings. OpenReview.net, 2018
Shishira R. Maiya, Max Ehrlich, Vatsal Agarwal, Ser-Nam Lim, Tom Goldstein, and Abhinav Shrivastava. frequency perspective of adversarial robustness. CoRR, abs/211 00s/2111.0006..1 2021.
AKM Iqtidar Newaz, Nur Imtiazul Haque, Amit Kumar Sikder, Mohammad Ashiqur Rahman, and Selcuk Uluagac. Adversarial attacks to machine learning-based smart healthcare systems. In GLOBECOM 2020-2020 IEEE Global Communications Conference, pp. 1-6. IEEE, 2020.
Tianyu Pang, Kun Xu, Chao Du, Ning Chen, and Jun Zhu. Improving adversarial robustness via promoting ensemble diversity. In International Conference on Machine Learning, pp. 4970-4979. PMLR, 2019.
Adam Paszke, Sam Gross, Soumith Chintala, Gregory Chanan, Edward Yang, Zachary DeVitto, Zeming Lin, Alban Desmaison, Luca Antiga, and Adam Lerer. Automatic differentiation in pytorch. 2017.
Rahul Rade and Seyed- -eyee-Mohsen Moosavi-Dezfooli Helper-based adversarial training: Reducing excessive margin to achieve better accuracy vs. robustness trade-off. ICML 2021 Workshop on Adversarial Machine Learning, 2021.
Edward Raff, Jared Sylvester, Steven Forsyth, and Mark McLean. Barrage of random transforms for adversarially robust defense. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 6528 6537, 2019.
Giulio Rossolini, Federico Nesti, Gianluca D' Amico, Saasha Nair, Alessandro Biondi, and Giorgio Buttazzo. On the real-world adversarial robustness of real-time semantic segmentation models for autonomous driving. IEEE Transactions on Neural Networks and Learning Systems, pp. 1-15, 2023. doi: 10.1 2.0.33355......
12

---

Published as a conference paper at ICLR 2025
Andrei A. Rusu, Dan Andrei Calian, Sven Gowal, and Raia Hadsell. Hindering adversarial attacks with implicit neural representations. Kamalika Chaudhuri, Stefanie Jegelka, Le Song. Csaba Szepesvári, Gang Niu, and Sivan Sabato (eds.), International Conference Machine Learning, ICMI. 2022, 17-23 July 2022, Baltimore, Maryland, USA, volume 162 of Proceedings of Machine Learning Research, pp. 18910-18934 PMLR, 2022.
Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recognition. In Yoshua Bengio and Yann LeCun (eds.), 3rd International Conference Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings, 2015.
James C Spall. Multivariate stochastic approximation using a simultancous perturbation gradient approximation. IEÉE transactions automatic control, 37(3):332-344 341, 1992.
Jacob Springer, Melanie Mitchell, and Garrett Kenyon. A little robustness goes a long way: Lever- aging robust features for targeted transfer attacks. Advances in Neural Information Processing Systems, 34:9759 9773, 2021.
Christian Szegedy, Wojeiech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus. Intriguing properties of neural networks. In 2nd International Conference on Learning Representations, ICLR 2014, 2014.
Florian Tramèr, Alexey Kurakin, Nicolas Papernot, Ian Goodfellow, Dan Boneh, and Patrick Mc- Daniel. Ensemble adversarial training: Attacks and defenses. In International Conference Learning Representations, 2018.
Dimitris Tsipras, Shibani Santurkar, Logan Engstrom, Alexander Tumee, and Aleksander Madry. Ro- bustness may be odds with accuracy. In International Conference on Learning Representations, 2018.
Zekai Wang, Tianyu Pang, Chao Du, Min Lin, Weiwei Liu, and Shuicheng Yan. Better diffusion models further improve adversarial training. In Andreas Krause, Emma Brunskill, Kyunghyun Cho, Barbara Engelhardt, Sivan Sabato, and Jonathan Scarlett (eds.), International Conference Machine Learning, ICML 2023, 23-29 July 2023, Honolulu, Hawai, USA, volume 202 of Proceedings of Machine Learning Research, pp. 36246-362633 PMLR, 2023a.
Zekai Wang, Tianyu Pang, Chao Du, Min Lin, Weiwei Liu, and Shuicheng Yan. Better diffusion models further improve adversarial training. In International Conference on Muchine Learning, pp. 36246-36263. PMLR, 2023b.
Zifan Wang, Yilin Yang, Ankit Shrivastava, Varun Rawal, and Zihao Ding. Towards frequency-base explanation for robust cnn. arXiv preprint arXiv:2005.0m 03110, 2020.
Futa Waseda, Sosuke Nishikawa, Trung-Nghia Le, Huy H Nguyen, and Isao Echizen. Closer look at the transferability of adversarial examples: How they fool different models differently. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1360,1368, 2023.
Sven-Ake Wegner. Lecture notes on high-dimensional data, arXiv preprint arXiv:2101.05841 2021.
Yuancheng Xu, Yanchao Sun, Micah Goldblum, Tom Goldstein, and Furong Huang. Exploring and exploiting decision boundary dynamics for adversarial robustness. In The Eleventh International Conference on Learning Representations ICLR 2023, Kigali, Rwanda, May 1-5, 2023, 2023.
Huanrui Yang, Jingyang Zhang, Hongliang Dong, Nathan Inkawhich, Andrew Gardner, Andrew Touchet, Wesley Wilkes, Heath Berry, and Hai Li. Dverge: diversifying vulnerabilities for enhanced robust generation of ensembles. Advances in Neural Information Processing Systems, 33:5505-5515, 2020.
Ruijie Yang, Yuanfang Guo, Junfu Wang, Jiantao Zhou, and Yunhong Wang. Common knowledge learning for generating transferable adversarial examples. Frontiers Comput. Sci., 19(10);911339, 2025. doi: 1007/01 4744-344-4053344
13

---

Published as a conference paper at ICLR 2025
Zhuolin Yang, Linyi Li, Xiaojun Xu, Shiliang Zuo, Qian Chen, Pan Zhou, Benjamin Rubinstein, Ce Zhang, and Bo Li. Trs: Transferability reduced ensemble via promoting gradient diversity and model smoothness. Advances in Neural Information Processing Systems, 34:17642-17655, 2021.
Mehmet Kerim Yucel, Ramazan Gokberk Cinbis, and Pinar Duygulu. Hybridaugmentt+: Unified fre- quency spectra perturbations for model robustness. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp- 5718-5722 2023.
Sergey Zagoruyko and Nikos Komodakis. Wide residual networks. CoRR, abs/1 0ets/1605.000 076.00116666 2016.
Hongyang Zhang, Yaodong Yu, Jiantao Jiao, Eric P. Xing. Laurent El Ghaoui, and Michael I. Jordan. Theoretically principled trade-off between robustness and accuracy. In Kamalika Chaudhuri and Ruslan Salakhutdinov (eds.), Proceedings of the 36th International Conference on Machine Learning, ICML 2019, 9-15 June 2019, Long Beach, California, USA, volume 97 of Proceedings of Machine Learning Research, pp. 7472-7482. PMLR, 2019.
Wen Zhou, Xin Hou, Yongjun Chen, Mengyun Tang, Xiangqi Huang, Xiang Gan, and Yong Yang. Transferable adversarial perturbations. In Proceedings of the European Conference on Computer Vision (ECCV), pp. 452 467, 2018.
Yi Zhu, Chenglin Miao, Tianhang Zheng, Foad Hajiaghajani, Lu Su, and Chunming Qiao. Can use arbitrary objects to attack lidar perception in autonomous driving? Proceedings of the 2021 ACM SIGSAC Conference Computer and Communications Security, pp. 2021.
14

---

Published as a conference paper at ICLR 2025
A OMITTED PROOFS
A.1 PROOF OF INEQUALITY (6
Eyey Vi(FE,
Emny (I [[ - =yA - [t
= =Ewey E(Ey)E P(a.v) [(Fe(c)= = A FF(((x)) = y1
Then, we interchange the order of summation, and so the above equation is equal to
EOowED P(r.v) Emey [FF((()) = y FE(A(z)) = y6
= Ba,,u) (( = FF = y1}
For each (I, v). without loss of generality, let FE(((x)) = yo- For y * yo. T{ (F(()) = y A
FB(A(z)) = } O. For ye = yo. F(F(()) = A Fe(A(x)) vo} = I(Fe(()) = y}- So the above equation is equal to
E ((a)) [(F((z) =}}
E ( [[ [(FE(x)= S (FE(A(E)) FE(A(z)) v)}]
EANDD [(FF(()) SΛ (E(((())) + + +(F((()) A FE(A(x)) = =y}]
We split I((-) because FE(A(z)) # and FE(A(z)) equation is equal to
mutually exclusive. Then, the above
Vr(Fa) + E(.a)~D F(F((t) y FE(A()) =
2 Vr((F))
Overall, we can obtain the inequality (6): EyEy Vr VI(FE, Vr(FB)
B FREQUENCY SELECTION
Figure illustrates an example to show that, if we keep high-amplitude frequencies and remove low-amplitude ones, the image is changed slightly even with adding certain noise (i.e., we can still recognize the ground truth from the modified image). On the other hand, we keep the low- amplitude frequencies only, the semantic information almost missing. This observation suggests that high-amplitude frequency features are more strongly related to the semantic information of image.
C RANDOM NOISE BASED TRANSFORMATION
Random noise based transformation: This approach substitutes the identified non-robust frequen- cies with Gaussian noise. For an N N image, we take the non-robust frequencies based on the pre-specified threshold r. and replace them with random vector for each sub-model in our experiment. In particular, further increase the randomness, we perform this transformation for each epoch the training stage. If we select the top non-robust frequencies, the overall dimensionality of the edited random feature should be X (we concatenate those s-dimensional features together), where E is the number of epochs. For example, if N 32, N /2, and E 200, the overall dimensionality can be as large as 10S. Because these M features are random and have high dimensions, they are very likely to be nearly orthogonal with each other (this phenomenon in dighedimensional geometry can be proved by the central limit theorem (Wegner 202I)). As consequence, they tend to yield diverse training results for the sub-models.
15

---

Published as a conference paper at ICLR 2025
Original image
20% changed
50% changed
80% changed
Figure 4: The first and second rows are the figures by adding random noise to hish-amplttuee and low-amplitude frequencies, respectively. "20% changed" for the first row means we remove the 20% lowest-amplitudes frequencies, and add small noise to the remaining high-amplitude frequencies. "20% changed" for the second means remove the 20% highest-amplitude frequencies, and add small noise to the remaining low- -mmmliiie frequencies. "50% changed" and "80% changed" follow the same procedure "20% changed".
The implementation details are as follows. Giver an image x, we perform Fourier Transform on and also generated Gaussian noise ng- Then, we can obtain the low-amplitude frequencies and high-amplitude frequencies of x by setting an amplitude threshold. Next, we generate two masks (M1 and M2) to select high-amplitude frequencies and low-amplitude frequencies. We add the low-amplitude frequencies of no (i.e., M2(no)) to high-amplitude frequencies of x (i.e., M1 (x)), and obtain the transformation of (denoted TT))). Finally, we transform x (Ix) time domain by inverse Fourier transform and train the model with T (z).
D ALGORITHM OF FDT
Algorithm shows the overall framework of training ensemble model with FDT. It illustrates that our data transformation is performed at each iteration.
Algorithm Training ensemble model with FDT
Input: dataset X' x V, the number of sub-models M, and the epoch number E, Output: sub-model B1. B2, .BM
for i = 1 to E do
Run Fargeted-attade Transformation and obtain P1, P2, .. PM
for j j= to M do
train 3, on Ps end for end for
Algorithm 2 shows the details of targeted-attadt transformation method on the whole dataset. For each specific image z, we obtain the targeted class according to the allocation scheme mentioned in "Stage (1)". Then, we use targeted PGD attack to obtain the adversarial sample x'. After that, we perform Fourier Transform on high-amplitude frequencies of x by setting an amplitude threshold. Next, we generate two masks (M₂ and M2) to select high-an high-amplitude frequencies and low-anpli frequencies. We add the low-amplitude frequencies of x' (i.e., My (x') )) to high-amplitude frequencies of x (i.e., MM(x)), and obtain the transformation of (denoted as π (x)).. Finally, we transform (xx to time domain by inverse Fourier transform.
x'. and we can obtain the low-amplitude frequencies and
16

---

Published as a conference paper at ICLR 2025
Algorithm 2 Targeted-attack Transformation
Input:
dataset Pori, number M, steps s, class number k
Output:
Transformed data Pt, ......,,. PM
Divide the dataset Por: into k parts {C1. Cz. Cx} according to labels
Randomly partition the dataset Cj equally into disjoint & - parts {C Cja, C2,a-1}
Initialize P1, P2.... PM with empty set;
m <-000 0
for j = 1 to k do
for 1 to k k- do
Cj f calculate targeted attack example in C3. with label i + mod and perform data transformation each image;
for s 1 1 to A
do
m 4- +1 mod M;
Append C² 5,ز to P.i
end for end for end for
E IMPLEMENT
In this section, we provide more experimental details. our work, we utilize the CIFAR-10 (Krizhevsky & Hinton 2009), CIFAR- 100 (Krihevssky & Hinton| 2009). and 2en-
(Deng et al. (2009). In the testing process, the primary reason for selecting FGSM (Madry et al.) 2018). PGD (Carlini et al. 2019), BIM (Goodfellow et al. 2015), MIM (Dong al. 2018]. CW(Carlim &
Wagner 2017) as attack methods is to keep consistent with the baseline methods from the literature. Further, select AA (Croce & Hein 2020) because it is also a popular attack method and more
powerful than those base methods. To reduce the computational complexity of targeted attacks, we leverage the transferability of adversarial examples and utilize pre-trained simple network (VGG11 (Smmonyan & Zisserman, 2015) structure for targeted attacks.
Further, we introduce the implement of "FDT-random", "FDT-target" and "FDT-hybrid" here. For "FDT-random", we perform Fourier Transform on x and also on randomly sampled standard Gaus- sian noise 7lg- Then, we can obtain the low-amplitude frequencies and high-amplitude frequencies of x by setting an amplitude threshold. Next, we generate two masks (M1 and M2) to select high- amplitude frequencies and oow-ampittude frequencies. We add the low-amplitude frequencies of no (i.e., M2(no) )) to high-amplitude frequencies of x (i.e., M (z)), and obtain the transformation of r (denoted as π (x)). Finally, we transform (2)) to time domain by inverse Fourier transform and train the model with π (z). For "FDT-target", scheme mentioned in "Stage (1)". Then, we use targeted PGD attack to obtain the adversarial sample x'. After that, perform the same steps as with FDT-random (we substitute no with x'). For FDTT-hbrrd, we set two frequency selection thresholds 7i and 72 (7) < T2), and generate three masks to select the frequencies: M, for the amplituit frequencies (amplitude T2 ), M2 for the middle part (Ti < amplitude < ), and M3 for the small part (amplitude < 71). Next, we combine M₁ (x),, M2(x') and (n)) obtain the transformation (2x).
obtain the targeted class according to the allocation
F ADDITIONAL EXPERIMENTAL RESULTS
In this section, we provide more experimental results. Firstly. we extend our experiments to SVHN, Tiny-mmageNet-200. and WideResNet-28- 10 in Appendix F. We also conduct the ablation studies on weakness set allocation method, amplitude-basee based selection threshold and model architecture in Appendix F.I Then, we evaluate the performance of FDT under black-box attacks on the CIFAR- 10 and CIFAR- in Appendix F.2 Then we present the trade-off between clean accuracy and robust accuracy on the CIFAR- 100 using FDT method in Appendix F3 This trade-off sheds light on the effectiveness of FDT with changing the trade-off parameter. Additionally, in Appendix F44 compare the transferability across various sub-mut mobelsllss with the baseline methods. Furthermore, we compare method with more related methods in Appendix E5
17

---

Published as a conference paper at ICLR 2025
F.1 ABLATION STUDIES
In this section, we extend our experiments to additional datasets (SVHN, iny-ImageNet-2000 and architecture (WideResNet-28-1U) We also explore the ablation studies on weakness set allocation method, amplindde-basseee baste selection threshold and model architecture.
Table 3 presents the performance of ensemble methods trained with ResNet-20 on SVHN against several widely used white-box attacks. The experimental results demonstrate that all ensemble models achieve comparable levels of clean accuracy. Specifically, the FDT approach exhibits better robust accuracy than the other methods. These observations highlight the effectiveness of FDT in achieving favorable clean accuracy and robustness of ensemble models.
Table 3: Robust Accuracy (%) of different ensemble methods against white-box attacks on SVHN. The and stand for the lox norm the adversarial perturbation and the coefficient of C& attack respectively. The last column is the ensemble model trained with FDT-hybbrid
SVHN
ADP
GAL
DVERGE
TRS
FDTT-hybrid
clean accuracy
96.83
94.66
96.28
94.52
96.73 + 0.12
FGSM (e=0.01)
84.38
80.2
85.6
72.87
90.13 0.09
FGSM (c=0.02)
78.08
41.5
81.4
53.9
86.78 0.07
PGD (<= 0000)
51.01
50.1
53.31
54,43
59.42 + 0.07
PGD 0.022)
17.74
8.24
17.42
18.86
22.74 0.04
BIM (<= 0.01)
54.38
47.73
52.08
53.71
57.91 0.08
BIM (e = 0.02)
21.26
8.1
14.58
18.05
20.23
0.05
MIM (c= 0.01)
61.24
51.96
58.51
56.32
62.14 0.08
MIM (== 0.02)
24.84
5.14
23.22
21,95
25.37 + 0.04
AÁ (c= 0.01)
49.92
48.39
52.02
52.83
57.54 0.09
AA (e= 0.02)
16.13
6.90
16.95
17.48
20.12 + 0.05
C&W 0.10
55.81
49.94
66.82
52.74
72.14 0.11
We also extend our experiment to the sub-models trained with WideResNet-28-10 10 on CIFAR-10. Table4[show> shows the performance of the models facing various whitebox attacks. The results indicate that FDT maintains good performance even on more complex network structures. We also evaluated the robustness of ensemble of eight sub-models, with the results presented in Table 5
Table 4: Robust Accuracy (%) of different ensemble methods against white-box attacks on CIFAR- 10. The and À stand for the loo norm of the adversarial perturbation and the coefficient of C& W attack respectively. The architecture of sub-model is WRN-28-10 10.
CIFAR-10
ADP
GAL
DVERGE
FDT-hybrid
clean accuracy
92.99
82.14
94.32
94.18 0.06
FGSM (e=0.01)
60.04
44.94
71.01
80.64 0.05
FGSM (==0.02)
51.69
36.83
50.43
60.09 + 0.05
PGD (== 0.01)
11.09
22.10
44.25
64.64
0.07
PGD (e =0.22)
2.54
5.06
13.27
26.0 + 0.03
BIM (c= 0.01)
15.81
22.62
46.53
67.36
0.10
BIM (c =0.02)
4.50
5.43
17.38
32.36 + 0.06
MIM (c= 0.01)
18.18
25.97
44.21
64.36 + 0.08
MIM (c= 0.02)
4.72
7.81
12.83
25.64 0.05
AA (c= 0.01)
9.38
19.34
43.23
63.45 0.08
AA (c= 0.02)
1.17
3.93
12.49
25.23 0.04
C&W (A=0.1)
37.81
19.05
46.32
47.23 0.10
Table is is the result of ensemble methods trained with WideResNet-28-- on Tiny-ImageNet-2000. We test the robustness of different methods under widely used white-box attacks. Due the high
18

---

Published as a conference paper at ICLR 2025
Table 5: Robust Accuracy (%) of ensemble of eight sub-models against white-box attacks CIFAR-10. The and À stand for the loe norm of the adversarial perturbation and the coefficient of C& W attack respectively. The architecture of sub-model is WRN-28-10.
CIFAR-10
FDT-hybrid
clean accuracy
93.72 : 0.11
FGSM (c=0.01)
86.31 0.07
FGSM (==0.02
67.29 + 0.06
PGD (e= 0.01)
72.02 = 0.07
PGD (e = 00022
45.42+ 0.05
BIM (e= 0.01)
73.68 = 0.10
BIM (c = 0.02)
44.53+
MIM (== 0.01)
71.36 + 0.06
MIM (c= 0.02)
45.24 - 0.06
AA (E= 0.01)
70.45 + 0.08
AA 0.02)
44.23 = 0.07
C&W (A 0.1
72.37 + 0.11
time complexity of the TRS, we do not compare with it here. The experimental results show that all ensemble models achieve comparable levels clean accuracy while FDT-hybbrd achieves better robust accuracy than other methods.
Table 6: Robust Accuracy (%) of different ensemble methods against white-box attacks on Tiny- ImageNet-200. The and À stand for the 1x0 norm of the adversarial perturbation and the coefficient of C&&W attack respectively. The last column is the ensemble model trained with FDT-hybrid.
Iiny-ImageNet-200
ADP
GAL
DVERGE
FDT-hybrid
clean accuracy
49.88
45.7
51,46
64.21 + 0.06
FGSM (c 0.01)
10.46
1.24
22.82
21.73 + 0.04
FGSM (c 0.02)
4.38
0.59
18.42
19.28 0.04
PGD 0.01)
0.02
0.02
3.6
4.76 = 0.02
PGD (c = 0.02)
0.02
0.01
0.34
0.45 : 0.01
BIM (E 0.01)
0.07
0.02
3.35
4.81 = 0.03
BIM (c 0.02)
0.03
0.01
0.28
0.32::0000
MIM (E 0.01)
0.11
0.02
4.36
6.13 1 0.03
MIN (E 0.02)
0.03
0.01
0.41
0.48 + 0.00
AA (E 0.01)
0
0
0
2.66 + 0.02
AA 0.02)
0
0
0
0.02 0.00
CW (A 0.01)
2.36
0.13
9.54
19.47 0.06
Ablation study on model architectures. Table 7 presents the results across different model architec- tures, including ResNet20, ResNet50, WRN28-10 and WRN34-10. While larger models generally achieve higher clean and robust accuracy, the results suggest that our method consistently enhances robustness under various attack scenarios, demonstrating its applicability across diverse architectures.
Ablation study on allocation methods. Table 8 compares the performance of FDTT-hybrid with different weakness set allocation methods on CTFAR-10. The results indicate that our proposed allocation method achieves better clean accuracy and robustness under various attack scenarios than randomly uniform allocation.
Ablation study 7i and 72- Table 9 presents the results of FDT-hybrid with various combinations of selection thresholds Tı and 72 on CIFAR-10. The experiments reveal the impact of different thresholds on both clean accuracy and robustness under adversarial attacks. As 72 increases, robustness improves across all metrics, but clean accuracy decreases. For a fixed T2, increasing т generally leads to trade-off between clean accuracy and robustness. Setting 0.2 and 72 0.8 achieves relatively
19

---

Published as a conference paper at ICLR 2025
balanced performance, maintaining both competitive clean accuracy and robust accuracy under various attacks.
Table Robust Accuracy (%) of different model architectures against white-box attacks on Cifar10. The and À stand for the lor norm of the adversarial perturbation and the coefficient of C&W W attack respectively.
CIFAR10
ResNet20
ResNet50
WRN28-10
WRN34-10
clean accuracy
90.02
93.23
94. 18
94.63
FGSM (e 0.01)
72.24
76.65
80.64
81.04
FGSM (e 0.02)
58.04
58.59
60.09
60.92
PGD (c 0.01)
48.48
60.23
64.64
65.38
PGD (E 0.02)
20.01
24.35
26.00
27.42
BIM (c 0.01)
48.57
60.43
67.36
68.29
BIM (E 0.02)
16.63
23.57
32.36
33.86
MIM (c 0.01)
51.48
60.81
64.36
64.71
MIN (c 0.02)
20.09
24.54
25.64
26.42
AA 0.01)
51.56
60.48
63.45
64.01
AA (c 0.02)
19.42
24.21
25.23
26.39
CW (A 0.01)
56.08
56.55
57.23
57.52
Table 8: Performance of FDT-hybrid with different weakness set allocation method on CIFAR-10.
The other settings are consistent with those in Table
Allocation method Clean accuracy FGSM (e= 0.02)
PGD (e=0.02)
AutoAttack (e=0.02)
Uniform Random 89.32
56.20
18.24
17.89
Ours
90.20
58.04
20.01
19.42
Table 9; Performance of FDT-hybrid with different selection thresholds Ti and 72 CIFAR-10. The other settings are consistent with those in Table
Thresholds
Clean accuracy FGSM (e= 0.02)
PGD (e=0.02)
AutoAttack (e=0.02)
= 0.2, 72 0.7
91.03
56.62
17.74
17.60
Ti = =0.2, 0.2, 72 = 0.8
90.20
58.04
20.01
19.42
=0.2, T2 0.9
89.46
58.48
20.12
19.57
T = =0.4, 72 0.7
89.75
56.89
17.93
17.82
Ti 0.4, T2 0.8
89.08
58.21
20.01
19.47
= 0.4, 72 = 0.9
88.44
58.53
20.09
19.61
Ti 0.6, T2 = 0.7
89.62
53.35
15.27
14.63
T 0.6, 72 0.8
88.84
55.33
15.42
15.24
T 0.6, T2 = 0.9
88.12
55.46
15.83
15.47
F.2 RESULTS FOR BLACK-BOX ATTACK
In the black-box setting, the attacker's knowledge usually limited the original training dataset and has no information about the model. This setting represents a more practical attack scenario. The attacker can train a surrogate model to generate transferable adversarial examples and transfer them to the target ensemble model. We utilize single ResNet-20 model the surrogate model. Adversarial examples are generated on the surrogate model using the SPSA algorithm (Spall 1992). Figure shows the robust accuracy of ensemble models against black-box attacks under different degrees of perturbation. As we can see, FDT-hybrid ensemble training strategies outperform the other ensemble training strategy against black-box attacks both CIFAR 10 and CIFAR 100.
20

---

Published as a conference paper at ICLR 2025
Blackbox accuracy against transfer attack CIFAR10
Blackbox accuracy againsst transter attack on CIFAR100
I15
Rebust Accuracy (9%
CAL
Rebust Accuracy
Dusrge 7DThyer
Userge
87-
rotthybri
08
02
2.1
03-
001
0.02
5:3
2,01
0:00
D.53
2.03
0.01
0.03
0.00
0.05
0.00
0.07
perturbetian scale
perturbation scale
(a)
(b)
Figure 5: Robust Accuracy for different ensemble models against black-box attack with different
perturbation scale
F.3 TRADE-OFF OFF BETWEEN CLEAN AND ROBUST ACCURACY
In this section, we explore the trade-off between clean accuracy and robust accuracy by varying the frequency selection threshold т2 (as mentioned in Section 4.2 And we set Tj to be 0.1. To assess the adversarial robustness, as a benchmark. We train a set of ResNet-20 FDT-hybrid models on CIFAR-10 and CIFAR-100 with various frequency selection threshold E (0.4, 0.6, 0.8, 1.0, 1.2, 1.6]. Figure shows that the ensemble model has lower clean accuracy and higher robust accuracy with the increasing of 72.
utilize the PGD attack under (o0 perturbations of size
0.01
Trade-off between clean accuracy and robust accuracy
Trade-off between clean accuracy and robust accuracy CIFAR1CO
CIFRR1
6.510
Robust Accuracy
Robust Accuracy
0.505
0.500
6.655
0.06
0:00
2.00
0.93
0.60
0.04
Clean Accuracy
Clean Accuracy
(a)
(b)
Figure 6: (a) shows the trade-off on CIFAR-10 while (b) on CIFAR-100. From left right, decrease the trade-off parameter 72 for FDT.
F.4 TRANSFERABILITY ACROSS VARIOUS SUB-MODELS
To further investigate the diversity between sub-models, we conduct an analysis by generating adver- sarial examples using one sub-model and evaluating their accuracy on other target sub-models. The transferability of these adversarial examples among sub-models is visualized in Figure 7 considering different ensemble training methods on the CIFAR 10 dataset. We generate adversarial examples from "base model" and test the accuracy of "target model" .hee experimental results indicate that FDT exhibits comparable performance to DVERGE and TRS in reducing the transferability adversarial samples across different sub-models. This demonstrates that FDT not only enhances the diversity of weaknesses within the dataset but also weakens the transferability of adversarial examples between sub-models.
21

---

Published as a conference paper at ICLR 2025
Vanilla
ADP
GAL
100.00
90.76
92.26
99.89
61.10
46.39
100.00
36.45
68.04
Target model
Target model
Targes sorgemmoon model
91.95
100.00
90.90
61:48
99.76
46.15
45.29
100.00
48.89
91.92
89.69
100.00
58.05
56.82
74.91
39.77
100.00
Base model
Base model
Base model
(a)
(b)
(c)
DVERGE
TRS
FOT-hybrid
100.00
25.86
25.50
100.00
38.96
35.58
Target model
Target model
Target, model
28.32
99.99 .9
28.66
46.58
100.00
36.47
34.41
100. .0
35.22
model
model
model
(d)
(e)
(I)
Figure 7: Pair-wise adversarial transferability between sub-models against PGD attack with e 0.02 on CIFAR-10. The value represents the success rate of adversarial examples generated by the base model attacking the target model.
F.5 COMPARE WITH ADVERSARIAL TRAINING
We use target attacks in our data transformation, which differs significantly from adversarial training. First, we employ a simple pre-trained network (VGG11 in experiment) to compute adversarial examples, thereby accelerating the training process. Second, we only utilize the low amplitude part of the adversarial examples for data transformation, which helps maintain the model's s clean accuracy. We compare our method with several popular approaches (Wang et al. 2023b Rade & Moosavi-Dezfooli 2021 Xu al. 2023) on CIFAR-10 using AutoAttack under to perturbations (e = 8/255). Wang et al. (2023b) generated training datasets using a diffusion model, followed by adversarial training on these datasets. For fairness, compare our method with the version proposed
by Wang et al. (2023b) that uses 50k generated images. Rade & Moosavi- Dezzool (2021) used "helper example" to help the adversarial training. Xu al. (2023) proposed Dynamics- Aware Robust Training, which encourages the decision boundary to adjust way that prioritizes increasing smaller margins. We use WideResNet-22 generated data. The results in Table TO indicate that, although the robustness of our method is not highest, maintains clean accuracy with almost no decline. Moreover, our method does not require additional generated data adversarial training, and even with the need for ensembling, the training efficiency remains relatively high. This suggests potential way to enhance robustness while minimizing the decrease in clean accuracy.
as the sub-model and ensemble eight sub- sub-models without using
To further illustrate our method's advantage, we conduct additional experiments to compare the "robustness-cless" accuracy" trade-off curves of our method and AT under different settings. Fig. g compares the trade-off curves obtained by HAT Rade & Moosavi-Dezfoole (2021) with that of FDT. hybrid. For HAT, we fix 7 0.25 and vary
0.51, 2.5, 3.0, 4.0, 5.0 (3 is the coefficient of the robustness loss, and higher 3 indicates higher robust accuracy); for FDT-hybrid, we fix 71 = 0.2 and vary 72 E {0.5, 0.7, 0.9. 1.1, 1.3, 1.5). We observe that HAT's robustness declines rapidly when the 3 parameter is small (as increasing the clean accuracy). This result shows the significant advantage of our method when a clean accuracy above 90% required.
22

---

Published as a conference paper at ICLR 2025
Trade-off between clean accuracy and robust accuracy
HAT
Robust Accuracy
FDT-hybrid
20
15
84
90
Clean Accuracy
Figure 8: It shows the trade-off curves CIFAR-10. From left to right, decrease the trade-off off parameter for FDT, and decrease the trade-off parameter 33 for HAT.
Table 10: Clean accuracy and robust accuracy (%) of different methods against AutoAttack under Lo0
perturbations 8/255) on CIFAR-I0.
CIFAR-10
clean accuracy
robust accuracy
Wang al. 2023b)
86.15
55.71
Rade & Moosavi Deztooli
2021
84.90
49.08
Xuu et al.
2023)
85.55
54.69
OURS (FD hhbridd
93.72
34.61
23


Reviews:
[{"review_id": "Zvy7ky4WLX", "reviewer": "Reviewer_2rgn", "summary": "The paper presents a Fourier-based ensemble training method to counter adversarial transferability by diversifying non-robust frequency components across sub-models. Using random noise and targeted attack transformations, the method achieves improved robustness and competitive accuracy on datasets like CIFAR-10, outperforming current ensemble techniques.", "questions": "Refer to the weakness.", "limitations": "", "rating": 5, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "- The use of Fourier transformations to manipulate frequency components for adversarial robustness is innovative and less explored in ensemble methods.\n- The proposed method addresses the transferability of adversarial attacks more effectively by diversifying non-robust features across sub-models.\n- Unlike simultaneous training methods, the approach’s independent training of sub-models reduces GPU memory requirements and simplifies the training pipeline.", "weaknesses": "- The proposed targeted-attack transformation method, while effective, introduces a layer of complexity to the training pipeline. By replacing specific non-robust frequency components in the data with adversarially targeted features, the transformation becomes computationally intensive and intricate to implement, especially when scaling to larger datasets or real-time applications. This additional complexity may limit the ease of adoption for other researchers or practitioners looking to implement this technique without substantial computational resources.\n- The experiments are primarily conducted on smaller architectures, like ResNet-20, with limited exploration on larger or more complex models. Without extensive testing across a variety of architectures, it’s unclear whether the proposed Fourier-based transformations are universally effective or if they are more suitable for specific model types.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "kQZ29NFhAL", "reviewer": "Reviewer_hPDX", "summary": "Ensemble training is a commonly used technique to enhance a model's adversarial robustness. However, recent studies suggest that this approach may not be as effective, as adversarial examples can often affect multiple sub-models due to a phenomenon called \"transferability.\" This limitation arises because all sub-models are typically trained on the same dataset, leading them to share similar vulnerabilities. To address such issue, the author propose an effective data transformation framework to improve the diversity of training\ndata used by different sub-models for robust ensemble training. In the end of the paper, they present empirical results demonstrating the effectiveness of their method against common adversarial attacks, while also maintaining clean accuracy.", "questions": "Here are few questions and suggestion I want to provided for your paper:\n\n1. No detailed explanation of the feature extractor. In Definition 3.1, you provide a detailed definition of what a \"useful detail extractor\" is. However, the feature extractor itself is only given as $θ: X → \\mathcal{R}^k$. Based on the information provided, I can't see how it differs from a standard classification model. I hope you can add more details to improve the readability of the paper\n\n2. $\\hat{y}$ is not an appropriate mathematical symbol in my opinion. $\\hat{y}$ is typically used to denote the prediction of y by convention, but in your paper, you are using it to denote the one-hot vector of y, which is quite confusing.\n\n3. In your Definition 3.2, I am confused by the second formula you provided. The first formula indicates that the feature extractor is robust if the expectation $ y^t \\cdot \\theta(A(x)) > \\frac{1}{k} $, but the second formula does not contain $y^t$ at all. I'm not sure whether I misunderstood your formula or if there is a mistake here.\n\n4. The format of your formula 8 doesn’t seem correct to me. The equation is broken in the middle, leaving part of it followed after the plain text and the other part in equation mode.\n\n5. I am curious how you find the \"high-amplitude frequency features\" from your image. I think I am having this question since I didn't understand how your feature extractors work.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 1, "contribution": 2, "strengths": "- The proposed framework is innovative, intuitively reasonable, and easy to understand. \n- The method comes with solid theorical backing\n- The experiments are comprehensive and well-supported their findings", "weaknesses": "- The paper's readability is poor. I’ve provided specific feedback in the following sections and strongly recommend improvements in clarity and structure.\n- The ensemble learning with proposed method still shows limited competitiveness with adversarial training, which restricts its practical applicability. Although the appendix shows that this method maintains higher clean accuracy than popular adversarial training (AT) methods, it still falls significantly short in robust accuracy compared to these approaches.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "ikvbpzhFXu", "reviewer": "Reviewer_XA1w", "summary": "This paper proposes an ensemble training approach to tackle the adversarial transferability problem by leveraging Fourier transformation and a weakness allocation strategy to diversify non-robust features across sub-models. The authors introduce a novel data transformation framework involving frequency selection and frequency transformation, aiming to improve the ensemble model’s robustness against adversarial attacks without sacrificing clean accuracy. Experimental results show that their method, particularly the FDT-hybrid, outperforms several existing ensemble methods in robust accuracy on benchmark datasets like CIFAR-10 and CIFAR-100​.", "questions": "1. Could the authors elaborate on how the choice of frequency thresholds (τ1 and τ2) affects robustness and clean accuracy?\n2. Is the computational efficiency of the proposed approach scalable to larger models or more complex datasets, given the Fourier transformations applied?", "limitations": "", "rating": 6, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The paper addresses a critical challenge in adversarial machine learning, focusing on enhancing ensemble robustness without excessive overhead.\n2. The frequency-based approach for diversifying non-robust features is innovative and leverages insights from signal processing.\n3. Experimental results are thorough, showcasing improved robustness across various adversarial attacks and comparison with multiple baseline methods.", "weaknesses": "1. The paper’s complexity may hinder reproducibility, especially given the intricate weakness allocation and frequency transformation processes.\n2. The dependency on specific frequency thresholding could limit adaptability across different datasets or tasks.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "CW0wK3Iobp", "reviewer": "Reviewer_QFba", "summary": "This paper introduces a data transformation method aimed at improving the adversarial robustness of ensemble models. Instead of reducing the influence of non-robust features, the authors propose to reduce the transferability of attacks by increasing the diversity of non-robust features. The robust and non-robust features are identified by frequency selection. The weakness set is allocated to sub-models for training to increase diversity. Through the experiments on various datasets, the results demonstrate the superiority of the proposed method over other baselines.", "questions": "1. Please clarify the motivation of amplitude-based selection.\n2. Please provide more ablation studies of the proposed method.\n3. Please clarify the difference in the result in Table 2.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "1. This paper is well-written and easy to follow.\n2. The motivation of diverse non-robust features seems natural for a better trade-off between clean and robust accuracy.\n3. The comparison with other baselines shows the effectiveness of the proposed method.", "weaknesses": "1. The motivation of amplitude-based selection is not well explained. There is no theoretical or empirical evidence to support this design in this paper.\n2. There is no ablation study to verify the effectiveness of all the components, such as weakness set allocation, amplitude-based selection, new dataset construction, etc.\n3. The results of baselines seem strange. For example, the results in Table 2 differ from those in Table 1 of TRS.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)