Synergy-of-Experts: Collaborate to Improve Adversarial Robustness
Anonymous Author(s)
AffiliationAddress email
Abstract
Learning adversarially robust models requires invariant predictions to a smallneighborhood of its natural inputs, often encountering insufficient model capacity .
There is research showing that learning multiple sub-models in an ensemble couldmitigate this insufficiency, further improving the generalization and the robustness.
However, the ensemble’s voting-based strategy excludes the possibility that the truepredictions remain with the minority .
Therefore, this paper further improves theensemble through a collaboration scheme—Synergy-of-Experts (SoE).
Comparedwith the voting-based strategy, the SoE enables the possibility of correct predictionseven if there exists a single correct sub-model.
In SoE, every sub-model fits itsspecific vulnerability area and reserves the rest of the sub-models to fit othervulnerability areas, which effectively optimizes the utilization of the model capacity.
Empirical experiments verify that SoE outperforms various ensemble methodsagainst white-box and transfer-based adversarial attacks.
Introduction
Deep models have been widely applied in various real-world applications including high-stakesscenarios (such as in healthcare, finance and autonomous driving).
An increasing concern is whetherthese models make adversarially robust decisions [1, 2].
Recently, there are research revealing thatan adversarially robust method requires invariant predictions to a small neighborhood of its naturalinputs thus often encountering insufficient model capacity [3, 4].
This limits the further improvementof robustness and has the undesirable degradation of generalization [5].
Learning multiple sub-models in an ensemble [6, 7] can mitigate this insufficiency.
Remarkably,there are research [8, 9, 10] proposing to minimize the vulnerability overlaps between each pair ofsub-models and improving both robustness and generalization over a single model.
However, thevoting-based ensemble may waste the limited capacity of multiple models.
In the example of three sub-models (see Figure 1(b)), the adversarial input that lies in the black areascan fool the ensemble successfully, i.e., more than half of sub-models must correctly classify theadversarial input.
Therefore, the ensemble’s voting-based strategy excludes the possibility that truepredictions remain with the minority .
In other words, learning an ensemble requires more than half ofthe sub-models to fit the same vulnerability areas, which leaves the following question unansweredwhether
we could only leverage a single sub-model to fit a vulnerability area and reserve the rest ofthe sub-models to fit other vulnerability areas.
Inspired by mixture-of-experts (MoE) [11], we propose to learn a collaboration among multiple submodels to optimally utilize the limited capacity.
As shown in Figure 1(c), the adversarial input that
[Figure]
lies in the vulnerability overlaps of all sub-models can undoubtedly fool the collaboration.
Comparedwith the ensemble in Figure 1(b)), collaboration enables the possibility of correct predictions evenif there exists a single correct sub-model.
Besides, learning a collaboration could enable everysub-model to fit its vulnerability area, which could collectively fix broader vulnerability areas.
Then,sub-models could collaboratively choose a trustworthy one to make the final predictions.
Classic MoE methods assume that the problem space is separable, and the separability is irrelevantto the learned model [11].
However, the non-i.i.d adversarial inputs [12] depend on the learnedmodels and are hard to be classified by a learned gate in MoE. To tackle the above challenge, wepropose Synergy-of-Experts (SoE), which explicitly builds the relationship between learned modelsand adversarial inputs.
Specifically, each sub-model has dual heads: one outputs a vector of predictedprobability f θ ( · ) ; another outputs a scalar that measures the confidence of the prediction.
In theadversarial training phase, given an adversarial input x , each sub-model chooses an easy one(s) tofeed itself.
The other head is meanwhile updated by comparing the predicted probability on the truelabel— f yθ ( · ) (a scalar).
In the inference phase, given an input, SoE chooses a sub-model with thelargest confidence as the representative to output the overall prediction.
We highlight our key contributions as follows.
• We provide a new perspective on learning multiple sub-models for defending againstadversarial attacks.
We show that the collaboration could make better decisions than theensemble (Proposition 1), which implies collaboration may fix broader vulnerability areas.
• We propose a novel collaboration framework—SoE (see Section 3.2).
In the training phase,SoE minimizes the vulnerability overlap of all sub-models;
In the inference phase, SoE could
effectively choose a representative sub-model to make correct predictions.
We also provide
a comprehensive analysis illustrating the rationale of SoE in Appendix.
• Empirical experiments corroborate the SoE outperforms various ensemble methods [10, 8,9, 13] against white-box and transfer attacks.
Related Work
Adversarial attack.
Adversarial attacks aim to craft the human-imperceptible adversarial inputto fool the deep models.
Adversarial attacks could be roughly divided into white-box attacks inwhich the adversary is fully aware of the model’s structures [1, 14, 15, 16, 17, 18, 19, 20, 21, 22,23, 24, 25, 26] and black-box attacks in which the deep models are treated as black boxes to theadversary [27, 28, 29, 30, 31, 32, 33, 34, 35, 36].
This paper focuses on building effective defenseand select both white-box and black-box attack methods as our robustness evaluation metrics.
Adversarial defense.
Defending adversarial attacks is a challenging task and researchers haveproposed various solutions.
Certified defense tries to learn provably robust deep models against normbounded (e.g., ℓ 2 and ℓ ∞ ) perturbations [37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49].
Empiricaldefense.
leverages adversarial data to build effective defense such as adversary detection [50, 51,
52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68] and adversarial training (AT),
inwhich AT stands out as the most effective defense.
Researchers have investigated various aspectsof AT, such as improving AT’s robustness or generalization [5, 69, 70, 71, 72, 73, 74, 75, 76, 77,78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 3], fixing AT’s undesirable robust overfitting [90, 91],improving AT’s training efficiency [92, 93, 94, 95, 96, 97], understanding/interpreting AT’s uniquetraits [98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 49, 108, 109], etc.
Besides, researchers havealso actively investigated robust-structured models [110, 111, 112, 113, 114, 115, 116].
Nevertheless,the above research thoroughly investigated a single model; this paper focuses on the collaborationamong multiple models for adversarial defense.
Ensemble methods for adversarial robustness.
The most relevant studies are the ensemblemethods.
Ensemble methods such as bagging [6] and boosting [7] have been investigated forsignificantly improving the model’s generalization.
Motivated by the benefits of ensemble methods inimproving generalization, researchers introduced an ensemble to improve the model robustness [10,9, 8, 117].
Tramèr et al.
[117] proposed to reduce the adversarial transferability by training a singlemodel with adversarial examples from multiple pretrained sub-models.
Pang et al.
[8] introducea regularization method—ADP—to encourage high diversity in the non-maximal predictions ofsub-models.
Kariyappa et al.
[9] improved the ensemble diversity by maximizing the introducedcosine distance between the gradients of sub-models with respect to the input.
Yang et al.
[10]proposed to distill non-robust features in the input and diversify the adversarial vulnerability.
Thesemethods reduced overlaps of vulnerability areas between sub-models [10].
Compared with votingstrategy in ensemble, mixture-of-experts (MoE) assumes that the problem space can be dividedinto multiple sub-problems through a gate module [11, 118].
However, in adversarial training, theadversarial samples, which depend on the learned models, are not i.i.d.
A vanilla MoE is hard toidentify the best performing sub-models for each adversarial sample without the information aboutthe learned models.
Collaboration to Defend Against Adversarial Attacks3.1 Superiority of Collaboration
This section shows a collaboration , in theory, could make better decisions than an ensemble .
Ensemble.
Suppose that there are M learned sub-models { f θ 1 , f θ 2 , ..., f θ M } , given an input x , Msub-models make predictions (cid:8) f θ 1 ( x ) , f θ 2 ( x ) , ..., f θ M ( x ) (cid:9) .
The ensemble outputs a final predictionensemble( x, f θ 1 , ..., f θ M ) by the voting-based strategy:
[Equation]
where 1 is the indicator function and K denotes the number of classes.
Note that the ensembleoutputs the predicted label y that agrees with the majority predictions of the sub-models.
Definition 1 (best-performing sub-model) .
Given an input x and its label y , the best-performingsub-model achieves the lowest objective loss on the data ( x, y ) among all M sub-models:
[Equation]
105 Note that the best-performing sub-model is w.r.t.
the input data ( x, y ) , i.e., different input datacorrespond to different best-performing sub-models.
Collaboration.
Suppose that there are M learned sub-models { f θ 1 , f θ 2 , ..., f θ M } .
Given an inputx , sub-models make predictions (cid:8) f θ 1 ( x ) , f θ 2 ( x ) , ..., f θ M ( x ) (cid:9) .
The collaboration tries to output afinal prediction collaboration( x, f θ 1 , ..., f θ M ) by the best-performing sub-model:collaboration( x, f θ 1 , ..., f θ M ) =
f
θ best ( x ) .
(3) 111 Proposition 1.
Given M learned sub-models, the predicted accuracy of the collaboration is upperbounded that of the ensemble, i.e.,
[Equation]
[Table]
x
[Table] [Table]
+a24M9J53DE8ux9OHFpiP0mS4tVeL1/QSpoc6jQop6CAdNS8/DM66MODiQgOGFxwwjZ0hPS1UYYGn7guYuICQpaMMyTIkTaiLEYZOrFjmoe0a6esS3vhGUq1SafY9AekVLFLGo/yAsLiNFXGI+ks2N+8Y+kp7jah1Ui9HGI5RsT+pfvM/K9O1MIxwJGswaKafMmI6szUJZKvIm6ufqmKk4NPnMB9igeETan8fGdVakJZu3hbXcZfZaZgxd5McyO8iVtSg8s/2zkNGvulcqV0cFYpVI/TVmexjR0UqZ+HqOIUNdTJ+xoPeMSTcqHcKLfK3Ueqkkk1m/g2lPt3Z3Waew==</latexit>
[Figure]
x <latexit sha1_base64="Pbc+1s8Sc82XeqeLL9Ole7+uVlw=">AAACxHicjVHLSsNAFD2Nr1pfVZdugkVwVRKp6LIoiMsW7AO0SJJO69DJg8lELEV/wK1+m/gH+hfeGVNQi+iEJGfOvefM3Hv9RPBUOc5rwZqbX1hcKi6XVlbX1jfKm1vtNM5kwFpBLGLZ9b2UCR6xluJKsG4imRf6gnX80amOd26ZTHkcXahxwnqhN4z4gAeeIqp5d12uOFXHLHsWuDmoIF+NuPyCK/QRI0CGEAwRFGEBDyk9l3DhICGuhwlxkhA3cYZ7lEibURajDI/YEX2HtLvM2Yj22jM16oBOEfRKUtrYI01MeZKwPs028cw4a
[Table]
+6j8z/6mQtAl1sqRo41RQqRlbn5C6J6oq8uf6lKkEOIXESdygeEXaU8qPPutLEqnbZW0vFX1SmZOXeyXMTvMpb0oDNn+McBM31ilmtbOxXy7XtfNQFLGIJqzTPTdSwizoa5H2FezzgUfO0a+1Gu31P1YZyzQK+Le3uDeEfoAI=</latexit> f ✓ M (˜ x ✓ M ) <latexit sha1_base64="AGt4k7VJmkQM5psYLcza9FRQ7aE=">AAAC63icjVHLSsNAFD3GV31XXboJFkU3JZGKLotu3AgVrAqthiSdtoN5kUzEEvoH7tyJW3/Arf6H+Af6F94ZU1CL6IQkZ86958zce53I44kwjNcRbXRsfGKyMDU9Mzs3v1BcXDpJwjR2Wd0NvTA+c+yEeTxgdcGFx86imNm+47FT53Jfxk+vWJzwMDgWvYid+3Yn4G3u2oIoq7jetrKm6DJhW9lhv7/RFNxrsey6f/GV3rSKJaNsqKUPAzMHJeSrFhZf0EQLIVyk8MEQQBD2YCOhpwETBiLizpERFxPiKs7QxzRpU8pilGETe0nfDu0aORvQXnomSu3SKR69MSl1rJEmpLyYsDxNV/FUOUv2N+9Mecq79ejv5F4+sQJdYv/SDTL/q5O1CLSxq2rgVFOkGFmdm7ukqivy5vqXqgQ5RMRJ3KJ4TNhVykGfdaVJVO2yt7aKv6lMycq9m+emeJe3pAGbP8c5DE62ymalvH1UKVX38lEXsIJVbNA8d1DFAWqok/cNHvGEZ83XbrU77f4zVRvJNcv4trSHDyaGoB4=</latexit> ` ( f ✓ best (˜ x ✓ 1 ) ,y )
[Table]
<
[Table]
<
[Table]
[Table] [Table]
<
[Table]
+60oSqdtlbU8XfVKZk5d5OciO8y1vSgAs/xzkNqvv5QjF/cFbMlY6TUaexjR3s0TwPUcIpyqiQ9w0e8YRn7VK71e60+89ULZVoNvFtaQ8fRn6a2Q==</latexit>
PGD (a2) collaboration training
g  ( x ) <latexit sha1_base64="7ctncwgFuMdaXoGwpIQz8wNf/kc=">AAACznicjVHLTsJAFD3UF+ILdemmkZjghrQGo0uiG5eYyCMBQtoywMTSNtMpkRDi1h9wq59l/AP9C++MJVGJ0Wnanjn3nDtz73Ujn8fSsl4zxtLyyupadj23sbm1vZPf3avHYSI8VvNCPxRN14mZzwNWk1z6rBkJ5oxcnzXc20sVb4yZiHkY3MhJxDojZxDwPvccSVRr0J22oyGfFe+Ou/mCVbL0MheBnYIC0lUN8y9oo4cQHhKMwBBAEvbhIKanBRsWIuI6mBInCHEdZ5ghR96EVIwUDrG39B3QrpWyAe1Vzli7PTrFp1eQ08QReULSCcLqNFPHE51Zsb/lnuqc6m4T+rtprhGxEkNi//LNlf/1qVok+jjXNXCqKdKMqs5LsyS6K+rm5peqJGWIiFO4R3FB2NPOeZ9N7Yl17aq3jo6/aaVi1d5LtQne1S1pwPbPcS6C+knJLpdOr8uFykU66iwOcIgizfMMFVyhipru+COe8GxUjbExM+4/pUYm9ezj2zIePgC26JOV</latexit>
Proof.
Given an ( x, y ) ∈ D , if the ensemble’s prediction is correct, at least one sub-model makescorrect prediction, i.e., 1 f θ best ( x )=
y holds; therefore, the collaboration’ prediction is correct.
Ifthe collaboration’s prediction is correct, there exists a case that the majority of sub-models makeconsistent but wrong predictions, while a single sub-model’s prediction is correct; then, ensemble’sprediction is wrong.
Therefore, Proposition 1 holds.
From Proposition 1, a collaboration could achieve an equal or higher performance than an ensemble.
Compared with the ensemble, the collaboration requires the identification of the best-performingsub-models using label information.
Next, we will introduce a realization of our collaborationframework.
3.2 Realization of collaboration for defending against adversarial attacks
Notation.
We firstly introduce the needed notations.
Suppose X and Y denote input space andoutput space, where Y = { 1 , ..., K } for a K -class classification problem.
There are N samplesin the dataset D
=
{ ( x, y ) } ,
where x ∈ X and y ∈ Y
Let d inf ( x, x ′ ) =
∥ x − x ′ ∥ ∞ denotes theinfinity distance metric, and B ϵ [ x ] = { x ′ ∈
X | d inf ( x, x ′ ) ≤ ϵ } is the closed ball of of radius ϵ > 0centered at x .
To search for adversarial data within norm ball B ϵ [ x ] , [5] proposed a projected gradientdescent (PGD) method that iteratively searches for adversarial data ˜ x
( x refers to natural data).
f θ ( x )outputs a K-dimensional predicted probability, i.e., ˆ p ( x ) =
[ˆ p 1 ( x ) , ..., ˆ p K ( x )] .
Goal of collaboration.
1) ensure the correct prediction of the best-performing sub-model for agiven input, and 2) select the best-performing sub-model among all sub-models to make predictions.
First, intuitively, every sub-model in a collaboration should maximize its expertise to fit its areas andleave the remaining areas fitted by others.
As a result, the collaboration can minimize the vulnerabilityoverlaps of all sub-models.
Section 4 shows “minimizing the vulnerability overlap of all sub-models”is “minimizing the objective loss of the best-performing sub-models”.
Therefore, during the trainingphase, the given data should always be allocated to the sub-model that has the lowest objective loss.
Input prediction the approximated confidence training dataset σ ; 1: do 2: do 3:
M do 5:
Calculate the approximated confidence, i.e., g ϕ j (˜ x θ
Minimize BCE loss ℓ ϕ = BCE( g ϕ j (˜ x θ i ) , ˆ p y
(˜ x θ i )) to update module g ϕ j ; 7:
Collect sub-model
i ’s cross entropy (CE) loss on data ˜
x
θ i : ℓ (cid:0) f θ j (cid:0) ˜ x
θ i (cid:1) , y (cid:1) ; 8: end for 9:
Calculate surrogate loss on data ˜ x
θ i : ˆ ℓ m = − σ ln (cid:80) Mj =1 exp (cid:18) − ℓ ( f θj ( ˜ x θi ) ,y ) σ (cid:19) ; 10:
Update performing
11: end for 12: end for
In other words, the sub-models always choose the easiest data to learn.
In the example of Figure 2(a),i ) Data 3 ⃝ is misclassified by both sub-models.
The blue sub-model is near Data 3 ⃝ and has the lowestobjective loss.
We assign the blue sub-model to fit Data 3 ⃝ .
ii ) Data 2 ⃝ is correctly classified by thepink model but wrongly classified by the blue model; for ease of effort, we assign the pink model tofit Data 2 ⃝ , because the collaboration can correctly be classified Data 2 ⃝ by selecting the pink modelas the representative.
iii ) Data 1 ⃝ is correctly classified by both models.
The blue model is far from
Data 1 ⃝ and takes the lowest effort on fitting it; therefore, we assign the blue model to fit Data 1 ⃝ .
Second, to select the best-performing sub-model, we construct dual-head structured sub-models .
Asshown in Figure 2(b), our sub-model has dual heads: 1) predictor (P) head: f θ predicts the labelprobability f θ ( x )
=
ˆ p ( x ) =
[ˆ p 1 ( x ) , ..., ˆ p K ( x )] (a vector); 2) evaluator (E) head: g ϕ approximatesthe confidence of the prediction ˆ p ( x ) .
Note that we use g ϕ to approximate the true label probability ˆ p y ( x ) , which denotes the true confidenceof a given prediction ˆ p ( x ) .
Meanwhile, the largest confidence corresponds to the lowest objective loss,and vice versa (see theoretical proof in Proposition 2).
Therefore, the best-performing sub-modelscould be identified using the approximated confidence ( g ϕ ( x ) ) by the E head.
To defend against adversarial attacks, the collaboration needs to learn from adversarial data.
Algorithm 1,2 along with Figure 3 articulates how to learn such the collaboration.
In particular, thetraining of our framework SoE is as follows:
the sub-model training: the sub-models fit adversarial samples from their own or other
sub
-models;
the collaboration training: the sub-models fit adversarial samples from the collaboration.
The sub-model training.
As shown in Figure 3(a1), given a natural training data x and its labely , we use the PGD method to obtain M adversarial variants { ˜ x θ i } Mi =1 of M sub-models as existingbaselines do.
For each adversarial variant ˜ x
θ i , we assign the best-performing sub-model to learn andupdate its feature extractor and label head.
This process corresponds to Lines 1–8 and Lines 9–10 in
Algorithm 1.
Note that in Line 9, we use a surrogate loss to approximate the sub-model assignmentprocess (reasons see Eq.
5 and 6).
As shown in Figure 3(b), given an adversarial variant ˜ x θ i , wepropose to use the binary-cross-entropy (BCE) loss between the predicted label probability on thetrue label (i.e., ˆ p y
(˜ x θ i ) ) and the approximated confidence (i.e., g ϕ (˜ x θ i ) ) to update each sub-model’s
E head.
This process corresponds to Lines 5–6 in Algorithm 1.
The collaboration training.
During the sub-model training, we learn the most adversarial data fromeach sub-model using the sub-models performing best.
The most adversarial samples cannot attack Input duel
} Mi
{ g i } Mi = f θ i outputs the label prediction and g dataset the collaboration successfully.
However, there may exist harmful samples (which may not be themost adversarial for any client) that are unexplored and can attack all sub-models.
Therefore, the submodel training may converge without a full exploration of the adversarial samples.
(The experimentalverification about this could be found in Section 4.4.)
To defend these potential adversarial samples,we propose the collaboration training shown in Figure 3 (a2) and Algorithm 2.
Firstly, we proposeto generate the adversarial samples that can worsen the outputs of the collaboration.
In particular,for each data sample x , we output the prediction ˆ p ( x ) whose confidence g ϕ ( x ) is the highest.
Weperturb x to fool the prediction using PGD method.
Then we minimize a surrogated loss to fit thisadversarial data ˜ x .
This process is detailly shown in Algorithm 2.
During the inference phase shown in Algorithm 3 in Appendix, once M sub-models are properlylearned, SoE chooses a representative sub-model whose confidence is the highest among all submodels, and then outputs this sub-model’s prediction.
3.3 Analyses of SoE
Optimizing the best-performing sub-models.
We firstly show that minimizing the vulnerabilityoverlap of all sub-models is equal to minimizing the objective loss of the best-performing sub-models.
For ease of optimization of the best-performing sub-model, we provide a surrogate loss.
The vulnerability overlap of all sub-models refers to the set of adversarial data (˜ x, y ) that aremisclassified by all sub-models, i.e., all sub-models’ objective loss is higher than a certain degreeδ : min θ i ∈{ θ 1 ,...,θ M } ℓ ( f θ i (˜ x, y ))
> δ, ˜ x ∈ ˜ D , where ˜ D denotes the vulnerability overlap of allsub-models.
To reduce the vulnerability overlap of all sub-models, we only need to reduce objective loss of asingle model, which is equal to minimizing the loss of the best-performing sub-model, i.e.,
[Equation]
where ℓ best (˜ x θ i , y ) =
min j ∈{ 1 , 2 ,...,M } ℓ (cid:0)
[Equation]
where
˜ x θ i is the adversarial data generated by the sub-model f θ i .
While directly performing the outer minimization in Eq.(5) may cause a trivial solution (e.g., there isonly one optimized sub-model), for ease of the optimization of Eq.(5)
(Corresponding to Lines 9–10in Algorithm 1), we provide a surrogate objective as follows. [Equation] where ˆ ℓ m [Equation]
˜ x
[Equation]
i , y [Equation] = − [Equation]
ln
[Equation]
Mj [Equation] and σ > 0 is a pre-defined hyper-parameter. In Eq.(6), we approximate the objective min θ j ∈{ θ 1 ,θ 2 ,...,θ M } ℓ (cid:0) f θ j
(˜ x θ i ) , y (cid:1) using a smooth surrogated maximum function due to
[Equation]
The proof of Eq.(7) is in Appendix.
The best-performing sub-model has the highest confidence.
We show that a sub-model withthe highest confidence achieves the minimum of the objective loss among all sub-models, i.e., thebest-performing sub-model.
Proposition 2.
Given an input x , the sub-model that has the highest confidence corresponds to thebest-performing sub-model, i.e.,
[Equation]
206 where confidence corresponds to the probability of the true label in a given ˆ p ( x ) .
To learn the Ehead to approximate the confidence of ˆ p ( x ) , we could simply compare its output with the predictedprobability on the true label (i.e., ˆ p y ( x ) ), then update the E head by gradient descent (corresponds to
Lines 6–7 in Algorithm 1).
Given an input to the collaboration, our dual-head structured sub-modelscan collaboratively decide the best-performing sub-model by comparing the values of the E head(corresponds to Algorithm 3 in Appendix).
Note that the E head may be susceptible to adversarial attacks in the white-box setting.
In ourimplementation, we use a simple linear structure to regress the confidence.
Experimental results onadaptive attacks demonstrate the reliability of our framework.
Experiments
In this section, we conduct experiments on a benchmark dataset to verify the effectiveness of ourmethod in defending against white-box and transfer attacks.
Then we provide ablation studiesto demonstrate the significance of the collaboration training described in Algorithm 2.
1 Theexperimental results about black-box attacks and more discussions could be found in Appendix.
4.1 Experimental Setup
Following the work in [10], we compare our method with various related methods, including ADP [8],
GAL [9], DVERGE [10] and MoRE [13].
We use ResNet-20 [119] as sub-models in all methods forfair comparisons, and we use CIFAR10 as the data set, a classical image dataset [120] that has 50,000training images and 10,000 test images.
4.2 Performance on White-box attack
As there are mainly two threat modes in the adversarial attack setting: white-box attack and black-boxattack.
White-box attack refers to that attackers know all the information about the models, includingtraining data, model architectures, and parameters, while black-box attackers have no access to theinformation about the model’s structures and parameters and rely on surrogate models to generatetransferable adversarial examples.
We compare our method with 4 baselines on defending against white-box attacks using a subsetof CIFAR10 .
We use 50-step PGD with five random starts and the step size of ϵ/ 5 to attack allmethods as in [10].
We learn the models in adversarial training and evaluate the robustness undervarious attacks with the same ϵ .
For example, to evaluate the robustness under white-box attack withϵ = 0 .
01 , we first learn the models in adversarial training with ϵ = 0 .
We evaluate all methodsfollowing the setting in [10].
In particular, we randomly select 1000 samples under different ϵ .
Forthe PGD attack, we select the cross-entropy loss to update the perturbations to search for adversarialsamples.
In addition to the robustness, we also report the performance of all methods on clean datawith the adversarial training under different ϵ . ϵ (robust/clean) 0 .0 .0 .0 .0 .0 .0 . 07 GAL 49.5/87. 9.0/76.0 DVERGE 67.3/85.
10.0/75.3 MoRE 67.9/88. 12.3/77.8 ADP 67.7/89. 20.3/63.0 SoE 72.0 /88. 24.0 /74.1 SoE (adaptive) 70.9/88.
From Table 1, SoE achieves a better robustness performance under white-box attack.
The resultsverify that collaboration significantly improves the utilization of the limited model capacity.
Therefore,
SoE can fit more adversarial data and have a relatively smaller vulnerable area.
Performance on Adaptive Attacks.
We follow the suggestions in [121] and conduct twodifferent adaptive attacks to fool the dual heads simultaneously.
For the first adaptive attack, we attack the E head to minimize the confidence of the best-performing sub-model.
Inparticular, we maximize l 1 = BCE( g ϕ j ( x ) , 1) , where j = arg max i ∈
[ M ] g ϕ i ( x ) means thej -th sub-model is identified as the best-performing sub-model.
For the second adaptive attack, we try to achieve a mismatch between the correct predictions and the highest confidence.
Specifically, we maximize l 2 = − log (cid:104)
f θ j ( x ) y ∗ g ϕ j
( x ) + 1 − g ϕ j
( x ) (cid:105) , where j =arg max i ∈
[ M ] − log (cid:104)
f θ i ( x ) y ∗ g
ϕ
i
( x ) + 1 − g ϕ
i ( x ) (cid:105) .
We conduct experiments by maximizingthe weighted loss ℓ adp= ℓ ( f θ ( x ) , y ) + λ · ℓ 1 and ℓ adp= ℓ ( f θ ( x ) , y ) + λ · ℓ 2 with varying λ .
Therobustness with respect to λ on the stronger adaptive attack (i.e., the attack achieves a higher successrate) is shown in Figure 4.
[Table]
254 Though both adaptive attacks could attack the predictor and the evaluator simultaneously, the evaluatorin our method is robust to the adversarial samples because of its sample structure.
As seen in Figure 4and Table 1, our method is slightly degraded by adaptive attacks and still outperforms baseline models.
258 4.3 Performance on Transfer Attack
Due to the transferability of adversarial examples, transfer adversaries can craft adversarial examplesbased on surrogate models and perform an attack on the target model.
In our experiments, we followthe transfer attack setting in [10] and select 1000 test samples randomly.
We use hold-out baselineensembles with three ResNet-20 sub-models as the surrogate models to generate adversarial samples.
In particular, we use three attack methodologies: PGD with momentum [35], SGM [29] which addsweight to the gradient through the skip connections of the model, and M-FGSM [36] which randomlyaugments the input images in each step.
For each sample, three adversarial variants are using thethree attack methods.
Only when the model can classify all kinds of adversarial variants can themodel successfully defend against adversarial attacks.
We show the results of all methods in Table 2.
Furthermore, we also use a more challenging setting following the work in [10].
We use hold-outbaseline models with 3, 5, and 8 ResNet-20 sub-models as the surrogate models.
Meanwhile, wegenerate adversarial samples with cross-entropy loss and CW loss [15].
For each sample, we generate30 adversarial variants, and only if the model classifies all the 30 variants can the model defend thetransfer attack successfully.
The results are shown in Table 3. [Figure] [Table] [Figure]
[Table]
[Figure] [Table] [Figure] In our experiments
[Figure]
GAL is hard to learn stably in adversarial training.
In Table 2, when 0 .
01 ≤ ϵ ,
SoE achieves a better performance compared with baselines.
With the increase of ϵ , the volumeof ϵ -ball increases exponentially.
The performances of all methods get worse significantly becauseof insufficient model capacity.
Similar results can also be found in Table 3.
Since SoE addressesmore adversarial data using a collaboration mechanism, it achieves a relatively better robustnessperformance as ϵ increases.
4.4 Ablation Studies
To make full use of the limited modal capacity of an ensemble, SoE proposes the sub-model training(shown in Algorithm 1), which makes the adversarial samples from sub-models be learned by thebest-performing ones. The sub-model training enables the most adversarial attacks of sub-modelscould be successfully defended. However, learning from the most adversarial samples of all submodels could converge without a full exploration of the vulnerable area.
Therefore, we propose thecollaboration training (shown in Algorithm 2), which fits the adversarial samples unexplored before.
To verify the significance of the collaboration training, we conduct ablation studies under two transferattacks.
In particular, we train two kinds of models to defend against the attacks: 1).
SoE ; 2). SoE without the collaboration training. The two transfer attacks are generated as stated in Section 4.3,1). attacks with 3 adversarial variants; 2).
attacks with 30 adversarial variants. We show the robustness of two methods in Figure 5. From Figure 5(a) and 5(b), when 0 . 01 ≤ ϵ ≤0 .
04 , SoE without the collaboration training achieves a similar robustness compared with SoE. Itmeans that the sub-model owns relatively sufficient model capacity and the adversarial samplescould be largely explored by the sub-model training. However, when ϵ ≥ 0 . 04 , the robustness of SoE without the collaboration training degrades significantly.
In this case, the need for model capacityincreases dramatically. The most adversarial samples identified by the sub-model training cannotcover all cases, and our proposed the collaboration training mitigates it effectively.
Conclusion
In this paper, we study an essential question in the field of adversarial attacks that when we shouldcollaborate.
( i )
If a single model can handle everything, there is no need for multiple models.
( ii )
If a single model can only handle a part of the whole, collaboration among multiple models makessense.
Adversarial defense is a typical task that falls into the circumstance ( ii ) because a single modelhardly fits adversarial data.
We provided a collaboration framework—SoE—as the defense strategyover ensemble methods, and empirical experiments indeed verified the efficacy of SoE. Future workincludes applying our collaboration framework to other areas such as kernel methods, fairness, andfederated model, etc.
Checklist
For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper’s contribu-
tions and scope?
[Yes] We discuss it in Appendix
(b) Did you describe the limitations of your work?
[Yes] The limitations are in Appendix(c)
Did you discuss any potential negative societal impacts of your work?
[Yes] The societal impacts
are in Appendix
(d) Have you read the ethics review guidelines and ensured that your paper conforms to them?
[Yes]
If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results?
[Yes]
(b) Did you include complete proofs of all theoretical results?
[Yes] The proofs are in Appendix.
If you ran experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main experimentalresults (either in the supplemental material or as a URL)?
[Yes] The source codes are madepublicly available.
(b) Did you specify all the training details (e
g., data splits, hyperparameters, how they were chosen)?
[Yes] The training details are in Appendix.
(c) Did you report error bars (e
g., with respect to the random seed after running experiments
multiple times)?
[Yes]
(d) Did you include the total amount of compute and the type of resources used (e
g., type of GPUs,
internal cluster, or cloud provider)?
[Yes] The devices of computing are in Appendix.4.
If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...
(a) If your work uses existing assets, did you cite the creators?
[Yes]
(b) Did you mention the license of the assets?
[Yes]
(c) Did you include any new assets either in the supplemental material or as a URL?
[N/A]
(d) Did you discuss whether and how consent was obtained from people whose data you’re us-
ing/curating?
[Yes]
(e) Did you discuss whether the data you are using/curating contains personally identifiable
informa-
tion or offensive content?
[Yes] We discuss our used data in Section 4 and Appendix.
If you used crowdsourcing or conducted research with human subjects...
(a) Did you include the