On the Diversity of Adversarial Ensemble Learning
Authors: Jun-Qi Guo
Meng-Zhang Qian
Wei Gao
Zhi-Hua Zhou
Venue: ICML 2025 poster (2025)
Keywords: Adversarial learning
Ensemble learning
Diversity
Primary area: general_machine_learning->everything_else
TL;DR: This work focuses on fundamental problems of diversity in adversarial ensemble learning, proposes the first diversity decompositions in adversarial environment and introduces a new robust ensemble method to enhance these diversities.

Abstract: Diversity has been one of the most crucial factors on the design of adversarial ensemble methods. This work focuses on the fundamental problems: How to define the diversity for the adversarial ensemble, and how to correlate with algorithmic performance. We first show that it is an NP-Hard problem to precisely calculate the diversity of two networks in adversarial ensemble learning, which makes it different from prior  diversity analysis. We present the first diversity decomposition under the first-order approximation for the adversarial ensemble learning. Specifically, the adversarial ensemble loss can be decomposed into average of individual adversarial losses,  gradient diversity, prediction diversity and cross diversity. Hence, it is not sufficient to merely consider the gradient diversity on the characterization of diversity as in previous adversarial ensemble methods. We present diversity decomposition for classification with cross-entropy loss similarly. Based on the theoretical analysis, we develop new  ensemble method via orthogonal adversarial predictions to simultaneously improve gradient diversity and cross diversity. We finally conduct experiments to validate the effectiveness of our method.

On the Diversity of Adversarial Ensemble Learning
Jun-Qi Guo 1
Qian
Wei Gao
Zhi-Hua Zhou
Abstract
2019b; Gowal al., 2021; Pang et al., 2021; Wang et al.,
2023; Peng et al., 2023; Bartoldson et al., 2024).
Diversity has been one of the most crucial factors the design of adversarial ensemble methods. This work focuses on the fundamental problems: How to define the diversity for the adversarial ensemble, and how to correlate with algorithmic performance. We first show that it is an NP-Hard problem to precisely calculate the diversity of two networks adversarial ensemble learning, which makes it different from prior diversity analysis. We present the first diversity decomposition under the first-order approximation for the adversarial ensemble learning. Specifically, the adversarial ensemble loss can be decomposed into average of individual adversarial losses, gradient diversity. prediction diversity and cross diversity. Hence, it is not sufficient merely consider the gradient diversity on the characterization of diversity as in previous adversarial ensemble methods. We present diversity decomposition for classification with cross-entropy loss similarly. Based on the thcoretical analysis, we develop new ensemble method via orthogonal adversarial predictions to simultaneously improve gradient diversity and cross diversity. We finally conduct experiments to validate the effectiiveness of our method.
Ensemble learning combines multiple learners rather than one single learner with better performance, which has been paid much attention in the adversarial robustness learning. Sequential robust ensemble has been constructed via the boosting framework (Abernethy et al., 2021; Zhang et al., 201 9a; 2022; Guo al., 2022), and parallel ensemble has also developed for robust learning (Pinot et al., 2020; Sen et al., 2020; Yang et al., 2021; 2022; Deng & Mu, 2024).
Diversity has always been one of the most crucial factors in the design of ensemble methods (Zhou, 2012; Wood et al., 2023). For adversarial ensemble learning, Pang et al. (2019) took prediction disagreements of base learners as diversity, while Yang al. (2020) defined the diversity via losses of base learners on exchanged adversarial examples. Another idea is to consider the misalignment of gradient directions for diversity (Kariyappa & Qureshi, 2019; Dabouei et al., 2020; Huang et al., 2021; Bogun et al., 2022).
There are some fundamental problems open for adversarial ensemble learning. For example, how formally define the diversity of adversarial ensemble, and what's 's more, how to correlate diversity definition with algorithmic performance from a theoretical view. This work studies fundamental problems of diversity in the adversarial ensemble learning, and the main contributions are summarized as follows:
1. Introduction
We first show that it is an NP-Hard problem to precisely calculate the diversity of two neural networks in the adversarial ensemble, since diversity is heavily relevant to intrinsic structures and output predictions of models simultaneously. This challenge makes it different from traditional diversity on output predictions (Zhou, 2012; Wood et al., 2023). Sun & Zhou (2018) indicated the importance of structure diversity for decision trees, whereas it remains open for neural networks.
General machine learning models may be misled heavily by examples with adversarial perturbations (Szegedy et al., 2014; Goodfellow et al., 2015), which raises some serious concerns about reliability of models, particularly in high- risk applications such healthcare, finance and autonomous driving (Finlayson et al., 2019; Deng et al., 2021; Fursov et al., 2021). Various robust methods have been developed against adversarial examples in recent years (Zhang et al.,
'Equal contribution NNational Key Laboratory for Novel Soft- ware Technology. Nanjing University, Nanjing, China; School of Artificial Intelligence, Nanjing University, Nanjing, China. Corre- spondence to: Wei Gao <gaow @ lamda.jju. mandd..j..edu.cn...
We present the first diversity decomposition under the first-order approximation in the adversarial ensemble learning. Specifically, the adversarial ensemble loss is decomposed into average of individual adversarial losses, predictin diversity, gradient diversity and cross diversity. It is not sufficient to only consider gradient diversity on the characterization of diversity as in prior
Proceedings of the 42nd International Conference Machine Learning. Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).

---

On the Diversity of Adversarial Ensemble Learning
adversarial ensemble methods. We present similar de- composition for classification with cross-entropy loss, which is commonly used for neural networks.
for integer A: 0. For two non-negative real numbers and
b with b 1, we define the KI--divergence
KL(a, b) a aaa/// + (1- In(())nnnn1)))) a) b))
Based on theoretical analysis, we develop the AdvEOAr adversarial ensemble method! via the orthogonal of adversarial predictions of base learners, which could improve gradient and cross diversity simultaneously. We finally conduct empirical studies validate the effectiveness of our AdvEOAp method.
and for two probability vectors a = (a1, a2, am) and
b = (b), b2, ... bm): we define the KI.-divergence as
m
KI b) Σ a ln(a///b))
The rest of this work is constructed as follows: Section 2 presents some preliminaries. Section 3 provides diversity analysis. Section develops our method. Section conducts experiments. Section 6 concludes with future work.
3. Theoretical Analysis on Diversity
Given learners fi (ar),, f' (a:). this work focuses on the simplest ensemble method (Dietterich, 2000; Zhou, 2012)
2. Preliminary
f(x)
172 Σ
fz) m
Let X' S IR' and y denote the instance and label space, respectively, where y/ = {0. 1} for binary classification and y S for regression. Let be an underlying distribution over the product space X' x V, and we have training data
We begin with the challenge on the analysis of adversarial diversity, and then present diversity decomposition w.r.t. squared loss and cross-entropy loss, respectively.
S, = {(x1,))), (22, 1M (Zn: yn)}
3.1. Main challenge on analysis of adversarial diversity
where each sample is drawn i.i.d. from distribution D.
For traditional non-adversarial ensemble learning, it is easy to make the arror-ambiguity decomposition over example (x, y) w.r.. squared loss from (Krogh & Vedelsby, 1994; Zhou, 2012) as follows:
Define the perturbation set AS w.r.t. ip norm and e > 0 as
4 = (8 E : ||ōl|, = (15, + ((2))))) sc
(g(()) yy²
(Jf(2) f(x)))
which shows instances' imperceptible perturbation (Szegedy et al., 2014). Let F= {f X' + R} be a hypothesis space, and loss function L y x y + R is introduced to measure performance. Given f E F, we define the adversarial perturbation w.r.t. example x as
(f(x) v)
ensemble loss
average loss
ambiguity
where the ambiguity can be viewed ensemble diversity, Wood et al. (2023) further presented bias variance-diveeetne decomposition for ensemble learning, simplified by
8 E arg max sea; ((f(x 5).
and æ + 87 is called the adversarial example w.r.t. æ. We define the expected adversarial loss as
ensemble loss bias variance diversity.
It is natural consider some similar decompositions the adversarial diversity learning. However, this remains some challenges shown by the following theorem.
aady (f, D) E max (1(f(x +).
BEA;
and define the empirical adversarial loss w.r.t. data Sn, as
Theorem 3.1. For squared loss, it is an NP-hard problem to precisely calculate the diversity w.r.l. example (æ, y) in the adversarial ensemble learning as follows:
1 Σ n
jady (J, (J,
max (I[[(x;; ()), SEA;
1 2
Σ max (z) y) mmx dEA,
(fxx((+++)))))))))) v)²)
BEA,
We finally introduce some useful notations in this work. Denote by (--) the inner product of two vectors, and e; is unit vector with i-th element 1. Write [k] 21, ... ,
where the error ambiguity decomposition is considered for two neural networks fi( and fz(œ) with ReLU activation, and f(x:) = (fi(z) + f2(2)))22
"ode available at http://gggiaabcooommGn/////////
2

---

On the Diversity of Adversarial Ensemble Learning
Theorem 3.1 shows that, even for the ensemble of two neural networks, it is NP-hard problem to make error ambiguity decomposition in the adversarial ensemble learning. The main challenge is that the adversarial example is heavily dependent on the intrinsic structure of neural network, and diversity analysis is relevant to intrinsic structure, model prediction and adversarial perturbation simultaneously. This different from traditional diversity analysis on model
CIFAR10 1/ nomm ball)
CIFAR10 1/, norm ball)
1.20rt0e 169
3.:35
1116-
2.22
advesuttd
Alversa ensombilies
Predition shondyy
Grod ot chersty
10 Cram divertty
Number Base Leamers
CIFAR1O (I. norm ball!
CIFARIO il norm ball)
predictions (Zhou, 2012; Wood et al., 2023). We also notice the importance of structure diversity for decision trees (Sun & Zhou, 2018), whereas it remains open for neural networks.
Acvertar mnsmmbilee
Red clon diversity Grad enc sirrrky
2.00
Cross diverity
10
Number
Number
The proof idea involves the reduction of 3-SAT problem. Specially, we consider a 3-SAT problem with k clauses, and each clause is disjunction of literals. We construct neural network g(x) with @(k) layers and (kk) width, where each literal can be regarded as input node of gxx), and each disjunction and conjunction can be replaced with the max and min operators, respectively. The max and min operators can be constructed by ReLU activation. We construct two neural networks fi fxx,)))) =o) = min(g(x). xo) and (a,, vo) = min(g(2:), -ro) by adding an auxiliary variable Ty. The detailed proof is given in Appendix A,
Figure I. An illustration of diversity decomposition Theorem 3.2 on dataset CIFAR10 of perturbation balls with l2 and loo norm, respectively. Here, we consider CNNs as base learners.
2 [fs(()lalf{())> - ul - - yl
cross diversity
where 1/p + 1/9 =
which is partially motivated from previous work on l or to norm (Katz et al., 2017; Weng et al., 2018), while our work generalizes to Ip norm with p= = 1,2, ... 00.
In this theorem, the adversarial ensemble loss is decomposed into the average of individual adversarial losses, prediction diversity, gradient diversity and cross diversity. Gradient diversity measures the dispersion of gradients concerning the mean of gradients w.r.t. lg norm, and it exactly becomes the variance as for = 2. Prediction diversity can be understood as the traditional diversity such as ambiguity (Krogh & Vedelsby, 1994; Zhou, 2012). Cross diversity can be viewed as a cross between functional outputs and gradients. Gradient and cross diversities are highly relevant to intrinsic structures of functions and functional outputs. The detailed proof is given in Appendix B.I.
3.2. Diversity decomposition w.r.t. squared loss
Previous ensemble methods generally take the first-order approximation for adversarial loss function, and focus on gradient diversity (Kariyappa & Qureshi, 2019; Dabouei et al., 2020; Huang et al., 2021). This is partially because of the NP-hardness for adversarial diversity in Theorem 3.1. Following the first-order Taylor approximation, we have
f(x+0)= = fa 6)
From Theorem 3.2, it is also observable that gradient and cross diversities take more important roles on adversarial ensemble learning as for larger e (i.c., radius of perturbation ball), yet prediction diversity dominates as for smaller e. Also, the decomposition is relevant to the 1p norm distance of perturbation ball. Thus, the characterization of diversities in adversarial ensemble learning is more complicated than that of traditional non adversaaalll ensemble learning.
We now present the first decomposition for the adversarial ensemble loss w.r.t squared loss as follows:
Theorem 3.2. For ensemble f = ΣT= fs/m, we present the decomposition of adversarial ensemble loss under the first-order approximation over example (xr, y) as follows:
Figure 1 presents an intuitive illustration for the diversity decomposition of Theorem 3.2. Here, we consider dataset CIFAR10 with two classes, and focus on the perturbation ball with the popular l2 and L.0 norm. More experiment details are given in Appendix B.2, and we try understand the trends of loss functions and diversities as the number of CNN-base learners increases.
max (1 + 5)- 2)
max m (f(x)()))) SEA:
average of individual adversarial losses
m
(f(z)
m
[f -1f(()))
As can be seen from Figure 1, we have smaller adversarial ensemble loss as for larger prediction, gradient and cross diversities w.r.t. 12 and los norm perturbation balls, when we
m
m
1=1
prediction diversity
gradient diversity
3

---

On the Diversity of Adversarial Ensemble Learning
keep average of individual adversarial losses stable. Gener- ally, gradient and cross diversities are larger than prediction diversity and take more important roles. This is nicely in accordance with our Theorem 3.2, and diversity empirically plays an important role in adversarial ensemble learning.
CFAR10 ( norm ball
CIFARLO II, norm
11.20
1.15
1:10
Paringh
Rusi agaaa Value
Aiversatte loss
2.60
2.50
sg CoO Valu
edversauiiis
Advetrattal sneembi 100
1.20
Adrervatd anembb 1o0
2.30
22 120
1.8g
10 220
Number of CNN Base Leamers
Number Base Leamers
We focus on the first-order approximation as previous adversarial ensemble methods (Kariyappa Qureshi, 2019; Dabouui et al., 2020; Huang et al., 2021), and is interesting to explore the second-order (or higher-order) approximation from gradients and Hessian matrices. The main challenge is how to obtain the closed-forn solution of adversarial loss via second-order approximation, because it is relevant to the roots of high-order polynomials (Forsythe & Golub, 1965; Moré Sorensen, 1983; Fortin & Wolkowicz, 2004). More discussions on this issue are given in Appendix B.3.
Figure 2. An illustration of the influence of average cos value
dataset CIFAR10 of perturbation balls with 2a and to norm, respectively. Hore, we consider CNNs as base learners.
where a = (v5- and (V6+1)/(a+ V V5a). We study two ensembles: one ensemble of fi and f2; the other ensemble of fs and f1. For example (xx,) y) = ([L, 0j. 1) and
perturbation set A = {ō ||0 S 2). we have
cos (1 V [2())) = cos V fa())) fa()))
Relevant to previous adversarial ensemble methods
also with the same average of individual adversarial losses
Most previous ensemble methods consider the first-order approximation of adversarial loss functions (Kariyappa & Qureshi, 2019; Dabouei et al., 2020; Yang et al., 2021; Bogun al., 2022), and the diversity is measured by the average of values over pairs of gradients of base learner. Specifically, the diversity w.r.t. instance is given by
2 (fi( + 5) y)
(fi(x + +5) -y)
max SEA
max BEA
2
2
1=1
However, the adversarial ensemble losses are different from
max(fi(2 + 6)/2 8)/2 - y) = 8,
SEA
m
m
max (fxx(f((x= 5)/2 f4( + 6)/2 2 y)² 7.2. AEA
COS (V)(()) V[()))
m(m - 1)
1=1+1
Here, we consider the 12 norm in perturbation set A, and similar analysis could be made for nn-norm. More details are presented in Appendix B.4.
where cos (z),,, (z)) denotes the cos value of the angle between f(()) and (z)( We could derive the relationship between our gradient diversity and previous diversity via the cos functions as follows:
In addition to Example 1, we could also present empirical studies on adversarial ensemble losses versus the cos values over dataset CIFAR10, and the experiment details are given in Appendix B.2. Figure 2 shows the curves of adversarial ensemble loss, the averages of cos values and individual adversarial losses with l2 and loo norm.
Gradient diversity- = m 2
m2
[VA
(S cos (z) (x))
m
From Figure 2, is clear that adversarial ensemble losses keep decreasing when we increase number of CNN- learners, whereas it almost remains constant for the averages of cos values and individual adversarial losses. Therefore, it is not sufficient to characterize the diversity of adversrrial ensemble by merely considering the average of cos values as in (Dabouei et al., 2020; Bogun et al., 2022).
It is feasible to enlarge gradient diversity by decreasing cos functions, which is nicely in accordance with previous work (Dabouei et al., 2020; Bogun et al., 2022). Meanwhile, it is noteworthy of other important factors on gradient diversity such as gradient norm, rather than only one factor, which can be shown by following examples.
Example 1. There exist two ensembles of the same averages of cos values and individual adversarial losses, but with different adversarial ensemble losses.
3.3. Diversity decomposition w.rt. cross-entropy loss
We study the decomposition of adversarial ensemble w.r.t. cross-entropy loss for binary classification, and also follow the first-order approximation f( se (Z)+Vf(())) 8. For base learners with logit outputs (i.e., the logarithm of the ratio of probabilities), we have the probability of the positive class over æ + 5 as follows:
Proof. We focus on 2-dimensional instance space X' C R22 and label space y C R. and consider
fi()) = σι + 22, fa(x)) aboi + brz,
f2(x) = I1 - -322
fa(x)) = abıı .bz
( + 6) = (1 +
+ V)) (1)
4

---

On the Diversity of Adversarial Ensemble Learning
CIFAR10 b norm ball
CIFAR10 D, norm bell)
CIFARIO ( norm
norm
5.55
5.70
Anerage Coo Vale
2.40
3.59
3.50
aevernatia loss
Average
Avereg valu
2440
Mtterans rnsorntl let
Does deanity
Adversural crovn boo
Advassu crrar tie hoo
3.30
2.30
10 12..
Number of CNN Baon Learners 5.20
5
12
Number
Number CNN Base Learners
5.00
GFAM10 UI norm ball!
Revera 111
(I. norm ball)
Figure 4. An illustration of influence of the average cos value on dataset CIFAR10 perturbation balls w.r.t. l2 and too norm. respectively. Here, we consider CNNs as base learners.
5.60
0.50
Averags adveriarii 18s5
Advamar craumbre co
1:0
CNN
diversity and cross diversity. The cross diversity shows the diversity of base learners according to their KL divergence to ensemble f. which is relevant to function outputs and gradients, as shown in Lemma 3.3. is evident that the decomposition of Theorem 3.4 is quite different from that of Theorem 3.2 because of different loss functions. The detailed proof is presented in Appendix. C.2.
Figure 3. An illustration of diversity decomposition in Theorem 3.4 on dataset CIFAR10 of perturbation balls with l2 and loo norm, respectively. Here, we consider CNNs as base learners.
The cross-entropy loss over (x + y. is given by
We also present Figure 3 to illustrate the decomposition of Theorem 3.4, and some experimental details are given in Appendix B.2. As can be seen, adversarial ensemble loss gets smaller for larger cross and gradient diversities w.r.t. l2 and t60 norm perturbation balls, when maintain the average of individual adversarial losses stable. In addition, it is not sufficient to characterize the diversity of adversarial ensemble by only merely considering the average of cos values as done in (Dabouei et al., 2020; Bogun et al., 2022). We also present some empirical studies on the influence of average cos values on adversarial ensemble loss shown in Figure 4, and more details are given in Appendix C.3.
- V(
+ +nn11 + (xx + ())))
as in (Bishop Nasrabadi, 2006), and recall that
89 E arg max [(f(= 8).
We have the following closed-form solution for p (a + 8*), and the detailed proof is presented in Appendix C.1. Lemma 3.3. For function f and example (æ,, y). we have
where probability of positive class (-(-) and perturbation 67 are defined by Eqns. (1) and (2), respectively.
4. Our AdvEOAp Method
Motivated from our theoretical analysis in Theorem 3.4, we develop robust ensemble method for multi-class learning, and the basic idea is to train multiple deep neural networks for adversarial ensemble via a regularization by considering gradient diversity and cross diversity simultaneously.
Based on this lemma, we have
Theorem 3.4. For ensemble f E fa/m, we have the decomposition of adversarial ensemble loss under the first first-order approximation over example (x, y) as follows:
For multi-class learning with к classes, the base learner f = (fi, f2, fk): X' + KK maps each instance to a x-dimensional logit vector. The predicted probability vector ( = (py, (z),P%,
m maxsEA; {(5 () max {ff(x + 8), SEA,
m
P1,) (2z), ... Py. (z)) can be calculated from f (a) via softmax function as follows
averuge of individuul adversarial losses )). ps, (āx,)
m (x) -1V
m KL(p,(ā) 7)
cxp (fi(zz)
PIz (x)
(2) = - exp (f,
(3)
m
m
1=1
gradient diversity
cross diversity
This section also focuses on the first-order approximation f(x 5) R f(x) + (2)6 where Jz(x) is the Jacobi matrix of f (Rudin al., 1964). Denote by
where 1/p + 1/q = 1, r e(y py(ā;) py(āt)) - 1), P-() is defined by Eqn. (1), and , = x + 57 is the adversarial example with 87 from Eqn. (2).
8, E arg max (ff(x x 8), v)
(4)
In this theorem. the adversarial ensemble loss is decomposed into the average of individual adversarial losses, gradient
For m base learners fi,... fm with f, = (fa1, 8zx),
5

---

On the Diversity of Adversarial Ensemble Learning
Algorithm The AdvEOAp method
Probability
Leamerl
Probasiliit
Learner2
Input: training dataset S, number of base learners m, learning rate 1j and the SGD iterations T Initialize: base learners fi,"..," f
class class 2. class 3 class 4
class class
for = 1 to 'I' do
Partition training dataset S into batches S for k E [b] do
Probabliity
Learner3
Probabliity
Ensemble
Generate adversarial examples for every base learner and example by the PGD-attack
class class 2 class 3
class
cluss
class
class class
for j = 1 to do
Update learner fy by gradient descent in Eqn. (5)
Figure 5. An illustration of orthogonality by optimizing Eqn. (5).
end for end for end for Output: Ensemble model f f= -ド= f3/m
we have cross diversity for multi-classification
(L(pf(0 Pj).Ps. Pr, (x +7,))
Cross diversity
m
where py (.) and 67 are given by Eqns. (3), respectively. From some algehraic derivations in Appendix D.1, we have
larger volume and entropy in I' (x, y). and hence improve gradient diversity and cross diversity simultaneously.
(r, (2)0, - {2x
Given training sample S = {1)1,)1)... yi).... ( Bn)) we present the final objective optimization as
Gradient diversity
m
(r, fy (x= + 83,) f + 87))
m
E(Jz(a; + 5), y1) AT (2..
max BEA
(5)
where r = (p + 87) Ey-
i= j=1
For multi-class learning, we could simultaneously improve cross diversity and gradient diversity by diversifying the output predictions of adversarial examples of base learners. This motivates to develop
where is a regularization T. (-). We could obtain mutual orthogonal base learners f1, fm from the optimization of Eqn. (5), and the details given in Appendix D.2.
to tradeoff adversarial loss and
ensemble algorithm via orthogonal adversarial predictions, i.e., we orthogonalize the outputs over adversarial examples for base learners to improve diversity. The orthogonal idea is inspired by (Pang et al., 2019), which limited only on clean examples, while our work generalizes to adversarial examples.
Figure 5 gives an illustration for the orthogonality of base learners. Here are three base learners in multi-class learning of 4 classes, and the ground-truth class is class 1. The output predictions are mutually orthogonal except for the ground- truth class 1. Therefore, the ensemble of three base learners could make correct prediction robustly even if three base learners are misled to different classes.
Specifically, introduce regularization for orthogonal adversarial output predictions of fi,.... f.
Pf (ar + 87,) +a log log(det(ATA)),
(x = H
On the optimization of Eqn. (5), we take adversarial training method with stochastic gradient descent from (Madry et al., 2018). We generate adversarial examples w.r.t. all base learners by PGD-attack, and calculate gradients to update the parameters for each base learner of deep neural network. Algorithm presents the detailed description for our Ad- versarial Ensemble training with Orthogonal Adversarial Predictions, which is short for AdvEOAP. For Algorithm 1, the time complexity takes m-times as that of training a single neural network adversarially. In addition, it takes (m²)) computational cost to calculate the regularization and gradients. More details are given in Appendix D.3
m
where H(-) is the function of information entropy and A P (x 5, ... Pr. (x + 8 (k)m))xm
Here, (-(() is an (K - (KK-1)-dime dimenssioaa vector obtained by removing the y-th element of (-) and normalizing with 4 norm, and det(A" A A) shows the square of volume of polytope spanned by py (x 57, , (x 5% 6 in (Bernstein, 2009). Intuitively, the orthogonal probabil- ity vectors Pf (x: ) ",,),... Py., (ær + 8 could yield
as
6

---

On the Diversity of Adversarial Ensemble Learning
Table Comparison of classification accuracies (mean + std %) over adversarial examples generated by different adversarial attacks.
/o indicates that our Adv is significantly better/worse than the corresponding methods (pair-wise t-test 95% significance level).
Methods
FGSM
PGD10
PGD20
PGD40
AutoPGD
MORA
AutoAttack
Our AdvEaw
96.413=0.124
95.676:20.039
95.648::00056
95.504=20.098
94.907=001187
94.672::0.209
94.072::::23
GAL
10.271-H0.709
00.001++0.002+
00.002=0.005
00.000++0.000
00.000+00000
00.004+000055
00.000+0.000
ADP
10.888+:1.9314
00.000==0.000.
00.000=0.000+
00.000=+0.000+
00.000++0.000
00.004=0.0000
00.000=0.000
MNIST
AdvADP
95.333:10.0998*
95. 1..6119::0061
95.059:10.089
93.032:10.11e
93.298:10.109
93.172::00033
92.948:10.062.
DVERGE
74.903:11.055
37.506:14.292
4.91
05, 2.633
00.00110.002+
00.299=0.361
00.000110000w
PDD
10..4461:1.0 w
06.0411:2.66 640
04.059 2.904
02.256::2.792
01.163:11 1.295
04. 4.000000000
00.000110.000+
TRS
91. 91.044=008922
86,544 1.300
86.709=1..0 1.044
15.9544+3.283+
80.902:t-4 905
79.628=5.477
78.615=559997.
iGATADP
83.814112.408
79.892:114445
1.732
79.778:+2.028
59.357
50.208=6.439
48.5 48.5899-5.1788
Our AdvEoxr
82,743:10.265
$1.770-10.245
81.752:10.248
80.543-10.290
81.467:10.079
80.643-10.332
80.506-10.327
GAL
15.404=66.709 00.352:0.4447
00.170=0.215
00.315:+0,306
00.000=+0.000
00.009=0.005
00.000=00000+
ADP
22.287:41.7444
00.00111.0002
00.001
00.000:10.000
00.000010.000+
00.009=00005
00.000110000+
F-MNIST
AdvADP
82.728=0.331
79.167 0.3388
79.074=0.387
80.080=+0.408
78.461+003772+
77.833=0031344
77.732++0.245
DVERGE
48.24221:1.566
27. 27.140-12.8778
15.846-3.0011
22.053
17.222:14.775
D0.1261:.60555
15.035:15.5660
PDD
18.740++4.9255 1
17.958=4.85
19.043
12.161+:5.521
14.408
00.319:+0.466w
TRS
70.767:10.466
69.330.10.12 1
69.285=0.1055
67.6441::0.444
68.719::000988
67.800=0.1133
67. 0..200
iGATADR
66.278:+2.05 +
63. 13.13390.236e
62.967=0.1499
02.8822:033799
61.920=:0.088
51.869
48.750==5.633
Our AdvFowr
55.718:+0.245
53.076++0.249
52.996++0.255
$2.903++0.299
5..994++02234
48.318++0.065
47.884+0.000
GAL
12.370:12.959.
00.00711..0134
00.000=0.000
00.00001:00000+
00.000
00.002 +.04
00.000110.000+
ADP
23.100:#0.7574 00.000::0.010+
00.001
00.000=+0.0000
00.000:
00.001=0.003+
00.000=10.000+
CIFAR10
AdvADP
55.478:10.214
47.116:10.2788
16.802-002844
46.573:10.332+
44. 0,1122
42.904=0.235
12:
DVERGE
28.536+00.822 0..358+
0..830=0.343+
04.690=:02269
02.246
02.868=0.177 178
01.748:+0.166
PDD
23.750:14.005 14.116:14.5%
14.8% 4.556
17.400:12.710
2.3555
09.526
01.000-10.626
TRS
39.350++0.4024 37.453=0.4400 37.548+00431
37.263++02259
36.690+004440
32.828=00433e
32.588++0.433
iGATADR
19.990:10.509.
18.07510.035
18.000=0.056
13.707:12.566
17.260110099
12.365 1.04+
12.090
5. Experiments
PDD: aon-adversarial training to diversify the feature representations via dropouts (Huang et al., 2021);
We conduct experiments on three datasets MNIST of 70000 images and 784 dimensions, F-MNIST of 70000 images and 784 dimensions, and CIFAR10 of 60000 im- ages and 3072 dimensions. Three datasets have been well- studied in previous works (Strauss al., 2017; Kariyappa & Qureshi, 2019; Yang al., 2021; Deng & Mu, 2024). We compare our method with the state-of-the-art methods adversarial ensemble learning as follows:
TRS: GAL by preserving the smoothness of base learn- ers (Yang et al., 2021);
ADP: AdvADP by allocating globally adversarial examples base learners (Deng & Mu, 2024).
For all datasets, the perturbation size is set as 0.2, 0.05 and 0.03 under -oom ball, respectively, as done in (Croce & Hein, 2020: Deng & Mu, 2024). For all methods, we select ResNet20 base learners with learners number 3 and 8 for MNIST, F-MNIST and CIFAR10, respectively. More settings are given in Appendix E. All experiments are performed on a server with 64 CPU cores (2 Intel Xeon Gold 6430 CPUs) and NVIDIA GeForce RTX 4090 GPU, running Ubuntu 24.04 with 1TB main memory.
GAL: Non-adversarial training via the diversity of cos values of gradients (Kariyappa & Qureshi, 2019);
ADP: Non-adversarial training via the diversity of the orthogonality of predictions (Pang et al., 2019);
AdvADP: Adversarial training via the diversity of the orthogonality of predictions (Pang et al., 2019);
DVERGE: Adversarial training on the exchange of adversarial examples base learners to diversify the
5.1. Performance under adversarial attacks
We take accuracy to measure performance on adversarial examples generated by seven popular adversarial attacks, i.e., FGSM (Goodfellow et al., 2015), PGD10, PGD20,
adversarial vulnerability (Yang et al., 2020);
Download from https//paperwwithcole..oo/////// http:///iitcoeeeecmmtaatsstttt

---

On the Diversity of Adversarial Ensemble Learning
Table Comparison classification accuracies (mean std %) over adversarial examples generated by EOT and BPDA attacks. e indicates that our AdvEoAr is significantly better/worse than the corresponding methods (pair-wise t-test 95% significance level).
Attacks
Datasets
Our AdvEuw
GAL
AdvADP
PDD
DVERGE
TRS
iGATxar
MNIST
88. 0,366
01.178+ 1,958
85.724 1633
03.913345.471
75.535-1-.9811 38.358-43.1477
1..221:11.2888
E01
F-MNIST
62.416=00.778
02:201 + 1.866
0,5137==0,477
08.5234::18115
36.162:42.390+
56,592::1.11 10
$7.893::2.147
CIFATIO
02.003==0.504
01.490=:0.179
002387=002238
08.535::0.006
20.571::0.500+
31.629::1
16:408:42.6977
MNIST
95.382:00.09 00.002:#0.005.
92.713:::01337
02.757
14.187:4:7.8666
85.02241:.7588
71.763:43.041
BPDA:
F-MNIST
80.080=00226
00.352:=0.447
0.307
17.958:::.873
25.846:::.019
66.409:1:0:450
62,967:1:0.1999
CIFATIO
49.530=00.00
00.000==00000+
0.286
14.8962:4.56
04.730:10.344
33.417::0.310
10.693::1.776
BPDAs
MNIST
95.676=0. 104
00.000=:0.000+
93.646:20.1336
12.398:::4.848
19.385::8.467
86.783:1:34466
84.261:::..677
F-MNIST
0031 01.099::0:233+
80.832::0.38
30,152:48.997
40.672+42.473+
68.032-10.366
73.444::0.657
CIFATIO
53.092:#0.200 00.000::00000
47.858::0.1744
25.277:t:4.4499
05.608 +00.43
37.383-10.355
14.500+H2.6608
BPDAa
MNIST
00.000:10.000+ 0..10
94.857
09.165::0
57.491:::..6409
88.9933-228873
91.126::0.6722
F-MNIST
01.309:0.409 00.343--02206
82.106-:0.466
30.265-4-91199
52.1577-:1.6558
68.900-H0.3300
75.948
CIFATIO
52.440=00226 00.000=:0.000.
46,240::0.252+
23.690:::.737+
05.347:::0.382
36,480:1:0.211
13.600
BPDA4
MNIST
94,596:00.15 135
00.022::0.020
92.830:
04,941 4:4.88
02,635:::1 .482
81.5955--..2556
44 44.669:E7. 900
F-MNIST
80.241 00.026::0.045
80.757:+003888
05,706-t:7.5955
50.324:1-4. 4.693
67.526:1:0.667
066674
CIFATIO
53.383=0.1100
00.273-:00033
53.397.10.110 160
12.650 1.642
26.573:+0.046
00.5077-0.3999
14.41:::1.111
Table 3. Comparison of accuracies stt))) for our AdvEOAr
1:30
MNISST
1,00
PHNIST
with and without regularization I'(-) under the APGD attack.
3.88
0,90
3.80
20.3
6,80
99.30
Our methed
0.70
Our methoc
Our AdvEosp
MNIST
F-MNIST
CIFARIO
3.
AddA07
ACHADT
without ('aO]
93,513+-0.046
80.317:t0.159
50.833++0,10 102
0.85
0.33
0,50
-4-
TRS
16A7
with F-
94.907+-0.187
31.2012-0.111
51.997+40.23
0.24
0.40)
Perturaation Size
Perturbation Size
Improvement
1,394=0,149f
1.150++0.140"
1.163=0.2571
Figure 6. Influence of perturbation sizes under the PGD20 attack.
PGD40 (Madry et al., 2018). AutoPGD (Croce & Hein, 2020), MORA (Gao et al., 2022) and AutoAttack (Croce & Hein, 2020). All methods are evaluated over 50 runs with different random initializations, as summarized in Table 1.
where EOT considers adversarial perturbations insensitive to transformations, and BPDA considers potential gradient risks and designs corresponded attacks. More details are given in Appendix F.2. We implement EOT and four BPDA attacks, and experimental comparisons are summarized in Table 2. It is obvious that our AdvEOAP method achieves better performance than other adversarial ensembles, which shows the robustness of our method to gradient risks and adversarial perturbations insensitive to transformations.
From Table 1, it is clear that our AdvEOAP method achieves significantly better performance than GAL, PDD and TRS, since it wins at most times and never loses. This is because such methods merely focus on cos values as the diversity measure, yet ignore other factors such as cross diversity and individual adversarial losses, which is consistent with the ensemble decomposition in Theorem 3.4.
We also present some ablation experiments to verify the effectiveness of the regularization (a(()) in Eqn. (5), which essentially considers gradient diversity and cross diversity via orthogonal adversarial predictions. Table 3 shows some experimental comparisons for our AdvEOAp with and with- out regularization. It is clear that our method takes better performance with regularization, which nicely shows the importance of diversity on the design of ensemble methods.
Our AdvEOAP is also better than DEVERGE and iGATADP: since DVERGE exchanges adversarial examples of base learners and iGATADP allocates adversarial examples of ensemble, without the consideration of diversities of base learners. Our AdvEOAP outperforms ADP and AD, since such methods consider diversity over clean examples, rather than adversarial examples. It is quite different study diversity in the adversarial ensemble learning, which is heavily relevant to intrinsic structures and output predictions of models simultaneously as in Theorem 3.1.
We study the influence of different perturbation size over datasets MNIST and FMNIST, as shown in Figure 6. It is observable that our AdvEOAP achieves better or comparable performance than other ensemble methods for different size of perturbations, in particular for larger e. This shows the effectiveness of our methods for larger perturbation size.
We further study two additional adversarial attacks BPDA (Athalye et al., 80) and EOT (Athalye et al., 2018b),
8

---

On the Diversity of Adversarial Ensemble Learning
MNIST
MNIST
Our AdvE-GAP Cela
ADMoon-add) AdVADP
DVERGE
TRS IGAT
1.20
Buaniig Trrm (veconds)
GALinon adx))
POD
3.40
20.30
Average
0.60
Gradiant diverty
Choss disenity
0.50
10
32
Epach
Epoch
WNIST
F.MNIST
CIFARLO
FMNIST
FMNIST
Datasets
Adversanallos
1.200
Figure 7. Comparisons of running time (scconds/epoch) for our AdvEaAr and other adversarial ensemble methods.
0.30
adversafd
0.50
Adeessria
Grodiett dieisty
Cross dirursty
0.50
Epach
100
MNIST
2.0
FM
CIFRRO
CIFARTO
With reg Aariation Whout Inteal
Frequuncyy
Winh B4 mmssrrguubrcettn
1:20
R.c.
3.05
0.4
0.300
Avcroge adrassrial
Diverssy
02:
Grudlient sity
0.0
1.0
0.75
Coasa dirurslty
Chsses
Number Misted Casses
0.70
100
150 Epoch
200
250
Epoth
Figure 8. The frequency of number of misled classes.
Figure 9. The curve of adversarial losses and diversities during the training process, where we normalize according the average of individual adversarial losses.
We finally compare the training time of AdvEOAp with other methods in Figure 7. As can be seen, our AdvEOAP takes comparable training time to other adversarial-trainigg en- sembles Ad,ADP, DVERGE, PDD, TRS and iGAT, but with more time than the aon-adveraaiial training methods GAL and ADP, which obviously take smaller adversarial prediction accuracy as shown in Table 1.
ensemble learning. We prove the NP-Hard problem on the precise calculation of diversity for networks in adversarial ensemble learning, and give the first diversity decomposition under first-order approximation. Specifically, adversarial ensemble loss can be decomposed into average of individual adversarial losses, prediction diversity, gradient diversity and cross diversity. We consider similar diversity decompo- sition for classification with cross-entropy loss. Based on theoretical analysis, we develop new ensemble method via orthogonal adversarial predictions to improve gradient and cross diversity simultaneously. An interesting future work is to explore other adversarial ensemble algorithms with better robustness and generalization from our theoretical analysis.
5.2. Orthogonality and convergence analysis
We now illustrate the orthogonality of base learners for our AdvEOΛp, which could mislead base learners different classes under adversarial attacks. Figure& 8 summarizes the frequency of number of misled classes with 8 base learners over two datasets MNIST and FMNIST. It is clear that base learners of our AdvEOAp predict with more different classes than that of AdvEOAP without regularization.
Acknowledgements
We also present the convergence analysis on adversarial ensemble loss, average of adversarial losses, well gradient diversity and cross diversity during the training process. Figure 9 presents the convergence curves over three datasets MNIST, FMNIST and CIFAR10. is clear that our AdvEOAP method could decrease adversarial ensemble loss, and simultaneously increase gradient diversity and cross diversity. This is nicely in accordance with our diversity decomposition in Theorem 3.4.
The authors want to thank the reviewers for their helpful comments and suggestions. This research was supported by National Key R&D Program of China (2021ZD01 (1TZDD0112022) and NSFC (62376119).
Impact Statement
This work shows the NP-Hardness of the calculation for diversity in adversarial ensemble learning and presents the first diversity decomposition with first-order approximation. It further develops new ensemble method for adversarial defense and validates it through a series of experiments.
6. Conclusion
Diversity has always been one of the most crucial factors on the designs of ensemble methods. This work focuses on the fundamental problems of diversity in the adversarial
There are many potential societal consequences of our work, none of which we feel must be specifically highlighted here.

---

On the Diversity of Adversarial Ensemble Learning
References
Deng, Y., Zhang, T., Lou, G., Zheng, X., Jin, J., and Han, Q.-I. Deep learning-based autonomous driving systems: A survey of attacks and defenses. IEEE Transactions on Industrial Informatics, 17(12):7877-7912, 2021.
Abernethy, J., Awasthi, P., and Kale, S. A multiclass boost- ing framework for achieving fast and provable adversarial robustness. CoRR, 0as/2103.01276, 2021.
Dietterich, T. G. Ensemble methods in machine learning. In Proceedings of the 1st International Workshop on Mul- tiple Classifier Systems, pp- 1-15, Cagliari, Italy, 2000.
Andriushchenko, M., Croce, F., Flammarion, N., and Hein, M. Square attack: A query-efficient black-box adversarial attack via random search. In Proceedings of the 16th European Conference on Computer Vision, pp. 484-501, Virtual Event, 2020.
Finlayson, S. G., Bowers, J. D., Ito, J., Zittrain, J. L., Beam, A. L., and Kohane, I. S. Adversarial attacks on medical machine learning. Science, 363(6443)
2019.
Athalye, A., Carlini, N., and Wagner, D. Obfuscated gra- dients give a false sense of security: Circumventing de- fenses to adversarial examples. 35th International Conference on Machine Learning, pp. 274-283, Stockholm, Sweden, 2018a.
Forsythe, G. E. and Golub, G. H. On the stationary values of a second-degree polynomial on the unit sphere. Journal of the Society for Industrial and Applied Mathematics, 13 (4): 145:0000-, 1965.
Proceedings of the
Fortin, C. and Wolkowicz, H. The trust region subproblem and semidefinite programming. Optimization Methods and Software, 19(1):1-67, 2004.
Athalye, A., Engstrom, L., Ilyas, A., and Kwok, K. Synthe- sizing robust adversarial examples. In Proceedings of the 35th International Conference on Machine Learning. pp- 284 293, Stockholm, Sweden, 2018b.
Fursov, I., Morozov, M., Kaploukhaya, N., Kovtun, E., Rivera-Castro, R., Gusev, G., Babaev, D., Kireev, I., 7a- ytsev, A., and Burnaev, E. Adversarial attacks on deep models for financial transaction records. In Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining, pp. 2868-2870 Virtual Event, 2021.
Bartoldson, B. R., Diffenderfer, J., Parasyris, K., and Kailkhura, B. Adversarial robustness limits via scaling- law and human-alignmenn studies. In Proceedings of the 41 International Conference on Machine Learning, pp. 3046-3072, Vienna, Austria, 2024.
Gao, X., Xu, C.-Z., et al. Mora: Improving ensemble ro- bustness evaluation with model reweighing attack. In Advances in Neural Information Processing Systems 35, pp. 26955-26965 New Orleans, LA, 2022.
Bernstein, D. S. Matrix Mathematies: Theory, Facts, and Formulas. Princeton University Press, 2009.
Bishop, C. M. and Nasrabadi, N. M. Pattern Recognition and Machine Learning, volume 4. Springer, 2006.
Goodfellow, J., Shlens, J., and Szegedy, C. Explaining and harnessing adversarial examples. In Proceedings of the 3rd International Conference on Learning Represen- tations, San Diego, CA, 2015.
Bogun, A., Kostadinov, D., and Borth, D. Saliency diver- sified deep ensemble for robustness to adversaries. In Proceedings of the 36th AAAI Workshop Adversarial Machine Learning and Beyond, Virtual Event, 2022.
Gowal, S., Rebuffi, S.-A., Wiles, O., Stimberg, F,, Calian, D.A. A., and Mann, T. A. Improving robustness using gener- ated data. In Advances in Neural Information Processing Systems 34, pp. 4218 443-3423, Virtual Event, 2021.
Croce, F. and Hein, M. Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-fiee free attacks. In Proceedings of the 37th International Con- ference on Machine Learning, pp. 2206-2216 Vienna, Austria, 2020.
Guo, J.. Teng, M.-Z.,, Gao, W., and Zhou, Z.-H. Fast provably robust decision trees and boosting. In Proceed- ings of the 39th International Conference Machine Learning, pp. 8127-8144 8144, Baltimore, MD, 2022.
Dabouei, A., Soleymani, S., Taherkhani, F., Dawson, J., and Nasrabadi, N. M. Exploiting joint robustness to adver- sarial perturbations. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 1122-1131, Seattle, WA, 2020.
Huang, B., Ke, Z., Wang, Y., Wang, W., Shen, L., and Liu, F. Adversarial defence by diversified simultaneous training of deep ensembles. In Proceedings of the 35th AAAI Con- ference on Artificial Intelligence, pp. 7823-7831, Virtual Event, 2021.
Deng, Y. and Mu, T. Understanding and improving ensem- ble adversarial defense. In Advances in Neural Infor- mation Processing Systems 37, pp. 58075-58088 New Orleans, LA, 2024.
Kariyappa, S. and Qureshi, M. K. Improving adversarial robustness of ensembles with diversity training. CoRR, abs/1901. 90.9911,,99,,, 2019.
10

---

On the Diversity of Adversarial Ensemble Learning
Katz, G., Barrett, C., Dill, D. L.,, Julian, K., and Kochender- fer, J. Reluplex: An efficient smt solver for verifying deep neural networks. In Proceedings of the 29th Inter- national Conference on Computer Aided Verification, pp. 97-117, Heidelberg, Germany, 2017.
Strauss, T., Hanselmann, M., Junginger, A., and Ulmer, H. Ensemble methods as defense to adversarial perturbations against deep neural networks. CoRR, abs/ (e/1109993422, 2017.
Sun, and Zhou, Z.-H. Structural diversity of decision tree ensemble learning. Frontiers of Computer Science, 12(3): 560-570, 2018.
Krogh, A. and Vedelsby, J. Neural network ensembles, cross validation, and active learning. In Advances in Neural Information Processing Systems 7. pp. 231, Denver, CO, 1994.
Szegedy, C.,, Zaremba, W., Sutskever, I., Bruna, J., Erhan, D., Goodfellow, L., and Fergus, R. Intriguing properties of neural networks. In Proceedings of the 2nd International Conference on Learning Representations, BanfI, Canada, 2014.
Madry, A., Makelov, A., Schmidt, L., Tsipras, D., and Vladu, A. Towards deep learning models resistant to adversarial attacks. In Proceedings of the 6th Interna- tional Conference on Learning Representations, Vancou- ver, Canada, 2018.
Wang. Z.,, Pang, T., Du, C., Lin, M., Liu, W., and Yan, S. Better diffusion models further improve adversarial training. In Proceedings of the 40th International Confer- ence on Machine Learning, pp. 36246-36263, Honolulu, Hawaii, 2023.
Moré, J. J. and Sorensen, D. C. Computing trust region step. SIAM Journal on Scientific and Statistical Comput- ing, 4(3):553-572, 1983.
Pang, Xu, K., Du, C., Chen, N., and Zhu. J. Improving adversarial robustness via promoting ensemble diversity. In Proceedings of the 36th International Conference on Machine Learning, pp. 4970-4979, Long Beach, LA, 2019.
Weng. L., Zhang, H., Chen, H., Song, Z., Hsieh, C.-J., Daniel, L., Boning, D., and Dhillon, I. Towards fast computation of certified robustness for relu networks. In Proceedings of the 35th International Conference on Machine Learning, pp- 5276-5285, Stockholm, Sweden, 2018.
Pang. T., Yang, X., Dong, Y., Su, H., and Zhu, Bag of tricks for adversarial training. In Proceedings of the 9th International Conference Learning Representations, Virtual Event, 2021.
Wood, D., Mu, T., Webb, A. M., Reeve, H. W., Lujan, M., and Brown, G. A unified theory of diversity in ensemble learning. Journal of Machine Learning Research, 24 (359): 19, 2023.
Peng, S., Xu, W., Cornelius, C., Hull, M., Li, K., Duggal, R., Phute, M., Martin, J., and Chau, D. H. Robust principles: Architectural design principles for adversarially robust cnns. In 34th British Machine Vision Conference, pp- 739-740, Aberdeen, UK, 2023.
Yang. H., Zhang, J., Dong, H., Inkawhich, N., Gardner, A., Touchet, A., Wilkes, W., Berry, H., and Li, H. Dverge: diversifying vulnerabilities for enhanced robust genera- tion of ensembles. In Advances in Neural Information Processing Systems 33, pp. 5505-5515, Virtual Event, 2020.
Pinot, R., Ettedgui, R., Rizk, G., Chevaleyre, Y., and Atif, Randomization matters how to defend against strong adversarial attacks. Proceedings of the 37th Interna- tional Conference on Machine Learning, pp. 7717-7727, Vienna, Austria, 2020.
Yang, 7., Li, L.,, Xu, X., Zuo, S., Chen, Q., Zhou, P., Ru- binstein, B., Zhang, C., and Li, B. Trs: Transferability reduced ensemble via promoting gradient diversity and model smoothness. In Advances in Neural Information Processing Systems 34, pp. 17642-17655, Virtual Event, 2021.
Robbins, H. and Monro, S. A stochastic approximation method. Annals of Mathematical Statistics, pp. 400-407, 1951.
Rudin, W. et al. Principles of Mathematical Analysis, vol- ume 3. McGraw-hil hill New York, 1964.
Yang, 7., Li, L., Xu, X., Kailkhura, B., Xie, T., and Li, B. On the certified robustness for ensemble models and beyond. In Proceedings of the 10th International Conference on Learning Representations, Virtual Event, 2022.
Sen, S., Ravindran, B., and Raghunathan, A. Empir: En- sembles of mixed precision deep networks for increased robustness against adversarial attacks. In Proceedings of the 8th International Conference on Learning Represen- tations, Addis Ababa, Ethiopia, 2020.
Young, L. An inequality of the hölder type, connected with stieltjes integration. Acta Mathematica, 6711)2251-28, 1936.

---

On the Diversity of Adversarial Ensemble Learning
Zhang, D., Zhang, H., Courville, A., Bengio, Y., Ravikumar, P., and Suggala, A. S. Building robust ensembles via mar- gin boosting. In Proceedings of the 39th International Conference on Machine Learning. pp. timore, MD, 2022.
Bal-
Zhang. H., Cheng, M., and Hsieh, C.-J. Enhancing cer- tifiable robustness via deep model ensemble. CoRR, abs/1910. 14655, 2019a.
Zhang, H., Yu, Y., Jiao, J., Xing, E., El Ghaoui, L., and Jordan, M. Theoretically principled trade-off between robustness and accuracy. In Proceedings of the 36th International Conference Machine Learning, pp. 7472- 7482, Long Beach, LA, 2019b.
Zhou, Z.-H. Ensemble Methods: Foundations and Algo- rithms. CRC Press, 2012.
12

---

On the Diversity of Adversarial Ensemble Learning
maX
X1
P1.1
min
X7
P1.r
max
Q2
04
Figure 10. The structure of transformed neural network. The ( (-) and min(-) functions can be constructed by ReLU functions. The yp2 fully connected layer with -1, 1 or weights. The yos fully connected layer with or weights.
A. Proof of Theorem 3.1
Definition A.1 (3-SAT problem). Given boolean variables and clauses in conjunctive normal form CNF formula with each clause's size at most 3, is there an assignment to the variables to make the CNF formula to be satisfied?
We present a key lemma for NP-hardness of adversarial squared loss, and the basic idea is a reduction from 3-SAT problem.
Lemma A.2. For some ReLU neural network g(-) and example (æo. yo), an Na-hard problem to precisely calculate the following adversarial squared loss
max BE
[l((1 -))
Proof. It is sufficient to prove the NP-hardness of solving maxsEд; (gg( + 6) -))²} = y for some 7 0. Let
s = C1 A C2 A...A ....ACC C, be a 3-SAT formula over a variable set V= (r,,,,- ... Jr Each C; = af v q. v qe is a disjunction with three literals qf of, qu, and each literal is variable from V or their negations. The 3-SAT problem is to determine whether there exists an assignment a V > {0, 1) for true cb.
We will show that any 3-SAT formula can be transformed into neural network over sample (æo, yo) = (0, (0.-1) in polynomial time, well as the following sufficient and necessary condition:
s satisfiable > max (g(( + 8) - --))²) (c/ SE
(6)
We will construct the ReLU neural network g(x) = 93 o p4 P3 p0 (œ) with input æ = (01, T2, ... ,I,) E R" as
shown in Figure 10. Specifically, we present the detailed constructions as follows:
We construct function (or (xx) IR" IR" with the j-th element
Ij > c/r 1/r for
[=)), max(2y+ +e/r 1/p /p.0) -m=
Ij
for
- - < <I, S e/13/p
-//1
for
8, 5 c//
For xn = and ||5||, < c, we can achieve the maximum or minimum of the elements of 9 (I0 + 8) independently. is easy construct (p(()) with a ReLU neural network because of three operators +, and max(-, 0). Intuitively, the i-th element of you (mo + d) can be viewed the variable in 3-SAT problem, and its value -e/r1/0 and c//1/p can be viewed as the false and true of variable e, respectively.
We construct (x) (x, -x), and this follows that, for j E [2r], 2o 0 and | Se.
-e/r 1/ < [122 o p1 (x + 8)1; S <
We achieve the maximum or minimum for the first r elements of و 4o (xp + 5) independently. The last elements are always the opposite of the first r elements. Intuitively, the last r elements yo2 1p (x + 8) can be viewed as the negation of variables "1,""" My in 3-SAT problem.
13

---

On the Diversity of Adversarial Ensemble Learning
We construct SP3 (a)( Wæ with W (wi; wf; wi:..... we: wg; w / RRaxx2, Here, we construct three vectors w. wf, w (0, 13 1 for clause C; = of V of q for i E [s]. For [3], the wo is the unit vector with j-th and (j r)th element if of is variable Uj and its negation -, respectively. For [3s], xo 0 and ||ö||z S we have
-e/r /v S (ps o p2 o yp (mo + 8)1, S //rl/p
Intuitively, every three elements of (p3 o p2 & (0o + 8) can be viewed as three literals of a clause in 3-SAT problem. We achieve independently the maximum or minimum for 53 u22 o y0 (xz + 5) whose corresponding literals are different variables.
We construct pa(at) = (maxf21, 22,, 23). max( Is, Ts). ... max -3s-1,
1, 33s F), where max.:...) = m and min('.... = m J. can constructed via ReLU function (maxf 0} function) as max a b} = max - b, 0} + and minfa, b) = max{6 - a, 0. +
For j E [s). 20 = 0 and ||81p I c, we have
-e/r//0 s pp P2 o (20 o)], S c/r'/r
We can achieve the maximum for the j-th element of you 4g opp2 (mo - 8) if and only if the 2)-th,, (3j 13j-11 1) or 3j-th elements of (P3 (22 1p (zo 6) are maximal. Intuitively, the max function can be viewed as the V operator, and the i-th element of 4p o Pp u2 42 (20 + ö) can be viewed as the clause C₁ in 3-SAT problem.
We construct (a)) min (11 .2,
xz} min m (min/mint...... ,T,-2), I. 13 2x via ReLU function. For
Zo and ||1|, S we have
c/ 1/8 S 45
po o (2o + 6) S c/r2/p.
(8)
We achieve the maximum and only all elements of packpy (Iu +0) are maximal. Intuitively, the min function can viewed as the A operator 3-SAT problem. The -e/r1/p and e/r3/p value of yss o o o FP 422 o (2 + 6) can be viewed as the false and true of the CNF formula Φ = C1 C2 A C., respectively.
For (Zo, vo) = (0,-1) 11 and é < 1, it remains to show that, from Eqns. (6) and (8),
Φ is satisfiable + there is an s.t. g = e/r1/p.
(9)
- 1f 6 = CiA==CACCAA--A C. is satisfiable, then there is a satisfiable assignment a. We set the i-th element of ô as 2/1
and - //p if a<(x;) true and false, respectively. Then, we discuss g( step by step as follows:
The i-th element of 4p (6) is - -//12/p and c/r1/p if the i-th element of 5 is -e-re- -e/1// and e/12/p, respectively;
The first r elements of 222 1 (6) are equal to yp (6) while the last elements are the opposite of the first r elements;
The (3(1 - 1) + ji element of Pa P2 o up (6) is the I-th and (1+r)) r)-th element of (22 p1p1 (6) if literal of is variable ty and -ty, respectively;
Every element of (p1 iPs P2 (8) is //r since there is at least one true literal in qf o7 of for every satisfiable C of V of V qf and hence there is at least one element with e/ in every three elements of (ps u22 pp (8);
The final output sss o 4p4 o Pp3 o u22 o yp (5) e/r since yss takes the minimum of ypu o 933 o 622 o (8))
- If g()) = e/1 we discuss ô as follows:
Every elements of f yps p2 42 (ō) is 1/ from (p(()) [- ///
At least one element is equal to 1/r in every three elements of (os 62 o 9 ( since i1 takes the maximum of every three elements. Without loss of generality, let all (3(i 1) + 1) elements be c/rl/v for i [m]:
14

---

On the Diversity of Adversarial Ensemble Learning
X1
X
8
9
Xy
X
mia
Xy
9
min
x
Xy+1
Xy
h1
h2
h
Figure 11. The structure of the transformed neural network. The sp block is network similarly in the proof Theorem A.2.
For the first elements y22 o ypi (6), the I-th element will be equal to e/r1/p if literal of variable er, For the last r elements of y22 yp (6), the (1+ r)-th element will be equal /r if literal of is the negation of variable my
The I-th element of #0 (6) is c/rl/p and -e/r// if the I-th element of (22 ppi (ō) is e/rl/n and -e/r /p, respectively;
The 1-th element of is c/rl/p and -c/rl/P if the 1-th element of (6) is c/r and - -c/r/// respectively.
Let be true and false if the i-th element of ò is c/rl/p and -e/r"/p respectively. This assignment makes the CNF formula satisfiable, since there is at least one element in every three elements of PP o PP 4o1 (ō) with e/r1/p.
Proof of Theorem 3.1. It is sufficient to prove the NP-hardness of solving
max (a5(x 6) y) max ((f(x - SEA
-} =2 for some
<> 0.
BED
Let Φ C1ACAA. C. be 3-SAT formula over variable set V = {ui, , J. Each C, = qf V V qe is a disjunction with three literals 9. .f q² and each literal is a variable from or their negations. We will show that any 3-SAT formula can be transformed into two neural networks f1. f2 and sample (æo, y0) = (0,-1) -1) in polynomial time, as well as the following sufficient and necessary condition:
ø is satisfiable
T
max (f) + 1)²} max ((f(0) + 1) (E/r + +1)- BE
(10)
2
BEA
We will construct the ReLU neural network fi (x)) and f2(x)) with inputs æ E gr+1 in Figure 11. For & C1AC2A- construct neural network g(x):r) shown in Lemma A.2, where Fur the first elements of æ. We construct
fi fz()) min(g((1..) (((r+1)) with
e/r2/p for Ir+1 e/T2/p
c ((5+41) = max(#141 + T.pp 0) max "1/p" 1/p
e
0)
E
Ir+1 for
-e/r < Ir41 S 1/
-c/12/p for
5441 S -e/p//p.
easy to construct [1(m) via ReLU neural network from Eqn. (7), and we have
is satisfiable <
max (8) +) (c/r2/2 11 Sea;
(11)
This is because -c/72/0- < fz( < e//2/2 from Eqn. (8), and for E (0, 1), ø is satisfiable if and only if there exists an s.t. (0) e/11/p. If (8) e//2/v, then & is satisfiable from Eqn. (9), since we have g( = e/rl/p from (ō) min minngg(ó (0())..))) (6)++1)) and (66,+1) S 1/p, P, For the inverse direction, o is satisfiable, then there is an 8 s.t. g(61.r) = c/r'/n from Eqn. (9). This follows that (ō) e/r for da+1 c/rl/p.
In similar manner, we construct f2(x) = m ( ), 77( +1)} and have
@ is satisfiable + max (f2 + 1)²) (c/r2/p + 1)2.
(12)
SEA,
15

---

On the Diversity of Adversarial Ensemble Learning For odd function n())), have the ensemble [(x= (/i(æ:) f2(x))
1
f(x) = (min (1 (nin(o(ass)))
- (1( = (1(#=+1)~1(24+>7) = 0.
We have - - - [(x) s 0, and it holds that (f(6) + 1)2 < for e E (0, 1]. and the equality holds for 0. We have
max {(f(8) + 1) = || Se
(13)
For the proof of Eqn. (10), we have Y (e/r//v + 1)2 - 1 if Φ is satisfiable from Eqns. (11)-(13), For the inverse direction, i = (c/r1/p + +1² - 1, then we have, from Eqn. (13),
max (O)(0) + 1))} (e/T + 13? SEA;
(14)
2
From f(()) E 1le//r//,,///// and 1]. we have
(fi(6) + 1)2 (e/r + 1)² and (f2(0) + 1)2 s (E/r +1)
This follows that, from Eqn. (14)
max (0) + 1) (c/r2/2 + 1)² and max ((f(()))))))))) 1) (c/r2/r 1)2; SEA;
therefore, & satisfiable from Eqns. (11)-(12). This completes the proof.
B. Appendix for Section 3.2
B.1. Proof of Theorem 3.2
We begin with some useful lemmas as follows:
Lemma B.1 (Hölder's inequality (Young, 1936)). For two real vectors a = (' ;aa)) and b = (b1, bb).. we have
Σ labbil < ||ا|اي|bb|| for positive and with 1/9-11
where the equality holds if and only if aala lP B|b;1 for E [d] w.r.. some positive constants a and B
Lemma B.2. For vectors w. E Rd, we have
max w" = c/|w||y and |
min w=o = -c -c|lw]ly |
Proof. For every d with |8]|p I c, we have, from Lemma B.I,
w
w ||0|||||||| S Se|[w|l
Notice that the above equality holds if we choose 8" (0ī, o2 5% with
sign((w;ee (Iw,1% 1 11)) 1/P
where sign(w;) is equal to -1, if wy is negative, zero or positive, respectively. This follows that TH E||w||y
max |
We also have, by letting 8' -8
min
nH 8'8
min
Th (-"(--))- = min -wT -- 18
max TT8' = -
Se
1-8 11
8 1811. Se
which completes the proof.
16

---

On the Diversity of Adversarial Ensemble Learning
Lemma B.3. For linear finction (w)) w x + b, we have, for positive p, q with 1/p + 1/9 1,
(w T (xc + 6) + -----9 y)²² = (1w0 x + b - yl + [l1
max 5
Proof. We have the adversarial squared loss
(w- (x + +) + y)² S
max
max (w x + y w 5)2
[8] p 8 < |l01 and
From Lemma B.2, we have- -11w|1ae S w"d
max (w" (x + 5) + y)²
max 8 8
(w Tx + b + w w0² = (lw" x +
yl + |201g))²
S
which completes the proof.
Proof of Theorem 3.2. We have the adversarial loss for fy and f. from Lemma B.3,
max (1f( + 6) -)) max ( -) (1f(x)) vl + [V(())), V/)==) y) = (1/(()) - vl [V llp))
d
;
max (15 + 8) - -) = SEA,
max SEA,
This follows that
max (f) -1)
max (ffx +-))++0) BEL;
- =
m j=1
SEAS
1 Σ (1)( y + ( (1](x0) - yl + |+
((f(z) ))) w >/ |
/V(())
(IV1))())1 [ 2e
(I wl -() - v1)
1 m
Σ (f)()) - y)
m
m > (f)(z) (0(x))-00 -((=)()))))--)) y)²
Gradient Diversity Cross Diversity
m j=1
We also have
m m Σ (1((a) - 1000-y)--00 (f(x)) - y)²
Σ [x(2)2 - f(x)² _ Σ (2(()) - f(x))2
m
1 m
((x)))
m
m
j=1
which completes the proof.
B.2. Training Details for Diversity Decomposition
We consider the base learners convolutional neural network with two convolutional layers and one MLP layer of 100 neurons. The first convolutional layer has 24 filters with kernel size 5, while the second convolutional layer has 24 filters of kernel size 5. We take the ReLU activation function, and the input and output sizes are 3x32x32 and 1, respectively.
We select the 5-th and 6-th class on datasets MNIST and F-MNIST to train the base learners independently, and take the SGD method (Robbins Monro, 1951) with batch size 256 and learning rate 0.01. We consider the PGD-attack (Madry et al., 2018) to calculate the adversarial ensemble loss and average of individual adversarial losses. The perturbation size set to 8/255 and 128/255 for Lox and l2 norm, respectively.
17

---

On the Diversity of Adversarial Ensemble Learning
B.3. Discussions on the Second-Order Approximation For the second-order approximation, have
0( [(x) V , H())ō
where (() is the Hessian matrix of f at point æ. This follows that
(f(x 6) y)² max BEAp
max
1 5" (f())+
00 H()) -1y
-
w)
BEA;
and hence, it is sufficient to consider two problems as follows
max f(x) + Vf(="² 8 H(r) y and ;
min f(x) Vf(2)" BE
2 8t H(x)6-1 y.
We focus on the special case p 2, and the above two problems can be formalized as
1 5" B5 5 for some b E R" and symmetric matrix B E Raxx
min S 2
(15)
This is known the trust region subproblem (Forsythe & Golub, 1965; Moré & Sorensen, 1983; Fortin & Wolkowicz, 2004), and we have
the optimal solution 5* = B- if there is no solution on the boundary {ō ||0112 S c} in (15), from the positive definiteness of B and | <
the optimal solution 8 -B + a 1) if there is solution the boundary {8:110112 S e) in (15) and a² > -À1
Here, a1 is the smallest eigenvalue of B and a" is the solution of
0+)2 =c²
(16)
where "*. An are eigenvalues of B and 71, In are the elements of Q² with eigendecompositiinn B = QAQ".
the optimal solution 8t = 8, + T= if there is solution on the boundary {ō ||01|2 S <} in (15) and a S < in (16). Here, dg is the solution of
||óo|| <, and is eigenvector of B with eigenvalue A and r E IR satisfies || T2| =
BB T)ō0 = b s.t.
Here, the main challenge is how to obtain the closed-form solution of adversarial loss via second-order approximation, since it is relevant to the roots of high-order polynomials Eqn. (16).
B.4. Discussions of Average of cos Values for 1,0 Norm
We focus on 2-dimensional instance space X' C R2a and label space y C R, and consider
(xx) = #1 + 02; [2(x)) = F1 3u2, fa(x) = abıı + buz and fa(æ) = abæi bor
where = (V5 1)/2 and b = /2 + v5/5. We study two ensembles: one ensemble of Si and fz; the other ensemble of f3 and f4. For example (z., v) ([1, (), 1) and perturbation set A (ō || S 1}. we have
(z), Vf((z))
(V), v((=) cos (V)(())) (/()))
(m),, f2()))
[V] IVAA()]|
and we also have the same average of individual adversarial losses from Lemma B.3 as follows
(f.(x: + 5) 1--y y)
(f.(z + 5) y)
max SEA
max AEA
2
2
3
18

---

On the Diversity of Adversarial Ensemble Learning
However, the adversarial ensemble losses different from
max (i (a + 6)/2 f2( 81/2 - y)² 8 and A=م
max(S3 8)/2 fa(ar + 8)/2 y)² 7.2.
8EA
Thus, there exist two ensembles of the same averages of cos values and individual adversarial losses, but with different adversarial ensemble losses for 2o norm.
C. Appendix for Section 3.3
C.1. Proof of Lemma 3.3
We have the cross-entropy loss
((f( + -(
+ In(1 (x[++)p((()+++((())))))
and the adversarial cross-entropy loss
max {(f(x + 6) max - v(f(2)) V)(x)" + In(1 v(f(=))VV(())))) V\(=)
(17)
SEA;
For EAp, we have, from Lemma B.1
> V (V
We get the maximum of Eqn. (17) when
V [f for == 0
- (>>](>> for y/ = 1,
and the optimal adversarial perturbation 8" with (a()) = --2y (=:)lle We finally have
f(x) f()) V =f(x) (2y 1)] (())lle,
and the probability of the positive class of the adversarial example 5
adv P
1 +xxx V[())=0)))
+ (+exp((([(()) (2y 1))l)(()))l
which completes the proof.
C.2. Proof of Theorem 3.4
We have
In(1 - v/IV - In(1 + exp(f(x) VV
1 In
1 1 (z) - / (z) (ge)
- [l(
1 1+exp(f(2))
- (/ 1 - (z))oe))
In
-
1 [=)
In
(xcxp(f())-//) 1 //VV
1 1+exp(f(2) -VV
In 1 + - -
1 5/ [/(()))))))
1 exp(-( -
- -'(lV/(==)lla)))
-xexp(-Ufxx (f)(z) -/ /(x))
KK(pp aid + Pf,+ ado (x)
[+]] - f(0) + V/
19

---

On the Diversity of Adversarial Ensemble Learning
This follows that, from Lemma 3.3,
m
max 181I.
(2) 0),
max {(g(x + d).8) 18
171 ==1
Σ In(1 + - - =(b((r)))))))))))))))){{)))))) V
(n(1 exp(f(x)) - ( f/>|[](
aado + (Pf 12 1)s - Σ (II [V
which completes the proof.
C.3. Discussions of Average of cos Values for Cross-Entropy Loss
Example 2. There exist two ensembles of the same averages of cos values and individual adversarial cross-entropy losses, but with different adversarial ensemble losses for cross-entropy loss and l2 norm.
Proof. We focus 2-dimensional instance space X S IR² and label space y S IR, and consider
f1(x) Iı + T2. f2(x) x1 3x.2 fa(x) abæı baz and fa(x) abæı bb
where
2 and b= 2ln ex + v2)) + exp(1 + v10)) - -1)
v5 -1
-
We study two ensembles: one ensemble of fi and fz; the other ensemble of fs and fa. For example (æx, v) = ([1, 0]. 1) and perturbation set A
{ō: ||0112 I 1}, we have
(V/(()), V(((2)) (V [V
(V))()), V((ma) cos V[
V f (m), ff((=))
and we also have the same average of individual adversarial losses from Lemma 3.3
(Lrl + + - y)²
(Ji(x + 5) y)
In(1 + exp(1 + v2)) + In(1 + exp(1 + v10)
max SEA
max SEA
2
2
2
However, the adversarial ensemble losses are different from
max + 8)/2 f2 + 8)/2 y)² ~ 2.4999 and
max(fs(# 8)/2 fxxx + 08/2 2 - u13 ~ 2.3738
SEA
BEA
which completes the proof.
Example 3.
There exist two ensembles of the same averages of cos values and individual adversarial cross-entropy losses, but with different adversarial ensemble losses for cross-entropy loss for loc norm.
Proof. We focus 2-dimensional instance space A' C IR2 and label space y C IR, and consider
(x æ1 + 22,
fz(z) 21 3xz, fa(x) abæı brz and fa(x) abæı brz
where (V5-) 1)/2 and b In( ( exp(3)) exp(5)) 1)/v5. We study ensembles: one ensemble of
fi and f2; the other of fa and fa. For example (x, v) = ([1, 0), 1) and perturbation set A = {6: || S 1}. we have
(z), Vf(())
(V)) f(()
cos (.) vf2(z))
cos (V)(()) (x))))
[V/3<)]|
20

---

On the Diversity of Adversarial Ensemble Learning
and we also have the average of individual adversarial losses from Lemma 3.3
2 max 2
(Ji(æ 6) -y)²
(Jt(a: + y)² In(1 exp(3)) In(1 + exp(5))
max BE
SEA
However, the adversarial ensemble losses different from
(f + 6)/2 f2(2 6)/2 y)² 3.0486 and
max + 60/2 fa(x+ + +8/22 y)² 2.37388 dEA
which completes the proof.
D. Appendix for Section 4
D.1. Proof of the Extended Diversity
For simplicity, we abbreviate f f(t and (āt) to f and Py, respectively. Let Pe. [K] be the k-th element of and py. respectively. We have the adversarial cross-entropy loss for multi-classification
Σ expl(as
{(f(tg).)= = -fo + log
This follows that
{(f())))))
m
(( -f
m Σ Ji log
1
exp(Fe)
log Σ Eespt(is)) ): (18)
m
We also have
Σ exp(Fa) loog Σ exp(fsx)
K Σ k1
exp(Frn) E exp(fit
Σ exp(fa)
log
log
Σ exp(fjx)
Amml
(ki( log
fanh/ EK= exp(Six)
x f
K
Σ
Pfe (fx - S1,k - KL(Py,Pr,)
R1 E cxp(Fk)
FkL/ Ek= exp(fi)
J
Am
and this follows that, by setting = -pp(tt) - -
log Σ expp(fss) log
> (r,f((ā1,) - f(ē,))
Σ Exxp(fa)
K
Σ
KL(py(E)) Ps, (ā,,)) (19)
m
m
5=1
We have, from the first-order approximation f(0g) = f(x) Jz(()8
(r. f(tf)) = 1 Σ
Jr. (x) -z(2)07)
j=1
which completes the proof by combining with Eqns. (18)-(19).
D.2. Proof of Orthogonality
We now show the orthogonalizztion of the predictions of base learners by optimizing Eqn. (5) as follows. Theorem D.1. For K multi-class learning, let fi, ... fm be m base learners with cross-entropy loss being at least B, and K 1 is multiple of m. We have the minimizer of Eqn. (5) over example (x, v) as
exp(- B)
for k =
Pr. (xc + +7,))
1 l-exp(-B) K-1
for 8;
(20)
otherwise
where ". 8m a partition of set KK,(K and p., (T) the kith element of p (,
21

---

On the Diversity of Adversarial Ensemble Learning
Table 4. Comparison of training time (times(s) per epoch) for our AdvEOAr with and without the regularization r.
Our AdvEAAP
MNIST
F-MNIST
CIFARIO
without regularization (-)
40.08
40.11
123.84
with regularization (+)
41.45
41.32
125.48
Proof. We first have Pf. (æ + 87 exp( B): since the adversarial cross-entropy loss is at least B for each base learner. The regularization in Eqn. (5) is defined as
(2 v) H H Ps, (01; /mm) + log(V (#f), ,, (m.)
The Vip, (zz,)..... .... (. achieves its maximum if and only if the non-label probability vectors of each individual
network are mutually orthogonal (Bemstein, 2009). The H[ Py, (it,) //m) achieves the maximum if and only if the mean of non-label probability vectors of individual networks are uniform. It is obvious that Eqn. (20) satisfies the two conditions simultaneously. Thus, Eqn. (20) is the minimizer of Equ. (5).
D.3. Other Details of Algorithm
PGD-attack for I, norm perturbation ball
The PGD-attack generates adversarial examples iteratively for loc -nrm perturbation ball as follows
II AS
(x +a sign sign(Ve(()(z*
where [[ denotes the projection and æ + A = fa + 880 )... The 0 is initialized as æ, and T is used as the adversarial example of base learner fz. For other lp norm perturbation ball, PGD-attack generates adversarial examples as
II *+
(o a la...aavie (V((( v)))
where is the function that maps the gradient to the update direction
(p(w) sign(aa.)((wu)l//1e))
Calculating the volume of polytope
For m vectors 31, ... Bm E Rd and X = (x), (mm) E Rdxm, we have, from matrix theory (Bernstein, 2009),
V² 3m) det(X"
where det(X* X) is the determination of the matrix X x. For matrix A E Rax*, we also have
i det((A) dct())
dai
where aij the i-th row and j-th column element of A. We finally optimize the objective Eqn. (5) for neural networks with SGD method (Robbins & Monro, 1951).
Time Complexity of Algorithm 1
The time complexity of Algorithm takes m-times as that of training a single neural network adversarially (m the number
of neural networks in the ensemble). addition, takes (m²²) computational cost for the regularization with its gradient. In practice, the regularization takes much smaller computational cost than that of training neural networks, as in Table
22

---

On the Diversity of Adversarial Ensemble Learning
Table Hyperparameters of ensemble methods used in experiments. Parameters that applicable left hlank.
Parameter
GAL
ADP
AdvADP
DVERGE
PDD
TRS
iGAT(ADP)
0.5
2
2
0.01
2
B
0.5
0.5
5
0.5
E. Appendix for Section 5
E.1. Experimental settings
For iGATADP: ADP. take 150, 150 and 480 epoches for MNIST, F-MNIST and CIFAR10 for convergence; while for other ensemble methods, take 60, 60 and 250 epoches for MNIST, F-MNIST and CIFAR10, respectively. For adversarial examples in training process, we take PGDI0 with 10 steps with step-size 0.04, 0.01 and 0.008 for MNIST, F-MNIST and CIFAR10, respectively. We set a = 0.02 and = 10 for our method, and Table summarizes parameter setting for others.
E.2. EOT and BPDA attacks
We take the Backward Pass Differentiable Approximation (BPDA) attack (Athalye et al., 2018a) for potential gradient risks and designs attacks. We design four different attacks as follows:
BPDA₁: : For potential gradients vanishing risk (i.e., small gradient of ensemble from different gradient of base learner), we instead use the k times of the average logits of the base learners as the logit of the ensemble, where k E [1, m] and m is the number of base learners. We evaluate all possible values of k E [1 and report the lowest adversarial accuracy observed.
BPDA2:
For potential incorrect gradients from random gradients of single base learner, consider the attack of deleting base learner and using other gradients of the ensemble.
For potential incorrect gradients from random gradients of base learners, we could consider the attack of selecting randomly half base learners at each step for attack.
BPDAs:
BPDA+: For other potential gradient risks, we consider the black box attack (Andriushchenko et al., 2020).
We also take the Expectation over Transformation (EOT) attack (Athalye al., 2018b) to add adversarial perturbations insensitively in transformations. We implement 20 times rotations randomly within -30 to +30 degrees.
23


Reviews:
[{"review_id": "BSYwbLPQ5F", "reviewer": "Reviewer_gZFG", "summary": "This paper considers the problem of defining and correlating diversity with algorithmic performance in adversarial ensemble learning. The authors first prove that precisely calculating the diversity in adversarial ensemble learning is an NP-Hard problem. They then propose a new diversity decomposition for adversarial ensemble learning, breaking down the adversarial ensemble loss into four components: the average of individual adversarial losses, prediction diversity, gradient diversity, and cross diversity. This decomposition highlights the insufficiency of considering only gradient diversity, as done in prior works. The authors also extend this decomposition to classification tasks using cross-entropy loss. Based on their theoretical analysis, they develop a new ensemble method called AdvE$_{oap}$, which improves gradient and cross diversity simultaneously by orthogonalizing adversarial predictions of base learners.\n\nAfter rebuttal:\n\nThe authors have address my questions. I will keep my score.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "[Q1] … a gap between NP-hard problem to first-order approximation …. only consider first order approximation since the complete analysis is a NP-hard problem.\n\n[A1] We will clarify that the first-order approximation is motivated from previous first-order approximation methods for robust learning [Kariyappa & Qureshi 2019; Dabouei et al. 2020; Huang et al. 2021; Bogun et al., 2022], and it is difficult to make similar analysis for higher-order (even for second-order) approximations without closed-form solutions of adversarial examples and loss functions, which is relevant to the roots of polynomials of $2d$ degree ($d$ is the input dimensionality) [More & Sorensen, 1983; Fortin & Wolkowicz, 2004].\n\n\n[Q2] In the theorem of decomposition (Theorem 3.2 and 3.4), It is weird to me to you equally, since those are only first order approximation.\n\n[A2] We will clarify that we consider the first-order approximation in the decomposition theorems, and we will take the approximately equal rather than exactly equal.\n\n[Q3] It would be better if the authors could provide experiments on larger dataset, such as TinyImageNet.\n\n[A3] We will clarify that we consider datasets Minist, F-minist and Cifar10 for fair comparisons with previous ensemble methods, and we have been conducting experiments on larger datasets with complex network architectures. Here are some preliminary results:\n\nAdversarial accuracy for CIFAR100 with 5 ResNet18\n\n| Methods | PGD20 | AutoAttack |\n|-|-|-|\n| Our Method | 25.16 | 24.39 |\n| AdvADP | 23.690 | 21.22 |\n\nWe could present more experimental results in the following days due to computational resources. For example, it is about 4 days to train a deep neural network on Tiny-ImageNet with four A6000 GPUs, while we only have a single NVIDIA GeForce RTX 4090 GPU to train multiple deep neural networks for diversity.\n\n[Q4] The paper could benefit from a more detailed discussion of the limitations of the proposed method, particularly in scenarios where the assumptions of first-order approximation may not hold.\n\n[A4] We will add a figure to discuss the limitations of the proposed method with and without first-order approximations, and the basic idea is to consider the residual terms on the first-order approximation in experiments and analyze its influence.", "claims_and_evidence": "In general the Claims are clear. The claims made in the paper are well-supported by both theoretical analysis and empirical evidence. \n\nYet I think the following discussions could be improved.\n\n1) There is a gap between NP-hard problem to first-order approximation. It is a very weak argument that we only consider first order approximation since the complete analysis is a NP-hard problem. There are still many things we can do in between.\n\n2) In the theorem of decomposition (Theorem 3.2 and 3.4), It is weird to me to you equally, since those are only first order approximation", "methods_and_evaluation_criteria": "The proposed methods are well-suited for the problem of adversarial ensemble learning. The authors use first-order Taylor approximation to decompose the adversarial ensemble loss. The evaluation criteria include classification accuracy under various adversarial attacks (PGD, AutoPGD, AutoAttack etc.), which are standard benchmarks in adversarial robustness research. The experiments are conducted on widely used datasets, and the results are compared against several baseline methods, ensuring a comprehensive evaluation.\n\nIt would be better if the authors could provide experiments on larger dataset, such as TinyImageNet.", "theoretical_claims": "The theoretical claims are sound and well-justified. The authors provide detailed proofs for the NP-Hardness of calculating diversity (Theorem 3.1) and the diversity decomposition for both squared loss (Theorem 3.2) and cross-entropy loss (Theorem 3.4). The proofs are rigorous.\n\nAgain, as I mention before, it is weird to me to see equality when using first order approximation.", "experimental_designs_or_analyses": "The experimental design is appropriate for validating the theoretical claims. \n\nAfter the authors proposed the decomposition, they provide experiments to see which components are more relevant. This helps to justify their propose method AdvE$_{oap}$.\n\nThen, the authors conduct extensive experiments on three datasets (MNIST, F-MNIST, and CIFAR10) and compare their method against several state-of-the-art adversarial ensemble methods. The use of multiple adversarial attacks (PGD, AutoPGD, Autoattack etc.) ensures that the evaluation is comprehensive.", "relation_to_broader_scientific_literature": "To my knowledge, no.", "essential_references_not_discussed": "To my knowledge, no.", "other_strengths_and_weaknesses": "Weaknesses:\n\nThe paper could benefit from a more detailed discussion of the limitations of the proposed method, particularly in scenarios where the assumptions of first-order approximation may not hold.", "other_comments_or_suggestions": "no", "questions_for_authors": "See above.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "FnoyJdnkvv", "reviewer": "Reviewer_wKSP", "summary": "This paper addresses the critical role of diversity in adversarial ensemble learning, demonstrating that precisely calculating diversity for neural networks under adversarial perturbations is NP-hard due to structural and predictive interdependencies. By decomposing adversarial ensemble loss into four components—average individual losses, prediction diversity, gradient diversity, and cross diversity—the authors reveal the insufficiency of prior gradient-centric approaches. They propose AdvEOAP, a method that enforces orthogonal adversarial predictions through KL divergence regularization and entropy maximization, simultaneously improving gradient and cross diversity. Empirical validation on MNIST, F-MNIST, and CIFAR10 shows state-of-the-art robustness, with AdvEOAP outperforming baselines by up to 80% under AutoAttack, while ablation studies confirm the necessity of diversity components. The work establishes a theoretical foundation for adversarial ensemble diversity and offers a practical framework for enhanced robustness.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "[Q1] …The NP-hardness proof (Theorem 3.1) assumes ReLU activations and specific perturbation bounds. How generalizable are these results to other activation functions (e.g., sigmoid) or arbitrary perturbation norms?\n\n[A1] We will clarify that our results hold for $l_p$ norms with $p = 1, 2, \\cdots, \\infty$ in Theorem 3.1, and it is feasible to generalize to other activation functions (e.g., sigmoid) based on universal approximation [Kidger et al. 2020], i.e., a 3-SAT problem can be reduced to a diversity problem of ReLU neural networks as in Theorem 3.1, and network employing other activation functions (e.g., sigmoid) can approximate any ReLU network with arbitrary precision.\n\n[Q2] First-Order Approximation Limits: The loss decomposition (Theorems 3.2/3.4) relies on first-order Taylor expansions. How might higher-order terms affect the validity of the decomposition…\n\n[A2] We will clarify that the first-order approximation is motivated from previous first-order approximation methods for robust learning [Kariyappa & Qureshi 2019; Dabouei et al. 2020; Huang et al. 2021; Bogun et al., 2022], and it is difficult to make similar analysis for higher-order (even for second-order) approximations without closed-form solutions of adversarial examples and loss functions, which is relevant to the roots of polynomials of $2d$ degree ($d$ is the input dimensionality) [More & Sorensen, 1983; Fortin & Wolkowicz, 2004].\n\n[Q3] Scalability: Experiments are limited to ResNet20 on small/medium datasets (MNIST, CIFAR-10). Can AdvEOAP scale to larger architectures (e.g., Vision Transformers) or high-resolution datasets (e.g., ImageNet) without prohibitive computational costs?\n\n[A3] We will clarify that we consider ResNet20 on datasets Minist, F-minist and Cifar10 for fair comparisons with previous ensemble methods, and we have been conducting experiments on larger datasets with complex network architectures. Here are some preliminary results:\n\nAdversarial accuracy for CIFAR100 with 5 ResNet18\n\n| Methods | PGD20 | AutoAttack |\n|-|-|-|\n| Our Method | 25.16 | 24.39 |\n| AdvADP | 23.690 | 21.22 |\n\nWe could present more experimental results in the following days due to computational resources. For example, it is about 4 days to train a deep neural network on Tiny-ImageNet with four A6000 GPUs, while we only have a single NVIDIA GeForce RTX 4090 GPU to train multiple deep neural networks for diversity.\n\n[Q4] Computational Overhead: Training time and resource requirements for AdvEOAP need to be quantified in theory and experiments.\n\n[A4] We will clarify that, for AdvEOAP, the training time takes $m$-times as that of training a single neural network ($m$ is the number of neural networks in ensemble). In addition, it takes $O(m^3)$ computational cost for the regularization with its gradients. For resource requirements, AdvEOAP needs $m$ times the space complexity of a single neural network. \n\nHere are some experiments as follows:\n\nFor MNIST dataset:\n\n| Methods | GAL | ADP | AdvADP | DVERGE | PDD | TRS | $\\text{iGAT}_{\\text{ADP}}$ | Our $\\text{AdvE}_{\\text{OAP}}$ |\n|-|-|-|-|-|-|-|-|-|\n| Time per Epoch (s) | 23.45 | 7.14 | 45.36 | 40.65 | 70.72 | 241.8 | 22.93 | 41.45 |\n\nFor F-MNIST dataset:\n\n| Methods | GAL | ADP | AdvADP | DVERGE | PDD | TRS | $\\text{iGAT}_{\\text{ADP}}$ | Our $\\text{AdvE}_{\\text{OAP}}$ |\n|-|-|-|-|-|-|-|-|-|\n| Time per Epoch (s) | 23.45 | 7.15 | 45.50 | 39.52 | 69.48 | 219.32 | 23.94 | 41.32 |\n\nFor CIFAR10 dataset:\n\n| Methods | GAL | ADP | AdvADP | DVERGE | PDD | TRS | $\\text{iGAT}_{\\text{ADP}}$ | Our $\\text{AdvE}_{\\text{OAP}}$ |\n|-|-|-|-|-|-|-|-|-|\n| Time per Epoch (s) | 69.47 | 15.18 | 134.54 | 172.93 | 291.07 | 626.33 | 100.51 | 125.48 |\n\nAll methods require approximately 4.4GB memory except TRS with 10.3GB memory due to storing the Hessian matrix.\n\n[Q5] Parameter details: Hyperparameters for comparison methods are not fully disclosed. While AutoAttack is included, hyperparameters for weaker attacks (e.g., FGSM step size) are not fully disclosed, limiting reproducibility. \n\n[A5] We will clarify more parameter details in Appendix: For the FGSM, PGD10, and PGD20 attack methods, the hyperparameters are set as\n\n| Method | Steps | Step Size |\n|-|-|-|\n| FGSM | 1 | $\\epsilon$ |\n| PGD10 | 10 | $\\epsilon/3$ |\n| PGD20 | 20 | $\\epsilon/6$ |\n\nHere, $\\epsilon$ is the perturbation size.\nFor the AutoPGD, MORA, and AutoAttack attack methods, we set the number of steps to 20, and keep all other parameters in the original reference or the provided code defaults. \n\nWe will add a formal description for our algorithm in Appendix to clarify orthogonalization implementation, and release the code to ensure reproducibility.", "claims_and_evidence": "The claims in the paper are supported by rigorous theoretical and empirical evidence. The NP-hardness of calculating adversarial ensemble diversity is proven via a reduction from the 3-SAT problem, demonstrating that precise diversity computation requires solving an NP-hard optimization due to structural and predictive interdependencies. The proposed adversarial loss decomposition (into average individual losses, prediction/gradient/cross diversity) is validated theoretically through first-order Taylor approximations (Theorems 3.2 and 3.4) and empirically via experiments on MNIST, F-MNIST, and CIFAR10, showing gradient/cross diversity dominate under large perturbations. The effectiveness of AdvEOAP is confirmed by ablation studies (e.g., 1.1–1.5% accuracy drop without orthogonality regularization) and superior robustness over baselines (e.g., +40% accuracy vs. DVERGE under AutoAttack). Critiques of prior gradient-centric methods are supported by constructed counterexamples (Example 1) and empirical trends (Figure 2), where adversarial loss decreases despite unchanged gradient alignment. While the analysis relies on first-order approximations and scalability to larger models remains unexplored, the core claims are substantiated by theoretical proofs, controlled experiments, and statistically significant results across 50 runs.", "methods_and_evaluation_criteria": "The proposed methods and evaluation criteria are aligned with the problem of adversarial ensemble learning. The AdvEOAP method, which enforces orthogonal adversarial predictions via KL divergence and determinant-based regularization, directly addresses the need for diverse vulnerability patterns among base learners-a critical requirement for robustness against adaptive attacks. By optimizing both gradient diversity and cross diversity, the method systematically tackles the limitations of prior gradient-centric approaches. The evaluation criteria are appropriate and comprehensive.", "theoretical_claims": "The theoretical claims in the paper are correct under their stated assumptions but face limitations in generality. The NP-hardness proof for diversity calculation (Theorem 3.1) is valid for ReLU networks under specific perturbation bounds but does not generalize to all activation functions or norms. The loss decomposition theorems (3.2 and 3.4) rigorously hold under first-order Taylor approximations but neglect higher-order effects, limiting applicability to small perturbations (ϵ) or highly nonlinear regimes. Overall, the core proofs are logically consistent but rely on simplifications (e.g., linearized adversarial perturbations) that may not fully capture real-world model behaviors, particularly for deep networks with complex interactions.", "experimental_designs_or_analyses": "The experimental designs and analyses are methodologically sound but have minor limitations. The authors evaluate AdvEOAP on MNIST, F-MNIST, and CIFAR-10—standard benchmarks for adversarial robustness—using diverse attacks (FGSM, PGD, AutoAttack) to cover gradient-based, iterative, and adaptive threat models. The inclusion of ablation studies (Table 2) effectively isolates the impact of orthogonality regularization, showing a 1.1–1.5% accuracy drop when removed. Statistical rigor is ensured via 50 runs with reported mean ± std. deviations, and Figure 7’s training dynamics validate the theoretical link between diversity and robustness. However:\n\n1. Scalability gaps: Experiments are limited to ResNet20 on small/medium datasets; larger architectures (e.g., Vision Transformers) and high-resolution datasets (e.g., ImageNet) are untested, leaving scalability in doubt.\n\n2. Attack parameter details: While AutoAttack is included, hyperparameters for weaker attacks (e.g., FGSM step size) are not fully disclosed, limiting reproducibility.\n\n3. Computational costs: Training time and resource requirements for AdvEOAP (e.g., orthogonalization overhead) are unquantified, which is critical for real-world deployment.\n\nOverall, the experiments robustly validate the method’s core claims but leave practical scalability and efficiency as open questions.", "relation_to_broader_scientific_literature": "The paper’s contributions significantly advance the adversarial robustness literature by addressing critical gaps in ensemble diversity theory. Prior works (e.g., Kariyappa & Qureshi, 2019; Dabouei et al., 2020) focused on gradient misalignment as the primary diversity metric but lacked theoretical grounding for its sufficiency. This work bridges that gap by proving that diversity in adversarial settings is inherently NP-Hard (extending Katz et al.’s (2017) NP-Hardness results for neural network verification) and introducing a multi-component diversity decomposition—building on classical ensemble theories (Krogh & Vedelsby, 1994; Zhou, 2012) but adapting them to adversarial perturbations. The AdvEOAP method’s orthogonality regularization draws inspiration from Pang et al. (2019), who used prediction disagreements, but innovates by combining KL divergence and determinant-based constraints to enforce structural diversity. These contributions align with broader trends in robustness research (e.g., Yang et al.) that emphasize diversifying vulnerability patterns but provide the first formal framework linking diversity components to adversarial loss. By unifying theoretical rigor (NP-Hardness, decomposition) with practical algorithm design, the paper addresses a longstanding challenge in adversarial ensemble learning: how to systematically define and optimize diversity for robustness.", "essential_references_not_discussed": "N/A", "other_strengths_and_weaknesses": "Enter any comments on other strengths and weaknesses of the paper, such as those concerning originality, significance, and clarity. We encourage you to be open-minded in terms of potential strengths. For example, originality may arise from creative combinations of existing ideas, removing restrictive assumptions from prior theoretical results, or application to a real-world use case (particularly for application-driven ML papers, indicated in the flag above and described in the Reviewer Instructions).", "other_comments_or_suggestions": "N/A", "questions_for_authors": "Q1. Theoretical Generality: The NP-hardness proof (Theorem 3.1) assumes ReLU activations and specific perturbation bounds. How generalizable are these results to other activation functions (e.g., sigmoid) or arbitrary perturbation norms?\n\nQ2. First-Order Approximation Limits: The loss decomposition (Theorems 3.2/3.4) relies on first-order Taylor expansions. How might higher-order terms affect the validity of the decomposition under large perturbations (e.g., ϵ>0.1) or highly nonlinear regimes?\n\nQ3. Scalability: Experiments are limited to ResNet20 on small/medium datasets (MNIST, CIFAR-10). Can AdvEOAP scale to larger architectures (e.g., Vision Transformers) or high-resolution datasets (e.g., ImageNet) without prohibitive computational costs?\n\nQ4. Computational Overhead: Training time and resource requirements for AdvEOAP need to be quantified in theory and experiments. \n\nQ5. Attack Parameter Transparency: Hyperparameters for comparison methods are not fully disclosed. This might affect reproducibility and comparative analysis.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "jmZuZLbna4", "reviewer": "Reviewer_y8GR", "summary": "This work explores the role of diversity in adversarial ensemble learning, focusing on its definition and impact on algorithmic performance. The authors demonstrate that precisely calculating diversity is NP-Hard, distinguishing it from traditional diversity analysis. They introduce the first diversity decomposition in adversarial ensemble learning, decomposing adversarial ensemble loss into average of individual adversarial losses, prediction diversity, gradient diversity, and cross diversity—challenging previous methods that considered only gradient diversity. Extending this decomposition to classification with cross-entropy loss, they propose a novel ensemble method based on orthogonal adversarial predictions to enhance both gradient and cross diversity. Empirical results confirm the effectiveness of their approach.\n\n## update after rebuttal\nI have carefully read the authors' rebuttal and the feedback has well addressed my questions and concerns. Thus, I would like to insist on the score of accept . Thanks.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "[Q1] This work proposes the diversity with the first-order approximation, which is somewhat limited. The neural network may not exhibit strong linearity in local regions.\n\n[A1] We will clarify that the first-order approximation is motivated from previous first-order approximation methods for robust learning [Kariyappa & Qureshi 2019; Dabouei et al. 2020; Huang et al. 2021; Bogun et al., 2022], and it is difficult to make similar analysis for higher-order (even for second-order) approximations without closed-form solutions of adversarial examples and loss functions, which is relevant to the roots of polynomials of $2d$ degree ($d$ is the input dimensionality) [More & Sorensen, 1983; Fortin & Wolkowicz, 2004].\n\n[Q2] What is the time complexity of Algorithm 1?\n\n[A2] We will clarify that the time complexity of Algorithm 1 takes $m$-times as that of training a single neural network ($m$ is the number of neural networks in the ensemble). In addition, it takes $O(m^3)$ computational cost for the regularization with its gradients.\n\n[Q3] Can the decomposition be applied to other types of loss functions?\n\n[A3] We will clarify that this work makes the decompositions for two commonly-used loss functions, squared loss for regression and cross-entropy loss for neural network in classification, and it can be applied to other loss functions such as Poisson loss as in [Wood et al., 2023].", "claims_and_evidence": "The claims made in the submission are supported by clear and convincing evidence.\n\n1. The authors establish that calculating diversity precisely is NP-hard.\n\n2. They introduce the first diversity decomposition in adversarial ensemble learning.\n\n3. They propose a novel ensemble method based on orthogonal adversarial predictions to enhance the proposed diversity.\n\nEvidences\n\n1. The NP-hardness of the problem is demonstrated through a reduction from the 3-SAT problem, as proven in Theorem 3.1.\n\n2. Theorems 3.2 and 3.3 present the diversity decomposition with respect to squared loss and cross-entropy loss, respectively. Experimental results further validate the correctness of the decomposition, as shown in Figures 1 and 3.\n\n3. Empirical results confirm that the proposed method enhances diversity (Figure 7) and improves robustness (Table 1).", "methods_and_evaluation_criteria": "The proposed diversities and ensemble method make sense for adversarial ensemble learning.\n\nThe proposed diversity measures are derived through decomposition, clearly illustrating their impact on adversarial ensemble loss. As demonstrated in Theorems 3.2 and 3.3, greater diversity leads to a lower adversarial ensemble loss. \n\nBuilding on this insight, the authors emphasize the orthogonality of adversarial predictions. Empirical results further validate the effectiveness of the proposed method, showing enhanced diversity (Figure 7) and improved robustness (Table 1).", "theoretical_claims": "I have reviewed all the proofs, including the NP-hardness in Theorem 3.1, the diversity decompositions in Theorem 3.2 and 3.4, and others. \n\nThe NP-hardness is demonstrated through a reduction from the 3-SAT problem. The authors build two neural networks given a 3-SAT problem and show the value of the diversity of the neural networks can determine the satisfiability of the 3-SAT problem. The proof of the decomposition is relatively simple, mainly involving some equation transformation techniques and solving for adversarial perturbations.\n\nI found no errors.", "experimental_designs_or_analyses": "I have checked the experiments and found them to be sound and valid. \n\nThe experiments in Section 3 validate the correctness of the proposed diversity measures and highlight the limitations of previous approaches. Furthermore, the experiments in Section 5 confirm the effectiveness of the proposed method, demonstrating increased diversity (Figure 7) and enhanced robustness (Table 1).", "relation_to_broader_scientific_literature": "This work defines the diversity for adversarial ensemble by decomposing the adversarial ensemble loss to the average of individual losses and diversities. While such decomposition has been explored in non-adversarial environment [1,2], this work introduces the first decomposition in adversarial environment. \n\nPrevious works define the diversity via gradient [3,4]. The proposed decomposition shows that it is not sufficient to only consider gradient to characterize the diversity.\n\n[1] Zhou, Z.-H. Ensemble Methods: Foundations and Algorithms. CRC Press, 2012.\n\n[2] Wood, D., Mu, T., Webb, A. M., Reeve, H. W., Lujan, M., and Brown, G. A unified theory of diversity in ensemble learning. Journal of Machine Learning Research, 24 (359):1–49, 2023.\n\n[3] Dabouei, A., Soleymani, S., Taherkhani, F., Dawson, J., and Nasrabadi, N. M. Exploiting joint robustness to adversarial perturbations. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 1122–1131, Seattle, WA, 2020.\n\n[4] Huang, B., Ke, Z., Wang, Y., Wang, W., Shen, L., and Liu, F. Adversarial defence by diversified simultaneous training of deep ensembles. In Proceedings of the 35th AAAI Conference on Artificial Intelligence, pp. 7823–7831, Virtual Event, 2021.", "essential_references_not_discussed": "To the best of my knowledge, this paper cites all essential works related to traditional diversity or adversarial diversity.", "other_strengths_and_weaknesses": "Strengths:\n\n1. This work is well-structured and easy to follow, with clear proofs and informative supplementary materials. The proof of NP-hardness in Theorem 1 constructs two neural networks by introducing an auxiliary variable, which is skillful.\n\n2. The authors demonstrate that the proposed diversities can answer what the diversity is in adversarial ensemble learning, both theoretically and empirically. They also show that previous diversity measures may fail to capture true diversity, as illustrated in Example 1 and Figure 2.\n\n3. The analysis of diversity in cross-entropy loss suggests that attention should be given to the predictions of adversarial examples of base learners. This insight is both natural and valuable, and experimental results further validate it.\n\n4. Extensive experiments are conducted, demonstrating the effectiveness of the proposed method. As shown in Figure 7, the proposed method enhances the proposed diversities, leading to improved adversarial accuracy.\n\nWeaknesses:\n\n1. This work proposes the diversity with the first-order approximation, which is somewhat limited. The neural network may not exhibit strong linearity in local regions.\n\n2. The proof of the decompositions is relatively simple and does not involve any special techniques.", "other_comments_or_suggestions": "1. The author has scattered the introduction of related work throughout the main text. It would be better to add a “Related Work” section in the appendix to summarize previous studies.\n\n2. It would be better to investigate how the hyper-parameters affect the performance of the proposed method.\n\n3. It would be better to conduct experiments on other l_p norms.", "questions_for_authors": "1. What is the time complexity of Algorithm 1?\n\n2. Can the decomposition be applied to other types of loss functions?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "kHTaIlMeWr", "reviewer": "Reviewer_ybcr", "summary": "This paper investigates the role of diversity in adversarial ensemble learning, addressing two key questions: how to define diversity formally in adversarial scenario and how diversity correlates. To address this questions, the paper introduces a first-order approximation and proposes a novel diversity decomposition within four components: average of individual adversarial losses, prediction diversity, gradient diversity, and cross diversity. Empirical evaluations validate the effectiveness of proposed AdvEOAP method.\n\n## Update After Rebuttal\nDear Author,\n\nThank you for your detailed response and clarification. My concerns have mostly been addressed, and I will raise my score.\n\nPlease remember to organize the rebuttal materials mentioned above in your next revision, as this will greatly enhance the clarity of the paper.\n\nBest regards,\\\nReviewer ybcr", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "[Q1] It would benefit to examine the performance of the authors’ proposed method with Expectation over Transformation (EOT) and Backward Pass Differentiable Approximation (BPDA).\n\n[A1] We will clarify that this work does not consider the EOT and BPDA attacks, because the EOT attack is designed for random networks [Athalye et al., 2017], while the BPDA attack is considered for networks with non-differentiable operations [Athalye et al., 2018]. In contrast, our model is deterministic without any non-differentiable operations.\n\n[Q2] The practical applicability of the method in real-world scenarios is not fully explored, such as large-scale Imagenet-1K and Vit architecture.\n\n[A2] We will clarify that we consider ResNet20 on datasets Minist, F-minist and Cifar10 for fair comparisons with previous ensemble methods, and we have been conducting experiments on larger datasets with complex network architectures. Here are some preliminary results:\n\nAdversarial accuracy for CIFAR100 with 5 ResNet18\n\n| Methods | PGD20 | AutoAttack |\n|-|-|-|\n| Our Method | 25.16 | 24.39 |\n| AdvADP | 23.690 | 21.22 |\n\nWe could present more experimental results in the following days due to computational resources. For example, it is about 4 days to train a deep neural network on Tiny-ImageNet with four A6000 GPUs, while we only have a single NVIDIA GeForce RTX 4090 GPU to train multiple deep neural networks for diversity.\n\n[Q3] Understanding the computational overhead introduced by the proposed method is essential, especially when compared to other baseline methods.\n\n[A3] We will clarify that, for AdvEOAP, the training time takes $m$-times as that of training a single neural network ($m$ is the number of neural networks in the ensemble). In addition, it takes $O(m^3)$ computational cost for the regularization with its gradients.", "claims_and_evidence": "The major claims are well supported by empirical or theoretical evidences.", "methods_and_evaluation_criteria": "The methods and evaluation criteria make sense to me.", "theoretical_claims": "I only reviewed the general formula derivation without verifying the detailed proofs.", "experimental_designs_or_analyses": "The experimental designs make sense to me.", "relation_to_broader_scientific_literature": "Not applicable.", "essential_references_not_discussed": "Not applicable.", "other_strengths_and_weaknesses": "Strengths:\n\n1. The NP-Hardness proof is well-motivated and distinguishes adversarial ensemble learning from traditional ensemble theory.\n2. The paper is technically sound, with rigorous mathematical formulations and proofs provided for key theorems that underpin the main claims. \n3. This paper is well-written, making it easy to follow.\n\nWeaknesses:\n\n1. It would benefit to examine the performance of the authors’ proposed method with Expectation over Transformation (EOT) and Backward Pass Differentiable Approximation (BPDA).\n2. The practical applicability of the method in real-world scenarios is not fully explored, such as large-scale Imagenet-1K and Vit architecture.\n3. Understanding the computational overhead introduced by the proposed method is essential, especially when compared to other baseline methods.", "other_comments_or_suggestions": "None", "questions_for_authors": "Refer to strengths and weaknesses.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: This paper introduce an ensemble method that enhances robustness by orthogonalizing adversarial predictions to increase both gradient and cross diversity.

Strengths consistently noted by reviewers include:
- Reviewer wKSP and Reviewer y8GR praised the novelty and theoretical rigor of the NP-Hardness proof and diversity decomposition.
- Reviewer gZFG appreciated the sound experimental method and impact of orthogonal prediction regularization.
- Reviewer ybcr highlighted the technical depth, clarity, and robustness improvements over prior adversarial ensemble methods.

Reviewers acknowledged and appreciated the authors’ responses. Before the rebuttal, Reviewers raised concerns regarding the limitations of relying on first-order approximations in theoretical decomposition, the lack of experiments on larger-scale datasets and architectures, the absence of analysis under advanced attack settings like EOT and BPDA. Major concerns were thoroughly addressed in the rebuttal with additional theoretical clarifications, new empirical results under EOT/BPDA, training time comparisons, and scalability discussions.

Therefore, AC recommends acceptance of the paper.
Decision: Accept (poster)