Appropriate Balance of Diversification and Intensification Improves Performance and Efficiency of Adversarial Attacks
Authors: Keiichiro Yamamura
Issa Oe
Nozomi Hata
Hiroki Ishikura
Katsuki Fujisawa
Venue: Accepted by TMLR ()

Abstract: Recently, adversarial attacks that generate adversarial examples by optimizing a multimodal function with many local optimums have attracted considerable research attention. Quick convergence to a nearby local optimum (intensification) and fast enumeration of multiple different local optima (diversification) are important to construct strong attacks. Most existing white-box attacks that use the model’s gradient enumerate multiple local optima based on multi-restart; however, our experiments suggest that the ability of diversification based on multi-restart is limited. To tackle this problem, we propose the multi-directions/objectives (MDO) strategy, which uses multiple search directions and objective functions for diversification. Efficient Diversified Attack, a combination of MDO and multi-target strategies, showed further diversification performance, resulting in better performance than recently proposed attacks against around 88% of 41 CNN-based robust models and 100% of 10 more advanced models, including transformer-based architecture. These results suggest a relationship between attack performances and a balance of diversification and intensification, which is beneficial to constructing more potent attacks.

Published in Transactions on Machine Learning Research (May/202)
Appropriate Balance of Diversification and Intensification Improves Performance and Efficiency of Adversarial Attacks
Keiichiro Yamamura
@kyudai..
Graduate School of Mathematics Kyushu University
Issa Oc
issa-oo Qkyudaaa
Graduate School Mathematics
Kyushu University
Nozomi Hatal
n.hata @kyudai..jp
Institute of Mathematics for Industry Kyushu University
Hiroki Ishikura
ishikuro.h. ww.. ac.jp
Institute of Innovative Research Tokyo Instiutte of Technology
Katsuki Fujisawa
fujisawa k. aa Omn, titech, ac.jp
Institute of Innovotive Research Tokyo Institute of Technology
Reviewed OpenReview: https: // https://openrevieuu openreview. net/ forum? id= mK6TuminII
Abstract
Recently, adversarial attacks that generate adversarial examples by optimizing a multimodal function with many local optimums have attracted considerable research attention. Quick convergence to a ncarby local optimum (intensification) and fast enumeration multiple dif- ferent local optima (diversification) are important to const ruct strong attacks. Most existing white-box attacks that use the model's gradient enumerate multiple local optima. based on multi-restart; however, our experiments suggest that the ability of diversification based on multi-restart is limited. To tackle this problem, propose the multi-directions//ojjeiiiee (MDO) strategy, which uses multiple search directions and objective functions for diversi- fication. Efficient Diversified Attack, combination of MDO and multi-target strategies, showed further diversification performance, resulting in better performance than recently proposed attacks against around 88% of CNN-based robust models and 100% of 10 more advanced models, including transformer-basel architocture. These results suggest a rela- lionship between atlack performances and balance of diversification and intensification, which is beneficial to constructing more potent attacks.
1
Introduction
Deep neural networks (DNNs) have demonstrated excellent performance in several applications. However, DNNs are known to misclassify adversarial examples generated by tiny perturbing inputs that are imper- ceptible to humans Szegedy et al. 2014). Vulnerabilities caused by adversarial examples can have fatal
"Present affiliation NTT Computer Data Science Laboratories, NTT Corporation. Prreeent affiliation NTT Communication Science Lahoratories, NTT Corporation.

---

Published in Transactions on Machine Learning Research (May/2024)
Class
Closs
Closs
Objective fimetion
DUR
Robust accuracy (%)
CW
AutoAE
ACG
51
CE
APGD
AutoAttack
EDA
60-
CAA
19
Ours (EDA)
Gradient direction
Conjugate gradient direction
1000
2000
3000
4000
50000
6000)
Total number of inferences per image
Figure 2: Illustration of the MDO strategy with two search directions and three objective functions on a toy model. The different search directions obbjctive functions search for different regions.
Figure 1: EDA recently proposed attacks ad- versarially trained CIFAR-10 model Engstrom et al. 2019h.
consequences, especially in safety-critical applications such as automated driving (Gupta et al. 2021 facial recognition (Adjabi al., 2020), and cybersecurity (Liu et al., 2022b). Adversarial training (Madry et al, 2018) is one of the most effective defenses, which uses adversarial examples during training. Generating many adversarial examples faster is beneficial for the security of DNNs because adversarial training requires many adversarial examples. Our ultimate goal is to find many adversarial examples quickly, with proper control of both extensive and intensive searches.
Adversarial attacks optimize a challenging nonconvex function to find adversarial examples. We focus on white- bhx attacks that use gradient-basee optimization algorithms, assuming access the outputs and gradients of the DNN. Higher objective function values increase misclassification chances, creating adversarial example candidates out of local optima. The objective function of this problem is multimodal because its maximization involves complex DNN. Because multimodal function has myriad of local optima, quick convergence to a nearby local optimum and fast enumeration of multiple different local optima are important. These are referred to as intensification and diversification, respectively (Glover & Samorani 2019).
Many existing gradient-based attacks are considered to achieve some degree of intensification because the objective value can be improved by moving the gradient direction within a neighborhood. Whereas many existing attacks diversify the search based the multi-restart (Dong et al., 2013 Madry et al. 2018 Croce
& Hein 2020b
Liu et al. 2022c few studies have examined other diversification strategics. Therefore, further research on diversification is needed.
Diversification and intensification involve trade-offs, particularly under a strict computational budget. Thus, we have to balance both of them appropriately. However, an appropriate control method for this balance has yet. to be developed. This study the first to systematically explore and control this balance by coordinating multiple search directions and objective functions based on diversification indices. Using the proposed methodology, we can discover adversarial examples that clusive even to current state-of-tho-ar methods. Our experiments using diversification indices in Section 3.1 suggest that the attacks using multi- start strategy are likely to converge in the same cluster, which does not yield high diversification performance. However, the experiments in Section 3.1 also suggest that different local solutions may be enumerated using different search directions and objective functions. Inspired by this observation, we propose the multi- directions/ rrrrctions/objeee /objectives (MDO) strategy that uses multiple search directions and objective functions. Figure is a toy example of an attack using the MDO stratogy.
To implement this strategy effectively, propose the Automated Diversified Selection (ADS) and search framework called the MDO framework in Section 3.2 and Section 3.3 respectively. The MDO strategy needs to utilize an appropriate combination of search direction and objective function for effective diversification. This is because some combinations may explore similar areas. ADS selects search directions and objective functions that are likely to search different areas based on the Diversity Index (DI) (Yamamura et al. 2022) enhance diversification. Unlike existing approaches for attack selection (Mao et al.] 2021 Liu et al. 2023b), which rely on direct indicators like loss value and attack success rate, ADS employs an indirect indicator such as DI.
2

---

Published in Transactions on Machine Learning Research (May/202) /204)
The MDO framework explicitly considers the diversification and intensification phases through ensemble and composite of attacks to balance them appropriately. The diversification phase finds promising starting points via extensive search, and the intensification phase enhances attack performance through intensive search in the vicinity of these points.
We evaluated the proposed methods with robust accuracy, computation cost, and DI because the attack with an appropriate balance of diversification and intensification is expected to be strong and fast. The experimental results in Sections 4.3 and 4.4 suggest that ADS contributes to search diversification and suc- cessfully finds the appropriate combination in both diversification and intensification. The MDO framework also realizes an appropriate balance of diversification and intensification, resulting in a promising attack performance. As shown in Section 4.2 the MDO strategy found adversarial examples for some inputs in less time than the multi-target (MT) strategy (Gowal et al. 2019), one of the promising attacks. These results imply that the combination of MDO and MT strategies can realize stronger and faster attacks.
Motivated by these results, we experimentally investigated the attack performance of the combination of MDO and MT strategies, called Efficient Diversified Attack (EDA). Experimental results in Section 4.1 show that EDA exhibits higher diversification performance than the attack using the MDO strategy alone, resulting in better attack performance in less computation time than the recently proposed attacks under the standard setting of perturbation bound (e) in the Robust leaderboard Croce et. al. (2021). Given the difference in robust accuracy among recently proposed attacks, EDA exhibits sufficiently large improvements in robust accuracy and runtime. The above experimental results suggest that appropriately enhancing diversification leads to higher attack performance. The major contributions of this study are summarized below.
(MDO) strategy and its implementation: A novel search strategy using multiple search directions and objective functions, realized by Aulomated Diversified Selection (ADS) and the MDO framework. ADS chooses the gradient-basee based search direction and the objective function that im- proves the likelihood of misclassification so that the DI-based index is maximized, thereby achieving efficient diversification and avoiding getting stuck during intensification. The MDO framework finds promising start- ing points in the diversification phase and enhances the objective value during the intensification phase. 2. Efficient Diversified Attack (EDA): A faster and stronger attack using MDO and MT strategies. Un- der the standard setting of perturbation bound in RobustBench leaderboard, EDA showed the best attack performance among recent allacks against around 88% of 41 CNN-based robust models trained on three representative datasets and 100% of 10 more advanced models, including transformer-base architecture, trained on ImageNet (Russakovsky et al. 2015).
1. paaltiiddrretiios//ojcttttt
2 Preliminaries
2.1 Problem settings
Let g: D - RC be a locally differentiable C-classifier, Vorg E D be a point with e as the correct label, and dDDxD + R be a distance function. Given E > 0, the feasible region S is defined as the set of points z E D that are within distance of from Forg, i..., S (ar 1 d(x,, Forrg S Then, we define an adversarial example as Tady S satisfying arg max,=1.....C (x A c. The following expression is formulation of one type of adversarial allack, the untargeted atlack, where the atlacker does not specily the misclassificatiin target.
find x E S s.t. max max_9((x) - gx(x:) > 0 i=I...
(1)
Problem is solved through maximization of the objective function L(g(z), c) within the feasible region S. This maximization aims to reduce the probability that æ is classified in class e by g. Therefore, æ with a high objective value L(g(x),c) c) is more likely to be misclassified by g. When d(æadv, Forg) is small, the norm of the adversarial perturbation is also small. The targeted attack aims at maximizing the probability that Fadv is classified in a particular class + e by solving max L (g(x), c,t. For adversarial attacks on image classifiers, D = [o, 1|" and d(v, w) := || wllps (p 2, o0) is typically used. This study focuses on the untargeted attack image classifiers using d(v, w) = || w boo referred to loo attacks.
3

---

Published in Transactions on Machine Learning Research (May/2024)
2.2 Related work
General white-box attacks. In the white-box attack, the initial point sampling determines x0 first. Then, the step size update rule and the search direction ô = 5(L) updates the step size r7th) and the moving direction g(k), respectively. Subscquently, the search point x (R+++) is updated by the following formula.
x sampled by x, ((+++)) Ps (x)) + (7) (1))
(2)
where k is the itcration, and Ps is projection onto S. The moving direction (() is usually computed based on the gradient )L(g(z()),c).. According to equation 2 allack methods are characterized by the initial point sampling x, step size update rule 7) search direction ö, and objective function L. The tuple a = (x, 17. 6, L) is referred to as an attack a. Algorithm | shows the procedure of the general white-bos attacks a without multi-restart. We assume that the white-box attack a returns a set of best search points Xa and a set of classífication labels with the highest probability except for the correct classification labels during the search 11e The basic white-box attacks iteratively update the search point xx()) lines 5 and 6 in Algorithm| if Lew(g( ),)) Lew(g( (cw(g(x())), c). Lcw denotes CW loss proposed by Carlini & Wagner (2017). After the N iterations of updates, the attack procedure is finished. For the attacks with a multi-restart strategy,
[ to search for adversarial examples. At each iterations k., the best search point x" is updated
Algorithm General white-box attacks without multi-restart (Attack)
Require: a = (x, 6, L): an attack, N: maximum iteration, Ps: a projection function, DNN, I: test
samples
Ensure: X": a set of best, search points, ITs: a set of classification labels with the highest probabilities except for the correct classification class
for l. I do Forg f v; I, Y correct classification label corresponding æ;
3: xx()) t- initialize by x, x= + =x) frest t- LCw (g(x:))))
4: for k O..... 1 - do
Update 1/th)) and (k)) by update rule v and search direction [(++1]_ +
5:
P Ps (x + ttk) -(())
6:
7:
Update x" and fhest
I1 + I11 (arg sa Bi (x()+1)))
end for
10: X X2 (x*}
11: end for
Algorithm is repeated from different initial points. The returned X² should contains R best search points if the number restarts R.
Commonly used attack components. Projected Gradient Descent (PGD) (Madry et al. 2018) is a fundamental whitc-box attack. PGD uses fixed step size (Max) and moves to the normalized gradient direction (ŌPGn). Auto-PGD (APGD) (Croce & Hein 2020b) is a variant of PGD using heuristic (Nλрс)) for updating step size and moving to the momentum direction (θAPGD). In addition, some studies use cosine annealing (Neo=) (Loshchilov & Hutter (017)) for updating step size. Gradual reduction of step size, such as 7APGD and Toos, showed better results than fixed step size. Auto-Conjugate Gradient attack (ACG) (Yamamura et al. (222)) uses MAPGD and moves to the normalized conjugate gradient direction (6ACG). Whereas the sort of steepest directions, such as ÔPCD and ÒAPGD, are suitable for intensification, the conjugate suddient-based direction is suitable for diversification. For the initial point, uniform sampling from S or input points (Xorg) is usually used. Assuming the many restarts, Output Diversified Sampling (ODS, xODS) and its variant, which considers the output diversity of the thrcat model (Tashiro et al. 2020:
Liu et al., 2022ch, outperformed naive random sampling. For the objective functions, cross-enlropp (CE)
loss (Goodfello et al., 2015) (LCe) and margin- losses such as CW loss (Lew), a variation of CW loss scaled by the softmax function (Lscw), and Difference of Logits Ratio (DLR) loss (Croce Hein 2020b)
(LDRR) are often used. We denote the targeted version of these losses as LT, e.g., L for targeted CE loss.
See Appendices A.T and A.2 for mathematical formulas of attack components.
4

---

Published in Transactions on Machine Learning Research May/2024)
Robustness evaluation. The robustness of DNNs is usually evaluated using AutoAttack (AA)
Croce
& Hein (2020b), combining four different. attacks, including the MultiTargeted (MT) Gowal et al. (2019) strategy. MT strategy runs targeted attacks on each of multiple target classes and thus requires higher com- putation costs. The high computational cost of AA has motivated the research community to pursue faster
allacks for adversarial training and robustness evaluation (Gao et al. 2022 Xu et al., 2022). Composite Adversarial Attack (CAA) (Mao et al. 2021) and AutoAE (Liu et al.
2023b combine multiple attacks by
solving an additional optimization problem, requiring the pre-execution of candidate methods on relatively large samples. Consequently, they are still computationally expensive. Adaptive Auto Attack (A²) Liu et al. (0022c) demonstrated state-of-the-art (SOTA) attack performance in less computation time by improving the initial point sampling and discarding the hard-to-attack images. A is based on PGD with CW loss, which uses a single search direction and objective function. Some studies have investigated how to switch either the search direction or the objective function based on case studies to improve attack performance (Yamamura) ct al.] 2022] Antoniiuu et al., 2022). However, to the best of our knowledge, this is the first study that inves- tigales the combination of multiple search directions and objective functions based on search diversification. See Appendix A.3 for more information.
Quantification of search diversity. Yamamura el al. (2022) proposed the Diversily Index (DI) to quan- tify the degree of diversification during the attacks. DI is defined as
DI(X, == M WM I. M
h(0; do,
(3)
where M- = supill - yll2 x, y E S} is the size of the feasible region, X is the set of search points, and h(0; X) is the function of e based on the global clustering coefficient of the graph G(X.0) = E(0) (X, {(,,,) u. E X, -lllu-lla s 0)). By definition, 0 S h(0; X) S 1 holds. Thus, 0 DT(X, S also holds. DI quantifies the degree of density of any point set as value between 0 and based on the global clustering coefficient of a graph. DI tends to be small when the point set forms clusters. Because of this feature, DI is suitable to evaluate how clustered the point set is. We use the weighted average of DI to choose pairs of search directions and objective functions that are likely to search for different areas. This study uses the same value of M as in Yamamnra et al. 2022 For simplicity, M) is denoted by DI(X) in this paper.
3 Mutticdirections//ajetttisss strategy
3.1
Motivation
We hypothesize that diversification contributes to attack performance. This section empirically demonstrates that allacks with multiple search directions (8) and objective functions (L) can achieve more efficient diver- sification than attacks with a single ô and L. Four search directions and seven objective functions are used in this experiment. Nmax iterations of attacks are performed from R initial points for each pair of search direction and objective function. The search point with the highest CW loss (Lcw) value is collected at the end of the attack, which starts at each initial point. The obtained set of these search points is denoted as X We focus on this set of search points to analyze the characteristics of the attack using a single 8 and L. For this experiment, used 10,000 images from CIFAR-10 (Krizhevsky et al. 2009) as test samples and attacked the robust model proposed by Sehwag et al. 2022 results.
See Appendix C for more information and
Notation. Let = (LCE, Low, Lscw, U g = 4,5,6) be set of objective functions, and D = (ŌPGD, ŌAPGD. JACG, dxez} be a set of scarch directions. We have proposed 8Nem and LG-DLR. in this study. ÔNes is the search direction of Nesterov's accelerated gradient (NAG) (Nesterov) 2004) normalized by the sign function to accommodate foo attacks. We refer to LG-DLR.. as generalized- DLR (G-DLR) loss with the denominator of DLR loss extended from Ix, (x: 9ng to 9n. (x) Ir, (æ);; ma E Y denotes the class label that has the q-th largest value of g(z). Mathematical expressions of δN and LG-DLR. can be found in Appendix B Given an initial point sampling and step size update rule n, we define a set of attacks
5

---

Published in Transactions on Machine Learning Research (May/2024)
Di of uniformly sampled points
(4)
LCE Low Lsoo LDLR LeMMR LG-DERS LO.DLR,
DI of uniformly sampled points
CCE
AG-DURA @ Lgew
APGD
Lew
GG-DLRS
ACG
&oss
DLLB
LE-DLR6
ÖPGD
APGD
ACG
Now
2D visualization of X using UMAP. Points of the same color in (a)/(b) represent points obtained using the same objective function /scarch direction, respectively.
Figure 4:
Figure 3: Violin plot of DI(X;). The upper/lower figure represents the attack with the same objective function/search direction, respectively.
1. Run E (x.1) with MADS iterations (lines 1-4 in Algorithm I)
2. Get best search points
3. Compute D (Eq. 4) (lines 5-8)
4. Compute (a) argmax De (line 9)
class
class
class
class
class
Uatey X
Vace, X
Vaneja Xi
DI: high p² low
DI: low f high
DI: high p FiAI, high
DI: high f411 low
DI: low p, high
DI: high p: high
Figure 5: The procedure of ADS. The black circle represents a single search point, and DI tends smaller when the search points form clusters. Regions of different colors represent different classification classes, and the more scarch points are distributed around multiple regions, the higher the degree of diversification in the output space, ie., the value of P...
as A(x, 11 = =a= (x, DE (
L}. In the following paragraphs, we analyze the characteristics of allacks a E AAxxons, )ros) which use single 8 and L,, based on the experimental results with Nmax 30 and R 10.
Limited diversification ability of attacks using single and L. We quantify the diversity of X; using DI to reveal the diversification ability of the altack a, which uses a single and L. Figure shows the violin plot of DI (Xf) for 10,000 images and all attacks a. The mean and standard deviation of the first, second, and third quartiles were 0.190 0.019, 0.223 + 0.023, and 0.269 + 0.033, respectively. Figure 3 and these DI values suggest that the diversity of the best point set X is relatively low. Thus, the diversification ability of seems to be limited.
Attacks using multiple d and can lead to efficient diversification. Figure 4 shows the best point set X² embedded in two-dimensional space using Uniform Manifold Approximation and Projection (UMAP) (Melnnes et al. 2018). Dimensionality reduction methods, such as UMAP, preserve the maximum possible distance information in high-dimensional spaces possible. Figure (a) depicts the points obtained by the altack using the same L in the same color. Figure (a) shows that sets of best search points obtained from searches with different tend to form different clusters. Figure | (b) depicts the points obtained by the attack using the same in the same color. Similarly, sets of best scarch points obtained by the attack using different 8 also tend form different clusters. Based on these observations, possible to efficiently search for different. local solutions using different 8 and L, or appropriate combination of both.

---

Published in Transactions on Machine Learning Research (May/2024)
Algorithm 2 Automated Diversified Selection (ADS)
Require: J: small samples, A(x, candidate attacks, NADS: maximum iterations, Ps: projection, g: DNN, M: the size of feasible region, da: number of attacks select Ensure: {(ō.;,Lm (,,,aa))33 )) The pairs of search direction and objective function
Choose the best
1: for a = (x, 1), 6, L) A(x, do Attack(a, NADS, Ps. J. / "Algorithm
2: X XIIII
Choose the best
3: end for
A fe A(x, ) |e| na A
(a) Ensemble
(b) Composite
(c) MDO Framework
5: for e A do
Pf + Usge | Vi 1,....177
7: Compute D by equation !
end for
Figure 6: Illustration of the ensemble, compos- ite, and the MDO framework.
9: Get [(6a; D/= from {a]) = arg max D
eEA
3.2 Automated Diversified Selection
The analyses in Section 3.1 suggest that the MDO strategy can efficiently search for different local solutions. However, this streeggy is ineffective unless the combinations of 8 and I. are properly determined because dif- ferent attacks may search similar regions. To address this issue, we propose Automated Diversified Selection (ADS), which selects the combinations of search directions and objective functions based on the degree of diversification in input and output spaces. Algorithm 2 and Figure 5 show the ADS procedure. Ideally, we aim to identify the combination of 5 and L, that facilitates the most extensive exploration through numerous iterations of candidate attacks on the entire dataset. In practice, however, computation time should be min- imized. Therefore, we atlempt to approximate the combination of and I capable of exploring extensive regions by assessing the diversity of the best solutions derived from a limited iteration attacks on small samples. The ADS procedure outlined below.
First, NADS iterations of attack candidates a E A(x, 1) executed the image set J C D (lines 1-3 in 22 In this study, J is 1% of the images uniformly sampled from all the test samples. Subsequently, the set of best scarch points X and class labels IIş {arg max,yic; Og (x(x)) obtained. X and I are constructed as described in lines 10 and 8 of Algorithm | respectively. The observations in Section 3.1 suggest that the degree of diversification may be greatly reduced when the selected attacks employ the same objective function. Therefore, a set of the candidate combinations of na attacks is defined as = |a C A(x, n)) [[ )/=ı1 na} so that each attack uses a different (line in Algorithm| 2). The weighted average of the DI is caleulated for all e A quantify the diversity of the best point set Ua as follows:
) E Y l = ,,..,,.,Aaa. ) are
J P. DI(UaseXf).
D
(4)
1=1
where P. = Ubge 11;1 is the number of types of classification labels with the highest prediction probability excluding the correct label (lines 5-8 in Algorithm 2h. A high DI indicates a high diversity of Uye and a high P indicates a high output diversity. Finally, ADS outputs L(;)) ); as the appropriate combinations of and L, where {a = arg max.ea(x.n) D (line 9 in Algorithm 2 The algorithm is designed on the assumption that the best search points found through appropriate diversification are unlikely to form clusters. Therefore, ADS selects combinations based on DI, which can directly quantify whether point set contains clusters or not. Because there is no general definition of appropriate diversífication, selection method based on indices other than DI is also possible. The experimental results suggest that selecting combinations based on DI is reasonable approach.

---

Published in Transactions on Machine Learning Research (May/2024)
MDO framework (Algorithm 3)
Diversification phase
Intensification phase (lines 16-24)
Image
Image Label
(lines 5-11)
Attack success
ADS (Algorithm 2)
{ai))= (E
ADS (A (Algorithm 2)
{aj fa (5 A) 1j=1
Classifier
1x,
Attack failure
AXx
R2
Go
Initial point (x)
Step size (n)
Search direction (6)
Objective function (L)
Candidate attacks
Data
Attack (A (Algorithm 1)
(Image, Label, & Classifier)
Component of attack a
Attack components
(8 and L)
Figure 7: Flowchart of MDO framework.
3.3 Search framework for the MDO strategy
Considering the difference in directions reported by Yamamura et al. (2022), we propose a search framework called the MDO framework consisting of diversification and intensification phases. MDO framework is a combination of ensemble and composite. Figure iilluttrates the procedure of the ensemble, composite, and MDO framework. An ensemble realizes the diversílication phase, and the intensification phase is based on a composite. The diversification phase aims find appropriate starting points via extensive search within the feasible region, and the inten- sification phase aims to improve the objective value through intensive scarches of nearby areas around the best point obtained during the diversification phase. As demonstrated in Figure [2] DI of best search points obtained by an ensemble/a composite tends to be relatively high/low. The high/low DI value indicates that the corresponding search procedure is extensive /inttensieeee Thus, an ensemble is suitable for diversification, and a composite is suitable for intensification. The pscudocode of the MDO framework described in Algo- rithm 3 The overview of the MDO framework is also illustrated in Figure Assume that the total number of the MDO framework iterations is N na. Inspired by the APGD step size control, the diversification phase takes [0.41N] na iterations, and the intensification phase uses the remaining iterations.
performance between PGD and CG-based search
Diversification phasc. The diversification phase uses large step sizes to scarch broader arca. First, ADS is execuled with fixed step size of 2s and initial point sampling x determine the pairs {(5a;; La; ) 1n j= where (ai, ag....,,",.. C A(xinit, M)x) (line 4 in Algorithm 3b. For each pair of 6ug and Lay, N = [ iterations of the attack are executed with an initial step size of 2s starting at an initial point selected by Xuit (lines 6-11 in Algorithm 3). The images whose adversarial example is found are excluded from the attack target (line 10 in Algorithm 3) because we assume the robustness evaluation. The step size is updated by JАPGD. When the total iteration is Niter; APGD step size control allocates at least [0.22. Nater iterations with a step size of 28 and [(0.4] - 0.22) = a9 iterations with a step size of 28 or e. Inspired by this, the diversification phase uses the total iterations of N1 for each attack and the checkpoints of W = {[0.22 x N]] for step size control 7APGD to achieve diversification with large step sizes. The diversification phase aims to identify promising starting points through exlensive exploration. Therefore, we need to thoroughly explore the neighborhood of these identified points to enhance the objective value further. To achieve this, we introduce the intensification phase subsequently.
Intensification phase.
The intensification phase takes the solution with the highest Lew value found by the diversification phase as the initial point and searches for different local solutions within range not far from the initial point. In APGD step size control, the step size is set to e/2 after the search with step size
8

---

Published in Transactions on Machine Learning Research (May/2024)
Algorithm 3 MDO framework
Require: Σ: test samples, g: DNN, allowed perturbation size, n.: number of pairs to select, NADs: search iterations in ADS, M: the size of the feasible region, N: search iterations per attack
Ensure: Xudva; adversarial examples
1: X Xadve <
/* Diversification phase */
3: J + uniformly sampled 1% images of I. [[6,,, LL,))]== < ADS A(xmii,, 1jxx), NADS, Ps.g, M, Ma) Ni < [0.11,, N2 c N Ni
6: for = =.,...,, do
7: a' + (Ximx, MAPGD: Sey,Lug
X" ",,I'''' Attack (a', N1, 9. Algorithm /
9: Update Xadva by X
10: Update I by excluding images that succeeded in the attack.
11: end for
12: 1 Intensification phase */
13: Ianb +
14: J uniformly sampled 1% images of
15: [(ōn,: [ <- ADS (J', ((Xestt: THix), (atk),Na, NADS, Ps.g, M, ma)
for ==,...,,, do 17: a' -Xxeett Teos, day, Lay) {Use NAPGD instead of Mooos when ōa, 8xco
Xa _II' + Attack((a N2, Ps. Ps.g,IIun g,Ib)) /* /*Aloorithm
19: Update Xaávs by X=
Extract images whose highest CW loss value is close to 0 for further intensification. */
21: Lunb f- Lanth n EI 0..05
I max Lew (ow(g(x") c.)) x* Exx
X" _I Attack(a', N2, Ps, 9, Imub) Algorithm / 23: Update Xadva by x"
24: end for
of c. Based on this, the initial step size is set to e/2. First, ADS executed with a fixed step size of fj 8/2 to determine the pairs [(Sa,, -- (line 15 Algorithm 3 where (ā,, ă2, an.) C A(xbests 7hx), and Xhest denotes the initial point sampling that uses the solution with the highest Lew value as the initial point. The experimental results in Table demonstrate the effectiveness of the ADS execution in this phase (line 15 in Algorithm 3). For cach pair of (6a,- Lay), N₂ N Nı iterative scarches are performed with the initial point determined by Xhests and the initial step size of e/2, using step size update rule Teons (lines 16-19 in Algorithm 3/. Xbest enables composite execution of the attack sequence. The step size update rule Noos lcads to a larger step size than NAPGD in nature. Thus, Toos is expected to avoid getting stuck during intensification employing the steepest descent-based search directions. On the contrary, Meoe may result in insufficient intensification using a diversified search direction such as dACG- For this reason, 4APGD used update the step size when the search direction da, is equal to ÒACG, aiming for better intensification performance (line 17 in Algorithm 3). Subsequently, the same scarches performed the images whose highest. Lew values are greater than or equal to -0.05 to accelerate the intensification (lines 20-24 in Algorithm 3). Lines 20-24 are introduced to avoid insufficient optimization, one cause of attack failure pointed out by Pintor et al. 2022). Same as in the diversífication phase, successfully perturbed images, i.e. æ; E Ioub s.t. max exxx Low (g(x"), c)))) > 0, are excluded (line 23 in Algorithm
3.4 Efficient Diversified Attack
The comparisons of the MDO framework and MT in Section 4.2 suggest that combining MDO and MT strategics would lead to a faster and stronger attack. This result motivates us to propose Efficient Diversified Attack (EDA), combination of MDO and MT strategies. EDA implemented as an ensemble of the MDO framework and targeted attack a (Xiity MAPGD, dGD, I.C)) with N iterations. We selected this targeted
9

---

Published in Transactions on Machine Learning Research (May/202) (024)
Efficient Diversified Attack (EDA, Algorithm 4)
Image
MDO strategy
Image Label
(lines 2-4 of Algorithm 4)
Attack success
Attack success
MDO framework (Algorithm 3)
Classifier
MT strategy (lines 5-18 of Algorithm 4)
Attack failure
Attack failure
Target Selection (lines 6-13)
Targeted Attack (Algorithm
Attack success
Attack (A Agoorith 1)
Component of an attack
Attack failure
Figure 8: Flowchart of EDA.
Considering computational efficiency, we estimate the most. likely cause misclassification target class T; for the targeted attack through a few iterations of the multi-targeted attack. This procedure referred
attack configuration based on small experiments in Appendix G.1
a small-scale search (lines 6-13 in Algorithm 4 See Appendix F for validation of the larget selection scheme based on the small-scale search. The pseudocode of EDA is described in Algorithm Figure provides overview of EDA. EDA executes the MDO framework first (lines 1-4 in Algorithm 4 Same in the MDO framework, successfully perturbed images are excluded because we assume the robust evaluation (lines in Algorithm 4 Subsequently, the target class T; is selected based on the small-scale search from top- classes (lines 9-13 in Algorithm 4 Existing researches support the effectiveness of the multi-targeted attack with top-K classes Croce & Hein (2020b): Gao et al. (2022). Finally, the targeted attack a² with selected target T, is executed (lines 14-18 in Al Algorithm 4). Lines 1-4 the MDO strategy, and lines 5-18 represent the MT strategy. MT strategy is independent of the MDO strategy. Thus, the execution order of MDO and MT strategies is not important.
4 Experiments
The efficacy of the proposed methods was examined through a series of experiments involving an lo attack against loo defense models listed in RebustBench (Croce et al.] 2021
Dataset and models. We used 41 models and 21 differont defenses including 25 models trained on CIFAR-10, 11 on CIFAR-100 (Krizhevsky et al.) 2009). and five on ImageNet (Russakovsky et al. 2015 We also used 10 additional models trained on ImageNet with three different. defenses test the EDA 's performance against more advanced models, including transformer based architectures. We performed too attacks 10,000 images with 8/255 for CIFAR-10 and CIFAR-100 models and 5,000 images with == 4/255 for ImageNet models. These values are used in the RobustBench leaderboard. The test images were sampled in the same way as in RobustBench. The text presents results against nine representative models for CNN-based models. Complete results for CNN-based models are described in Appendix D
Computer specification. The experiments were conducted with two types of CPUs and single type of GPU, The CPUs used in the experiments were Intel(R) Xeon(R) Gold 6240R CPU @ 2.40GHz and Intel(R) Reo Silver 4216 CPU @ 2.10GHz. The GPU used in the experiments NVIDIA GeForce RTX 3090.
The used models publicly available robusthench v1.1.
10

---

Published in Transactions on Machine Learning Research (May/2024)
Algorithm 4 Efficient Diversified Attack (EDA)
Require: test set, g: DNN, s: allowed perturbation size, na number pairs select, K: number of target classes, NADS: search iterations in ADS, M: the size of the feasible region, N: search iterations per attack N,: number of iterations small-scale search
Ensure: Xadvs; Adversarial examples
1: X gadvs <- 0
2: 1 Rum MDO framework (Algorithm %
3: Xadva &- MDO framework(Z,, 9, Ma, NADS, M, N)
Update by excluding images that succeeded in the attack.
5: /* Begin multi-targeted strategy */
/* Select target class based on small-scale search */
7: T. + Tı, (Ai 1,.... , [Z])
8: a' <- (Xinit, Mooo. kGD. L(w)
9: for 1 I....K do
10: /* RRun targeted attack with the target class m. /
11: X" ",11" t Attack((a,, NN, Ps. 9. Allgorith /
12: Update target class 7, for images æ, that record the highest loss value
13: end for
a' (Xinit, WAPGD, den, L(w)
15: /* Rum largeted atlack with the larget T; //
16: X"" ,11"" + Attack(a", .N, Ps. g, g. /*Aloorithm|
17: /* End multi-targeted strategy */
18: Update xadvs by x=
Table 1: Comparison in robust accuracy. The lower the robust accuracy, the higher the attack performance. The lowest robust accuracy is in bold. The " "#query" row shows the number of queries in the worst-case
per image. The robust accuracy of AA is the value reported by RobustBench. MDO represents the MDO
framework. OOM (Out of Mcmory) indicates that execution could not be completed due to memory error.
Dataset
No.
Defense
clean
AA
ACG
MT...
CAA
AutoAE
A
MDO
#query
6.1k
500
504
800
3.6k
1k
502.24
802.24
Sehwag et al.
2022
84.59
55.54
56.19
55.54
55.52
55.51
53.53
55.58
55.49
CIFAR-10
Carmon et al.
2019
89.69
59.53
60.10
59.54
59.50
59.47
59.44
59.46
59.40
8/255)
Rebulfi et
2121
88.54
64.25
64.80
64.28
64.23
64.19
64.24
64.32
64.20
4
[Rice et al.
2020
53.83
18.95
19.48
18.99
18.96
18.91
18.89
18.97
18.88
CIFAR-100
Sitawarin
2021
62.82
24.57
25.69
24.55
21.56
24.52
24.56
24.65
24.50
(s 8/255)
Gowal al.
2020
69.15
36.88
37.84
36.95
36.90
36.86
36.87
36.96
36.81
7
Salman al.
2020
52.92
25.32
26.40
25.24
25.27
OOM
25.22
25.22
25.11
ImageNet Net
Engstrom et ..
2010
62.56
29.22
29.34
20
OOM
29.32
29.20
29.01
(s
4/255)
9
Wong et al.
2020
53.62
26.24
28.46
26.40
26.56
OOM
26.42
26.22
26.12
Summary (# bold 41 models)
0
0
2
36
When we compared the performance of each attack on model A and dataset B, all compared attacks were run on the same device. The runtime comparison is thus fair.
Hyperparameters. The parameters of the ADS n, 5 and NADS 4, which are the mumber of pairs of 8 and L and the number of iterations, respectively. These parameters were determined based small-scale experiments in Appendix D.1 The paramcter of the MDO framework is N 100. The total number of the MDO framework iterations is n. x N 500. EDA has parameters N, 10 and K in addition to ADS and the MDO framework parameters. The experiments used K = 9, 14, and 20 for CIFAR- 10, CIFAR-100, and ImageNet, rospectively. Unless otherwise noted, the initial point sampling (xmit) in ADS, the diversification phase, and the targeted attack was Prediction Aware Sampling (xPAs, PAS), variant of ODS described in Appendix E
11

---

Published in Transactions on Machine Learning Research (May/2024)
Table 2: The comparison in the robust accuracy against robust models trained on ImageNet, including transformer-based architectures. The robust accuracy of AA is the value reported in RobustBench. The
lowest robust accuracy is in bold. "sec (ratio)" columns show the runtime in seconds and the ratio of runtime to EDA.
clean
AA
CAA
Aa
EDA
Defense
Model
arc
acc
acc
sec (ratio)
a0c
sec (ratio)
acc
sec (ratio)
Liu et al.
20ZJA
ConyNeXt-B
76.02
55.82
55.86
20,244
(1.4)
55.84
37.437
(2.7)
55.78
14,073
(1.0)
et. al.
21244
Swin-B
76.16
56.16
56.14
25,913
[1.9)
56.06
20,620
(1.5)
56.04
13,988
(1.0)
Debenedetti
2023
XCYT-L12
73.76
17.00
17.70
25,196
(2.0)
17.51
20,011 (1.6)
17.14
12,705
(1.0)
Debenedeiti
2723
XCFT-M12
74.04
45.
45.44
13,187
(1.7)
45.22
11,067
(1.4)
45.14
7,884
(1.0)
Debeuedelt el
2123
XCIT-S12
72.34
41.78
41.88
8,690
(1.G)
41.74
6,560 (1.2)
41.60
5,548
(1.0)
Singa et sl.
2023
ConyNeXt-B+ Con-Steem
75.90
56.
56.12
21,650 (1.5)
56.14
35,122 (2.5)
56.02
14,399 (1.0)
Singh et ml
2123
ConvNeXe S+ Stenmmttem
74.10
52.42
52.44
14,184 [1.3)
52.38
24,512 (2.3)
52.30
10,601 (1.0)
Simghh et al 2023
ConvNeXi-T++ CoLT---CovSeen Stem
72.72
49. 16
49.42
8,444 (1.2)
49.44
19,391 (2.7)
49.38
7,311 (1.0)
Singh et al. 2023
WUT-B+CouvSteen
76.30
54.66
54.72
22,262 (1.9)
55.02
17,141 (1.4)
54.62
11,962 (1.0)
Singn et
2023
VII-S- ConvStem
72.56
48.08
48.18
8,912 (1.7)
48,08
5,928 (1.1)
48.02
5,355 (1.0)
4.1 Comparison with recent attacks
The comparative experiments of EDA with the standard version of AA, CAA, AutoAE, and A were con- ducted to investigate the performance of EDA. The parameters of the existing methods were the default values in their official codes. In addition, MT ans, a step size variant of MT-PGD (Gowal et al. 2019), was compared with the MDO framework to analyze the characteristics of the MDO strategy. In our notation, MT cos is expressed as (Korg: Teos: δGD, L(w). The parameters of MT were the number of target classes, K 9, and iterations per target class, Nr 56. The MDO framework and MT thus spent almost the same number of queries. To investigate the stability of the compared methods including A and EDA, we ran five times with different random seeds for the experiments on 41 CNN-based models. The results suggest a stable performance. T the mean of five runs. See Table 11 in the Appendix for the experimental results including standard deviations. The remaining experiments were conducted with a single fixed random soed.
EDA showed SOTA performance in less runtime under the standard setting in Robust The summary in Table ! shows that the attack performance of EDA exceeds that of recently proposed methods for around 88% of 41 CNN-based robust models. We could not finish AutoAE exccution against ImageNet models due to the out-of-mmmory error. As shown in Figure [ EDA showed lower robust accuracy with fewer numbers of worst-case queries than AA, CAA, Auto F,, and A3. In addition, Figure shows that EDA is 1.07-2.28, 1.46-2.87, and 18.74-38.61 61 times faster than A3, CAA, and AutoAE on average. A stops the scarch before the given query budget, depending on the models. Because of this, A spent less time than EDA for some models, specifically for CIFAR-10 models. The reasons for better performance and less runtime of EDA are the complementarity of MDO and multi-targeted strategies (Section 4.2 and higher degree of diversification of EDA (Figure 12 which lead to efficient diversification and intensification. The differences in robust accuracy among CAA, AutoAE, and A are approximately 0.05%. Considering these advances in robust accuracy by recent attacks, EDA's performance improvements, around 2 to 30 times speed-up up and approximately 0.01 to 0.21% improvement in robust accuracy, are sufficiently large. As shown in Table 2 EDA also demonstrated belter performance against. 100% of 10 advanced models, including transformer-based architectures. We discuss the complexity of the compared methods in the number of queries in Appendix D.2 in detail. In the following analyses of EDA, mainly focus the comparison with A3, which is sufficiently fast and strong among baseline methods.
EDA showed higher transferability. We chose three recent models trained on CIFAR-10, including Semi (Carmon et al. 2019), MMA (Ding et al. (020). and LBGAT (Cui et al.) 2021). Table 3 shows that EDA tends to exhibit the highest performance among baseline methods in the transfer scenarios from more robust models to less robust models. EDA showed the second-best. performance in the other scenarios.
12

---

Published in Transactions on Machine Learning Research (May/2022)
38.61
EDA
Table 3: Comparison in robust accuracy for transfer set- ting. "A-BB" indicates the transfer attack from model A
A²
CAA
to model B.
18.74
Transfer
clean
AA
CAA
AutoAE
EDA
Semi-
84.36
71.82
73.15
73:24
70.46
68.14
1.001.07 2.53
1.001.562.00
1.002.281.45
Semi-I.BGAN
88.22
71.90
74.76
74.66
73.46
70.29
MMA -Semi
89.69
83.23
85.01
84.85
85.16
83.08
CIFAR-10
CIFAR-100
ImageNet
MMA >LBCAT
88.22
79.77
82.16
81.83
52.42
80.32
LBGAT SSmi
89.69
76.17
90.24
80.32
79.94
27.21
LRGAT--MMA
84.36
70.16
72.07
72.09
Figure 9:
The average ratio of computation time EDA.
Best robust accuracy
LBGAT
MMA
Semi
in RobustBench leaderboard
52.86
59.53
Image
Noisex 1
Image
Noisex
Image
Noisex !
Image
NoiseX
Original
CAA
2-norm: 5.9547
2-norm: 5.5645
2-norm: 5.8908
2-norm: 5.3621
A
2-norm: 5.6698
2-norm: 5.2672
2-norm: 5.7469
2-norm: 4.9245
EDA
2-norm: 5.7201
2-norm: 5.527
2-norm: 5.6659
2-norm: 5.7124
Figure 10: Visualization of adversarial examples (a = 4/255 on ImageNet). "2 "2-norm" denotes the perturba- tion size terms of Euclidean distance. EDA and CAA succeoded in the attack, but A3 failed.
Visualization of the adversarial examples. Figure 10 shows the visualization of the generated ad- versarial examples. EDA and CAA, which induced misclassification, perturbed the original image more extensively than A in terms of Euclidean distance. This indicates that EDA explored a broader area than A3, leading to successful attack. The DI-based analysis in Section 4.1 also suggests higher diversification performance of EDA than A3, Additionally, EDA perturbed the image to lesser extent than CAA, yet both methods led to misclassification. This suggests that EDA is more likely to generate adversarial examples that are closer to the original image than CAA. This study does not focus on the perturbation size in the Euclidean norm, but it represents an advantageous property for the imperceptibility of perturbations.
Performance evaluation with different perturbation bounds E. As shown in Figure 11 for all datasets, the performance difference between the compared attacks is very small when epsilon is small. Regarding the runtime, EDA spent slightly less time than A3. When epsilon is large, A tends to have
13

---

Published in Transactions on Machine Learning Research (May/2024)
CIFAR-10 (Sehwag et al., 2022)
CIFAR-100 (Sitawarin al., 2021)
ImageNet (Engstrom al., 2019)
in rabbst acouracy (%)
AA-EDA
Difforance in rabust acouracy (5)
A?-EDA
0.2
in rabust acouracy
Difference
0.2
1Nfference
0.0
0.0
0.0
0.2
16
Perturbation bound (s = x/ 255
Perturbation bound (e x/255)
Perturbation bound (s x/255)
CIFAR-10 (Sehwag et al., 2022)
CIFAR-100 (Sitawarin al., 2021)
ImageNet (Engstrom al., 2019)
AA
Execurtion time (4)
4000
Execurtin time (4)
Execution time (6)
16000
EDA
20000
3000
10000)
2000
5000
1000
16
16
2
Perturbation bound (s = x/255)
Parturbation bound (c = x/255)
Perturbation bound (e x/255)
Figure 11: Difference in robust accuracy and runtime with different perturbation bounds s. The upper part shows the difference in robust accuracy, and the lower part shows the runtime in seconds. The positive value of the difference in robust accuracy indicates the EDA's performance is higher than the compared method. For the execution time, the lower value is better.
the highest attack performance. However, regarding runtime, AA and EDA tend to decrease epsilon increases, while A tends to increase. EDA showed better performance than AA for all perturbation bounds. In terms of overall runtime and performance, EDA is considered to achieve a good balance between the two. Appendix G.4 provides further results as a table. To further analyze the attack performance of EDA for large perturbation bounds, we compare the robust accuracy of A and EDA with a sufficiently large value of parameter N. N is set so that the runtime of EDA does not exceed the runtime of A3. We report the reprosentative results corresponding to Figure 11 For the model proposed by [Schwag et al. (2022) and trained CIFAR-10, EDA achieved 19.91% robust accuracy in 718 seconds with N = 300, and A achieved 20.07% in 868 seconds. For the model proposed by Sitawarin et al.] (2021) and trained on CIFAR-100, EDA showed 6.14% in 11,024 seconds with N = 2000, and A showed 6. 6.1 in 19,993 seconds. For the model proposed by Engstrom et al. (2019) and trained on ImageNet, EDA showed 7.38% robust accuracy in 6,752 seconds with N 1000, and A showed 7.44% in 11, 11,717 seconds. These results suggest that EDA 's performance may be competitive or slightly better than A3 if the computation budget in the sense of runtime is almost the same. Thus, the lower attack performance of EDA than A in the experimental results presented in Figure [D can be attributed to the slower convergence of EDA's search rather than its earlier convergence. Given that the runtime of A increased as the perturbation size increased and that the attack performance of EDA with increased runtime was equal to or better than A3, the number of queries required for attack convergence may increase as the perturbation size increases.
The appropriate balance of diversification and intensification is one of the reasons for EDA's 's high performance. According to Figure 12 EDA and MDO framework showed higher DI values than A3, The point set with higher value of DI is less likely to form clusters. This suggests that EDA searches larger area than 3 with almost the same or shorter runtime, resulting in EDA's high attack performance. A comparison of the MDO framework with ADS (ADS) and EDA in Figure 12 reveals that EDA exhibits a greater minimum DI than ADS. The distinction between ADS and EDA lies in the presence of MT stratogy. Consequently, the results presented Figure 12 suggest that the MT strategy may facilitate the diversification of images that are not adequately diversified by the MDO strategy. Whereas A uses a single 5 and I., FDA and MDO framework use multiple 8 and L.. Thus, this result. suggests that the attack
14

---

Published in Transactions on Machine Learning Research (May/2024)
Table 4: Ablation study of ADS.
MDO framework
0.3
No.
ADS
R-ADS
RAND
55.58
55.58
55.61
2 59.46
59.53
59.56
64.32 3
64.54
64.23
0.0
18.97
18.99
18.98
R-AD5
RAND
AD6
Composte EDA
A²
MT
5 24.65
24.68
24.71
36.96 6
37.53
37.05
Figure 12:
Violin plot of D1. The model is
Carmon
25.22
25.44
25.22
ct al.
2019
8
29.20
29.26
29.56
26.22 9
26.36
26.26
10
MTcue
MDO
Winagas
MTcos MDO
PGG ACC APGG Öges
Lce
Lew
Lscw
DLR
L.
GS
-300
-001
100
100
2000
300
0.0
1.0 0.0
LOUR LG- LG-DLRA
LG-
1.0
Difference in # query (MT MDD)
Figure 13: The difference between MT cos and the MDO framework in #queries to find adversarial ex- amples. The model is (Carmon et al. 2019
Figure 14: The ratio of 8 and L selected by ADS. LS and GS refer to the intensification and diversification phases, respectively.
with multiple ö and L has a higher diversification ability than the attack with a single ô and L. Although gradient-based attacks can achieve some degree of intensification by adjusting step size, the diversification and intensification are trade-offs given the computation budget. Figure 12 shows that MT cos showed higher DI Chan EDA. However, MT showed lower allack performance than EDA, as shown in Table This result suggests that MT cos performs insufficient intensification, and EDA has appropriate balance between diversification and intensification.
4.2 Analyses of the MDO strategy
The combination of MDO and MT strategies would lead to an efficient attack. Figure 13 shows the difference between the number queries spent by MTons and the MDO framework for images that both succeeded in attack. The images that MT and the MDO framework spent the same number of queries were excluded. The positive value means MT cos spent more queries than the MDO framework. Figure 13 indicates the presence of images that can be successfully attacked with fewer queries by oT..s/ /the MDO framework but require more queries by the MDO Hamework/MT..c.
The MDO strategy mainly contributed to the attack performance of EDA. We examined the ratio of adversarial examples generated only by the MDO framework, the targeted attack a"", and both methods to entire images that EDA succeeded but A failed. In the case CIFAR-10, the percentages of adversarial examples generated only by the MDO framework, aT and both methods are 58.98 58.98%, 6.71%, and 34.31%, respectively. These values are averages over the 25 models trained on CIFAR-10. Similarly, the percentages 35.02 35.02%, 8.28%, and 56.70% for CIFAR-100 and 25.90%, 15.51%, and 58.59% for ImageNet. Here, the percentages in CIFAR-100 are averages analysis suggests that the MDO strategy contributes to the attack performance more than the MT strategy, and it can find adversarial examples that are difficult to find for existing altacks. In addition, EDA showed larger improvements in robust accuracy in half the runtime of A3 for models trained on ImageNet, and the MDO framework showed promising results for the same modcls. These results indicate that the MDO strategy may have advantages in the attack for models trained on ImageNet, which is more practical regarding image size and the number of classification classes.
11 models and those in ImageNet over five. This
15

---

Published in Transactions on Machine Learning Research (May/2024)
4.3 Analyses of ADS
The combination of 5 and I. selected by ADS brings a higher degree of diversification. We compared the attack performance of the MDO framework with three selection algorithms, including ADS, Reverse ADS (R-ADS), which finds the pairs minimizing equation | and uniform sampling (RAND). ADS
in Figure 12 and Table | represents the MDO framework with the selection algorithm ADS, and the same is applied to R-ADS and uniform sampling. Figure 12 shows that the DI of the best scarch points obtained by the MDO framework with ADS tends to be higher than that obtained by the MDO framework with R-ADS and uniform sampling, indicating ADS may select the pairs that enhance the diversification. According to Table 4 the MDO framework with ADS showed lower robust accuracy than that with R-ADS and uniform sampling. In addition, the MDO framework with R-ADS showed significantly higher robust accuracy on several models. Although the performance difference between ADS and uniform sampling is smaller than between ADS and R-ADS, ADS can select the appropriate pair more stably than uniform sampling. These results indicate that ADS selected the pairs that diversify the search more, leading to stronger atlacks.
The percentage of å and L selected by ADS. As shown in Figure 14 during the diversification phase, ADS tended to choose DLR loss and the search direction of ACG. In the intensification phase, CW loss and directions other than ACG were more likely to be selected. Yamamura et al. (2022) reported that ACG showed superior diversification ability compared to APGD. In addition, ACG showed better attack performance with DLR loss, whereas APGD showed better performance with CW loss. These experimental results suggest that ADS selected appropriate pairs in both the diversification and intensification phases. Similar trends were observed regardless of the datasct.
The role of D in the intensification phase. The correlation between D and the highest CW loss value was checked to investigate the influence of ADS on the intensification phase. The results show that D and the highest CW loss value are positively correlated, with correlation coefficients of 0.57 on average. As discussed in Section 4.4 attack performance is significantly degraded if ADS is not performed in the intensification phasc. These experimental results suggest that ADS can select appropriate pairs for diversification and intensification. We further discuss the individual influence of the two terms, pe and DI, that appear in De calculation on the overall ADS search performance in Appendix G.3
Influence of restricting candidate combinations. To investigate the impact of restricting the can- didate combinations in ADS (line 4 in Algorithm 2 we compare robust accuracy and runtime with and without this restriction. Table 5 shows that restriction on the candidate combinations docs not necessarily have a positive impact on allack performance. However, in terms of runtime, this restriction provides a sig- nificant advantage. This advantage is due the difference in the number of D calculations. D calculation requires the Euclidean distance between search points whose computation cost depends on the size of the images. Therefore, the runtime becomes more significant without this restriction for ImageNet, where the image size. is larger.
4.4 Ablation study of the MDO framework
Comparison with an ensemble and composite. We executed na attacks selected by ADS us- ing the MDO framework, ensemble, and composite. The ensemble and composite executed the attacks aj = (xPAS, Toos, daz, Lay) for j I,..., na. The initial step size and number of iterations of each a, were set to 28 and N 100, respectively. Table @ that the attack performance of the MDO framework is higher than that of the ensemble and composite in most cases. Table T also shows that the attack perfor- mance of the MDO framework is higher than AA for several models in fewer queries. The violin plot for DI in Figure 12 shows that the degree of diversification is higher for the ensemble, the MDO framework, and the composite, in descending order. These experimental results indicate that the MDO framework is one of the effective implementations of the MDO strategy because it can achieve an appropriate balance of diversification and intensification than ensemble and composite.
16

---

Published in Transactions on Machine Learning Research (May/2024)
Table 5: Influence of restricting candidate combinations.
Table 6: Ablation study of the MDO framework.
w/o restriction
w/ restriction
A
MDO
MDO framework w/o ADS
No.
acc
sec
8cc
(sec)
No.
Ensemble
Composite
framework
in the intensification phase
55.62
672
55.58
285
55.68
55.90
55.58
62.80
2 59.46
2,308
59.46
2,112
195
2
59.70
59.59
59.46
69
64.23 3
15,300
64.32
187
113
3
64.53
64.43
64.32
70.71
4
18.98
487
18.97
299
187
18.97
24.40
G
24.61
1,420
24.65
1,186
233
24.74
24.83
24.65
32.51
36.92
9,229
36.96
9,175
53
6
37.19
37.17
36.96
43.87
7 25.20
6,190
25.22
1,430
4,759
25.46
25.48
25.22
29.76
8 29.28
6,492
29.20
1,973
4,518
29.64
29.68
29.20
36.74
9
26.40
7,765
26.22
2,930
4,834
26.84
26.78
26.22
30.66
Table 7: Influence of further intensification (lines 21-23 of Algorithm
w/o lines 21-23
w/o lines 22-23
w/ lines 21-23
Atlack subset (line 21) Extra attack iteration (lines 22-23)
No.
SCC
acc
SCC
1
55.58
574
55.63
340
55.58
387
2
59.46
3,675
59.45
1,914
59.46
2,112
3
64.30
26,709
64.30
13,993
64.32
15,187
4
18.94
380
19.00
262
18.97
299
5
24.66
1,749
24.67
1,078
24.65
1,187
6
36.93
14,385
36.93
8,335
36.96
9,175
7
25.18
1,376
25.18
1,182
25.22
1,431
8
29.16
2,688
29.36
1,929
29.20
1,974
9
26.36
3,795
26.42
2,838
26.22
2,931
Influence of ADS in the intensification phase. TaalleBshonee the robust accuracy of the MDO frame- work with and without ADS exccution prior to the intensification phase. The results show that the attack performance significantly drops without ADS prior to the intensification phase. This suggests that the pairs suitable for diversification and intensification differ from each other. The EDA's performance may be further improved by considering the execution order of the attacks with selected and L. However, determining the execution order requires that (n.!)² permutations be considered, which is computationally expensive. The experimental results suggest that EDA achieves satisfactory trade-off between the computation cost. and the attack performance.
Influence of further intensification (lines 21-23 of Algorithm The intensification phase of the MDO framework is conducted subset of test samples reduce the computational cost. This section validates the efficacy of this heuristic for determining the subset of test samples and further intensification this subset (lines 21-23 of Algorithm 3. Table 7 indicates that limiting the attack target images during the intensification phase results in notable reduction in computational cost compared to a sccnario where no such limitations are imposed. Nevertheless, simply limiting the larget images may reduce allack performance by up to 0.2%. According to the comparison between "w/o lines 21-23" and "w/ lines 21-23", the performance degradation up to 0.04% when more attack queries are allocated the subset of test samples. The required computational overhead is not particularly significant compared to "w/o lines 22-23". Furthermore, the attack performance of "w/ lines 21-23" is the highest for some models.
17

---

Published in Transactions on Machine Learning Research (May/2024)
4.5 Additional results and ablations
Appendix D.5 describes the analysis of EDA for the model of Ding et al. (2020), which showed different trends. The analysis using the Euclid distance-based mcasure in Appendix D.4 showed similar trends to DI-based analysis. Appendix G shows the result of the ablation studies, including the hyperparameter sensitivity of EDA and performance evaluation with different perturbation bounds.
EDA is expected to be robust to hyperparameter settings to some extent. As described in Appendix performance than AA and A for randomized defenses.
H EDA also showed better
5 Limitations and assumptions
This study has the following assumptions and limitations. EDA assumnes to be able to access the outputs and gradients of the attacked models. Similar to the Adaptive Auto Attack (A²), EDA assumes to attack the set of images. These assumptions are reasonable for robustness evaluation and adversarial training. The number of combinations of search directions and objective functions grows exponentially. Thus, the number of search directions and objective functions to be selected should be manageable. Unlike A AA and EDA do not alter the number of attack queries during the attack process. Although we empirically find that EDA is robust to hyperparameter scttings, users should specify enough number of attack queries, espocially for the different perturbation bounds.
Expected situations in EDA work well This study experimentally verifies the performance of EDA, focusing on image classifiers using deterministic defenses such as adversarial training. Although EDA was tested with images of different resolutions, domains, and models of different architoctures, EDA workod in most settings. Therefore, it can be assumed that EDA has some generalization performance for image classifiers using deterministic defenses. Furthermore, EDA showed lower robust accuracy in less runtime than AA and A on several randomized defenses. Similar to AA, EDA is applicable to the models whose gradients are available. However, the performance of EDA against models out of image classifiers needs further investigation.
Expected situations in EDA do not work well EDA contains gradient-based attacks as some compo- nents. Therefore, EDA's performance may be degraded for models which cause incorrect gradient calcula- tions. As reported by Croce& & Hein (2020b), existing techniques like expectation over transformation might help improve the attack performance on these models. Also, the performance of EDA may be degraded when the MDO and MT strategies do not work well.
6 Conclusion
This study empirically confirmed that different local solutions can be efficiently enumerated using various search directions and objective functions. Based this observation, have proposed the MDO strategy and its implementation, including ADS and the MDO framework. The experiments robust models, including 41 CNN-based and 10 more advanced architectures, have demonstrated that the MDO strategy realized by the MDO framework has a higher diversification ability. In addition, EDA, a combination of MDO and MT strategy, showed higher attack performance than recently proposed attacks. Though more appropriate indices may exist, these results suggest that the attack designed basod on the DI shows an appropriate balance of diversification and intensification, resulting a stronger allack. We are interested in exploring simpler algorithms future work that perform as well better than the proposed method.
The experimental results in Section 3.1 provide future rescarch dircctions for further understanding of oplimization--assed adversarial attacks or adversarial training. The first direction is a systematic classi- fication of the optimization-based attacks. Although many optimization-baser attacks have been proposed, the appropriate method differs depending on the purpose, including training, whitc-box evaluation, and transfer-basee attacks. Systematic classification may help in selecting appropriate methods. The second di- rection is adversarial training using multiple attacks. The experimental results suggest that different attacks
18

---

Published in Transactions on Machine Learning Research (May/202) (024)
find different clusters of adversarial examples. This slight difference may affect the stability or efficiency of adversarial training. Adversarial training using EDA is also interesting topie.
Broader Impact Statement
Deep neural networks (DNNs) are known to be vulnerable to adversarial examples. A promising defense mechanism to address this vulnerability is adversarial training, where training is performed using adversarial examples. Many adversarial examples generated by strong attack methods are required to produce robust models through adversarial training. Therefore, developing strong and fast adversarial attacks helps improve the robustness of DNNs. The EDA, which is one of the main proposals of this research, can generate larger number of adversarial examples in shorter time and can be used for both the robustness evaluation of defense methods and data generation in adversarial training. Thus, this research siguificantly contributes to the security of DNNs. The positive impact of this research is twofold. First, we can make DNNs more robust through adversarial training using the data generated by the strong attack method EDA. Second, we can more accurately evaluate the robustness of the models. The potential negative impact of this research includes possible attacks by malicious users on systems containing DNNs. However, EDA is a white-bor attack that assumes the accessibility of model gradients. In addition, it is difficult to access the model gradients involved in real system. EDA is thus unsuitable for attacking real system. As described above, research benefits are more significant than the potential negative elfects. This study helps to improve the robustness of DNNs, allowing them to be more safely applied to a broader range of applications, including safety-critical applications.
Acknowledgments
This research project was supported by the Japan Science and Technology Agency (JST), the Core Re- search of Evolutionary Science and Technology (CREST), the Center of Innovation Science and Technology based Radical Innovation and Entrepreneurship Program (COT Program), JSPS KAKENHI Grant Number JP16H01707 and JP21H04599, Japan.
References
Sravanti Addepalli, Samyak Jain, and Venkatesh Babu Radhakrishnan. Efficient and elfective augmentation strategy for adversarial training. In NearIPS, 2022.
Insaf Adjabi, Abdeldjalil Ouahabi, Amir Benzaoui, and Abdelmalik Tameb-Ahmed. Past, present, and future of face recognition: review. Electronics, 9(8):1188, 2020.
Maksym Andriushchenko and Nicolas Flammarion. Understanding and improving fast adversarial training. In NewrIPS, 2020.
Maksym Andriushchenko, Francesco Croce, Nicolas Flammarion, and Matthias Hein. Square attack: A query-efficient black-box adversarial attack via random search. In ECCV, 2020.
Nikolaos Antoniou, Efthymios Georgiou, and Alexandros Potarnianos. Alternating objectives generates stronger base-based adversarial attacks. CoRR, abs/2212.07992,, 2022.
Nicholas Carlini and David A. Wagner. Towards evaluating the robustness of neural networks. In SP, 2017.
Yair Carmon, Aditi Raghunathan, Ludwig Schmidt, John C. Duchi, and Percy Liang. Unlabeled data improves adversarial robustness. In NearIPS, 2019.
Francesco Croce and Matthias Hein. Minimally distorted adversarial examples with a fast adaptive boundary attack. In ICML, 2020a.
Francesco Croce and Matthias Hein. Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks. In TCML, 2020b.
19

---

Published in Transactions on Machine Learning Research [May/202/)
Francesco Croce, Maksym Andriushchenko, Vikash Sehwag, Edoardo Debenedetti, Nicolas Flammarion, Mung Chiang, Prateek Mittal, and Matthias Hein. Robustbench: standardized adversarial robustness benchmark. In NewrIPS, 2021.
Jiequan Cui, Shu Liu, Liwei Wang, and Jiaya Jia. Learnable boundary guided adversarial training. In ICCV, 2021.
Edoardo Debenedetti, Vikash Sehwag, and Prateek Mittal, A light recipe train robust vision transformers. In SaTML, 2023.
Gavin Weiguang Ding, Yash Sharma, Kry Yik Chau Lui, and Ruitong Huang. MMA training: Direct input space margin maximization through adversarial training. In ICLR, 2020.
Minjing Dong, Xinghao Chen, Yunhe Wang, and Chang Xu, Random normalization aggregation for adver- sarial defense. NewrIPS, 2022.
Xingye Dong, Ping Chen, Houkuan Huang, and Maciek Nowak. A multi-restart iterated local search al- gorithm for the permutation flow shop problem minimizing total flow time. Computers & Operations Research, 40(2):627-66) 6322, 2013.
Logan Engstrom, Andrew lyas, Hadi Salman, Shibani Santurkar, and Dimitris Tsipras. Robustness (python library), 2019.
Yonggan Fu, Qixuan Yu, Meng Li, Vikas Chandra, and Yingyan Lin. Double-win quant: Aggressively winning robustness of quantized deep neural networks via random precision training and inference. In ICML, 2021.
Ruize Gao, Jiongxiao Wang, Kaiwen Zhou, Feng Liu, Binghui Xie, Gang Niu, Bo Han, and James Cheng, Fast and reliable evaluation of adversarial robustness with minimum minimum-margin attack. In ICML, 2022.
Fred Glover and Michele Samorani. Intensification, diversification and learning in metaheuristic optimization. Journal of Heuristics, 2019.
lan J. Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples.
In ICLR, 2015.
Sven Gowal, Jonathan Uesato, Chongli Qin, Po-Sen Huang, Timothy A. Mann, and Pushmeet Kohli. An alternative surrogate loss for pgd-bascte based adversarial testing. CoRR, abs/1910.09933 09338, 2019.
Sven Gowal, Chongli Qin, Jonathan Ucsato, Timothy A. Mann, and Pushmoet Kohli. Uncovering the limits of adversarial training against norm-bounded adversarial examples. CoRR, abs/2010.00599, 2020.
Abhishek Gupla, Alagan Anpalagan, Ling Guan, and Ahmed Shaharyar Khwaja. Deep learning for object detection and scene perception self-driving cars: Survey, challenges, and open issues. Array, 10:100057, 2021.
Dan Hendrycks, Kimin Lee, and Mantas Mazeika. Using pre-training can improve model robustness and uncertainty. In ICML, 2019.
Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. 2009.
Jiadong Lin, Chuanbiao Song, Kun He, Liwei Wang, and John E. Hopcroft. Nesterov accelerated gradient and scale invariance for adversarial attacks. In ICLR, 2020.
Chang Liu, Yinpeng Dong, Wenzhao Xiang, Xiao Yang, Hang Su, Jun Zhu, Yuefeng Chen, Yuan He, Hui Xue, and Shibao Zheng. A comprehensive study on robustness of image classification models: Benchmarking and rethinking. arXiv preprint ar XrXim:2302. 14301, 2023a.
Jiabao Liu, Qixiang Zhang, Kanghua Mo, Xiaoyu Xiang, Jin Li, Debin Cheng, Rui Gao, Beishni Liu, Kongyang Chen, and Guanjie Wei. An efficient adversarial example generation algorithm based on an accelerated gradient iterative fast gradient. Comput. Stand. Interfaces, 82:103612, 2022a.
20

---

Published in Transactions on Machine Learning Research MMy/22024)
Ruofan Liu, Yun Lin, Xianglin Yang, Siang Hwee Ng, Dinil Mon Divakaran, and Jin Song Dong. Inferring phishing intention via webpage appearance and dynamics: A deep vision based approach. In USENIX Security, 2022b.
Shengeai Liu, Fu Peng, and Ke Tang. Reliable robustness evaluation via automatically constructed attack ensembles. In AAAI, 2023b.
Ye Liu, Yaya Cheng, Lianli Gao, Xianglong Liu, Qilong Zhang, and Jingkuan Song. Practical evaluation of adversarial robustness via adaplive auto attack. In CVPR, 2022c.
Ilya Loshchilov and Frank Hutter. SGDR: stochastie gradient descent with warm restarts. In ICLR, 2017.
Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu, Towards deep learning models resistant to adversarial attacks. In ICI.R, 2018.
Xiaofeng Mao, Yuefeng Chen, Shuhui Wang, Hang Su, Yuan He, and Hui Xue. Composite adversarial attacks. In AAAI, 2021.
Leland McInnes, John Healy, Nathaniel Saul, and Lukas Großberger. UMAP: uniform manifold approxima- tion and projection. J. Open Source Softw., 3629):861, 2018.
Yurii Nesterov. Introductory Lectures Conver Optimization. Springer New York, NY, 2004.
Yuval Netzer, Tao Wang, Adam Coates, Alessandro Bissacco, Bo Wu, and Andrew Y Ng. Reading digits in natural images with unsupervised feature learning. 2011.
Tianyu Pang, Xiao Yang, Yinpeng Dong, Taufik Xu, Jun Zhu, and Hang Su. Boosting adversarial training with hypersphere embedding. In NeurIPS, 2020.
Dan Pelleg and Andrew W. Moore. X-means: Extending k-means with efficient estimation of the number of clusters. In ICML, 2000.
Maura Pintor, Luca Demetrio, Angelo Sotgiu, Ambra Demontis, Nicholas Carlini, Battista Biggio, and Fabio Roli. Indicators of attack failure: Debugging and improving optimization of adversarial examples. In Newr/PS, 2022.
Aitrestre-Alvise Rebuffi, Sven Gowal, Dan Andrei Calian, Florian Stimberg, Olivia Wiles, and Timothy A. Mann. Data augmentation can improve robustness. In NewrIPS, 2021.
Leslie Rice, Eric Wong, and J. Zico Kolter. Overfitting in adversarially robust deep learning. In ICML, 2020.
Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, Alexander C. Berg, and Li Fei-Fei. ImageNet Large Scale Visual Recognition Challenge. IJV,, 115(3):211-252. 2015.
Hadi Salman, Andrew Tlyas, Logan Engstrom, Ashish Kapoor, and Aleksander Madry. Do adversarially robust imagenet models transfer better? In NewrIPS, 2020.
Vikash Schwag, Shiqi Wang, Prateck Mittal, and Suman Jana. HYDRA: pruning adversarially robust neural networks. Tn NeurIPS, 2020.
Vikash Sehwag, Saeed Mahloujifar, Tinashe Handina, Sihui Dai, Chong Xiang, Mung Chiang, and Prateek Mittal. Robust learning meets generative models: Can proxy distributions improve adversarial robustness? In ICLR, 2022.
Naman D Singh, Francesco Croce, and Matthias Hein. Revisiting adversarial training for imagenct: Archi- tectures, training and generalization across threat models. ar preprint urXi::2303.01870 2023.
Chawin Sitawarin, Supriyo Chakraborty, and David A. Wagner. SAT: improving adversarial training via curriculum-based loss smoothing. In ATSecQCCS, 2021.
21

---

Published in Transactions on Machine Learning Research May/2202
Kaustubh Sridhar, Oleg Sokolsky, Insup Lee, and James Weimer. Improving neural network robustness via persistency of excitation. In ACC, 2022.
Christian Szegedy, Wojciech Zaremba, llya Sutskever, Joan Bruna, Dumitru Erhan, lan Goodfellow, and Rob Fergus. Intriguing properties of neural networks. In ICLR, 2014.
Yusuke Tashiro, Yang Song, and Stefano Ermon. Diversity can be transferred: Output diversification for white- and black-box attacks. In NeurIPS, 2020.
Yisen Wang, Difan Zou, Jinfeng Yi, James Bailey, Xingjun Ma, and Quanquan Gu. Improving adversarial robustness requires revisiting misclassified examples. In ICLR, 2020.
Eric Wong, Leslie Rice, and J. Zico Kolter. Fast is better than free: Revisiting adversarial training, In ICLR, 2020.
Dongxian Wu, Shu-Tao Xia, and Yisen Wang. Adversarial weight perturbation helps robust generalization. In NewrIPS, 2020.
Zhuoer Xu, Guanghui Zhu, Changhua Meng, shiwen cui, Zhenzhe Ying, Weiqiang Wang, Ming GU, and Yihua Huang. A2: Efficient automated attacker for boosting adversarial training. In NewrIPS, 2022.
Kelichiro Yamamura, Haruki Sato, Nariaki Tateiwa, Nozomi Hata, Toru Mitsutake, Issa Oe, Hiroki Ishikura, and Katsuki Fujisawa. Diversified adversarial attacks based conjugate gradient method. In ICML, 2022.
Dinghuai Zhang, Tianyuan Zhang, Yiping Lu, Zhanxing Zhu, and Bin Dong. You only propagate once: Accelerating adversarial training via maximal principle. In NeurIPS, 2019.
Jingfeng Zhang, Xilie Xu, Bo Han, Gang Niu, Lizhen Cui, Masashi Sugiyama, and Mohan S. Kankanhalli, Attacks which do not kill training make adversarial learning stronger. In ICML, 2020.
Appendix
This supplementary material provides additional information as follows.
The summary of abbreviations and mathematical notations defined in the main text (Tables and 9
2. More information about related work (Appendix
3. The proposed search direction and objective function (Appendix
Additional results of the analysis in Section 3.1 Appendix
5. Complete results of the experiments in Scction AAppendix
D
The details of Prediction Aware Sampling (Appendix
7. The details of the targeted attack used in EDA (Appendix
8. Ablation study of EDA (Appendix
Experiments on randomized defenses (Appendix
22

---

Published in Transactions on Machine Learning Research (May/202) (024)
Table Summary of abbreviations
Existing attack techniques
PGD
Projected Gradient Descent
MT-PGD
MultTTargotod-PGI.
APGD
Auto-PGD
ACG
Auto Conjugate Gradient attack
AA
AutoAttack
CAA
Composite Adversarial Attack
A
Adaptive Auto Attack
CE
Cross-entropy
DLR
Difference of Logit Ratio
ODS
Output Diversified Sampling
DI
Diversity Index
Proposed methods
G-DLR
Generalized-DDL
NAG
Nesterov's accelerated gradient
PAS
Prediction Aware Sampling
MDO
Multi-direetions//oojpe
ADS
Automated Diversified Sampling Search framework for the MDO strategy Efficient Diversified Attack
MDO framework
EDA
Others
DNN
Deep neural network
SOTA
State-of-the-art
MT
Multi-target
UMAP
Uniform Manifold Approximation and Projection
A More information about related work
A.1 Search directions
Projected Gradient Descent Projected Gradient Descent (PGD) (Madry al. 2018) is the most funda-
mental adversarial attack based on the steepest gradient descent. The search direction of PGD is computed
follows:
DTGD = sign (VI(g(a(())
(5)
Also, Fast Gradient Sign Method (FGSM) Goodfellow et al. 2015 updates towards the same direction.
Auto-PGD Auto-PGD (APGD) CCoce & Hein 2020b is PGD variant that adjusts the step size and
updates towards um direction in addition to the PGD's search direction. The search direction of APGD is defined as follows.
(k) Ps (k) + (x(a) (k)
8PGD):
(6)
(k) APGD ((2(k) - (k)) + (z) (k-)))
(7)
where a a cocfficient of momentum term. APGD uses 1 for the first itcration and a 0.75 for the remaining iterations. The step size (R) halved the following conditions are satisfied the wy e W iteration, with the initial value =
23

---

Published in Transactions on Machine Learning Research (May/202) 2024)
Table Summary of mathematical notations
Defined in Section 2.1
9: + RC
Locally differentiable C-classifier.
Borg D
Original input.
CE (1,2....,C}
The correct classification label of Zorg.
d: D) x D -R R
A distance function.
S
The feasible region.
L:RO x - R
Objective function (untargeted attack).
IF: RO x X x Y +R
Objective function (targeted attack).
Defined in Section 2.2
Ps: D S
Projection onto the feasible region S.
Initial point sampling.
Step size update rule.
Search direction direction/Update formula. Search point at iteration
(k) D
(k) E R
Step size at iteration k.
(k)) R"
Search direction at iteration An attack.
(z, 17, 8, L)
X
A set of best search points.
X
A set of best search points corresponds to image Z; A set of classification labels with the highest probabilities except for the correct classification class.
IIe
Defined in Section
Mg EY
The class label that has the q-th largest value of g(x). A set of update formulas.
D
C
A set objective functions.
A set of attacks with initial point sampling and step size update rule 7).
A(x,1)
Diversity Index of a point set X.
DI(X) = DI(X.M)
M is the size of feasible region.
Defined in Section 3.2
{a
combination of attacks.
1
<p (my wy
Lmax (g(2(1)-1)),)) Limx ()) and ,/ (2))))
The sequence of checkpoints W is computed based on the following gradual equation depending on the total number of itcrations Niter- Po = 0, Pi 0.22, Pj+1 = P; + max - P;-1 0.03, 0.06), w, = [p Niter In our notation, 7APGD denotes this step size updating rule.
Auto-Conjugate Gradient attack Auto- Conjugate Gradient (ACG) attack (Yamamura al. 2022) is inspired by the Conjugate Gradient method for nonlinear optimization problems. ACG performs
24

---

Published in Transactions on Machine Learning Research (May/202) 2024)
diversified search than the attacks based on the steepest descent. The search direction of ACG is as follows.
yla) VL(g(x(k-1)) )))) VL(g(z(()),
(8)
g(h) VI((x(k), (l(a y)
(9)
(y(l)) ACG
((k) (L(x)))) ,) g/ Xo0 ACG
(10)
A.2 Objective functions
Cross-entropy loss The untargeted version of cross-entropy (CE) loss is defined follows.
LCE(s(T), (z) + log
Σ
Exp
exp (1)((a))
(11)
j#c
Also, the targeted version CE loss is defined as follows.
L) = =g(()))
- log Σ jØt
exp (g)(2))
(12)
where 1 denotes the target label of misclassification. CE loss is known to be sensitive to the scaling of the logit, ie., the attack performance significantly varies depending the scaling of the logit (Carlini & Wagner 2017 Croce & Hein 2020b).
CW loss The untargeted version of CW loss is defined as follows.
Lew(g(æ), c) = max (x) - go(x)
(13)
JyC
Also, the targeted version of CW loss defined as follows.
( = gx(x)) (x)),
(14)
where denotes the larget label of misclassification.
Difference of Logit Ratio loss The untargeted version of the Difference of Logit Ratio (DLR) loss is defined as follows.
LDLR(O(x),c)- max 9j (x) gc(
(15)
FT. 9Fg
where Ty denotes the classification label with g-th highest value in g(xr). Also, the targeted version of DLR loss is defined as follows.
g((x) ge(x))
c, Jz1 (Ing (w) + INL (x))/2²
(16)
where / denotes the larget label of misclassification.
A.3 Comparison of existing attacks and EDA
Table 10 summarizes the characteristics of PGD-like like attacks, Auto Attack (AA) Croce & Hein 2020b)
Composite Adversarial Attack (CAA) Mao al. (2021), AutoAE Liu et al. (2023b) Adaptive Auto Attack (A3) Liu et al. 2022c). and EDA in terms of diversification, intensification, and computational cost. The allacks in Table 10 perform well in intensification because they include gradient-based allacks with ap- propriate step size management, Although PGD-like attacks have several variations, this section describes the representative one Table 10 PGD-like attacks and A3 use multi-rostart for diversification, and both attacks spend relatively short computational time. However, A outperforms PGD-like attacks because A uses better initial point sampling. AA considers multiple objective functions and multi-target attacks
25

---

Published in Transactions on Machine Learning Research (May/2024)
Table 10: Characteristics of PGD-like attacks, AA, A and EDA. multi-Z denotes the diversification using multiple objective functions, and multi-ò denotes the diversification using multiple search directions.
Diversification
Attacks
Intensification
Runtime
multi-L
multi-6 multi-restart
multi-target
PGD-like AA
short
long
CAA
V
long
AutoAE A
long short
EDA
short
for diversification in addition to multi-restart. Although AA achieves high attack success rate, AA is computationally expensive because AA consists of four attacks, including APGD with untargeted CE loss, APGD with targeted DLR loss, FAB attack (Croce & Hein 2020a), and square atlack (Andriushchenko iet al. 2020). CAA and AutoAE also combine multiple attacks by solving an additional optimization prob- lem. CAA executes MultiTargeted (MT) Ottack/Gowar et al. (2019) and CW attack Carlini Wagner (2017) as composite of attacks. AutoAE runs APGD with CE loss, APGD with DLR loss, FAB, and MT attack ensemble. In the sense that AA, CAA, and AutoAE different types of attacks, that they employ the diversification strategy based multi-ò. However, these methods use only a single search direction to solve (2) in the white-box setting. Therefore, we do not consider AA an attack with a diversification strategy based on multi-ō. In contrast, EDA, the proposed attack, uses all diversification strategics listed in Table 10 and achieves a higher attack success rate in a short computation time.
consider
The proposed search direction and objective function
B.1 Search direction inspired by Nesterov's Accelerated Gradient
Although some atlacks were inspired by Nesterov's accelerated gradient (Nesterov 2004 most of them apply constant value to the coefficient of momentum (Lin al.) 2020; Liu et al.) 2022a). However, the original Nesterov's accelerated gradient method determines the coefficient of momentum term by solving the quadratic equations. So then try to adopt Nesterov's accelerated gradient to foo attacks. Mathematically, dNes is computed by the following equations.
((k) is a positive solution of (2()) = (1 (1-
(17)
(K) 11- (p(k-1)-1)
(18)
pb)) + (p(k-1)32
(k) - (k) + 7(k) (z - x (k-1)
(19)
8 (k+1) t sign (
(20)
B.2 Generalized-Due DLR loss
We generalize DLR loss by extending the denominator of DLR loss from Ir, (x) Irg (x) to Fo. (æ) yr, (x).
Ky E Y denotes the class label that has the q-th largest value of g(x). More precisely, LG-DLR,y is defined as
LG..LL.gl(x c) = gc(x)) Irg (x)
(21)
Ir. (x) Ir, (x:
The motivation for proposing generalized-DIR loss to increase diversity in the output space. For the scarch points with larger values of G-DLR loss, the denominator cotooiinato/mmmrratooo /nmmerator will be smaller /laler// value, respec- tively. The small value of denominator gr, (a:) Dr, (2r) means that the values of gr, (æ), grg (æ),... -.. -r, (ar) are
26

---

Published in Transactions on Machine Learning Research (May/2024)
Andriushchenko et (PARN-1S)
Andriushchenko al. (PARN-18)
DI of uniformly sampled points
DI of uniformly sampled points
6
OPGD
APGD
ACG
O Nes
Len
Low Lsew LOLR LODIRA LGDLR. LEDLR.
Sehwag et al. (RN-18)
Sehwag al. (RN-18)
DI of uniformly sampled points
DI of uniformly sampled points
ÖPGD
OPPGD
d AOG
d Nes
LEE Low Liecw LOLR LGDDIAA LGDLLR LGDLL.
Engstrom et (RN-50)
Engstrom al. (RN-50)
Dt of uniformly sampled points
DI of uniformly sampled points
6
PGB
APGD
ACG
Nes
Los Low Lscww LOUR LE-MIRA LG-BERS LG-BER
Salman ul. (RN-50)
Salman et al. (RN-50)
DI of uniformly sampled points
Di of uniformly sampled points
6
6
OPGB
APGG
ACG
Ó Nes
LEE Lew Lscw LDLR LG-DLR4 LG-BRR LO-DERS
Figure 15: Violin plot DI obtained by attacking the models proposed by (Schwag ct al. 2022 An-
driushchenko & Flammnarion, 2020) for CIFAR-10 and (Engstrom el al. 2019 Salman et al. 2020 for
ImageNet.
close. Therefore, the large and small relationships between the predicted probability of classification classes are expected to be easier to variate than in the case of DLR loss.
Additional results of the analysis in section 3.1
C.1 Diversity index for the set of best search points
The main paper only includes the results for the model proposed by (Shway et al. 2022 This section describes the violin plot of DI for several models (Andriuscchenko & Flammarion 2020 Sehwag et al. 2022;
Engstrom et al. 2019 Salman et al. 2020). Figure 15 shows the violin plot of DI obtained by attacking the robust models. According to Figure 15 the best point sets obtained by attacks with single search direction and objective function have similar DI value trends.
C.2 Visualization of the best search points via UMAP
This section describes the 2D visualization of the best search points using UMAP for several models (An- driushchenko & Flammarion 2020 Sehwag et al. 2022 Engstrom et al. 2019 Salman et al.) 2020 Figure 16 shows the 2D visualization of the best search points obtained by attacking the robust models trained CIFAR-10. Figure 17 shows the 2D visualization of the best scarch points obtained by attacking the robust models trained on IetgeNet. According to Figures 16 and 17 the best point sets obtained by altacks with different search directions and objective functions tend to form different clusters. The points determined belong the same cluster due to clustering using the X-mcans (Pelleg & Moore (000) are also plotted close together in the visualization using UMAP. These results suggest that 2D visualizations using UMAP expected to reflect the actual distribution of search points.
27

---

Published in Transactions on Machine Learning Research (May/2024)
Andriushchenko et al. (PARN-18)
6 APGD
LCE
Le DLR,5
ACG
x
Low
LO.DER.
ÓPGD
LouR
lsow
LO.DLRA
Nes
Sehwag et al. (RN-18)
LCB
LaLDLR,
OPGGD
X
Lew Lo-DER.
X
ACG
LDLR
Leew
ÓPGD
Lamara
6 New
Figure 16: 2D visualization of the best search points obtained by attacking the models proposed by (Sehwag et al. 2022: Andriushchenko & Flammarion 2020). The dataset is CIFAR-10. The same color in the left/center figure represents points obtained using the same objective function/scane direction, respectively. The same color in the right figure shows the points determined by X-means to belong to the same cluster.
C.3 The reason why we used UMAP
The objective of the qualitative evaluation using UMAP is to know how the best points obtained by attacks using different objective functions/search directions are distributed and form different clusters. In order to achieve this goal, it nccessary consider the distance between any two points and the distance between clusters. We have tried quantitative evaluation. However, we finally chose qualitative evaluation using UMAP because quantitative evaluations based indicators such objective values DI difficult to achieve our objective for the reasons described below. First, in adversarial attacks, distant points may show the same objective value or close points may show very different objective values because the adversarial attack is a maximization problem with many local optimums. Therefore, quantitative evaluation using objective values is considered difficult. In addition, DI cannot consider the distance between clusters because DI shows low values when a point set forms one or more clusters. Another possible evaluation method clustering, such as k-means, but this qualitative evaluation as with UMAP. UMAP is a dimensionality reduction method that preserves the distance information in the original space much possible so it reflect important information, such the distance between any two points or clusters. Therefore, think that the qualitative evaluation by UMAP provides convincing results.
D Complete results of the experiments
Table shows the complete results of the experiments in Section
4.1 as described Table [ 'abbl
shows the runtime of CAA, AutoAE, A3, and EDA. Table 13 is the complete results of the experiments Sections 4.3 and 4.4 described in Table To investigate the stability of the proposed methods, report the mean and standard deviation from five runs with different random seeds against 41 CNN-based models in Section 4.1 and Appendix G.2 The results in Section 4.1 and Appendix G.2 suggest stable performance of the proposed methods. Owing to the computation cost, the remaining experiments were conducted with a single fixed random seed. Table 14 shows the quantified degree of diversification of A3, the MDO framework with ADS, the MDO framework with R-ADS, the MDO framework with uniform sampling, and EDA.A Table 15 provides the complete results for the transfcrability evaluation of EDA described in Table 3. Figure 18 shows the difference between MT cos and the MDO framework in #queries to find adversarial examples for some models. Figure 19 shows the violin plot of DI for several models.
28

---

Published in Transactions on Machine Learning Research (May/2024)
Engstrom et al. (RN-50)
APGD
LCE +
LG.DLR,5
ACG
Lew LeaDLR,
Ô PGG
LDLR
Leow
LG-DLBA
Salman et al. (RN-18)
APGD
LOB
LO-DUR.S
6 ACG
X Lew
LE-DLB.
Ö PC11>
LDLR
Leow
Nes
LO.DLRA
Figure 17: 2D visualization of the best search points obtained by attacking the models proposed by (Engstrom ct al. 2019 Salman et al., 2020). The dataset is ImageNet. The same color in the lcft, center figure represents points obtained using the same objective function sunctio/seeacc direction, respectively. The same color in the right figure shows the points determined by X-means to belong to the same cluster.
D.1 Hyperparameter determination
The values of na. NADS, and N₂ are determined based on the following preliminary experiments, as they should be as small as possible regarding computational cost. The initial step size is determined based on the step size rules APGD, a powerful heuristic. The step size of 1) = 2E allows the initial search to move from one end of the feasible region to the other, thus allowing broader search. The parameters 0.22 and 0.41, which determine the allocation of the number of iterations for the diversification phase and the intensification phase, are inspired by the checkpoints in the APGD's step size update. N₁ is the number of iterations to be searched with a step size of 2g to & in APGD. The experiments in Yamamura et al. (2022) suggest that the CG diversification performance is well achieved by moving in the CG direction according to this itcration allocation and step size assignment. Therefore, we chose these values for step sizes, Ni, and N₂.
Preliminary experiments to determine hyperparameters of ADS We conducted preliminary exper- iments on the following five models determine the hyperparameters of ADS. ResNet-18 (Sehwag all, (Rice et al. 2020 4. WideResNet- 34-10 Sitawarin el al. 2021 5. ResNet-50 Engstrom el al. 2019 These numbers correspond to the "No."
2022 2. WiideResNet-28-11 (Gowal et al. 2020), 3. PreActResNet-118 18
column in Table 16
D.2 Comparison in computation cost based on the number of queries
We compare the number of queries because the bottleneck in adversarial attack is (queries). In attack selection, CAA requires KNtx# samples 60tx #samples 60 #samples, where K = 20 is the population sizc, N = 3 is the policy length, t 2 1 is the number of itcrations for the candidate allacks, and #samples = 4,000 for CIFAR-10 and 1000 for ImageNet. In summary, attack selection in CAA requires more than 240,000 queries for CIFAR-IL 10 and 60,000 for ImageNet. For attack selection by ADS, AAAs #samples 112x #samples queries are required, where |A] 28 is the number of candidates, NADS is the number of iterations for candidates, and #samples=100 for CIFAR-10/100 and 50 for ImageNet. ADS thus requires 11,200 queries for CIFAR-10 and 5,600 queries for ImageNet.
29

---

Published in Transactions on Machine Learning Research (May/2024)
Table 11: Average robust accuracy over five runs, The lowest accuracies are in bold. RN: ResNet, WRN:
WideResNet, PARN: ProActResNot, A: A-EDDA The TEDA/A3r column is the same the "ratio" column
in Table
CIFAR-10 (s = 8/255)
model
clean
AA
ACG
MT.
CAA
AutoAE
A3
EDA
Andriushchenko & Flammarion
2020
PARN-18
79.84
43.93
45.
43.96
43.93
43.93
43 43.96:10.00
13.85:20.02
Addepall et al 2022
RN-18
85.71
52.48
52.87
62.48
52. 48
52.46
52. 02.40:0.002
52.43:0.003
Senwag et al. 2022
RN-18
84.59
55.54
56.19
55.54
55.53
55.51
00.03:1.01
55.49=0.01
Engstrom el a1 2019
RN-50
87.03
49.25
50.88
49.21
49.24
49.21
49 09.25=00002
49.10=0.003
Carmon el. Al. 2019
WRN- 28 10
89.69
59.53
60.10
59.54
59.50
59.46
09.44:+0.01
59.40=0001
Gowal el. al.
2020
WRN 28 10
89.48
62.80
63.18
62.86
62.78
62.80
62.77+00.01
62.75+00022
H Tendrycles <t m 20
WRN-28 SENN-28-11 10
87.11
54.92
55.71
54.90
54.88
54.81
54.85+00.01
54.77+0.02
Rebuth et ml.
202T
WRN-28-10
87.33
60.75
61.23
60.80
60.74
60.71
60.72:+0.01
60.64+0.01
Sehwaag al.
2020
WRN-28-10
88.98
57.14
57.68
57.18
57.16
57.10
57.14:10.02
57. 57.03-10.01
Sriclaar
2022
WRN-28-10
89.46
59.66
60.22
59.67
50.60
59.60
59 59.
59.46:10.02
Wang et al.
20720
WRN 28 10
87.50
56.29
57.45
56.38
56.29
56.23
56.28+00.01
56.15+00.02
Wi rt al.
2020
WRN-28-11 10
88.25
60.04
60.35
60.06
60.00
59.97
60.02+00.01
59.94+0.01
Ding ct ml.
2020
WRN-28-4
84.36
41.44
44.36
41.99
41.67
41.48
41.24:10006
0..74
Addepalli ot ma.
2022
WRN-34-10
88.71
57.81
58,43
57.77
57.77
57.74
57 07.
57. 57.69110.02
Sclwag al.
1122
WRN-34-10
86.68
60.27
61.00
60.31
60.26
60.26
60 60.22:10.01
60. 60.18:1.00
Sitawarmn
202
WRN-34-10
86.84
50.72
51.94
50.75
50.70
50.64
50 02200
50.5910.01
Zhang et 8..
2019
WRN-34-10
87.20
44.83
45.76
44.69
44.65
44.57
44 03
44.51=0.02
Zhang et al.
2020
WRN-34-10
84.52
53.51
541.00
53.52
53.48
53.48
03.40:00001
53.
Srichar
2122
WRN-34-15
86.53
60.41
60.93
60.43
60.39
60.35
60.38++0001
60.32+0.01
Gowal et al.
2020
W4-N-4-20
85.64
56.86
57.15
56.83
56.82
56.85
56.81+0.01
56.79=0.03
Pang et al.
2020
WRN-34
85.14
53.74
54.66
53.71
53.72
53.67
53.69++0.01
53.66+0.01
Rice et sl. 2020
WRN-34-20
85.34
53.42
5-1.33
53.39
53.36
$3.35
63.38-10.01
53.3110.01
Gowal et al.
2020
WRN-70-16
85.29
57.20
57.60
57.15
57.16
57.19
67.11=0.001
67.12=0.01
Gowal et al.
2020
WRN-70-16
91.10
65.88
66.29
65.96
65.87
65.80
65 00.05++0001
65.83=0.01
Rebulh
2021
WRN-70-16
88.54
64.25
64.80
64.28
64.23
64.19
64.24+0001
64.20=0.03
CIFAR.100 (< 8/255)
model
clean
A
ACG
MT
CAA
AutoAF
A
EDD
Rice et ml. 2020
PARN-18
53.83
18.95
19.48
18.99
18.97
18.91
18 80.10.00
18. 18.88.10.01
Hendrycks et a. 19
WRN-25 28- 10
59.23
28.42
29.51
28.43
28.44
28.35
28. 28.32++0.02
28.27
Rebulhi al. 222
WRN-28-10
62.41
32.06
32.73
32.07
32.05
32.02
32.00++0022
31.94+0.03
Addiepalli et ml
2122
WRN-34-10
68.75
31.85
32.38
31.80
31.80
31.83
81.80+0.02
31.78+0.01
Cui ct al.
2121
WRN-34-10
60.64
29.33
29.65
28.99
28.99
28.86
28 0.000.0
28. 83.0022
Sitawarin ct pl 20
WRN-34-10
62.82
24.57
25.69
24.55
24.57
24.52
24 24.56.10.03
24.0010.01
Wi. ct al.
2020
WRN-34-10
60.38
28.86
29.90
28.86
28.82
28.79
28. 28.70-10.02
28.76:10.01
Cui ct al
2021
WRN-34-20
62.55
30.20
30.83
30.03
29.99
29.86
29 84.10.0111
29. 20.8510.001
Gowal et a.
2020
WRN-70-16
60.86
30.03
30.54
30,00
30.00
29.99
29.97
29.9 00.06
Gowal et a. 2020
WRN-70-16
69.15
36.88
37.84
36.95
36.90
36.86
36 0.22
36.81
Rebuth et al.
2021
WRN-70-16
63.56
34.64
35.30
34.68
34.63
34.62
34 64.02-00001
34.55+0.01
ImageNet (5
4/255)
model
clean
AA
ACG
MTT
CAA
AutoAE
A3
EDA
Salman al.
2020
RN-18
52.92
25.32
26.40
25.24
25.28
0OM
25.22+:0.03
25.11=0.00
Engstrom al.
2019
RN-50
62.56
29.22
31.54
29.34
29.41
00M
29.32++0.05
29.01
Salman
2020
RN-50
64.02
34.96
36.26
34.68
34.77
00M
34.75+00.04
34.52+0.02
Wong el. al.
2020
RN-50
55.62
26.24
26.46
26.57
00M
26.42++0.04
26.12+0.10
Salman
2020
WRN-50-2
68.46
38.14
40.24
38.22
38.23
00M
38.26+00.02
38.03+0.02
For the entire attack procedure, the standard AA requires queries of 6100x #images. CAA requires at least #0a queries. AutoAE runs 32 iterations of APGD with CE loss, 63 iterations of APGD with DLR loss, 160 iterations of FAB, and 378 iterations of MultiTargeted attack with nine target classes. According to the official implementation of AutoAE, the MultiTargeted attack runs 378 iterations for cach target class. Therefore, AutoAE requires (32 63 4-08 x 9) x #images = 3, 657 x #images queries for the adversarial attack. EDA requires n, x (N1 + N2 + N2)x # images= 5 x 100x #images queries for the MDO framework, and K N, + N4x images= 190-300x #images queries for the targeted attack a'. Therefore, EDA requires 692.24 to 802.24x # images queries in total. We compared the runtime of EDA with that of A because A automatically terminates its search before the query limit.
30

---

Published in Transactions on Machine Learning Research MMy/2222)
Table 12: The computation time in seconds and ratio of runtime to EDA.
CIFAR-10
model
ciran
EDA
A
CAA
AutoA E
8/256)
ratio
ratio
sec
ratio
sec
radio
Andriushchenkoo & Flammation
2020
PARN-18
79.51
513=32
1.0
38242
0.74
999:2
1.95
7,782
15.17
Adciepalli es a..
2022
RN-11
85.71
625=25
1.0
13112
0.69
1,107=3
1.77
8,503
13.60
Selnwwy
RN-18
1,1211668
1.90
1,104.2
1.87
13.89
Engetron el
2019
RN-50
87.03
1854778
1,572=11
1.06
3,672=5
2.17
28,263
19.03
Carmon s6 m.
2019
WRN-28-11
89.69
3,3161165
1.0
4,22114
1:27
5,957
2.70
64,382
19.42
Gowal
2120
WRN-28-10
89.48
4,357=112
1.0
3,811=13
0.84
81,358
2.19
80,517
17.67
Heudeycks
201
WRN-228-10
2.719
0.87
8,45
19.91
Rebith of m1
2721
WRN-288-0
87.33
4,439+774
1.0
3,928+30
0.88
11, 11,22=111
2.52
79,203
17.78
Selwwag et aL
2020
WRN-28-10
88.98
3,255
1.0
2,082=500
0.82
8,761=7
2.69
61,273
19.75
Sridltar et al.
2022
WRN-28-10
89.46
3,356:355
1.0
3,2151119
0.97
8,708=5
2.61
61,021
19.08
Wang
WRN--28-00
87.50
3,285+78
2.732+4
0.83
8,088=7
2.64
63,062
19.38
Wii et ai.
1120
WRN-28-10
88.25
3,5024:39
1.0
3,273
0.93
8,920=8
2.55
61,568
18.41
Dinng ot
2020
WRN-228-4
84.36
095-29
1.0
2,017+900
2.90
1,50 -
2.20
12,828
18.46
Addepsill e5 Au
2022
WRN--34-10
88.71
4,225=+109
3.926+7
0.93
18,18
2.63
80,337
19.01
Seliway
2122
WRN--34100
86.68
4,172+-18
3,858-66
0.92
11 11,067-14
2.65
79,526
19.06
Sitawarih ot
A12
WRN-344-10
86.84
3,501+100
1.0
3,845+22
1.07
10,675++5
2.97
79,519
22. 14
Zhang gl
2019
WRN-311-10
87.20
3,219#32
1.0
3,500=12
1.09
10, 80,39
3.23
79,061
21.56
Zhang
2020
WRN--34-10
84.52
3,936-+
3,912+16
0.99
10,593++7
2.00
76,724
19.49
Sridhar
WRN--44-15
86.53
7,785+ 116
6,805+14
0.87
20, 170=t9
2.59
14,9077
19.15
Gowal
2020
WRN--34-20
85.64
14,693
1.0
13,463
0.92
40,072
2.73
301,256
20.50
Pang 15 mi.
2012
WRN-344-00
85.14
18,775-4-241
1.0
12,4364-21
0.66
31 31,712-25
1.60
242,403
12.92
Rice al 1121
WRN--34-20
85.34
11.255++3777
1.0
12,290++5
1.09
32, 478
2.80
244,587
21.73
Gownl
2020
WRN--70-16
85.29
21,790:1-430
20,587.111005
1.22
60,431 11
2.77
451,566
20.72
Gownl
2020
WRN-77016
91.10
24.8854-377
1.0
20,544+252
1.19
63,761 154
2.56
468,883
18.84
Rebbllli cl ml.
20221
WRN-70016
88.54
24,632.1479
1.0
29,075.1887
1.18
63,433
2.57
459,409
18.64
CIFAR-100
model
clean
EDA
A²
CAA
AutoAE
8/255)
ratio
8e0
ratio
ratio
Rice al.
2020
PARN-I8
53.83
497=74
1,531+9924
3.08
1,062+
2.14
14,409
28.99
Heudryclos et
2019
WRN 22---28-0 10
59.23
1,981
1.0
2,684110
1.35
5,508
2.78
75,410
38.07
Rebbiii et al.
WRN-28-10
62.41
2,701 4-86
1.0
3.04448
1.13
7,4944-60
2.77
98,909
36.62
Adciepall
WRN-34-110
2,7921195
3,046117
8,045.1:83
109,600
Cis
WRIN-34-10
60.64
3,075
3.002.17
0.98
7,121 1.76
2.32
95,323
31.00
S Sitawarin ct
2121
WRN-34-10
62.82
1,985
1.0
4,985+ 1/77
2.49
7,081 1-81-4-86
3.57
99,096
49.92
Wi et al.
2020
WRN-34-10
60.38
2,360131
1.0
3,2581558
1.38
7,073.166
3.00
96,679
40.97
WRN-34-22
8,027.1977
9.708
22,894
313,305
39.06
Gowal
2170
WRN-700116
60.86
13,00011:14
21,4227011
1.64
12,3255:3393
3.24
563,7747
43.17
Gownl ct ml
2120
WRN- 70-16
69.15
15,64111467
1.0
22, 122.121119699
1.43
49 49,0299--476
3.13
6663,166
40.48
Rebul et sl.
2172
WRN-70-16
63.56
10,17111001
1.0
21,516=190
1.39
15,071+-128
2.91
575,450
$7.19
ImageNet
mode]
clean
A
CAA
AusoAE
4/255)
ratlo
ВеC
ratio
ratio
Salmman et al.
2020
RN-18 18
52.92
1,667.1119
1.0
2,937.110
1.76
1,511113
0.91
OOM
OOM
Engstroo ct m
2019
RN-50
62.56
3. 8,150--136 136
1.0
9,380 188
2.97
4,776.1-16
1.51
OOM
OOM
Salmmnn et al.
RN-50
64.02
3,325:1302
1.0
9,9891224
2.83
4,997.227
1.42
0OM
OOM
Woog ml
RN.50
8,4721194
7,320.131
1.64
OOM
Sellinn et al.
2020
WRN-50-2
68.46
5,102#119
1.0
9,8861120
1.94
9,267:79
1.82
0OM
OOM
D.3 Mathematical definition of the best point sets of A and EDA
Mathematically, the best point sets of 3 and EDA are defined as follows. First, the best point set of A3
is defined as X 3 where aa= = (XADI. Teos δGD, Lcw) and XADI Adaptive Direction Initialization (ADI) proposed by Liu et al. (202cc). Subsequently, the best point set of EDA is defined as (Uxree X(' x. 0% a......,,]]. and a' (xPAS MAPGD: 8GD T.(w)-
where e" = (x, 8,, I.,)
D.4 Analysis of EDA using an index based on Euclid distance
DI takes small values when the point set forms a cluster, even if the Euclidean distance between any two points is large. Therefore, quantification by DI and quantification based on the Euclidean distance between points in the point set may have different characteristics. Therefore, in this section, to compare the diversification performance from a different perspective than DI, we consider quantifying the degree of diversification of the best point set based on the average value of the Euclidean distance between the centroid of the point set X and all points in the point set X. Mathematically, the average Fuclidean distance between all points in
31

---

Published in Transactions on Machine Learning Research (May/202)
Table 13: Comparisons with MDO fram)) MDO framewark(R-ADS), and MDO frame-
(RAND) robust accuracy to validate ADS. The abbreviations are the same as those used in the main lext. The lowest. robust. accuracies in bold.
MDO framework
Ensemble
Composite
CIFAR-10 (e 8/255)
model
RAND
R-ADS
ADS
ADS
ADS
Audriushchenno & Flammarion
2020
PARN-18
44.09
44.02
43.95
44.12
43.95
Addepalli 2022
RN-18
52.45
52.83
52.50
52.63
52.65
[Sehwag et al.
2022
RN-18
55.61
55.58
55.58
55.68
55.90
Engstrom
2019
RN-50
49.60
49.40
49.52
49.79
50.04
Carmon et al.
2019
WRN-28-10
59.56
59.53
59.46
59.70
59.59
Goowal
2020
WRN-28-10
62.82
62.99
62.83
62.92
62.86
Hendrycks et al
2019
WRN-28-10
51.94
55.02
54.87
55.09
55.14
Rebulli al.
2021
WRN-28-10
60.67
60.84
60.77
60.82
60.82
Sehwag et al.
2020
WRN-28-10
57.16
57.38
57.14
57.43
57.34
Srichar
2022
WRN-28-10
59.63
59.76
59.59
59.89
59.85
Wang el al.
2020
WRN-28-10
56.32
56.50
56.33
56.66
56.87
Wu et al.
2020
WRN-28-10 10
60.02
60.13
59.99
60.14
60.13
Ding ot al.
2020
WRN-28-4
43.54
44.88
43.24
43.49
44.11
Addepalli a..
2022
WRN-34-10
57.75
57.87
57.72
57.92
57.90
Schwag al.
WRN-34-10
60.26
60.38
60.21
60.52
60.44
Sitawarin
202
WRN-34-10
50.83
50.78
50.70
50.98
51.14
Zhang al.
2019
WRN-34-10
44.78
44.70
44.62
44.87
44.79
Zhang et al.
2020
WRN-31-10
53.51
53.50
53.55
53.60
53.52
Sridhar al.
2022
WRN-34-15
60.40
60.46
60.39
60.46
60.48
Gowal et al.
2020
WRN-31-20
56.86
56.85
56.88
56.90
56.84
Pang et
2020
WRN-344 34-20
53.77
53.85
53.81
54.00
53.96
Rice et al.
2020
WRN-34-20
53.47
53.48
53.42
53.52
53.50
Gowal
WRN--0-I
37.23
57.21
57.18
57.27
57.22
Gowal et al. 2020
WRN-70-16
65.86
66.02
65.85
66.04
65.99
Rebuth al.
2021
WRN-70-16
64.23
64.54
64.32
64.53
64.43
CIFAR-100
8/255)
Rice al.
2020
PARN-18
18.98
18.99
18.97
19.08
19.12
Hendrycks cl al.
2019
WRN-28-10
28.56
28.83
28.44
28.61
28.45
Rebulli al.) 2021
WRN-28-10
32.08
32.13
32.08
32.22
32. 32.18
Addepall a.. 2022
WRN-34-10
31.91
32.23
31.86
31.91
31.97
CCui et al. 2021
WRN-34-10
28.97
29.20
28.99
29.20
29.13
Sitawarin al..
2021
WRN-34-10
24.71
24.68
24.65
24.74
24.83
Wi et al. 2020
WRN-31-10
28.93
29.46
28.88
28.97
28.97
Cui et al.
2021
WRN-31-20
30.07
30.35
30.01
30.15
30.29
Gowal et a..
2020
WRN-70-16
30.06
30.12
30.05
30.11
30.18
Gowal al.
WRN-70-16
37.05
37.53
36.96
37.19
37.17
Rebulli et al.
2021
WRN-70-16
34.61
34.97
34.65
34.88
34.65
ImageNet (s 4/255)
(Salman et al.
2020
RN-18
25.22
25.14
25.22
25.16
25.48
Engstrom
2019
RN-50
29.56
29.26
29.20
29.61
29.68
Salman et al.
2020
RN-50
34.68
34.68
34.84
35.00
34.92
Wong al.
2120
RN-50
26.26
26.36
26.22
26.84
26.78
Salman et al.
2020
WRN-50-2
38.38
38.54
38.28
38.52
38.58
point set X is defined
E(X)
Σ
|
Z112,
(22)
X
FEX
where z is the centroid of the point set X, defined
[X Exex x. As shown in Figure
20 the value of
32

---

Published in Transactions on Machine Learning Research May/2024)
Table 14: The quantified degree of diversification. DI denotes the Diversity Index, and E denotes the metric defined by equation 22
RAND, R-ADS, and ADS represent MDO framowork(RAND)) MDO framework(R.
ADS), and MDO framework(ADS), respectively.
CIFAR-10
Medels
Aa
RAND
R-ADS
ALXS
EDA
Ie 8/285)
DI
DI
E
DJ
E
DI
DI
E
Amstriedsllamik F Flammuriion
2020
FARN-18
0.20:-0.09
0.66+:0.27
0.90+00.6
0.55-0200
0.26+0005
0.73++0.14
0.33-6.07
0
0.36+00.0
1.11=0.15
RN-18
0:22-0.10
0.60:0038
0.51:0.07
0.94:0.20
0:23:0.05
0.98+-011
0.56:0.05
10240.19
0.40:0.06
1
SS 26
RS-18
0:27-0.09
0.7240.38
0.31+0.05
0.33-.20
0.26+005
0.29+0.15
0.30-000
0.99+0119
0.38+000
1.15-0.15
RN-50
0.28-0.09
0.72+0.27
0. 0.33+11.06
0.20=0019
0,265000
0.76+:0.15
0.37-6077
1.04#0119
0. 0.30+3.0
1.17=0.16
WRN 18
0.08
0.602:0:25
0.30
LIM26118
0.260000
0.8120.14
0.38=6007
100690119
0.112:00
1.222:0.7
WRN-22-11
0:22-0.10
0.68+0.30
0.90-0.07
0.91-0211
0.20+005
0.71+0..11
1.440.22
041+0006
1.20-0.15
WRN-25-10
0.25
0.6490227
09:--020
02:2000
0.71
0.34-607
1001+0119
0.57=000
1.14-0.15
Keball 21 11
WRRS-25-10 10
0.110
066220030
1. 1.
0.8922221
02220006
0.752:1.12
0.9720118
0.572:100
1
WIN 28 10
0.05
0.641:026
0. 113.0
0.9920.19
0.201:0044
0.75110.11
1.38_.007
10010000
0.4110005
1.21
WRN-25-10
0.25=0000
066440.26
100
0.24
0,76
0.37-0.07
100
041+0006
0.2
WRN-25-10
0.28
0.683024
09
20:000
0,7020022
0
1
WiL <E m.
WIN 28 10
0,20
0.64110:27
0.321.1.06
0.98
20.0000
0,761...0
0.98_
100810199
0.4710006
1.25
WINN 28
0.10
0.9110.34
0. 0.5
0.0810.22
20 20.00000
0,9110.25
0.38_ 0.08
10210:22
0. 0.38.10.07
1.190.16
WRN-3L-10
0.27
0.6710.28
0.
0.37=00.5
1.21
0,29
0.65:10:26
0.32.00.55
0.9616
241004
0,7410,122
0.97
10220-188
0,381005
1.1510114
Lataario
WINN- 54 10
0.27 0.10
0.7210030
0. 0.52...06
0.0910.20
0610000
0.741..17
0.98_ 0.06
0.381000
1
WRN--SI-10
0.27-0.11
0.73+00311
0. 0.50
0.96:0.21
0 0:25:006
0.
0.70++0.16
0.52-0.07
100+0:21
0. 0.57:+0.06
1.18:0.17
WIN 34
0.25
0.6210.26
0.3110005
0
0,2610006
0.8210.17
0.03
0.9910020
0.38.10.06
1.21
Stylbar
WINN-
0.25
0,5710024
0.51110.77
0.9610.11
0.241006
0,7810,111
0.97-00,5
10610.19
0,411006
1.2110.16
WRN--34-20
0:21-0.11
0.55+0032
0.20=0007
0.05+6022
0.210006
0,73++0.15
0.34-0.07
10220221
0.58+0006
1.19-0.18
Tall et sl
1123
WIN 54 20
0.20_0.00
0.66100.0
0. 0.222.1.8
0.801..25
O. 10110.07
0.64110225
0.25
0.8710021
0.3111008
1.0R10.19
WRN-34 31-20
0.24
0.6410332
0 0.261.0.06
0.01::022
0.2210066
0.7110.17
0.33-0007
0.9810.21
0.3710066
1.14
WRN--90-16
0:22-0.10
0.56+-0.30
0.50+0.06
0
0
0.71+
0.38-0.05
1.400.30
0.30+0006
1.18-0.17
WRN-70-16
6:10-6.10
0.62+0332
0.26+0.06
0.91-0.22
0.22+005
0.36+00.15
0.31-6.77
1001+021
0. 0.36+000
1.10-0.16
Tobrihh ct all
4021
WRN-- 16-1-
0.23
0.59:00.2
0. 0.30:0..66
0.88.0020
0.2310006
0.7410.12
0 0.31
0.9610.11
0.57.10.06
1.00
CIFAR-100
Medels
RAND
R-ADS
ADS
EDA
[e 8/285)
DI
E
E
DJ
DI
E
E
axaa
PARN-18
0.34-0.15
0.63+0.31
0.52+0.06
0.97=00.0
0.25+0006
0.76+00.1
0.35-0.07
1.02+0211
0.30+0.06
1.15e0.20
WRN-25-10
021-
0.71:6030
0.20=11.7
0.:1:0222
0.226006
0.7641.36
0.31-6.88
0.9750211
0.36+006
1.18-0.18
WRN-22-10 to
0:32-0.15
0.81+0036
0.20+0007
0.94+6.25
0.25+0006
0.52+0.17
0.35-0.08
1.02+0.22
0.11+0006
1.20-0.20
WRN-SI-10
0.34-0.15
0.64+0.30
0.31+0007
0.87-6.21
0.2440066
0.56+0.13
0.36-0.00
1.0440211
0.2200066
1.21=0.18
WRN-81-10
0:27:0.11
0.6750.27
0.27:11.8
0.89200.22
0.20:0000
0.8220134
0.00=0.00
0.9720019
0.54=:007
10
WRN--SL-10
0.11-0.12
0.79++030
0.52-0.05
0.98-0200
02240005
0.75+0.17
0.35-6.07
102+003
0.36+0.06
1.14-0.00
Wi ml 28
WRN-81-10
0.28-0.12
0,73+0.32
0. 0.25+11.07
0.96=00.22
0.26#000
0. 0.70+11.15
0.32-0.88
0.99+0221
0.57+3.06
1.16=0.18
3721
WRN-31-20
022:=0.9
0.662:0.77
1. 0.25
0.12:40211
0.22:0066
0.75
0.32-0.0
0.9920221
0.55
+1120
WRK--
0.27-0.13
0.09100:44
0.28111377
0.0220222
0.1910000
0.0610133
0.38
0.9810.20
0.3743006
1.1920117
WRN-70-11
0.11-.14
0.7944033
0.51=0.07
091-0222
0.23-006
0.73+0.14
0.33-005
1001+021
0,40+0006
1.21-0.17
Metmila <t mb
3:21
WRN-77016
0.10-0.14
0.7646036
0. 0.30
0.5220022
0.2100066
0.742:0334
0.9020220
0.00:000
1.19:0011
InnegeNes
Modela
AP
RAND
R-ADS
ADS
EDA
[6 4/205)
DI
E
DI
E
D1
DI
E
DI
E
Saiman
2020
RN
0.26_00.7
2.6011011
0.34.0.07
3.5210.69
0,281005
2.3.10.42
0.08_0.08
3.66.10.00
0.43.0001
4.37100.6
0.2000.06
2.795083
8.920.588
0.110006
3.1840111
0.0000.06
A883061
0.4480144
4.3020.64
St Sin 05
RN 50
0.28..006
2.111090
0.35.1006
3.6120.65
0.3020066
3.00109477
0.37-0007
3.701000
0.221004
4.4320.53
Writg et
12
RN 50
0.27_0007
3.45.11.12
0.361.0.66
4.5910.82
0.3010066
5.76.0..58
0.07
4.811080
0.43.1.05
5.4310.99
WTNN-50-2
0.27-0.06
26630082
0.57100.0
3.60:0000
0.31:006
5.5310,58
0.38:0005
3.601061
0.43:0001
4.2210.65
equation tends to be larger for EDA than for A in most models where EDA has higher attack performance than A3. This difference is more pronounced than the difference in DT. While EDA shows similar trend for
all models, A shows a different trend in the value of equation 22 for some models. For example, as shown in Figure 20] the value of equation 22 for A tends to be larger for the model proposed by Ding et al. (2020) than for the other models. Given the high attack performance of A against these models, this suggests that the A3 diversification strategy may be more effective for these models.
D.5 Analysis of EDA for the model proposed by Ding et al.
The attack performance of EDA is significantly lower for the model proposed by Ding et al. (2020) compared to A This section discusses the reasons for this regarding diversification performance and computation time. As described in the main text, the value of D1 for the best point set tends to be higher for EDA and lower for A3, similar to the results for other models. On the other hand, the analysis in the previous section shows that for the model proposed by Ding et al.) (2020), the value of equation 22 for the best point set obtained by A tends to take larger values than the results for the other models. In addition, a comparison of the computation time for EDA and 3 shows that A takes more than three times longer than EDA. The above comparison suggests that the A can perform better diversification for the model than for other models. In summary, setting a longer computation time and increasing the number of multi-restart are
33

---

Published in Transactions on Machine Learning Research (May/202)
Table 15: Comparison in robust accuracy for transfer setting. "source-ttarge"" indicates that the transfer attack from the source to the target model. AutoAE exccuted only with CIFARI0 because of its long execution time.
CIFAR-10
Source Marget
clean acc (target modell)
robust acc
CAA
AutoAE
A EDA
(in Robust
Carmon
2019
Ding
84.36
41.44
71.82
73.15
73.24
70.46
Carmon et al.
2019
Cull et a..
2021
88.22
52.86
71.90
74.76
74.66
73.46
70.29
Ding et al.
2020
Carmon
2113
89.69
59.53
83.23
85.01
84.85
85.16
83.08
Ding ct R.
2020
Cui ct al.
121
88. 22
52.86
79.77
82.16
81.83
82.42
80.32
1il et al.
Carmon el
2019
89.69
59.53
76.17
80.24
80.32
79.94
77.24
Cuu ct al
2121
Ding et al.
2020
84.36
41.44
70.16
72.07
72.09
71.28
69.43
CIFAR-100
Rice 2020
Cui al.
2021
62.55
30.20
56.42
58.72
58.36
58.
Rioe al.
2720
Reballi
2021
63.56
34.64
58.28
60.45
59.74
55
Cul ot
Rioe ct al.
2020
53.83
18.95
44.51
45.87
44.50
42.86
Cuil eel al.
2021
Rebuth ai.
21721
63.56
34.64
53.58
56.24
55.35
54.02
Rebullh et al.
202
Rice et al.
2020
53.83
18.95
44.44
45.64
43.51
40.84
Rebulii et a.
2021
Cui et sl.
2021
62.55
30.20
50.15
51.98
51.26
48.45
ImageNet
Salman et al.
2020
Engstrom al
2019
62.52
29.22
53.74
55.88
55.20
52.76
Solman ct
2020
Wong
2020
55.64
26.24
47.22
47.90
46.40
45.12
Engstrom
2019
Salman
2020
68.46
38.14
63.62
65.34
64.68
64.46
Eagstirom
Wong et
2020
55.64
26.24
46.34
46.06
44.92
Wong et
2020
Salman et al.
2020
68.46
38.14
65.36
66.08
65.56
Wong et. al.
2020
Engstrom et al.
20019
62.52
29.22
56.40
58.02
57.80
56.72
Table 16: Results of the preliminary experiments determine the hyperparameters of ADS. The robust accuracy obtained by the MDO framework is described. The default parameters are in bold.
NADS
Datasel
No.
5
10
3
6
CIFAR-10
55.56
55.58
55.65
55.67
55.63
55.66
55.58
55.57
CIFAR-10
56.83
56.80
56.84
56.81
56.90
56.83
56.80
56.78
1000
19.01
18.87
19.03
19.01
19.16
19.10
18.87
18.90
CIFAR-10M 100
24.66
24.56
24.61
24.59
24.61
24.78
24.56
24.56
ImageNet
29.40
29.24
29.26
29.30
29.28
29.34
29.24
29.14
considered particularly effective in improving the attack performance for the model proposed by Ding et al.
22020).
D.6 Trends of search directions and objective functions selected by ADS
Figure bar chart displaying the ratio of times each search direction and objective function used by EDA. Figure 21 shows that ÒACG and LG-DLR, frequently used in the diversification phase, and ō is rarely used. In the intensification phase, Lcw, Lscw, and ôxes were more likely to be selected. This trend may reflect ACG's high diversification performance and NAG's high intensification performance. The potential reasons for these trends are: 1. P. and DI play different roles from each other, 2. the ACG's search direction may be similar to the steepest for small step sizes, and 3. the difference between Nesterov's acceleration gradient direction and gradient direction.
The role of Pe and DI. The pe measures the degroe of diversification the output space during the search. pair with the largest P is expected show a high diversity in the output space. In addition, from Yamamura et al. (2022), it be assumed that the ACG direction increases Pe while the steepest-like
34

---

Published in Transactions on Machine Learning Research (May/2024)
Andriushchenko et al. (PARN-18)
Ding et al. (WRN-28-4)
10a
MTcos
MDO
10
MT MDO
MTcos cos < MDO
#6 #imagess
MTcas MDO
600
400
200
200
400
100
200
200
400
Difference in query (MT
MDO
Difference in # query (MT
MDO
Schwag et al. (RN-18)
et al. (WRN-34-20)
107
MTcos MDO
10a
MT cos
MDO
MTcos < MDO
#images
MTcus
MDO
300
200
100
100
300
2000
-100
100
200
300
400
Difference # query (MToos
MDD)
Difference in # query (MT
MDD
Salman et al. (RN-50)
Engstrom (RN-50)
10a
10
MTeas
MDO MTeas
MTcne
MDO
aimagees
#images
MTcne
MDO
10*
-000
200
100
100
2200
300
3000
200
100
0
100
200
300
400
Difference in query (MT MDO
)
Difference
query (MT cos
MDO)
Figure 18: The difference between MT cos and the MDO framework in #qucrics find adversarial examples for some models. The attacked models are Andriushchenko & Flammarion (2020), /Shwwag al. 2022 , and
Ding et al. 20200) for CIFAR-10, Cui et al. [2021) h for CIFAR-100, and Salmann et al. (2020) and Engstrom et al. (009 for ImageNet.
Schwag et al. (RN-18)
Cui et al. (WRN-34-20)
63.
R-ADS
RAND
ADS Ensemble Composite EDA A MT...
R-ADS RAND ADS Ensemble Composite EDA A MT...
Ding et al. (WRN-28-4)
Salman et al. (RN-50)
R-ADS RAND ADS Ensemble Composite EDA A2 MT...
R-ADS RAND ADS Ensemble Composite EDA A² MT
Figure 19: Violin plot of DI. The attacked models are Schwag et al. 2022 and Ding et al. (2020) for CIFAR- 10, Cui et al. 2021 for CIFAR-100, and Salman et al. (2020) for ImageNet.
direction does not. From the above, considered that the pair with the maximum P likely to include the ACG direction. DI measures the diversity of the best point set obtained by the search. our use casc, DI represents the dissimilarity between the best points. That is, we expect that pairs with the largest DI more likely to enumerate dissimilar solutions. Intuitively, updates in diverse directions contribute to the
35

---

Published in Transactions on Machine Learning Research (May/2024)
Sehwag et al. (RN-18)
Cui et al. (WRN-34-20)
10.
63-
0.5
6//
60-
R-ADS RAND ADS
Ensemble Composite EDA A MT
R-ADS RAND ADS Ensemble Composite EDA A? MT
Ding et al. (WRN-28-4)
Salman et (RN-50)
10-
6.3
6U1
R-ADS RAND ADS Ensemble Composite EDA A3 MT
R-ADS RAND
Ensemble Composite EDA
MT
Figure 20: Violin plot of equation 22 The attacked models are |Sewwag et al. 2022 and Ding et al. (2200)
for CIFAR-10, Cui et al. (0221) for CIFAR-100, and Salman et al. 20200) for ImageNet.
CIFAR-10
CIFAR-I0
LS
GS
GS
0.0
0.2
0.4
0.6
0.8
1.0
0.0
0.2
0.4
LOw
Esew
LGBORR
LG-DLR
6PGD
Ax0G
ÓAPGD
5%m.
Lew
LOER
LO.DLR.
CIFAR-100
CIFAR-100
LS
LS
GS
GS
0.0
0.2
0.1
0.6
0.8
1.0
0.0
0.2
0.4
0.6
0.8
1.0
Log
Lsew
LG.DLR44
LC-DLR.G
8PGD
8ACG
APPID
8%ce
Lew
FDLR
LoaDER
ImageNet
ImageNet
1.5
LS
G5
GS
0.0
0.2
0.4
0.6
0.8
1.0
0.0
0.2
0.4
0.6
0.8
1.0
Lew
Leow Doan
La.tRA
Lo-min
8PGD
8xCG
ÓAPGD
Lew
LanerS
Figure 21: The average ratio of times each search direction and objective function used by EDA. LS and GS refer to the intensification phase and diversification phase, respectively.
enumeration of dissimilar solutions. Given that the search direction is gradient-dependde the pair with the largest. DI is likely to include variety of objective functions and update formulas.
36

---

Published in Transactions on Machine Learning Research (May/2024) (024)
ACG's search direction may be similar to the steepest for small step sizes. The reason for this
is as follows. According to equation 10 a(%) is close to the gradient Vigg(=(")) ), c) when 3(4) is close to O. From equation 9 (VL(g( c), y y is the numerator of ((6), Therefore, as y approaches O g(6) also approaches 0. When the step size is small, ||z(*) - -|(= is also small, so ((=-1) is likely to be close to o. As a result, the ACG's direction and the steepest. direction may be similar. The experiments conducted
by Yamamura et al. (0022) also support this claim.
The difference between Nesterov's acceleration gradient direction and gradient direction. Nes- terov's accelerated gradient (NAG) mothod updates the search point using the gradient of the point moved from the current search point to the momentum direction. Assuming that the objective function is multi- modal, the gradient at the current point is unlikely to be similar to NAG's search direction. Thus, the objective function is multimodal, search in the NAG's direction may find different local solutions from that in the gradient direction.
E Prediction Aware Sampling (PAS)
Motivation The hypothesis behind PAS is that starting the scarch with an initial point near multiple decision boundaries increases the likelihood of finding an adversarial example. When maximizing the inner product of random vector and logit, as ODS does, the distance to the decision boundary may be farther away than the initial point. However, when moving in the direction where the prodicted probability for the correct class is as small as possible, the inltial point is more likely to be closer to the decision boundary than the original point. We hypothesized that the attack's s success rate could be improved by starting the search at point closer to the decision boundary.
Prediction Aware Sampling One promising initial point sampling is ODS, which considers diversification in the output space. However, there is room for improvement because its sampling does not consider image- specific information. Based on the idea that the randomly sampled initial point close to decision boundaries makes the attacks easier to succeed, we propose Prediction-Awaz Sampling (PAS), variant of ODS. PAS maximizes the following function in the same way ODS to sample the initial point.
v(w,9, g, x) = w g(x) x exp (x)() (w ~ (
(23)
PAS samples the initial point by repeating the following updates for NPAS iterations.
V v V Veo(w: [v., 22
x Ps x + 7/AS sign
(24)
Same as ODS, PAS used NPAS = and Mes = E. Intuitively, maximizing equation 28 means moving the initial point closer to the decision boundary by reducing the prediction probability of the correct class e and, at the same time, moving the logit g(æ) closer the random vector w.
Experiments To test our hypothesis, we compared the success rate for each class in targeted attacks with nine target classes, using the input point, the point sampled by ODS, and the point sampled by PAS as initial points. In our notation, we compared the attack performance of (x, Meas; dGD, I2_w), E {xorg: XODS, x with 100 iterations for each target class and initial step size of 2s. The number of target classes K was set to 9. The following five models were used in the experiments. 1. ResNet-18 (Schwag et al., 2022 2. WideRessNt-228 10 (Gowal et al.) 2020), 3. PreActResNet-18 (Rice et al. 2020 4. Nete3- 2021), 5. ResNet-50 (Engstrom et al.) 2019). These numbers correspond to the "No." column in Table 17 The experimental results in Table 17 show that the attack with PAS can achieve higher attack success rates for many target classes than other initial point selections. The experimental results support our hypothesis that PAS brings the starting point closer to the decision boundary, resulting in a more successful attack. As described in Appendix G.2, the ablation results for the initial point of the EDA also indicate that the PAS contributes to the attack performance of the EDA and the MDO framework.
10 (Sitawarin et al.
37

---

Published in Transactions on Machine Learning Research (May/2024)
Table 17: Validation of PAS.
The lowest robust accuracy is in bold.
Dataset
No.
target
input
ODS
PAS
57.41
57.40
57.29
2
65.33
64.82
64.28
3
67.85
66.98
66.27
4
69.06
68.21
67.52
CIFAR-10
69.88
68.77
67.81
6
69.66
68.59
67.88
7
69.11
68.37
67.59
8
68.08
67.47
67.10
9
67.32
66.84
66.46
57.52
57.54
57.57
65.93
65.79
65.33
3
68.88
68.63
68.15
4
70.62
70.34
69.33
CIFAR-10 2
5
71.41
70.95
70.00
6
72.37
71.82
70.71
7
72.57
71.99
70.80
8
73.09
72.49
71.31
72.86
72.28
71.08
20.37
20.39
20.34
2
24.71
24.67
24.46
3
26.65
26.64
26.14
4
27.70
27.49
26.99
CIFAR-100 3
28.37
28.36
27.70
6 29.02
28.98
28.33
7
29.55
29.32
28.62
8
29.83
29.68
28.95
9
29.72
29.55
28.94
1
27.25
27.22
27.17
2 31.87
31.74
31.50
3
33.70
33.73
33.18
4
34.92
34.72
34.17
CIFAR-100 4
5
36.00
35.91
35.13
6
36.55
36.50
35.58
7
36.75
36.56
35.87
8
37.34
37.20
36.39
9
37.81
37.54
36.73
32.66
32.62
32.42
38.04
37.98
37.68
39.64
39.64
39.22
4
41.50
41.34
40.74
ImageNet.
5
41.68
41.48
41.00
6
42.82
42.80
42.20
7
43.10
42.92
42.46
8
43.22
43.02
42.52
9
43.20
43.12
42.58
38

---

Published in Transactions on Machine Learning Research MMay/202)
Table 18: Validation of targeted attack
Table 19: Ablation study of EDA.
The default parameter in bold.
Dataset
No.
EDA
EDA+
N,
# sampled images
Dataset
No.
10
15
1
3%
5%
7%
CIFAR-10
1
55.49
55.49
CIFAR-10
2
56.75
56.78
CIFAR-10
1
55.59
55.49
55.49
55.49
55.47
55.
55.50
CIFAR-100
18.86
18.87
CIFAR-10
56.73
56.76
56.76
56.76
56.81
56.81
56.79
CIFAR-100
18.85
18.86
18.86
18.86
18.87
18.88
18.87
CIFAR-100
4
24.47
24.48
CIFAR-100
4
24.46
24.47
24.48
24.47
24.49
24.49
24.47
ImageNet
5
29.00
29.02
ImageNet
5
29.04
29.04
29.04
29.04
29.04
29.08
29.04
F Targeted attack in EDA
Motivation The motivation for using a largeted allack to efficiently diversify the most likely prediction class of the adversarial example away from the correct class (diversification in the output space). CW loss and DLR loss are objective functions that generate adversarial examples misclassified into the class with the highest prediction probability among classes other than the correct class. In other words, they attempt to generate an adversarial example misclassified into the class whose decision boundary is closest to the current point. However, it is difficult to approach the decision boundary when the gradient is zero, even if the distance to the decision boundary is close because the gradient-basee attack moves in the direction of the gradient. In addition, Yamamura et al. (2022) reported that in the steepest gradient-based attacks, the class with the highest prediction probability among classes other than the correct class hardly changes during the search. Considering these factors, diversification in the output space could be effective, especially for attacks with untargeted losses. Some existing research also supports the effectiveness of diversification
in the output space (Tashiro et al. 2020 Gowal et al. 2019
Target selection Although a multi-target attack shows high performance by achieving diversification in the output space, existing methods (Gowal et al.] 2019 Croce & Hein 2020b) are computationally expensive because they assign equal number of iterations to each target. To reduce the computational cost of the multi-targel atlack, we propose Target Selection (TS), which estimates the easiest target class to allack based on a small-scale search. TS estimates the easiest target class to attack based on a small-scale search to reduce the computational cost of the multi-target attack. The objective is to roduce the computational cost by focusing the number of iterations on the selected target. The procedure of TS as follows: (1) Upper K classes with large logit values of initial points are selected as target candidates. performed for N₂ iterations for each target candidate. highest objective function value.
(2) Targeted attacks are (3) The output is the target candidate T' with the
Hyperparameters The parameters of the targeted attack are the number of candidate targets in target selection K 9, 14, 20, the number of itcrations N, = 10, and the number of iterations in targetod attack = 100. We chose N 100 because the number of iterations per targeted attack in AA to 100, which achieves reasonable tradc-off between computational cost and attack performance.
Experiments To investigate the validity of this target selection, we compare the attack performance of EDA+, which executes normal targeted attack after the MDO framework, with that of EDA. In this experiment, we performed (XPAS, Thos, δGD, LEw) with 100 iterations for each target class and the initial step size of 2s. The following five models were used in the experiments. 1. ResNet-18 (Sehwag et al. 2022 2. WideResNNtt-28- 10 (Gowal et al.) 2020), 3. PreActResNet-I (Rice et al. 2020 4. Net-3- 10 (Sitawarin et al. 2021), 5. ResNet-50 (Engstrom et al.) 2019). These numbers correspond to the "No." column in Table 18 Although the normal largeted altack requires K x N = 100K queries, our targeted attack scheme selects a single target class and thus requires N, x K + = 10K + 100 queries per image. Given the parameter of K = 9, 14, 20 2 2, our targeted attack scheme requires fewer queries than the normal targeted attack. Considering the results described in Table 18 our target selection may reduce runtime without significantly degrading the attack performance of FDA.
39

---

Published in Transactions on Machine Learning Research (May/200 (024)
Table 20: Ablation study of EDA. The default parameter in bold.
initial stepsize
NADS
Th
Dataset
No.
E/4
e/2
6
2
3
4
5
10
3
4
5
6
CIFAR-10
55.50
55.48
55.51
55.49 49
35.47
55.49
55.50
55.51
55.50
55.48
55.48
55.49
CIFAR-10
2
56.78
56.78
56.78
56.75
56.78
56.76
56.77
56.76
56.77
56.82
56.76
56.7 76.76
CIFAR-100 R-10
18.92
18.92
18.87
18.86
18.88
18.86
18.89
18.89
18.87
18.90
18.89
18.86
CIFAR-100
24.49
24.50
24.50
24.47
24.47
24.47
24.49
24.48
24.48
24.49
24.48
24.47
ImageNet
29.10
29.08
29.08
29.00
29.00
29.04
29.02
29.04
29.06
29.04
29.04
29.02
Table 21: Ablation for the targeted attack.
The default setting is in bold.
8GD
8GG
θAPGD
Dataset
No.
Lew
L'
DDR
LTw
L DLR
L/LRR
CIFAR-10
1
55.49
55.57
55.48
55.50
55.50
55.48
CIFAR-10
2
56.76
56.89
56.81
56.81
56.83
56.79
CIFAR-100
3
18.86
18.96
18.90
18.93
18.94
18.91
CIFAR-100
4
24.47
24.63
24.50
24.53
24.53
24.50
ImageNet
5
29.00
29.18
29.02
29.08
29.06
29.00
G Ablation study
G.1 Hyperparameter sensitivity of EDA
We investigated the impact of hyperparameter values on EDA's performance. In this experiment, the follow-
ing five models were used. 1. Res 1e (Sehwag et al.) 2022), 2. WideReeNNel-28-11 (Gowal el al. 2020), 3.
(Sitawarin et al. 2021). 5. ResNet-50 (Engstrom et al. 2019). Tables 19) and 20 show the robust accuracy obtained by EDA with cach parameter value. Although the attack performance of the MDO framework is different among different hyperparaneters, as described in Table T6. the EDA's performance is stable regardless of the hyperparameter setting. These experimental results imply that the attacks composed by different strategies could be robust to the hyperpa- rameter settings. In addition, we tested the EDA's performance with several targeted attacks. As described in Table 21 the search directions based on the steepest direction performed better than the conjugate gradient-basee direction. Also, the margin-based losses showed higher performance than the CE loss.
PreActResNet-118 18 (Rice et al. 2020),- 4. WideRes.Vet-34
G.2
The impact of initial point sampling on the performance of the MDO framework and EDA
Table 22 shows the robust accuracy of the MDO framework and EDA using different initial point sampling, including original input (Xorg), Output Diversified Sampling (ODS, xoDs), and Prediction Aware Sampling (PAS, xPAS). the MDO framework and EDA showed slightly beiter performance using PAS than using the original input. The difference in robust accuracy between the MDO framework using the original input and ODS larger than that using PAS and the original input. For EDA, the difference in robust accuracy among the three initial point sampling sults suggest that initial point sampling does affect attack performance but that differences in the framework and the pairs of objective function and search direction have a greater impact.
smaller than that of the MDO framework. These experimental re-
G.3 Influence of DI and on ADS's performance
To investigate the influence of two terms that appear in the computation of indicators used in ADS, we executed the MDO framework and EDA with the pairs of search direction and objective function selected using DI, Pe, and D The model under attack is a subset of the 41 CNN-based robust models used in the main text. The allowable perturbation size, test samples, and computing environment are the same as in the experiments deseribed in the text. The results of the experiments are shown in Table 23 DI, Pe, and D in Table 23 indicate that the pairs of search direction and objective function were selected using these indicators,
40

---

Published in Transactions on Machine Learning Research (May/200 /224)
Table 22: Robust accuracy of the ablation study for the MDO framework and EDA with different initial point sampling.
CIFAR-10
MDO framework
EDA
model
(s 8/253)
Xorg
Xons
XPAS
Xong
Xons
XPAS
Audriushchenko & Flammarion
PARN- 18
03.96:20003
44.05
43.95:1:002
43.85:10.02
43.86:00011
43.85=:0.22
Ardeppali
2022
RN-18
52.50+0.03
52.66
52.19::0003
52.41+0.011
52.15=0.002
52.43=0.03
1Schway al.
A722
RN-18
55.62+0.03
55.60+0.02
0..56++0.02
35.49-+0.01
55.50+0.01
55.49+0.011
Engstrom
2019
RN-50
49. 00.0030003
49.34=10.06
49.43:10.09
49.11400001
49.10=00001
49.10=0.03
Carmon et
2019
WRN-28-10
59.49-0.03
59.62-00.0 03
59.49++0.00
59. 10:40000000
59.45=0002
59.40=0.00
Grnwal et ml
27 222
WRN-28 10
62.84+0.02
62.99+00.55
62.82+00.03
62.75++0001
62.78+0.02
62.75+00022
H Headrycks et 0
201
WRN-28-10
51.8720.03
5184100.00 01
51.85400004
04.7740.011
54.77100001
51.77 02
Reballi el m..
WRN-28-10 10
00.78:0.04
60.83:+0.07
60. 00.75:100003
60.69:10.02
60.69:00.02
60.6440.01
Schwaag m]
2720
WRN-28-100
07.1
57.22:40.05
57.11+:0.02
57.01+40.2
57.07:0.03
57.03=0.00
Srathar m1
20/22
WRN-28-11 10
0..6
59.68 0.100
59.55
59.52+0.03
59.46-00.12
Wang al.
2020
WRN-28-10
56. 56.4810.06
56.52:10.24 24
50.29:1:0002
56 18.1001001
56.20=0.04
56.1520.02
Wi al.
7220
WRN-28-10
60.01:0.02
60.11=0.09
60.00 60.00:10.02
39.94+0.01
59.97:0003
59.94=0.01
Ding ef pl.
2012
WRN 28
43.33+0016
0..12 43.69=+0.1
41.70:+0.04
41.84=0.03
41.74=00.6
Addepall
2022
WRN-34-10
57.78
58.01=0.09
57. 03.00300.0
57. 00:10001
57.71=0.01
57.69=0.02
Schwag al. A022
WRN-344-10
60.27+0.02
60.30+0.02
60.27++0.05
60.17400.01
60.18+0.02
60.18+0.02
Stawarii cl. m1
2121
WRN-34-10
50.72:0004
50.69:10.08 0..3
50.72:10.00
50.59+0.02
50.60:10.02
50.59=0.01
Zhang et ai
WRN-34-10
01.6220.03
41.65::0.04 0..0
11.65:40000 0.06
04.51
41.52:0.02
41.51
Zhang et ml.
2120
WRN-344-00
53.53+0.03
53.56+00.33
53.50 53.50
0..43+0.001
53.44+0008
53.42+0.02
STidhar ml
2722
WRN-34-15
60.43+0.02
60.44:0.11
60.38-40.03
60.31+0.02
60.3240.03
60.32-0.01
Guswal cl 50.
21/0
WRN
56.80=0.02
57.08=10.05
56.86:10.002
36.79:10.01
56.81100011
56.79=0.08
Pang
2020
WRN-34-20
53 03.8520.03
53.80=00.08 03
03.82410.03
53.61110.00
53.67=0.01
53.6620.01
Rice et
2020
WRN-344200
53.47+0.02
53.566+0.09
533.46+-0.04
53.33+-0.01
53.344+0.01
53.34+00.01
Guwal el
2120
WRN-70-16
00002
57.28:10.05
57. 08.10:00002
57.12:40.00
07.14=0.01
57.12=0.01
Gowal et ai.
2020
WRN-70-16
65 0.000.0
65.91
65.87=+0.01
65.81+00.03
05.81
65.83=0.01
Rebulti et al.
2022
WRN-700-16
64.31 +0.04
64.36+ 0006
64.305+0.05
64.20++0.01
64.20+0.02
64.20+0.08
C1FAR-100
the MDO framework
EDA
model
(s 8/255)
Xorg
XP48
Xong
XODS
[Rice et al.
2020
PARN 18
18.97:0.01
19.04:10.09
18.98-+0.01
18.87:+0.01
18.66:00001
18.88=00.01
Headryckes et
WRN-28-10 10
28. 28.1620.02
28.60=:0.13
28.41+00.02
28.28:10.01
28.28:0.02
28.27=0.02
Rebutti et al.
122
WRN-228-10 10
32.22+0.03
32.45+00.2
32.07-+0.04
31.97+0.01
31.99+0.01
81.94+0.03
Adtlepalli el
2122
WRN-34-10
31.90:0.03
32.01:0.116
31.87:1:0022
31.78:10.01
31.78:10001
31.78=0.011
Cuu of al
WRN-34-100
29. 99.0022004
29.14:10.2 23
28, 96.90.005
28. 0.00
28. 01.00.2
28.83=0.00 02
Sitawarin et
2112
WRN-34-10
24.61 +0.04
24.67+0.07
24.66++0.03
24.50++0.01
24.48+0.01
24.50+0.01
Wu el ai.
20200
WRN-34410
28. 0..9520.04
29.17:0.11
28.89110.04
28.76:1001
28.77:1001
28.76=0.00
Cui
WRN-34-20
30 00.06:00.22
30.1140.10
30.00:10.02
29 00..6610001
29.86:20.01
29.85=0.01
Grwwal el ol.
20200
WRN-70 70-16
30.13:0.03
30.29:0.21
30.07:10.01
29.97:10011
29.99:00001
09.06=0001
Gowal
WRN-70-16
37. 37.0610.04
37.17
36.96110.01
36.8110.01
36.8310.01
36.81=0.01
Rebulti
202T
WRN-70-16
34.60-+0.04
34.81+0.09
34. 04.64400
34.57+0002
34.58+0.02
34.55+0.01
ImageNet
the MDO framework
EDA
model
(s 4/255)
Xarg
Xons
XPAS
Xong
XODS
XPAS
Salman al.
2020
RN-18
+0.05
0..21
25.25 0.002
25.12+0002
25.10+0.01
25.11+0.022
Engstrom
2119
RN-50
29.2911.06
29.77:10448 48
29.27:10.07
29.0111001
29.0010.02
29.01=0.01
Salmen
2220
RN-50
34.67:0.09
15.13:0.30
34.73:0.00
34.54+0.03
34.51=0.03
34.52=0.02
Wong
2120
RN 50
26.41:0.12
26.71+10.4
26.32-+0.18
26.13:+0.12
26.14:0.08
26.12:+0100
Salinen al
2120
WRN-50-2
38.1220.04
39.36::0.10
38.3110006
38.0110.02
38.03:00003
38.03=0.02
respectively. From the experimental results, the highest attack performance is expected when Pe is used to select the pairs. However, focusing on the MDO framework against CIFAR-10 and CIFAR-100 models, DI showed the highest number of models that performed better than the 2nd-best, Thus, the MDO framework is expected to show higher performance when the pairs are selected with DI than with pe for wide range of models. The attack performance of the MDO framework with D is often intermediate between that with DI and Pa. Thus, we used De to select the pairs in the text, considering performance and generalization. The reason for the better performance in selecting the pairs that maximize Ps in EDA can be altributed to its different nature from targeted attacks. Targeted attacks maximize the prediction probability of the target class. Thus, the class with the highest prodiction probability, excluding the correct prediction, is unlikely change during the search. In contrast, the pairs that maximize Pe have high probability that the class with the highest prediction probability, excluding the correct prediction, will change during the search. The pairs
41

---

Published in Transactions on Machine Learning Research (May/202)
Table 23: Influence of DI and P on ADS's performance. The lowest and second lowest robust accuracy of each attack is in bold and underlined, respectively.
defense
model
MDO framework
EDA
CIFAR-10
DI
p
17
DI
p
1)
Andriushchenko & Flammarion
2020
PARN-18
43.90
44.00
43.95
43.85
43.85
43.86
Addepalli et al.
(2022
RN-18
52.52
52.45
52.50
52.42
52.41
52.43
Sehwag et al.
2022
RN-18
55.61
55.59
55.58
55.50
55.50
55.49
Engstrom
20119
RN-50
49.50
49.31
49.52
49.11
49.09
49.12
Carmon et a..
2019
WRN-28-10
10
59.49
59.51
59.46
59.42
59.42
59.38
Schwag et al.
2020
WRN-28-10
57.10
57.10
57.14
57.03
57.01
57.06
Srichar ct al.
2022
WRN--2--10
59.50
59.54
59.59
59.45
59.46
59.47
Wu et al.
2020
WRN-28-10 10
60.02
60.04
59.99
59.96
59.96
59.95
Addepali li al.
2002
WRN-34-10
57.73
57.72
57.72
57.71
57.66
57.68
Sitawarin
2021
WRRN-34-10
50.75
50.85
50.70
50.62
50.60
50.58
Pang et al.
2020
WRN-34-20
53.81
53.74
53.81
53.65
53.65
53.65
Gowal et al.
2020
WRN-70-16
65.89
65.80
65.85
65.83
65.78
65.81
# bold
6
5
5
# underlined
5
3
CIFAR-100
Rice et al.
2020
PARN-18
18.91
19.01
18.97
18.88
18.88
18.89
Addepall
2022
WRN-34-100
31.81
31.85
31.86
31.77
31.77
31.78
Cui et al. 2021
WRN-34-10
28.93
28.96
28.99
28.85
28.82
28.81
Sitawarin et al.
2021
WRN-34-10
24.63
24.59
24.65
24.49
24.48
24.49
Wu al. 2020
WRNN-44-10
28.87
28.90
28.88
28.78
28.78
28.77
Gowal ct a.
2020
WRN-70-16
37.03
36.94
36.96
36.84
36.79
36.80
Rebulfi et al.
202T
WRN-70-16
34.65
34.64
34.65
34.57
34.54
34.55
# bold # underlined
3
0
5
2
4
5
ImageNet
Salman et al.
2020
RN-18
25.20
25.16
25.22
25.12
25.06
25.10
Engstrrom a
2019
RN-50
29.20
29.10
29.20
28.98
29.00
29.00
Salman et al.
2020
RN-50
34.64
34.62
34.84
34.52
34.50
34.56
Wong et al.
2020
RN-50
26.34
26.10
26.22
26.10
25.96
26.02
Salman al.
2020
WRN-50-2
38.32
38.30
38.28
38.04
38.00
38.02
# bold
4
2
0
# underlined
3
1
4
that maximize Pe are expected to be more complementary to the targeted attack than those that maximize DI.
G.4 Performance evaluation with different perturbation bounds
This experiment evaluates the performance of EDA at different perturbation sizes with 4/255 and 16/255 for CIFAR-10/ 100 and 2/255 and 8/255 for ImageNet. We used AutoAttack (AA) and Adaptive Auto Attack (A3) as bascline methods. AA is standard technique for robustness evaluation, and A showed superior performance in lerms of execution time and attack performance in the experiments presented in the main text. The experimental setup the same as the main text, except for the perturbation size. Table [4] shows the robust accuracy and exccution time. The performance difference between the different attacks is not that large for small E, even for models trained on ImageNet. When e is large, A3 shows the highest attack performance. However, the computation time of A tends to increase significantly compared
42

---

Published in Transactions on Machine Learning Research (May/202))
Table 24: Performance evaluation of EDA with different perturbation bound E.
CIFAR-10
4/265
16/255
Model
AA
A3
EDD
AA
A
EDA
BCC
Sec
SEC
88C
a0C
SEC
80C
86C
3.00
8ec
30c
aEc
Andrioshcbenko
Plammazion
2020
PARN-18
63.24
1,006
63.24
63.24
613
10:45
786
10.30
470
10.18
Artlepolll
2022
RN-18
71.08
71.08
705
17.59
758
17.53
203
Sehwaag m
222
RN-18
71.98
4,616
11.96
706
71.96
647
20.55
1,412
20.07
868
20.11
293
Empditom
RN-50
70.97
1,967
1,822
2119
WHN-28
77.59
38,346
77.58
6,241
77.59
4,012
20.35
10,906
19.96
5,995
20,00
1,429
Getral m.
A720
WRN-28.
78.63
45,270
78.60
7,045
78.00
5.547
24.95
15,441
24.40
2,291
24.56
2,292
2,697
Selwwag m
2124
WRN FRNN-2-111
75,80
37, 244
75.76
5,945
75.78
4,030
18.76
9,959
18.59
1,716
18.55
1,403
Spaliiat m
2022
WRN-28-- 28-10
77.77
38,391
77.74
6,209
77.74
4,150
20.30
10,944
20.10
6,036
20.18
1,168
Wang
1720
W-RN-28-10
74.32
74.51
4,150
17.89
9,055
17:48
5,353
1,336
et a.
222
WRN-28810
76.38
6,011
76.38
1,131
23.39
2,299
23,50
1,635
Ding st.
WRN--2 28-4
63.30
6,4711
63.36
8G4
12.15
1,540
12:27
13.72
Autlepalli
2122
1--0
75.26
16,846
75.25
7,160
75.26
5,101
20.40
13,667
19:91
2,221
19.89
1,798
Sahwag et ml
A722
WRN-34-10
75.75
7,581
75.77
4,902
24.47
9,489
24.06
2,029
WRN- WRN-34-10
9,583
5,161
1,334
Zhanng ml.
WYRN-34-10
60.15
43,120
69.11
6,930
09.13
4,499
8.38
0,229
8.08
13,233
8.18
968
Ziiing ot m. 31721
WBN-34-10
71.04
44,221
71.01
7.067
71.01
4,956
21.06
14,088
20.74
2,249
20.68
1,799
2122
WRN-34-15
75.34
76,527
75.33
11,393
75.34
9.502
24.75
26,993
24.48
3,986
3.807
Leowal ot m.
21220
WHN-34420
73.32
162,529
733.12
21,954
73.33
17. 8138
22.96
53,608
22.55
8,,119
22.57
6,545
WRN-34-20
72.06
14,227
15,510
4,185
21721
WHN--7016
73.23
210,017
73.22
301382
73.21
26,431
24.01
81,819
23.82
11,810
23.71
10.315
CIFAR-100
6
4/255
16/255
Model
AAA
A3
EDA
AA
A
EDA
acc
a0c
900
aCc:
5000
8.02
500
Rice et al.
2120
PAHN-18
33.337
2.372
33.36
4033
333.37
588
4.45
339
4.38
1,122
4.39
218
WRN-28-11
45.88
26,571
45.82
4,173
45.83
3.724
12.47
7,965
12.23
1,615
12.17
1,237
Addeppali
WHN-34-10
47.99
30,028
47.98
4,848
47.98
3.968
10.79
7,610
10.67
1.671
10.55
1,211
WRN-34-10
43.27
26,579
4,526
11.35
7,763
1,705
1.297
Sitawaria
2121
WHN-33-10
41.37
28,193
41.37
4,223
41.37
3,067
6.38
5,087
6.11
19.993
6.25
507
Rebulli et al
222
WRN-70-16
48:26
158,196
48.35 23
20,161
48.23
21.621
14.16
50,339
13.97
9.017
13.79
6,990
ImnageNet
2/255
8/255
Defence
Model
AAA
Aa
EDA
A.A.
A
EDA
ncc
ABC
900
8:00
800
8.00
Sahman al.
RN-18
38.44
1,581
34.70
2.194
8.16
3,990
8.12
1,153
Engettoom
2010
HNN-0
45.64
17,000
45.64
4,284
45.62
3,874
7.88
3,389
7.44
11,717
7.72
1.578
4,255
4,192
2,016
1.965
RN-50
35.22
5,819
33.16
5,657
7.66
20,439
7.50
2,290
Salmina et m
2120
WRN-50-2
50.32
8,017
50.28
6,788
12.56
3,506
12.60
2,377
43

---

Published in Transactions on Machine Learning Research (May/200 (024)
Table 25: Robust accuracy of randomized defenses. The lowest robust accuracy is in bold.
CIFAR-10 (s 8/255)
AA
A3
EDA
Defense
Model
clean
a0c
sec
8Cc
SBC
acc
Sec
RNA
RN--18
84.48
62.13
5,094
65.99
1,342
53.87
743
RNA
WRN-32
86.49
64.53
30,560
69.70
16,087
53.43
4,764
DWQ
PARN-18
82.11
53.81
6,469
56.63
3,161
48.00
897
DWQ
WRN-34
81.56
58.05
46,058
60.21
7,043
50.32
5,856
SVHN (s = 8/255)
DWQ
PARN-18
81.97
31.90
4,117
38.80
1,802
28.53
676
DWQ
WRN-34
88.65
37.20
29,969
43.87
5,438
35.23
4,027
CIFAR100
8/255)
RNA
RN-18
56.75
42.78
2,378
47.35
1,921
30.67
426
RNA
WRN-32
60.19
41.94
15,296
45.98
17,575
30.71
1,851
DWQ
PARN-18
55.96
32.61
3,559
33.72
1,491
26.51
607
DWQ
WRN-34
55.94
36.77
25,539
37.82
4,658
29.64
3,311
ImageNet
(e = 4/255)
DWQ
RN-50 (free)
49.44
35.62
2,386
45.08
109
24.18
829
DWQ
RN-50 (fgsmrs)
62.18
39.08
21,994
40.72
4,854
33.72
3,773
to other methods. The above results indicate that robustness evaluation can be effectively achieved by using these methods in different ways depending the acceptable computational resources.
H EDA's performance against randomized defenses
We investigated the efficacy of EDA against two randomized defenses, including Double- Win Quant (DWQ) (Fu et al. 2021) and Random Normalization Aggregation (RNA) (Dong et al. 2022 For the models trained on CIFAR-10/100 and SVHN (Netzer et al. 2011), we used E = 8/255 and 10,000 test im- ages. For the ImageNet, we used = 4/255 and 5,000 images, the same as the MobustBBnnch We report the results single run with a fixed random seed for reproducibility. The compared methods are AutoAttack (AA) and Adaptive Auto Attack (A3). A showed sufficiently fast and strong performance among recently proposed attacks against CNN-based robust models. For consistency with AA, calculated the robust ac- curacy using generated adversarial examples when the attack terminated and compared these values because the model with randomized defense may produce different outputs with each inference. Official implementa- tions of AA and EDA calculate robust accuracy in this way, but the official implementation of A calculates differently. Therefore, we slightly modified the implementation of A to calculate robust accuracy in the
same way. The parameters of the compared attacks were the same as in the main text. Table 25 shows the
robust accuracy and runtime of the standard version of AA, A3, and EDA. From Table 25 EDA showed. higher attack performance in less computation time. Although these are the results of single run, EDA is expected to be effective for randomized defenses.
44


Reviews:
[{"review_id": "NLhqXMfluA", "reviewer": "Reviewer_XVrq", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "## Strengths\n- The experimental results in Section 3.1 Motivation are quite interesting and make sense.\n\n## Weaknesses\n- This paper is very heuristics. The writing can be improved by simplifying mathematical notions. \n- The Automated Diversified Selection (ADS) is heuristic to me. Specifically, we need to search through many combinations of $n_a$ attacks to choose the optimal one with the smallest $D^e$. Moreover, the choosing of DI for measuring the diversity of adversarial examples is not well-justified. Even, DI has not described in the paper yet.\n- The proposed attack requires a testing set in advance, hence hindering its application. It would be better if we can attack single point solely.\n- Algorithm 3 is almost not understandable. It is unclear where MT is used in this EDA algorithm.\n- Although the proposed algorithms are complicated, they only slightly improve $MT_{cos}$ in Table 1. Also, it is unclear why the proposed algorithms are more scalable than the baselines because they look complicated, i.e., need to do at least two NAS and for each attack in the list of NAS, they use this attack to attack a point in the testing set.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper proposes Multi-directions/objectives (MDO) strategy and Efficient Diversified Attack (EDA) to improve adversarial attacks. Specifically, MDO uses multiple search directions and objective functions, whereas EDA is faster and more efficient by combining MDO with MT attack.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "- The paper needs a significant improving in presentation and writing. The mathematical notions need to be simplified to improve comprehensibility. \n- The motivations and high-level ideas of ADS, MDO, and EDA need to be presented and discussed.\n- Adversarial examples need to be visualized. Moreover, the adversarial examples need to be used in adversarial training to report the robust accuracies. This helps to demonstrate the merits of the generated adversarial examples.\n- Algorithms 2 and 3 need to be represented with more detailed explanations of the rationale of each step. Currently, they look heuristic and messy.", "broader_impact_concerns": "The broader impact is explicitly discussed in the paper.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "1QYtPX6jF4", "reviewer": "Reviewer_WpYh", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths: \n\n1. The method is well-motivated. Considering both diversity and intensity is important for improving the attack performance. \n\n2. The experiments are conducted on comprehensive datasets and models. The authors also report the standard deviation to validate the significance of performance gain.\n\nWeaknesses:\n\n1. The novelty is limited though novelty is not an important evaluation metric in TMLR. I understand that the authors first find a good starting point and then use APGD (the existing work) to find the adversarial data. The starting point is chosen based on different loss objectives which are existing works as well. \n\n2. Not many new insights are delivered from this paper though the proposed method works empirically. Prior works have tried to consider improving both diversity and intensity to boost adversarial attacks. Therefore, I found limited contributions to inspiring future works. It would be great if the authors could provide any.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper proposes an efficient and effective adversarial attack method called EDA. There are two critical stages in EDA: (1) The Diversification stage tries to use different objectives and directions to find a data point that has the largest C&W loss; (2) The Intensification stage updates the data point via APGD. The empirical results seem to validate the effectiveness of the proposed method.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "Please see the weaknesses above.", "broader_impact_concerns": "None.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "mhD6dp31eQ", "reviewer": "Reviewer_nGX2", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths\n\n1. It is an interesting observation that different attacks and hyper-parameters result in optimization clustering in certain regions.\n\n2. The evaluation is comprehensive, incorporating multiple existing baseline attacks and various defense techniques.\n\n\nWeaknesses\n\n1. The results on robust accuracy are marginal. As shown in the table, in most cases, the proposed attack can only reduce the robust accuracy by less than 0.2% compared to existing methods such as A^3. Similar observations are reported in Table 2. This improvement, when compared to existing attacks, is limited, and it is also challenging to determine whether the slight improvement results from the selection of random seeds and images for the attack.\n\n2. The paper is difficult to follow. One of the main algorithms is presented in Algorithm 2, consisting of 26 lines of pseudo code. However, the text lacks a detailed description of the entire procedure. Specifically, in lines 8 and 19, it merely says to \"run the attack following Algorithm 4,\" but Algorithm 4 is presented in the appendix. This setup makes it challenging for readers to grasp this part of the content. Ideally, the main text should be self-contained, eliminating the need to consult the appendix to understand the material.\n\n3. According to Table 1, the number of queries needed for the proposed attack is significantly lower than that for A^3. An explanation of why it requires fewer queries would be beneficial. Additionally, the submission lacks a study on different perturbation bounds. Evaluating the attack across various bounds would more effectively demonstrate the performance of the proposed attack in comparison to existing methods.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "No", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper proposes an adversarial attack that improves the attack success rate of existing methods. It observes that different attacks tend to cluster in certain regions (i.e., local minima) in the search space. This paper proposes searching for different local solutions using various attack methods and hyper-parameters. Specifically, it divides the adversarial example generation into two phases: a diversification phase and an intensification phase. In the diversification phase, the paper uses a large optimization step size (i.e., twice the perturbation bound) and applies different existing attack methods. In the intensification phase, it selects the perturbation with the largest loss from the last phase as the initialization and uses a smaller step size. The evaluation is conducted on three image datasets with 41 models and 21 different defenses. The experimental results show that the proposed attack can improve the attack performance to some extent.", "claims_and_evidence_yn": "No", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "See the above.", "broader_impact_concerns": "None", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: In their official recommendation, two reviewers lean toward accept and one reviewer leans toward reject this work. I read the paper in detail and also find that some claims in this work are not very well supported.

**1. Design Choices in the Algorithms:** The proposed multi-direction/objectives (MDO) strategy and eficient diversified attack (EDA) are the main contributions of this work. The motivation proposed in section 3.1 is intuitive and great, but the readers could struggle to fully understand the details and all the steps of the proposed algorithms. During the rebuttal, the authors put some effort into improving the methodology subsections (3.2 - 3.4), but there is still room for improvement.

There are many design choices in the proposed algorithms that make them complicated. The added descriptions in the revised paper are helpful, but clear evidence and further analyses are still needed to fully support these design choices (e.g., what are their concrete effects/impact for diversification and/or intensification). Neat ablation/validation and/or intuitive illustration (as in subsection 3.1) could also be very helpful to truly motivate each proposed design choice. On the other hand, are all the proposed design choices necessary for better diversification or intensification? If some designs only have a minor (or even no) impact, simplifying the algorithms could also significantly improve the exposition of this work.    

**2. Empirical Supports for Efficient Diversified Attack (EDA):** A key claim of the submission is that "(The proposed EDA method) is a faster and stronger attack using MDO and MT strategies." However, according to the new experimental results for multiple perturbation bounds, EDA is outperformed by A^3 for a larger perturbation bound, and achieves similar performance with other methods (AA and A^3) for a smaller perturbation bound. Therefore, the claimed stronger attack is not well supported.  

For the computational overhead, it is promising to see EDA requires a smaller number of queries than AA and A^3 in Table 1, and also a faster runtime in Table 22. However, the support and explanation for this efficiency is not enough. In the response, the authors simply attribute the fewer queries to "the complementarity of multi-targeted and MDO strategies (Section 4.2) and a higher degree of diversification of EDA (Section 4.1)". A deeper analysis and discussion could be required, especially since EDA is outperformed by A^3 for a larger perturbation bound (e.g., will the smaller queries be due to premature convergence?).

Another possible way to tackle this concern is to properly adjust the claimed contribution on "(The proposed EDA method) is a faster and stronger attack using MDO and MT strategies." to make it precisely reflect the performance of EDA. Adding a limitation section to clearly and explicitly discuss the pros and cons of MDO/EDA could also be very helpful for the readers to truly understand the proposed method.

This paper is an interesting work that studies an important topic (diversification v. intensification) for adversarial attacks, and I also appreciate the efforts the authors have made for a comprehensive comparison with other methods as well as the added motivation/description in the revised paper. Therefore I recommend **accept this paper but with a "minor" (somewhat like a major) revision**. In the revised paper, I expect the authors can properly address the concerns raised above.
Decision: Accept (poster)