Imbalanced Gradients: A New Cause of Overestimated Adversarial Robustness
Authors: Linxi Jiang
Xingjun Ma
Zejia Weng
James Bailey
Yu-Gang Jiang
Venue:  (2021)
Keywords: Adversarial Attack
Robustness Evaluation
Adversarial Defense
Deep Neural Networks

Abstract: Evaluating the robustness of a defense model is a challenging task in adversarial robustness research. Obfuscated gradients, a type of gradient masking, have previously been found to exist in many defense methods and cause a false signal of robustness. In this paper, we identify a more subtle situation called \emph{Imbalanced Gradients} that can also cause overestimated adversarial robustness. The phenomenon of imbalanced gradients occurs when the gradient of one term of the margin loss dominates and pushes the attack towards to a suboptimal direction. To exploit imbalanced gradients, we formulate a \emph{Margin Decomposition (MD)} attack that decomposes a margin loss into individual terms and then explores the attackability of these terms separately via a two-stage process. We examine 12 state-of-the-art defense models, and find that models exploiting label smoothing easily cause imbalanced gradients, and on which our MD attacks can decrease their PGD robustness (evaluated by PGD attack) by over 23\%. For 6 out of the 12 defenses, our attack can reduce their PGD robustness by at least 9\%. The results suggest that imbalanced gradients need to be carefully addressed for more reliable adversarial robustness.

Under review as a conference paper at ICLR 2021
IMBALANCED GRADIENTS: A NEW CAUSE OF OVER- ESTIMATED ADVERSARIAL ROBUSTNESS
Anonymous authors Paper under double-blind review
ABSTRACT
Evaluating the robustness of defense model is challenging task in adversar- ial robustness research. Obfuscated gradients, type of gradient masking, have previously been found to exist in many defense methods and cause false signal of robustness. In this paper, we identify more subtle situation called Imbal- anced Gradients that can also cause overestimated adversarial robustness. The phenomenon of imbalanced gradients occurs when the gradient of one term of the margin loss dominates and pushes the attack towards to a suboptimal direction. To exploit imbalanced gradients, we formulate a Margin Decomposition (MD) attack that decomposes a margin loss into individual terms and then explores the attackability of these terms separately via a two-stage process. We examine 12 state-of offtthe-rtt defense models, and find that models exploiting label smoothing easily cause imbalanced gradients, and on which our MD attacks can decrease their PGD robustness (evaluated by PGD attack) by over 23%. For 6 out of the 12 defenses, our attack can reduce their PGD robustness by least 9%. The results suggest that imbalanced gradients need to be carefully addressed for more reliable adversarial robustness.
INTRODUCTION
Deep neural networks (DNNs) are vulnerable to adversarial examples, which are input instances crafted by adding small adversarial perturbations to natural examples. Adversarial examples can fool DNNs into making false predictions with high confidence, and transfer across different models (Szegedy et al. 2014| Goodfellow et al.) 2015). A number of defenses have been proposed to overcome this vulnerability. However, a concerning fact is that many defenses have been quickly shown to have undergone incorrect or incomplete evaluation (Carlini and Wagner| 2017 Athalye ct al. 2018; Engstrom et al. 2018 Desato et al., 2018 Mosbach et al.) 2018 He et al. 2018). One common pitfall in adversarial robustness evaluation is the phenomenon of gradient masking (Papernot cet al. 2017 Tramer et al. 2018) or obfuscated gradients (Athalye et al. 2018), leading to weak or unsuccessful attacks and false signals of robustness. To demonstrate "real" robustness, newly proposed defenses claim robustness based on results of white-box attacks such as PGD (Madry et al., 2018), and at the same time, demonstrate that they not a result of obfuscated gradients. In this work, we show that the robustness may still be overestimated even when there are no obfuscated gradients. Specifically, we identify a new situation called Imbalanced Gradients that exists in several state-of-the-arr defense models and can cause highly overestimated robustness.
Imbalanced gradients is new type of gradient masking effect where the gradient of one loss term dominates that of other terms. This causes the attack to move toward a suboptimal direction. Different from obfuscated gradients, imbalanced gradients are more subtle and are not detectable by the detection methods used for obfuscated gradients. To exploit imbalanced gradients, we propose a novel attack named Margin Decomposition (MD) attack that decomposes the margin loss into two sepurate terms, and then exploits the attackability of these terms via a two-stage attacking process. We derive MD variants of traditional attacks like PGD and MultiTarreted (MT) (Gowal et al. 2019). and deploy these MD attacks to re-examine the robustness of 12 adversarial training- based defense models. We find that 6 of them susceptible to imbalanced gradients, and their robustness originally evaluated by the PGD attack drops significantly against our MD attacks. Our key contributions are:
We identify a new type of subtle effect called imbalanced gradients, which can cause highly overestimated adversarial robustness and cannot be detected by detection methods

---

Under review as a conference paper at ICLR 2021
for obfuscated gradients. Especially, We highlight that label smoothing is one of the major causes of imbalanced gradients.
We propose Margin Decomposition (MD) attacks to exploit imbalanced gradients. MD leverages the attackability of the individual terms in the margin loss in two-stage attacking process. We also introduce two variants of MD for existing attacks PGD and MT.
We conduct extensive evaluations on 12 state-of-the- them suffer from imbalanced gradients and their PGD robustness drops by more than 9% against our MD attacks. Our MD attacks exceed state-of-the the-art attacks when imbalanced gradients occur.
defense models and find that 6 of
2 BACKGROUND
We denote a clean sample by x, its class by y E {1,..., ,....,))) C} with C' the number of classes, and a DNN classifier by ʃ. The probability of x being in the i-th class is computed as p;(x) = ex プj= e*, where z, is the logits for the i-th class. The goal of adversarial attack is to find an adversarial example Xade that can fool the model into making a false prediction (e.g. I (Xadv) ) * y). and is typically restricted to be within a small c-ball around the original example (e.g. |aaae x 5).
Ad Adversarial Attack. Adversarial examples can be crafted by maximizing a classification loss e by one or multiple steps of adversarial perturbations. For example the one-step Fast Gradient Sign Method (FGSM) (Goodfellow et al., 2015) and the iterative FGSM (I-GGSM) attack (Kurakin et al. 2017). Projected Gradient Descent (PGD) (Madry et al. 2018) attack is another iterative method that projects the perturbation back onto the e-ball centered
when it goes beyond. Carlini and Wagner (CW) (Carlini and Wagner, (017) attack generates adversarial examples via an optimization framework. Whilst there exist other attacks such as Frank-Wole attack (Chen et al. (2018)), distributionally adversarial attack (Zheng et al., (219) and elastic-net attacks (Chen et al.) 2018b), the most commonly used attacks for robustness evaluations are FGSM, PGD, and CW.
Several recent attacks have been proposed to produce more accurate robustness evaluations than PGD. This includes Fast Adaptive Boundary Attack (FAB) (Croce and Hein 2019), MultiTargettd (MT) attack (Gowal et al.) 2019), Output Diversified Initialization (ODI) attack (Tashroo et al., (2020), and Auto.Attac (AA) (Croce and Hein 2020). FAB finds the minimal perturbation necessary to change the class of given input. MT (Gowal et al. 2019) is a PGD-based attack with multiple restarts and picks a new target class at each restart. ODI provides more effective initialization strategy with diversified logits. AA attack is a parameter- ensemble of four attacks: FAB, two Auto-PGD attacks, and the black-box Square Attack (Andriushchenko et al. 2019). AA has demonstrated be one of the stateof--he-art attacks to date (Croce and Hein 2020).
Adversarial LOss. Many attacks use Cross Entropy (CE) as the adversarial loss: (x,
(x, y) -log Py. The other commonly used adversarial loss is the margin loss (Carlini and Wagner 2017): margin (x, y) Zmaz - Zy: with Zmaz maxezy Zz. Shown in (Gowal et al. 2019). CE can be
written in a margin form (e.g. lxc(x, y) = log E 1-1 e².) -zy), and in most cases, they are both effective. While FGSM and PGD attacks use the CE loss, CW and several recent attacks such as MT and ODI adopt the margin loss. AA has one PGD variant using the CE loss and the other PGD variant using the Difference of Logits Ratio (DLR) loss. DLR can be regarded loss. In this paper, we identify a new effect that causes overestimated adversarial robustness from the margin loss perspective and propose new attacks by decomposing the margin loss.
"relative margin"
Adversarial Defense. In response to the threat of adversarial attacks, many defenses have been proposed such defensive distillation (Papernot et al.
2016), feature/subspace analysis (Xu et al. [0188] Liao et al.) (2088) (Samangouei et al. 2018 robust regularization (Gu and Rigazio| 2014 Tramrr et al. 2018 Ross and Doshi-
2017 Ma et al. (0188), denoising techniques (Guo et al.
2018). model compression (Liu et al. 2018: Das et al. 2018: Rakin et al. (2088) and adversarial
training (Goodfellow et al. 2015]
et al.) 2018). Among them, adversarial training via robust min-max optimization has been found to be the most effective approach (Athalye et al.) 2018). A
number of new techniques have been proposed to further enhance the adversarial training (Wang)
et al. (2019) /Zhang et al.) 2019) [Carmon et al. 2019: Alayrac et al. 2019: Wang and Zhang 2019
Zhang and Wang 2019: Zhang and Xu 2020
Wang et al. 2020 Kim and Wang 2020 Ding et al.
2

---

Under review as a conference paper at ICLR 2021
2018 Chan et al. 2020). We will discuss and evaluate these adversarial training-based defenses with our proposed attacks in Section
3 IMBALANCED GRADIENTS AND MARGIN DECOMPOSITION ATTACK
We first give a toy example of imbalanced gradients and show how regular attacks can fail in such a situation. We then empirically verify their existence in deep neural networks, particularly for some invversarially-triinne
models. Finally, we propose the Margin Decomposition attack to exploit the imbalanced gradients. Since CE and margin loss are the two commonly used loss functions for adversarial attack and CE can be written in a margin form (Gowal et al. 2019), here we focus on the margin loss present the phenomenon of imbalanced gradients.
Imbalanced Gradients. The gradient of the margin loss (e.g. Emargin (x, y) = Zenna - Z,y)) is the combination of the gradients of its two individual terms (e.g. Vx (Z Saeme--y.) Zyy) V VxZmax + Vx( (x(-))))
Imbalanced Gradients is the situation where the gradient of one loss term dominates that of other term(s), pushing the attack towards suboptimal direction.
Toy Example. Consider a one-dimensional classification task and binary classifier with two outputs 21 and Z₂ (like logits of a DNN), Figure Iillstraate the distributions of 21, 2g and O. The classifier predicts class when 2i 2 Ø2, otherwise class 2. We consider an input at x = 0 with correct prediction y 1, and maximum perturbation constraint e = 2 (e.g. perturbation ò E [-2, +2). The attack is successful if and only if 22 21- In this example, imbal- anced gradients occurs at Ø = O, where the gradients of the two terms V 22 and V 22) have opposite directions, and the attack is dominated by the term as x ) is significantly larger than 22 Thus, attacking æ with the margin loss will converge to +2, where the sample is still correctly classified. However, for successful attack, x should be perturbed to- wards -2. In this particular scenario, the gradient Z < 0 alone can provide the most effective attack direction. Note that this toy example was motivated by the loss landscape of DNNs when imbalanced gradient occurs.
Z2 21 around
Racosahic
7,29
x=
Figure 1: A toy illustration of im- balanced gradients at x = 0: the gradient of margin loss (22 2, is dominated by its -2 term, pointing to a suboptimal attack direction to- wards +2, where x is still correctly classified.
3.1 IMBALANCED GRADIENTS IN DNNS
The situation can be extremely complex for DNNs with high-dimensional inputs, as imbalanced gradients can occur at each input dimension. It thus requires a metric to quantitatively measure the degree of gradient imbalance. Here, we propose such metric named Gradient Imbalance Ratio (GIR) to measure the imbalance ratio for a single input x, which can then be averaged over multiple inputs produce the imbalance ratio for the entire model.
Definition of GIR. To measure the imbalance ratio, we focus on the input dimensions that are dominated by one loss term. An input dimension æ; is dominated by loss term (e.g. Zmmzz) means that 1) the gradients of loss terms at I; have different directions (V Zmaz Vz, (-zy) ) < 0), and 2) the gradient of the dominant term is larger (e.g. V Zmarl > I-,((=) According to the dominant term, we can split these dimensions into two subsets x., and x,, where x,, are dominated by the Amar term, while X,, are dominated by the -yy term. The overall dominance effect of each loss term be formulated r1 ||Vx,, (Zmaz zn) and r2 V (Zmax zy)ll Here, use the L1 norms instead of Lo Lo-norms (f.e. the number of dominated dimensions) to also take into consideration the gradient magnitude. To keep the ratio larger than 1, GIR is computed as:
GIR = max Fi
(I)
GIR is defined by the ratio between the norms of the gradients of two groups:
Note that the GIR metric is not a general measure of imbalance. Rather, it is designed only for assessing gradient imbalance for adversarial robustness evaluation. GIR focuses specifically on the
3

---

Under review as a conference paper at ICLR 2021
Gradient libalance Ratio
20.0
Attack Rate
50
17.5
15.0
Margin Loss
1
12.5
10.0
35
0
7.5
30
5.0
2.5
0.0
Natural
Madry
Sence
FeuScatter
Bilatersl
Advinter
Madry
Sense
FeuScatter
Biateer!
Advinterp
Figure 2: (a): Gradient imbalance ratio of 5 models. (b): Attack success rate of PGD-20 with different
losses. (c): The margin loss of the AdvInterp defense model on points x* X + a sign V (<Zz))))) where x is a natural sample and sign( vx( (x(-2z experiments are conducted on test images of CIFAR-10.
(-zy)) zy)) is the signed gradient of loss term Zy. All these
imbalanced input dimensions, and uses the L1 norm to also take into account the influence of these dimensions to the final output. The ratio reflects how far away the imbalance towards one direction than the other.
GIR of both Naturally- and Adversarial-trained DNNs. With the GIR metric, we next investigate 6 DNN models including naturally-trainet (Natural) model and 5 adversarially-trained models using standard adversarial training (Madry et al., 2018) (SAT), sensible adversarial training (Kim and Wang. 2020) (Sense), feature ceattering-based adversarial training (Zhang and Wang] 2019) (FeaScatter), bilateral adversarial training (Wang and Zhang. 2019) (Bilateral), and adversarial interpolation training (Zhang and Xu 2020) (AdvInterp). We present these defense models here because they represent different levels of gradient imbalance (a complete analysis of more models can be found in Appendix [D. Natural, SAT and Sense are WideResNet-34-11 models, while others are WideResNet-28-10 models. We train Natural and SAT following typical settings in (Madry et al.) 2018) while others use their officially released models. We compute the GIR scores of the models based on 1000 randomly selected test samples, and show them in Figure 2a One major observation that some defense models can have a much higher imbalance ratio than either naturally-trained or SAT model. This confirms that gradient imbalance does exist in DNNs, and some defenses tend to train the model have highly imbalanced gradients. We will show, Section 4 that this situation of imbalanced gradients may cause highly overestimated robustness when evaluated using traditional PGD attack.
Imbalanced Gradients Reduce Attack Effectiveness. When there are imbalanced gradients, the attack can be pushed by the dominant term to produce weak attacks, and the non-dominant term alone can lead to more successful attacks. To illustrate this, in Figure 2b we show the success rates of PGD attack on the above defense models (Natural has zero robustness against PGD) with different losses: CE loss, margin loss, and the two individual margin terms. We consider 20-step PGD (PGD-20) attacks with step size e/4 and = 8/255 on all CIFAR-10 test images. Intuitively, the two margin terms could lead to less effective attacks, they only provide partial information about the margin loss. This indeed the case for the low gradient imbalance model SAT. However, for highly imbalanced models Sense, FeaScatter, Bilateral and AdvInterp, attacking the Zmazz term produces even more powerful attacks than attacking the margin loss. This indicates that the gradient of the margin loss is shifted by the dominant term (e.g. -zg in this case) towards less optimal direction, which inevitably causes less powerful attacks. Compared between attacking CE loss and attacking -zy, they achieve a very close performance on imbalanced models. This shows stronger dominant
effect of -ny in CE loss (fcn((,)) y) log Dog(En e² Zy). It is worth mentioning that, while both GIR and this individual term-based test can be used to check whether there are significantly imbalanced gradients in a defense model, GIR alone cannot fully reflect the attack success rate. Figure 2c shows an example of how the -Z term leads the attack to suboptimal direction: the margin Toss is flat at the (-y) direction, yet increases drastically at an opposite direction. In this example, the attack can actually succeed if it increases (rather than decreases) Zy.
Gradients be Balanced by Attacking Individual Loss Terms. Here, we show that, interest- ingly, imbalanced gradients can be balanced by attacking the non-dominant term. Consider the AdyInterp model tested above example, the dominant term is -zy- Figure illustrates the
4

---

Under review as a conference paper at ICLR 2021
Gradient imbalance Ratio
Gradient Imbalan Ratio
Gradient Imbalance
1.
1.0
10
18
Perturbation Step
Perturbation Step
Perturbation Step
(a) PGD-20 with loss Zy
(b) PGD-20 20 with loss
(c) PGD-20 with loss Myy
Figure 3:
Changes in gradient imbalance ratio when apply PGD-20 (E
255 attack with the margin
loss (a), only the Zmar term (b), or only the -z, term (c), the AdvInterp model for 5 CIFAR-10 test images. The imbalance ratio is effectively reduced by attacking a single Zynax term.
GIR values of 5 randomly selected CIFAR-10 test images by attacking them using PGD-20 with different margin terms or the full margin loss. As can be observed that, for all three losses, the GIRs are effectively reduced after the first few steps, However, only the non-dominant term Zmac manages to stably reduce the imbalance ratio around 2. This indicates optimizing the individual terms separately can help avoid the situation of imbalanced gradients and the attack can indeed benefit from more balanced gradients (see the higher success rate of Zmaz in Figure 2b).
3.2 MARGIN DECOMPOSITION ATTACK
The above observations motivate to exploit the individual terms in the margin loss so that the imbalanced gradients situation can circumvented. Specifically, we propose Margin Decomposition (MD) attack that decomposes the attacking process with margin loss into two stages: 1) alternately attacking the two individual terms (e.g. Zmax or -Z,) at different restarts; then 2) attacking the full margin loss. Formally, our MD attack and its loss functions in each stage is defined as follows:
Xk+1
[x( + a sign( V 2 (xx: v))),
Zman
if k and mod =0
(x(x))))
-% Zyy
if and r mod
Zmax
Zy
if 2
where, II is the projection operation that projects the perturbed sample back within e-ball, k e {I....,K) K} is the perturbation step, r (I....,n] n.. is the r-th restart, mod is the modulo operation for alternating optimization, and f% defines the loss function used at the k-th step and r-th restart.
K The first The loss function switches from the individual terms back to the full margin loss at step 2 stage exploits individual loss terms rebalance the imbalanced gradients, while the second stage ensures that the final objective (e.g. maximizing the classification error) is achieved. Note that, not all defense models have the imbalanced gradients problem. A model susceptible to imbalanced gradients if there is substantial difference between robustness evaluated by PGD attack and that by our MD attack. In addition, to help escape the flat loss landscape observed in Figure [c] we initialize the perturbation in the first stage by perturbing one step with size 2 along the opposite direction of the other loss terms that are left unexplored.
We also propose a Margin Decomposition Multi- Marti-Tarrgeed (MDMT) attack, of our MD attack. The loss terms used by MDMT at different attacking stages are defined as follows:
version
Zt
if l < and r mod 2 = if k and mod 2 =
(xK, v)
-Zy
2t
Zy
if k
where, 2, is the logits of the target class * y- Like the MT attack, MDMT will attack each possible target class one at a time, then select the strongest adversarial example at the end. That is, the target different target class at each restart. The complete algorithms of
class ff y will be switched MD and MDMT can be found in Appendix A and an ablation study can be found in Appendix G
Initialization Perspective Interpretation of MD Attacks. Previous works have shown that random or logits diversified initialization are crucial for generating strong adversarial attacks (Madry et al. 2018 Tashiro et al. 2020). Compared to random or logits diversified initialization, MD attacks
5

---

Under review as a conference paper at ICLR 2021
can be interpreted as type of adversarial initialization, i.e., initialize at the ad sub-directions defined by the two terms of the margin loss. Moreover, rather than a single step of initialization, our MD attacks iteratively explore the optimal starting point during the entire first attacking stage.
EXPERIMENTS
We apply our MD attacks evaluate the robustness of 12 state-of-the-art defense models. We focus on adversarial training models, which are arguably the strongest defense approaches to date (Athalye et al. 2018 Croce and Hein| 2020), All the models are WideResNet variants (Zagonuyko and Komodakis 2016) and are trained against perturbation e 8/255 on CIFAR-10 10. For each defense model, we either download their shared models or retrain the models using the official implementations, unless explicitly stated. Further details about the models can be found in Appendix We apply current state-of-the attacks and our MD attacks to evaluate the robustness of these models in white-box setting.
Baseline Attacks and Settings. Following the current literature, we consider 6 existing attacks: 1) FGSM, 2) PGD. 3) Lix. version of CW attack (Madry et al. 2018 Wang et al. 2019). 4) MultiTargeted (MT) attack and two concurrently proposed attackss 5) AutoAttack (AA), and 6) Output Diversified Initialization (ODI). The evaluation is done under the same maximum perturbation training. For AA and ODI, we use the official implementation and parameter setting. For regular iterative attacks, we set the step size to a c/4 and the total perturbation steps to K = 40. For our MD and MDMT,
8/255 for
use a large step size a = 2 e in the first stage for better exploration and a = e/4 in the second stage to ensure a stable optimization for the final objective. For regular attacks PGD, CW and our MD, we use random restarts, while for more powerful attacks ODI, MT and MDMT, we use 20 restarts (MT attacks require more restarts to explore multiple target classes). A parameter analysis of our MD attack can be found in Appendix[H] H Adversarial robustness measured by the model accuracy on adversarial examples crafted by these attacks on CIFAR- 10 test images.
4.1 EVALUATION RESULTS
Table rreports the full evaluation result, where RST, UAT and TRADES are the top 3 best defenses. The SAT defense demonstrates 45% robustness consistently against either PGD stronger attacks such as MT, AA, ODI and our MD attacks. This indicates that SAT does not have imbalanced gradients and indeed brings consistent robustness, which is in line with other studies about SAT (Athalye et al. 2018 Croce and Hein 2020 Uesato et al.] 2018). While the rest 11 defense models are all developed based on SAT, they exhibit quite different robustness. Only 4 defenses including RST, UAT, TARDES and MART are indeed improved over SAT, while the other 7 defense models are actually not robust as SAT, according our MD or MDMT attacks. For the 4 improved defenses, their PGD robustness (e.g. robustness evaluated by PGD attack) can still be reduced by stronger attacks MT, AA, ODI or MD attacks. Considering that their robustness drops against our MD) attacks are within 5%, their drops may be caused by sufficient explorations such as more random restarts or better initialization rather than imbalanced gradients. Indeed, MT, AA, and ODI with more random restarts, multiple target classes, and better initialization can also reduce their robustness to the same level as our MD attacks.
Out of the 7 unimproved defenses, our MDMT attack can reduce the PGD robustness of 6 models (e.g. MMA, Bilateral, Ady-Interp, FeaScatter, Sense, and JARN-ATII) by at least 9%. On all 7 unimproved defenses, our MD attacks are always the most effective attacks compared to either classic attacks FGSM, PGD, CW, more recent attacks MT, AA and ODI. Note that, for 4 (e.g. MMA, Bilateral, Ady-Interp, and Sense) out of the 7 unimproved defenses, even state-of-the-arr attacks MT or AA evaluate them be more robust than SAT, which is not necessarily the case according to our MD attacks. Particularly, against the MT attack, the robustness of SAT is 45.34% while the robustness of Bilateral, Ady-Interp and Sense are 55.07% 61. 61.22% and 46.22% respectively. For the MMA defense, AA attack evaluates its robustness to be 45.69%, which is slightly higher than SAT's 45. 5..66% However, under our MD attacks, all models show much lower robustness than SAT (3%-10% lower). Next, we will investigate the imbalanced gradients problem in the unimproved defenses.
The PGD results with grid searched step size are reported in Appendix !| where it shows larger step size can help PGD when there are imbalanced gradients, yet still far less effective than our MI)

---

Under review as a conference paper at ICLR 2021
Table Robustness (%) of defense models evaluated by different attacks. The attacks are divided into 2 groups: 1) traditional attacks for robustness evaluation and our MD (column 3-6); and 2) more recent attacks and our MDMT (column 7-10). The defenses are also divided into 2 groups: 1) SAT or improved defenses (top rows); and 2) those that are not improved over SAT (bottom rows). Results in (+) in the MDMT column show the robustness decrease compared to the PGD attack.
Defense
Clean
FGSM
PGD
CW
MD
MT
AA
ODI
MDMT
RST
89.69
69.60
62.09
60.87
60.17
59.80
59.66
59.93
59.86 (-2.23)
UAT
86.46
68.31
61.08
62.11
59.36
56.72
56.94
57.98
56.65 (-4.43)
TRADES
84.92
60.87
55.00
53.69
53.10
52.67
53.18
52.68
52.78 (-2.22)
MART
83.09
61.43
56.10
53.02
51.84
51.12
51.05
51.15
51.07 (-5.03)
SAT
86.83
56.88
45.94
45.73
45.64
45,34
45.17
45.26
45.25 (-0,69)
Dynamic
85.35
55.19
46.36
45.53
43.93
42.75
42.88
43.03
42.69 (-3.67)
MMA
84.62
61.85
51.09
52.05
45.63
42.62
45.69
43.00
41.92 (-9.17)
Bilateral
90.73
71.10
60.95
57.82
39.82
55.07
37.96
38.65
37.21 (-23.74)
Ady-Interp
90.25
77.94
72.48
67.92
45.33
61.22
38.58
41,43
37.59 (-34.89)
FeaScatter
89.98
77.40
68.64
57.10
43.12
43.10
38.79
39.61
36.86 (-31.78)
Sense 91.51
72.71
59.86
57.67
40.64
46.22
36.10
38.15
35,25 (-24.61)
JARN-ATI
81.96
61.48
42.50
27.46
15.03
16.01
30.11
14.90
14.60 (-27.90)
attacks. A comparison of our MDMT attack to the 4 individual attacks in the AA ensemble can be found in Appendix [ where it shows our MD attack is superior to any of the 4 individual attacks 6 out of the 12 tested defense models, and our MD attack is the best across all defense models. More
evaluation results on defense models trained on CIFAR- 100 can be found in Appendix K
Efficiency Analysis. We compare the efficiency of the two best attacks identified in Table 1 our MDMT attack and AA attack. Ensemble attacks like AA are generally more powerful than standalone attacks, yet are also more time-consuming. To test this, here we also include the AA AA-Pattack which is an updated version of AA with an ensemble of 6 different attacks. We repeat the attack for 5 times on the entire CIFARIO 10 test set and report the average time cost in Table The time cost is measured with respect to single 2080TI GPU. As can be observed, our MDMT attack is at least 8 times more efficient than AA. Having two more attacks in the ensemble, AA+ is notably more cine-conuuming that the AA attack. The adaptation of our method to existing attacks only needs to replace the loss used the first half the iteration steps to the decomposed loss terms following Equation 2 thus does not increase the time complexity of the original attacks.
Table 2: Average time cost (in hours) of MTMD, AA and AA+ attacks on defense models Adv-Interp. FeaScatter and Sense over 5 repeats on the entire CIFAR-I0 test set. The best results are in bold.
Defense
MDMT
AA
AA+
Ady-Interp
2.01hrs
16.36hrs
20.7
FeaScatter
1.97hrs
15.97hrs
20.44hrs
Sense
2.28hrs
18.35hrs
23.44hrs
4.2 DEFENSE TECHNIQUES THAT MAY CAUSE IMBALANCED GRADIENTS
Here, we focus on 6 unimproved (compared to SAT) defenses: MMA, Bilateral, Ady-Interp. FeaScat- ter, Sense, and JARN-ATI, Their PGD-evaluated robustness has been reduced for > 9% by our MDMT attack.
Label Smoothing Causes Imbalanced Gradients. The PGD robustness of Bilateral, FeaScatter, and Ady-Interp decrease the most (e.g. 23% 34%) against our MDMT attack. This indicates that these defenses may have caused imbalanced gradients, also indicated by their high GIR values in Figure 2a] All three defenses use label smoothing as part of their training scheme to improve adversarial training, which we suspect is one common cause of imbalanced gradients. Given sample x with label y, label smoothing encourages the model to learn an uniform logits or probability distribution over classes j # y. This tends to smooth out the input gradients of x with respect to these
https//gihuu..cmm/rra3 /aauo

---

Under review as a conference paper at ICLR 2021
Table 3: Robustness (%) of WideResNet-33-100 models trained with/without label smoothing.
Vanilla
Label Smoothing
GIR
Defense
FGSM
PGD
MD
SAT
56.88
46.47
45.71
+ Label Smoothing
59.10
51.15
44.54
Natural
Madry
Natural
26.41
0.00
0.00
Label Smoothing
48.09
10.86
0.00
Figure 4:
Gradient Imbalance Ratio (GIR) of models trained with/without label smoothing.
classes, resulting in smaller gradients. In order confirm label smoothing indeed causes imbalanced gradients, we train a WideResNet-34-11 10 model using natural training ("Natural") ) and SAT with or without label smoothing (smoothing parameter 0.5). We report their robustness in Table and show their gradient imbalance ratios (GIRs) in Figure 4 According to GIRs, adding label smoothing into the training process immediately increases the imbalance ratio, especially in natural training. The PGD robustness of the naturally-traine
model also "increases" to 10.86%, which is still 0% under our MD attack. Using smoothed labels in SAT defense also "increases" PGD robustness by almost 5%. which in fact, decreases by 1%. These evidences confirm that label smoothing indeed causes imbalanced gradients, leading to overestimated robustness if evaluated by regular attacks like PGD. Interestingly. it appears that adversarial training can inhibit moderately the imbalanced gradients problem of label smoothing. This is because the adversarial examples used for adversarial training are specifically perturbed to the j # classes, thus helping avoid uniform logits over classes # to some extent.
Other Defense Techniques that may Cause Imbalanced Gradients. The other 3 unimproved defenses MMA. Sense and JARN-ATI adopt different defense techniques to improve robustness. MMA is a margin- defense that maximizes the shortest successful perturbation for each data point. MMA only perturbs correctly classified clean examples, and the perturbation stops immediately at misclassification (into j & class). In other words, MMA focuses on examples that are around the decision boundary (e.g. Zmaz zy)) between class y and all other classes j * y. During training, the decision boundary margin is maximized by pulling the boundary away from these examples. This process tries to maximize the distance to the closest decision boundary (e.g. towards the weakest class) and finally results in equal distances to all other classes. This tends to generate a uniform prediction over classes j * y, similar effect of label smoothing, and causes imbalanced gradients.
Similar to MMA, Sense perturbs training examples until a certain loss threshold is satisfied. While in MMA the threshold is misclassification, in Sense, it the loss value with respect to probability (e.g. Py = 0.7). This type of training procedures with a specific logits or probability distribution regularization has caused the imbalanced gradients problem for both MMA and Sense. Note that, Sense causes much severe imbalanced gradients than MMA. We conjecture it is because optimizing over probability threshold is much easier than moving the decision boundary.
JARN-ATI is also a regularization-base adversarial training method. Different from MMA or Sense, it regularizes the model's Jacobian (e.g. input gradients) resemble natural training images. Such an explicit input gradients regularization tends to reduce the input gradients to a much smaller magnitude and only keep the salient part of input gradients. The input gradients associated with other j + y classes will be minimized to cause an imbalance to that associated with class y. This has caused PGD to produce 27.90% more robustness than our MDMT attack. Note that, even the recent AA attack still produces 15.51 % overestimated robustness compared to our MDMT.
Correlation between GIR and Robustness. According the GIR scores shown in Figure 2a and Figure 6 (Appendix (F). models exhibit high GIR scores (e.g. Ady-Interp. FeaScatter and Bi Bilateral) generally more prone imbalanced gradients and potentially more vulnerable to our MD) attacks. However, GIR is not a measure for robustness nor should be used as an exact metric to determine whether one defense is more robust than the other. For example, MART demonstrates higher GIR score than Sense or JARN-ATI, ATT, however, according to our MDMT attack, it is 15.82% and 36.47% more robust than Sense and JARN-AY ATI, respectively. This is because the GIR score of model only measures the gradient situation of the model at its current state, which could decrease during the attack shown in Figure 3 and 5 (Appendix C). Our MDD attacks iteratively exploit and circumvent imbalanced gradients during the first attacking stage, thus can produce reliable robustness evaluation at the end.
8

---

Under review as a conference paper at ICLR 2021
4.3 AN ATTACK VIEW OF IMBALANCED GRADIENTS
As shown in Table T recent attacks ODI and AA are more effective than traditional attacks PGD and CW against imbalanced gradients. Here, we provide some insights into why these techniques are effective against imbalanced gradients. We consider attacking AdvInterp as an example and show how the gradient imbalance ratio (GIR) changes in different attacking processes.
Logits Diversified Initialization Helps Avoid Imbalanced Gradients. ODI randomly initializes the perturbation by adding random weights to logits at its first 2 steps. The random weights change the gradients' size, thus can also mitigate imbalanced gradients, as shown in Figure[5ajin Appendix C However, initialization only helps the first 2 steps, and the imbalance ratio still jumps in the following steps. Our attack provides more direct and efficient exploration of imbalance gradients, thus can maintain a low imbalance ratio even after the first few steps (see Figure 55 in Appendix C shown in Table our MDMT attack is consistently more effective than ODI.
As also
Exploration Beyond the c-ball Helps Avoid Imbalanced Gradients. AA is an ensemble of four attacks: two Auto-PGD attacks and two existing attacks FAB and Square. By inspecting the individual attacks, we found that the most effective method is FAB. FAB first finds a successful attack using unbounded perturbation size (e.g. > c), then minimizes the perturbation to be within the e-ball. As shown in Figure 5b in Appendix C the first few steps of exploration outside the e-ball can effectively avoid imbalanced gradients. This also why our MD attacks use large step size in the first stage. However, the imbalance ratio tends to increase when FAB attempts to minimize the perturbation (steps 10 16). We believe FAB can be further improved following our decomposition strategy.
Imbalanced Gradients are not Easily Detected or Circumvented by Existing Methods. We also show, in Appendix B that defense models with imbalanced Gradients can still pass the five checking rules of obfuscated gradients, and that many times restarts with random initialization or momentum method does not help escape imbalanced gradients in Appendix [ This makes imbalanced gradients more subtle and should be carefully checked to avoid overestimated robustness.
Black-box Attacks can be Improved by Cireumventing Imbalanced Gradients. Here we show gradient estimation based black-bos attacks can also benefit from our MD method when there are imbalanced gradients. We take SPSA for SPSA. This version of SPSA is denoted as SPSA+MD. For both SPSA and SPSA+MD, we use the same batch size of 8192 with 100 iterations, and run 1000 randomly selected CIFAR-10 test images. The attack success rates on Adv-Interp. FeaScatter and Sense models are reported in Table 4 Compared to SPSA, SPSA+MD can lower the robustness by at least 5.9%. This indicates that imbalanced gradient also has a negative impact on back-box attacks, and our method can be easily applied to produce more queries-efficien and successful black-box attacks.
example, and use the two-stage losses of our MD attack
Table Attack success rate (ASR) of the SPSA attack with or without our MD on three CIFAR-10 10 defense models. The ASRs are tested on the entire CIFAR-10 test set. The best results are in bold.
Attack
Adv-Interp
FeaScatter
Sense
SPSA
24.80% 40.30%
28.29 45.60 %
37.90
SPSA+MD
48.80%
CONCLUSION
In this paper, we identify Imbalanced Gradients, new situation where traditional attacks such as PGD can fail and produce overestimated adversarial robustness. We proposed a new metric to investigate the imbalanced gradients problem in current defense models. We also proposed new attack called Margin Decomposition (MD) attack to leverage imbalanced gradients via a two-stage attacking process. By evaluating 12 state-of-the of-the-art defense models, find that 6 of them are susceptible to imbalanced gradients and their PGD robustness suffers a significant drop against our MD attacks. We identified of possible causes of imbalanced gradients, and effective countermeasures. Our results indicate that future defenses should avoid causing imbalanced gradients to obtain more reliable adversarial robustness.

---

Under review as a conference paper at ICLR 2021
REFERENCES
Jean-Baptiste Alayrac, Jonathan Uesato, Po-Sen Huang, Alhussein Fawzi, Robert Stanforth, and Pushmeet Kohli. Are labels required for improving adversarial robustness? In NeurIPS, 2019.
Maksym Andriushchenko, Francesco Croce, Nicolas Flammarion, and Matthias Hein. Square attack: query-efficient black-box adversarial attack via random search. urXiv:/9/2.00044 2019.
Anish Athalye, Nicholas Carlini, and David Wagner. Obfuscated gradients give false sense of security: Circumventing defenses to adversarial examples. In ICMI, 2018.
Nicholas Carlini and David Wagner. Towards evaluating the robustness of neural networks. In S&P, 2017.
Yair Carmon, Aditi Raghunathan, Ludwig Schmidt, John C. Duchi, and Percy Liang. Unlabeled data improves adversarial robustness. In NewrlPS, 2019.
Alvin Chan, Yi Tay, Yew Soon Ong, and Jie Fu. Jacobian adversarially regularized networks for robustness. ICLR, 2020.
Jinghui Chen, Dongruo Zhou, Jinfeng Yi, and Quanquan Gu. A frank-wolfe framework for efficient and effective adversarial attacks. arXiv preprint arXiv:1811.10828 2018a.
Pin-Yu Chen, Yash Sharma, Huan Zhang, Jinfeng Yi, and Cho-Jui Hsieh. Ead: elastic-net attacks to deep neural networks via adversarial examples. In AAAI, 2018b.
Francesco Croce and Matthias Hein. Minimally distorted adversarial examples with fast adaptive boundary attack. 0rXiv:1907.02044 2019.
Francesco Croce and Matthias Hein. Reliable evaluation of adversarial robustness with an ensemble of diverse parameter- free attacks. arXiv:2003.01690, 2020.
Nilaksh Das, Madhuri Shanbhogue, Shang- Tse Chen, Fred Hohman, Siwei Li, Li Chen, Michael E Kounavis, and Duen Horng Chau. Compression to the rescue: Defending from adversarial attacks across modalities. In KDD, 2018.
Gavin Weiguang Ding, Yash Sharma, Kry Yik Chau Lui, and Ruitong Huang. Max-margin adversarial (MMA) training: Direct input space margin maximization through adversarial training. arXiv preprint arXiv: 18/2.0///2.026333 2018.
Yinpeng Dong, Fangzhou Liao, Tianyu Pang, Hang Su, Jun Zhu, Xiaolin Hu, and Jianguo Li. Boosting adversarial attacks with momentum. In CVPR, 2018.
Logan Engstrom, Andrew Ilyas, and Anish Athalye. Evaluating and understanding the robustness of adversarial logit pairing. arXiv preprint arXiv: 10272, 2018.
Ian J. Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. In ICLR, 2015.
Sven Gowal, Jonathan Uesato, Chongli Qin, Po-Sen Huang, Timothy A. Mann, and Pushmeet Kohli. An alternative surrogate loss for agd-based adversarial testing. arXiv preprint 2019.
Shixiang Gu and Luca Rigazio. Towards deep neural network architectures robust to adversarial examples. arXiv preprint arXiv:1412.5068, 2014.
Chuan Guo, Mayank Rana, Moustapha Cisse, and Laurens van der Maaten. Countering adversarial images using input transformations. In ICLR, 2018.
Warren He, Bo Li, and Dawn Song. Decision boundary analysis of adversarial examples. In ICLR, 2018.
Dan Hendrycks, Kimin Lee, and Mantas Mazeika. Using pre-training can improve model robustness and uncertainty. In ICML, 2019.
10

---

Under review as a conference paper at ICLR 2021
Jungeum Kim and Xiao Wang. Sensible adversarial learning, 2020. URL URL.attp:://ppeeeeeier Seet/foruumid=zz1E RVKwr
Alexey Kurakin, Ian Goodfellow, and Samy Bengio. Adversarial machine learning at scale. ICLR, 2017.
Fangzhou Liao, Ming Liang. Yinpeng Dong, Tianyu Pang, Xiaolin Hu, and Jun Zhu. Defense against adversarial attacks using high-level representation guided denoiser. CVPR, 2018
Qi Liu, Tao Liu, Zihao Liu, Yanzhi Wang. Yier Jin, and Wujie Wen. Security analysis and en- hancement of model compressed deep learning systems under adversarial attacks. In ASPDAC, 2018.
Xingjun Ma, Bo Li, Yisen Wang, Sarah M Erfani, Sudanthi Wijewickrema, Grant Schoenebeck, Dawn Song, Michael E Houle, and James Bailey. Characterizing adversarial subspaces using local intrinsic dimensionality. In ICLR, 2018.
Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. In ICLR, 2018.
Marius Mosbach, Maksym Andriushchenko, Thomas Alexander Trost, Matthias Hein, and Dietrich Klakow. Logit pairing methods can fool gradient-basee attacks. arXiv preprint arXiv:1810.1200 1244. 2018.
Nicolas Papernot, Patrick McDaniel, Xi Wu, Somesh Jha, and Ananthram Swami. Distillation as a defense adversarial perturbations against deep neural networks. In S&P, 2016.
Nicolas Papernot, Patrick McDaniel, Ian Goodfellow. Somesh Jha, Z Berkay Celik, and Ananthram Swami. Practical black-box attacks against machine learning. In Asia CCS, 2017.
Adnan Siraj Rakin, Jinfeng Yi, Boqing Gong, and Deliang Fan. Defend deep neural networks against adversarial examples via fixed and dynamic quantized activation functions. arXiv preprint arXiv: 1FX,:0//07.06714 2018.
Leslie Rice, Eric Wong, and Zico Kolter. Overfitting in adversarially robust deep learning. arXiv preprint arXiv: 2020.
Andrew Slavin Ross and Finale Doshi-Velez. Improving the adversarial robustness and interpretability of deep neural networks by regularizing their input gradients. In AAAI, 2018
Pouya Samangouei, Maya Kabkab, and Rama Chellappa. Defense-GAN: Protecting classifiers against adversarial attacks using generative models. In ICLR, 2018.
Ilya Sutskever, James Martens, George Dahl, and Geoffrey Hinton. On the importance initialization and momentum in deep learning. In ICML, 2013.
Christian Szegedy, Wojeiech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus. Intriguing properties of neural networks. In ICLR, 2014.
Yusuke Tashiro, Yang Song, and Stefano Ermon. Output diversified initialization for adversarial attacks. arXiv:2003.06878 2020.
Florian Tramèr, Alexey Kurakin, Nicolas Papernot, Ian Goodfellow, Dan Boneh, and Patrick Mc- Daniel. Ensemble adversarial training: Attacks and defenses. In ICLR, 2018.
Florian Tramèr, Alexey Kurakin, Nicolas Papernot, Ian Goodfellow, Dan Boneh, and Patrick D. McDaniel. Ensemble adversarial training: Attacks and defenses. In ICLR, 2018.
Jonathan Uesato, Brendan O' O'Donoghue, Pushmeet Kohli, and Aäron van den Oord. Adversarial risk and the dangers of evaluating against weak attacks. In ICML, 2018
Jianyu Wang and Haichao Zhang. Bilateral adversarial training: Towards fast training of more robust models against adversarial attacks. In ICCV, 2019.
Yisen Wang, Xingjun Ma. James Bailey, Jinfeng Yi, Bowen Zhou, and Quanquan Gu. On the convergence and robustness of adversarial training. In ICML, 2019.
11

---

Under review as a conference paper at ICLR 2021
Yisen Wang, Difan Zou, Jinfeng Yi, James Bailey, Xingjun Ma, and Quanquan Gu. Improving adversarial robustness requires revisiting misclassified examples. In ICLR, 2020.
Dongxian Wu, Shu-Tao Xia, and Yisen Wang. Adversarial weight perturbation helps robust general- ization. NeurIPS, 33, 2020.
Weilin Xu, David Evans, and Yanjun Qi. Feature squeezing: Detecting adversarial examples in deep neural networks. arXiv preprint arXiv:1704.01155, 2017.
Sergey Zagoruyko and Nikos Komodakis. Wide residual networks. In BMVC, 2016.
Haichao Zhang and Jianyu Wang. Defense against adversarial attacks using feature scattering- -aased adversarial training. In NeurlPS, 2019.
Haichao Zhang and Wei Xu. Adversarial interpolation training: simple approach for improving model robustness, 2020. /tLpttse///ppeneeii.e. http:///.we..eeeerooooc///aaaaaa
Hongyang Zhang, Yaodong Yu, Jiantao Jiao, Eric P. Xing. Laurent El Ghaoui, and Michael I. Jordan. Theoretically principled trade-off between robustness and accuracy. In ICML, 2019.
Tianhang Zheng, Changyou Chen, and Kui Ren. Distributionally adversarial attack. In AAAI, 2019.
12

---

Under review as a conference paper at ICLR 2021
A ALGORITHMS OF MT AND MDMT ATTACKS
Algorithm D and Algorithm 2 below describe the complete attacking procedure of our Margin Decomposition (MD) attack and its Multi- Multi-Targeted (MDMT) version.
Algorithm Margin Decomposition Attack
I: Input: clean sample x, label y. model f.
2: Output: adversarial example Xade
3: Parameters: Perturbation bound step size a, number of restarts n,, number of steps K.
4: Xade f-x X
5: for (1, n) do
6: Initialize x by one step of perturbation along the opposite direction of gradients.
for E {1, ..., K'
8: Update X2 by Eq. (2)
9:
if ((xadv) < {(xx) then
Nadv X6 11: end if
10:
12: end
13:
end for
14:
return Xadv
Algorithm 2 Margin Decomposition MultiTargeted attack
1: Input: clean sample x, class label y, class set T, model f.
2: Output: adversarial example Xade
3: Parameters: Perturbation bound PGD step size a, number of restarts n, number of steps K.
4: nr < [n/|T[], Xode f- x
5: for E [L....,ue] do
6: for E T do
7: Initialize Xo by one step of perturbation along the opposite direction of gradients.
8: for k E {l,..., K} do
Update X₁ by Eq. (??) 10: if (xxdv) < ((xx) then
9:
Xado X
12: end
end for
14:
end for
end for
16:
return Xado
B IMBALANCED GRADIENTS ARE DIFFERENT FROM OBFUSCATED GRADIENTS
Imbalanced gradients occur when one loss term dominating the attack towards suboptimal gradient direction, which does not necessarily block gradient descent like obfuscated gradients. Therefore, does not have the characteristics of obfuscated gradients, and can not be detected by the five checking rules for obfuscated gradients (Athalye al. 2018). Here, test all the five rules the four defense models that exhibited significant imba anced gradients: Adv-Interp. FeaScatter, Bilateral, and Sense. Note that all these models were trained and tested on CIFAR- 10 dataset.
One-step attacks perform better than iterative attacks. When gradients are obfuscated, iterative attacks more likely get stuck in local minima. To test this, compare the success rate of one-step attack FGSM and iterative attack PGD in Table 5 We see that PGD outperforms FGSM consistently on all the four defense models, i.e no obvious sign of obfuscated gradients.
Unbounded attacks do not reach 100% % success. Increasing distortion bound does not increase success. Larger distortion bound gives the attacker obfuscated, unbounded attack should reach 100% success rate. To test this, we run an "unbounded" PGD attack with 1. As shown in Table all models completely broken by this unbounded
ability attack. So, if gradients
13

---

Under review as a conference paper at ICLR 2021
attack. This again indicates that the overestimated robustness is caused by a different effect rather than obfuscated gradients.
Black-box attacks are better than white-box attacks. If a model is obfuscating gradients, it should fail to provide useful gradients a small neighborhood. Therefore, using a substitute model should be able to evade the defense, the substitute model was not trained to be robust small perturbations. To test this, we run black-box transferred PGD attack on naturally trained substitute models. We find that all four defenses robust to transferred attacks ("Transfer" in Table 5). We also attack the four defense models using gradient-free attack SPSA (Uesato et al. 2018). For SPSA, we use batch size of 8192 with 100 iterations, and run on 1000 randomly selected CIFAR-10 10 test images. We confirm that SPSA cannot degrade their performance. None of these results indicate obfuscated gradients.
Random sampling finds adversarial examples. Brute force random search within some e-ball should not find adversarial examples when gradient-baset attacks do not. Following (Athalye et al.) 2018), we choose 1000 test images on which PGD fails. We then randomly sample 10 points for each image from its E = 8/255-ball region, and check if any of them are adversarial. The results (e.g. "Random") ") shown in Table confirms that random sampling cannot find adversarial example when PGD does not.
All the above test results lead one conclusion that the robustness of the four defenses is not result of obfuscated gradients. This indicates that imbalanced gradients does not share the characteristics of obfuscated gradients, thus cannot be detected following the five test principles for obfuscated gradients. This makes adversarial robustness evaluation more difficult. Therefore, imbalanced gradients should be carefully addressed for more accurate robustness evaluation.
Table 5: Test of obfuscated gradients for four defense models that have significant imbalanced gradients following (Athalye et al. 2018): attack success rate (%) of different attacks. None of the above results indicates a clear sign of obfuscated gradients.
Defense
FGSM
PGD
Unbounded
Transfer
SPSA
Random
Ady-Interp Zhang and Xu 2020)
23.06
27.52
100.00
10.89
24.80
0.00
FeaScatter Zhang Wang
22.60
31.36
100.00
11.11
28.20
0.00
Bilateral (Wang and Zhang 2019)
28.90
39.05
100.00
9.23
36.00
0.00
Sense (Kim and wang 2020)
27.29
40.14
100.00
9.90
37.90
0.00
C CAN LOGITS DIVERSIFIED INITIALIZATION HELP CIRCUMVENT IMBALANCED GRADIENTS?
Figure 5 shows the GIR values of 5 randomly selected CIFAR-10 test images at the first 20 steps of ODI, FAB, or our MDMT attack. The FAB attack the most effective attack in the AA ensemble.
Gradient mmbalance Ratio
Gradient imbaa imbraannce Ratio
Gradient Imbralann ance Ratio
20
20
6
14
16
18
10 16 18
10
16 18
Perturbation
Step
Perturbation Step
Perturbation Step
(a) ODI
(b) FAB
(c) MDMT (Ours)
Figure 5: Gradient imbalance ratio at the first 20 steps of ODI (a), FAB (b) and our MDMT (c) attacks on the AdvInterp model for randomly selected CIFAR-10 test images.
14

---

Under review as a conference paper at ICLR 2021
D CAN RANDOM RESTART OR MOMENTUM HELP CIRCUMVENT IMBALANCED GRADIENTS?
As we discussed in Section 3 many times of random starts can potentially increase the probability of finding an adversarial example. Momentum method is another way to help escape overfitting
to local gradients (Sutskever et al.] (013). Here, we test whether random restart or momentum can help avoid imbalanced gradients. For random restart, we run 400-step PGD attack with 100 restarts (PGD 100x400). For momentum,
momentum iterative FGSM (MI-FGSM) (Dong et al., 2018 with 40 steps, 2 restarts and momentum 1.0. For both attacks, we set = 8/255 and step size = 2/255. We apply the two attacks on 1000 randomly chosen CIFAR 10 test images, and report the robustness
6able for the four defense models checked in Section B Compared to traditional PGD with 40 steps, the robustness can indeed be decreased by PGD L00 except Bilateral, an observation consistent with our analysis in Section 3 that more restarts can lower model accuracy. However, the robustness still highly overestimated compared that by our MDMT attack. This indicates that imbalanced gradients can exist in wide-spanned input regions, resulting in a low probability for random restart to find successful attacks. To our surprise, MI-FGSM performs even worse than traditional PGD. On three defense models (eg. Adv-Interp, FeaScatter, and Sense), it produces even higher robustness than PGD. This implies that accumulating velocity in the gradient direction can make the overfitting even worse when there are imbalanced gradients. This again confirms that the imbalanced gradients problem should be explicitly addressed to obtain more reliable adversarial robustness.
Table 6: Robustness (%) of four defense models that have significant imbalance gradients against PGD 100x 400 and MI-FGSM attack.
Defense
PGD
MDMT
PGD 400
MI-FGSM
Adv-Interp
72.48
37.59
70.70
73.25
FeaScatter
68.64
36,86
64.10
70.79
Bilateral
60.95
37.21
64.08
51.52
Sense
59.86
35.25
56.00
62.41
E 12 EXAMINED DEFENSE MODELS
We focus adversarial training models, which arguably the most effective defense models to date. The 12 selected defense models are as follows. The standard adversarial training (SAT) (Madry et al.) 2018) trains models on adversarial examples generated by PGD attack. Dynamic adversarial training (Dynamic) (Wang et al. 2019) trains on adversarial examples with gradually increased convergence quality. Max-Margin Adversarial training (MMA) (Ding et al. 2018) trains adversarial examples with gradually increased margin (e.g. the perturbation bound c). For MMA, we evaluate the released "MMA-32" model. Jacobian Adversarially Regularized Networks (JARN) adversarially regularize the Jacobian matrices, and can be combined with 1-step adversarial training (JARN-ATI) to gain additional robustness (Chan al.) 2020). For JARN, only evaluate the JARN-AT1 as JARN has already been completely broken in (Croce and Hein 2020). We implement JARN-ATI on the basis of their released implementation of JARN. Sensible adversarial training (Sense) (Kim and Wang] 2020) trains on loss-sensible adversarial examples (perturbation stops when loss exceeds certain threshold). Bilateral Adversarial Training (Bilateral) (Wang and Zhang 2019) trains on PGD adversarial examples with adversarially perturbed labels. For Bilateral, we mainly evaluate its released strongest model "R-MOSA-LA-S" Adversarial Interpolation (Ady-Interp) training (Zhang and Xu (020) trains on adversarial examples generated under an adversarial interpolation scheme with adversarial labels. Feature Scattering- based (FeaScatter) adversarial training ZZhang and Wang 2019) crafts adversarial examples using latent space feature scattering, then trains on these examples with label smoothing. TRADES (Zhang et al. 2019) replaces the CE loss of SAT by the KL divergence for a better trade-off between robustness and natural accuracy. Based on TRADES, RTS (Carmon al., 2019) and UAT (Alayrac et al. 2019) improve robustness by training with 10x more unlabeled data. Misclassification Aware Rsviasarial Training (MART) (Wang et al. 2020 further improves the above three methods with a misclassification aware loss function.
15

---

Under review as a conference paper at ICLR 2021
F GRADIENT IMBALANCED RATIO OF MORE DEFENSE MODELS
In this Section, we provide a complete analysis the gradient imbalance ratios (GIRs) of all 12 examined defense models and naturally trained model. The GIR values of these models are shown in F One immediate observation is that the GIR value of a defense model is positively correlated with its robustness drop against our MDMT attack in T Slightly imbalanced defense models SAT, TRADES and RST demonstrate minimum robustness drop, while the PGD-evaluate robustness of highly imbalanced defense models FeaScatter, Bilateral and AdvInterp can drop drastically against our MD attacks. This verifies that higher gradient imbalance can indeed causes more overestimated robustness by regular PGD attack.
Gradient Imbalance Ratio
20.0
15.0
10.0
5.0
a.0
Natural
Madry
MST
Dynomic
UAT
JARN AT1
Sence
MMA
MART
SeaStatter
Bliateral
Advinterp
Figure 6: Gradient imbalance ratios (GIRs) of 12 defense models and naturally trained model
("Natural"). All models are trained on CIFAR-10 dataset.
G ABLATION OF THE PROPOSED MD ATTACKS
In this section, we investigate the influence of three factors to our MD attack: 1) initialization method, 2) the second attacking stage, and 3) the stage ordering. We use AdvInterp as our target model, and conduct the following attack experiments on CIFAR- 10 test data.
Initialization Method. We compare the success rates of our MD attacks using random initialization versus the opposite direction initialization (see Algorithm [ and Algorithm 2). The results are reported in Table 7 As can be observed, the opposite direction initialization demonstrates a clear advantage over random initialization. Particularly, for MD attack, using opposite direction initialization can improve the attack success rate by 8%, while for MDMT attack, the success rate can also be improved.
The Second Attacking Stage. We further investigate the importance of the second stage of attacking with the full margin loss our MD attacks. Here, we fix the initialization method to the opposite direction initialization. The attack success rates with without the second stage are also reported in T We highlight that attacking the full margin loss via the second attacking stage can consistently increase the success rate. Especially for MD attack, 4.99% improvement can be achieved by the second attacking stage.
The Ordering of the Stages. To verify that the ordering of the two stages is suitable for MD attacks, we evaluate new version of our MD attacks with the two stages are switched: the first stage optimizes the full margin loss and the second stage explores the individual loss terms. The results are reported in Table (the last two columns). As can be observed, MD attacks become much less effective when the stages switched. This is because
16

---

Under review as a conference paper at ICLR 2021
Table 7: Attack success rates (%) of our MD and MDMT attacks with 1) different initialization methods, 2) with/without the second attacking stage, and 3) with/withhut stages being switched. Experiments are conducted on defense model AdvInterp and dataset CIFAR-IO.
Initialization
Second Attacking Stage
Switching Stage
Attacks
Random
Opposite
without
with
Yes
No
MD
46.32
54.67
49.68
54.67
48.41
54.67
MDMT
61.07
62.41
61.82
62.41
60.62
62.41
H PARAMETER ANALYSIS OF THE PROPOSED MD ATTACK
We further investigate the sensitivity of our MD attack to two parameters: 1) the number of perturba- tion steps, and 2) the step size. Here, we focus on the first attacking stage as the second stage is typical PGD attack, which has been thoroughly investigated in (Wang et al. 2019).
40. Number Steps for the First Stage. The total number of perturbation steps set
When we vary the perturbation steps of the first stage, the remaining steps will be given to the second stage. MD attack will reduce to the regular PGD attack if the perturbation steps of the first stage is set to O. Here, we vary the steps from 5 to 40 in a granularity of 5. The step size is set to 8/255 and 2/255 for the first and second attacking stage, respectively. The robustness of 4 defense models including Bilateral, Adv-Interp. FeaScatter and Sense are illustrated in Figure 7a As can be observed, the performance of our MD attack tends to drop at both ends, and the best performance is achieved at [20,30). 30.. Therefore, we suggest to simply use half of the perturbation steps for the first stage (e.g. switching to the second stage at the 2 -th step).
Step Size for the First Stage. We vary the step size used for the first stage from 2/255 to 16/255 in a granularity of 2/255. Following the above experiments, here we fix the number of steps in each stage to 20. The evaluated robustness (or model accuracy on the generated attacks) of defense models Bilateral, Ady-Interp and FeaScatter are illustrated in Figure 7b A clear improvement of using large step size in the first stage can be observed. Therefore, we suggest to use large step size for the first stage of exploration.
0.48
Bilateral
0.50
Bilateral
0.47
Sense
Sense
0.46
Adv Interp FeaScatter
0.48
FeaScatter
Adv-Interp
0.45
Accuracy (%)
Accuracy
0.44
0.43
0.44
0.42
0.41
0.40
10
15
20
25
30
40
2
12 Step size for the first stage
10
3.4
16
Number of steps for the first stage (/40)
(a)
(b)
Figure 7: Parameter analysis of MD attack: (a) the accuracies of 5 defense models under MD attacks with different number of perturbation steps in the first stage; (b) the accuracies of 5 defense models under MD attacks with different step sizes in the first stage.
I PARAMETER TUNING FOR PGD ATTACK
In this section, compare the attack success rates of the PGD attack with different step sizes. We vary the step size from 2/255 to 16/255 in a granularity of 2/255. Note that we only change the step size for the first half of the attacking iterations. The results on defense models including Bilateral, Ady-Int FeaScatter and Sense are reported in Table 8 As has also been confirmed in other works (Croce and Hein 2020
Tashiro al. 2020), larger step size does help obtain stronger
17

---

Under review as a conference paper at ICLR 2021
attacks, especially there imbalanced gradients (e.g. Adv-Interp, FeaScatter and Sense). However, these finetuned PGD attacks are still far less effective than our MD attacks (see Table T On SAT model, the best step size for PGD is 4/255, and larger step size than 4/255 even harms the attack.
Table 8: Adversarial robustness (%) of PGD attack with different step sizes on defense models Ady-Interp, FeaScatter, Sense and SAT trained on CIFAR-10. The results are computed on the entire CIFAR-10 test set. The lowest robustness (i.e. strongest attack) of each defense model is highlighted in bold.
Defense
2/255
4/255
6/255
8/255
10/255
12/255
14/255
16/255
Ady-Interp
72.48
72.20
71.87
71.24
70.52
69.34
67.41
65.20
FeaScatter
68.64
67.59
66.75
66.59
64.69
63.66
61.96
59.78
Sense
59.86
58.86
58.11
57.13
56.61
55.44
54.47
53.22
SAT
45.94
45,90
46.02
46.15
46.40
46.75
47.25
47.74
J COMPARISON TO THE FOUR INDIVIDUAL ATTACKS IN AUTOATTACK
In this section, we compare the model robustness evaluated by the individual attacks in the AA ensemble with our MD attacks. These experiments follow the same setting in Section 4 The results are shown in Table 9 As can be observed, our MDMT attack demonstrates a superior performance across all the defense models. Moreover, our MD attack which as efficient PGD attack can even achieve better performance than all individual attacks on out of 12 models.
Table 9: Attack success rates (%) of the 4 individual attacks (column 2-6) in AA attack and our MD attacks (column 6-7). The best results highlighted in bold. The second best results highlighted in underline
Defense
APGDCE
APGD DLR
FAB
Square
MD
MDMT
RST
61.47
60.64
60.62
66.63
60.17
59.86
UAT
59.86
62.03
58.20
66.37
59.36
56.65
TRADES
55.08
54.04
53.82
59.48
53.10
52.78
MART
55.52
52.51
51.55
57.45
51.84
51.07
SAT
46.40
46.56
46.38
53.13
45.64
45.25
Dynamic
45.81
45.86
43.64
53.49
43.93
42.69
MMA
49.40
50.18
47.38
55.48
45.63
41.92
Bilateral
58.26
43.11
41.36
59.07
39.82
37.21
Adv-Interp
69.36
49,43
40.60
66.87
45.33
37.59
FeaScatter
62.03
48.96
40.84
59.12
43.12
36.86
Sense
54,80
48,41
38.88
61.31
40.64
35.25
JARN-ATI 37.25
67.55
67.40
75.32
15.03
14.60
Table 10: Robustness (%) of defense models trained on CIFAR100 data against PGD, AA and our MDMT attacks. The best results (lowest evaluation robustness) are highlighted in bold.
Defense
PGD
AA
MDMT
AT-AWP (Preact ResNet-18)
30.70
25.35
25.37
AT-PT (WideResNet-344 34-)11
33.5
28.42
28.45
AT-ES (Preact ResNet-18)
28.1
19.07
19.12
K EVALUATION RESULTS ON CIFAR- 100 DATASET
Here, MD attacks 3 defense models trained CIFAR-100 dataset: 1) Adversarial
Training with Adversarial Weight Perturbation (AT-AWP) (Wu et al. 2020), 2) Adversarial Training
with Pre- Training (AT-PT) (Hendrycks al. 2019), and 3) Adversarial Training with Early Stopping
18

---

Under review as a conference paper at ICLR 2021
(AT-ES) (Rice et al., 2020). For AT-AWP and AT-ES, use their Preact ResNet-18 model, while for AT-PT, we use their WideResNett34-10 10 model. The results are reported in Table [T] Our MDMT attack achieved similar robustness evaluation to the AA ensemble attack. lowering the PGD evaluated robustness by at least 5%. While being as effective as AA attack, our MDMT attacks are more than 8 times faster than AA attack, we have shown in Table 2
19


Reviews:
[{"review_id": "S0OLMF2CWn", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The paper introduces and analyses the possibility that the effectiveness of PGD-based adversarial attacks might be reduced by imbalanced gradients between the terms of the margin losses commonly used. As a remedy, it also proposed a new scheme for PGD attack, where for the first half of the iterations a single-term loss is optimized, before falling back on the the usual margin loss. The authors test the hypothesis of imbalanced gradients, introducing a new metric, GIR, and the newly proposed attack in two versions, MD and MDMT, on several defenses based on adversarial training.\n\nPros\n1. Understanding why some adversarial defenses make some versions of PGD (e.g. with cross-entropy or margin loss) overestimate, sometimes even largely, robustness is an interesting direction, and the proposed explanation of imbalanced gradients is, as far as I know, novel.\n2. The proposed attacks are effective against many defenses.\n\nCons\n1. It is not clear what the proposed metric Gradient Imbalance Ratio (GIR) captures and how it is connected the scheme of MD. In fact, the GIR values for JARN-AT1, Sense and MMA, which are not robust, especially against MD, are smaller than for MART and similar to those for UAT, which are instead robust and little or not affected by gradient imbalance according to the results of the MD attacks. Moreover, FeaScatter and MART have similar GIR, but the improvement of MDMT compared to standard PGD is >31% for the former but only around 5% for the latter.\n2. In the comparison with other methods (Table 1) the results of AA seem worse than reported in the latest version of (Croce & Hein, 2020), which report lower values of robust accuracy than MDMT for the same defenses. Also, when considering only individual attacks, it'd make sense to include FAB, as in Sec. 4.3 and Appendix C the authors state that it is the most effective in AA and partially avoids the problem of imbalanced gradients.\n3. In the MD attack, the first half of the iterations have step size set to $2\\cdot \\epsilon$, which is than decreased in the second stage. This seems similar to what Auto-PGD does.\n\nOverall, the hypothesis the imbalanced gradients might be a cause of overestimation of robustness is reasonable and worth exploration, but the metric proposed by the authors does not clearly capture this, and the attacks which should overcome that issue are not clearly better than existing methods.\n\n---\nUpdate after rebuttal\n\nI thank the authors for the response. After reading it, the revised version and the other reviews, the concerns expressed in the initial review are still valid.\n\nThen, I keep the initial score.", "extra_scores": {}, "extra_text": {}}, {"review_id": "M0UbJbncWIZ", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper explores constructing adversarial examples in classification, in order to create better robustness metrics for general classifiers. An attack is defined as an epsilon-perturbation of the learned parameters which create a model whose performance is much degraded. The premise of this paper is to use gradient imbalance as a way of creating perturbation targets, which are claimed (and shown numerically) to better fool networks that are trained to withstand more traditional attacks, and can be used to create more robust models in general.\n\nI think the premise is well-motivated, and the results look promising. However, I am having quite a bit of trouble understanding the main method. From the margin loss given, $l(x,y) = z_{\\max}-z$, which seems similar to the loss in logistic regression, with gradient  $\\nabla \\ell(x,y) = yx^T(1-\\sigma(yx^T\\theta)$. (It is not entirely clear what $z_{\\max}$ and $z$ correspond to in the general case.) I am not clear as to why we would expect label balance, e.g. why would $1 = \\sigma(yx^T\\theta)$? It is also not clear why, when one term dominates the other, that it is a form of nonrobustness. \n\nAlternatively, the interpretation may be that a particular training sample is affecting the model more than the others. That would seem like a more reasonable metric of fallibility, but I don't think that's what the authors intend.\n\nOverall, I do think the paper can be improved if written more clearly, with terms and concepts defined. I guessed that $\\Pi$ is projection, and other terms like imbalance ratio are buried in text and hard to find. But, I find the method overall simple and elegant, so with continued discussions with the authors to clarify what exactly is happening, I would raise my score. ", "extra_scores": {}, "extra_text": {}}, {"review_id": "GGnitctB4Ee", "reviewer": "AnonReviewer4", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This work highlights the existence of imbalanced gradients as a phenomenon that may hinder optimization of gradient-based adversarial attacks and, thus, give a false sense of robustness. Imbalanced gradients may occur as the attack objective consists of the difference of two terms (typically, the outputs of the network on two different classes). When the gradients of these two terms have opposite directions, the attack optimization may get easily stuck in a suboptimal local optimum, thus decreasing the attack effectiveness.\nTo overcome this issue, the authors propose a simple variant that can also be applied to existing attacks, based on following the gradient of each term in the objective iteratively, rather than considering their combination.\n\nThe paper addresses an important issue in adversarial machine learning, i.e., the security evaluation of a defense. In fact, the performance of many proposed defenses has been overestimated due to the weakness of attacks used in the evaluation. One of the causes of this is the inability of gradient-based attacks to correctly perform the optimization.\n\nThe paper provides an exhaustive analysis and formulation of the problem of imbalanced gradients, for which also a heuristic metric is provided. This phenomenon is then evaluated on a set of state-of-the-art defenses and attacks. \n\nThe authors propose an attack variant inspired from PGD and MT attacks, using the difference of class logits as the objective (i.e., the so-called margin loss). Then, the optimization algorithm works as follows. For each restart, in the first half of the optimization process, only one of the two terms of the margin loss is alternatively used, while in the second half, the whole margin loss is used. The attack formulation and algorithm are clear.\nIt is important to remark that rather than proposing a new attack framework, the authors actually only propose an optimization variant of the existing PGD and MT attacks and run these attacks with the proposed changes. As correctly stated by the authors, indeed, their proposal can be viewed as an initialization strategy for adversarial attacks (rather than a completely novel attack).\n\nExperiments are reproducible and consider many state-of-the-art attack algorithms and defenses. However, the comparison with other attacks does not seem totally fair. The same hyperparameters (step size, number of iterations and restarts) are used for PGD, C&W and MD attacks (same also for ODI, MT and MDMT), and default hyperparameters are used for the AutoAttack framework. This is problematic, as each attack should be tuned independently, and the choice of hyperparameters should be justified (a common criterion for hyperparameter tuning may be selected, and a grid-search over the attack hyperparameters should be conducted). As shown in many other papers, the attack hyperparameters play a key role in correctly optimizing the attack objective (even single data points can require different hyperparameter tuning).\n\nAnother limitation of this work is that it is restricted solely to WideResNet architectures and CIFAR10.\n\nFurthermore, the computational efficiency of the attack is not discussed. Although it is not a primary aspect, it should at least be mentioned when it is compared to other attacks, since it is potentially slower as it might require more iterations and restarts. To be fair, attacks should be even compared under the same complexity (e.g., number of queries to the target model).\n\nIn the last column of Table 1, the performance difference between MDMT and PGD is reported. However, this could lead to an overestimation of the attack performance: for a fair evaluation, the MDMT attack should be compared to the stronger attack available at the state of the art, according to the experimental evaluation conducted by the authors, and not to PGD (which has been shown to perform worse than more recent attacks in many papers – e.g., AutoAttack).\n", "extra_scores": {}, "extra_text": {}}, {"review_id": "ClWDvhypd6k", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper identified the issue of Imbalanced Gradient, verified through some recent defense methods. Motivated by such an issue, a marginal decomposition (MD) attack is proposed to offer a stronger robustness measure. In general, the paper is well written, and the studied problem is interesting. The MD perspective explains why label smoothing may provide insufficient robustness.\n\nMy comments are listed below. \n\n1)  In Sec. 3.2, why are the first K/2 iterations used to maximize the individual margin term and then update the entire loss? What will happen if the scheduling is the opposite: updating the entire loss at the first K/2 iterations, then individual terms？Some ablation studies or explanation should be provided.\n\n2) Does the proposed stronger attack offer a stronger min-max defense? Suppose that the ordinary PGD attack is replaced by an MD attack during min-max training, will it offer better overall robustness? The general question to ask is: In addition to root cause analysis on the ineffectiveness of some existing defense methods, what are the additional benefits of the newly proposed MD attack?\n\n3) Does it seem that the MD attack has to run over more iterations than the PGD attack, leading to extensive computation cost?\n\n4) What is the possible effect of the MD attack on the generated perturbation pattern? In the black-box setting, will the MD attack be more query-efficient than a commonly-used PGD black-box attack?\n\n\nPost-rebuttal:\n\nI am mostly satisfied with the authors' response. After reading other reviewers' comments, I shared a similar concern on the marginal contribution. However, the newly added black-box result is a good addition to the paper. Thus, I keep my original rating toward the positive side. \n\n", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper identifies a subtle gradient problem in adversarial robustness-- imbalanced gradients, which can cause create a false sense of adversarial robustness. The paper provides insights into this problem and proposes a margin decomposition based solution for the PGD attack.

Pros:
- Novel insights into why some adversarial defenses may make some versions of PGD overestimate robustness.
- Proposes a method that is motivated by such findings of imbalanced gradients.
- The proposed attacks are shown effective across a wide range of defenses.

Cons:
- The proposed gradient imbalance ratio could be better motivated: i.e. how is it connected to the scheme of margin decomposition?
- Limited novelty in the attacks: i.e. variant of the existing PGD and MT attacks with some proposed changes.
- Various concerns with experiments (i.e. stepsize tuning, choice of hyperparameters).

Overall, the reviewers felt that there were some interesting ideas and directions presented in the paper; however, the reviewers also felt that the contribution was of marginal significance and more confidence in the various components (i.e. how the proposed metrics measure the imbalanced gradient effect and various concerns in the experiments) would have made the paper more convincing.
Decision: Reject