Revisiting adversarial training for the worst-performing class
Authors: Thomas Pethick
Grigorios Chrysos
Volkan Cevher
Venue: Accepted by TMLR ()

Abstract: Despite progress in adversarial training (AT), there is a substantial gap between the top-performing and worst-performing classes in many datasets. For example, on CIFAR10, the accuracies for the best and worst classes are 74% and 23%, respectively. We argue that this gap can be reduced by explicitly optimizing for the worst-performing class, resulting in a min-max-max optimization formulation.  Our method, called class focused online learning (CFOL), includes high probability convergence guarantees for the worst class loss and can be easily integrated into existing training setups with minimal computational overhead.  We demonstrate an improvement to 32% in the worst class accuracy on CIFAR10, and we observe consistent behavior across CIFAR100 and STL10.  Our study highlights the importance of moving beyond average accuracy,  which is particularly important in safety-critical applications.

Thomas Pethick Published in Transactions on Machine Learning Research (1/2023)
Revisiting adversarial training for the worst-performing class
phommss.pthhicckkeppl..
École Polytechnique Fédérale de Lausanne (EPFL)
Grigorios G Chrysos
grigortiss.drrpssseeep...dd
École Polytechnique Fédérale de Lausanne (EPFL)
Volkan Cevher
volkan. Makkam.ccuher@eeff..d
École Polytechnique Fédérale de Lausanne (EPFL)
Reviewed on OpenReview: https: openreview. net/ forun? id= wkecshl Yw!
Abstract
Despite progress in adversarial training (AT), there substantial gap between the top- performing and perso--prforming classes in many datasets. For example, on CIFAR10, the accuracies for the best and worst classes are 74% and 23%, respectively. We argue that this gap can be reduced by explicitly optimizing for the worst-performing class, resulting in min-max-max optimization formulation. Our method, called class focused online learning (CFOL), includes high probability convergence guarantees for the worst class loss and can be easily integrated into existing training setups with minimal computational overhead. We demonstrate an improvement 32% in the worst class accuracy on CTFAR10, and we observe consistent behavior across CIFAR100 and STL10. Our study highlights the importance of moving beyond average accuracy, which is particularly important in safety- critical applications.
1 Introduction
The susceptibility of neural networks to adversarial attacks (Goodfellow et al. 2014 Szegody et al. 2013 has been grave concern over the launch of such systems in real-world applications. Delense mechanisms that optimize the average performance have been proposed (Papernot et al. 2016 Raghunathan et al. 2018
Guo et al. 2017 Madry et al. 2017 Zhang et al. 2019 response, even stronger attacks have been
devised (Carlini Wagner 2016 Engstrom et al. 2018 Carlini
2019
In this work, we argue that the average performance not the only criterion that of interest for real- world applications. For classification, in particular, optimizing the average performance provides very poor guarantecs for the "weakest" class. This is critical in scenarios where require any class to perform well. It turns out that the worst performing class can indeed be much worse than the average in adversarial training. This difference is already present in clean training but we critically observe, that the gap between the average and the worst is greatly exacerbated in adversarial training. This gap can already be observed on CIFAR10 where the accuracy across classes far from uniform with 51% average robust accuracy while the worst class is 23% (see Figure D The effect is even more prevalent when more classes are present as in CIFAR100 where we observe that the worst class has zero accuracy while the average accuracy is 28% (see Appendix C where we include other datasets). Despite the focus adverarial training, note that the same effect can be observed for robust evaluation after clean training (see Figure
This dramatic drop in accuracy for the weakest classes begs for different approaches than the classical empirical risk minimization (ERM), which focuses squarely on the average loss. We suggest a simple al- ternative, which we call class focused online learning (CFOL), that can be plugged into existing adversarial training procedures. Instead of minimizing the average performance over the dataset we sample from an adversarial distribution over classes that. is learned jointly with the model parameters. In this way we aim

---

Published in Transactions on Machine Learning Research (1/2023)
Clean training
Adversarial training
1.0
test accuracy
Robust test accuracy
0.8
0.6
airpiane
automobile
bird
calt
deer
dog
frog
horse
ship
truck
airplane
automobile
bird
deer
dog
frog
horse
ship
truck
Figure 1: The error across classes is already not perfectly uniform in clean training on CIFARIO. However, this phenomenon is significantly worsened in adversarial training when considering the robust accuracy. That is, some classes perform much worse than the average. The worst class accuracy and average accuracy is depicted with red and black line respectively.
at ensuring some level robustness even for the worst performing class. The focus of this paper is thus on the robust accuracy of the weakest classes instead of the average robust accuracy. Concretely, make the following contributions:
We propose simple solution which relies on the classical bandit algorithm from the online learning literature, namely the Exponential-weighh algorithm for Exploration and Exploitation (Exp3) (Auer et al. 2002). The method is directly compatible with standard adversarial training procedures (Madry al. 2017), by replacing the empirical distribution with an adaptively learned adversarial distribution over classes.
We carry out extensive experiments comparing CFOL against three strong baselines across three datasets, where we consistently observe that CFOL improves the weakest classes.
We support the empirical results with high probability convergence guarantees for the worst class accuracy and establish direct connection with the conditional value at risk (CVaR) (Rockafellar et al. 2000 uncertainty set from distributional robust optimization.
2
Related work
the first to make the important observation that deep neural networks vulnerable to small adversarially perturbation of the input. Since then, there has been growing body of literature addressing this safety critical issue, spanning from certified robust model (Raghunathan et al. 20118). distillation (Papernot et al. 2016). input augmentation [Guo et al, 2017), to adversarial training (Madry et al. 2017 Zhang et al. 2019). We focus on adversarial training in this paper. While certified robustness is desirable, adversarial training remains one of the most successful defenses in practice.
Adversarial examples Goodfellow et al.
(2014); Szegedy et al.
(013)
Parallel work (Tin et al. 2021 also observe the non-umiform accuracy classes in adversarial training, further dtrenguhening the case that lack of class-wise robustness is indeed an issue. They primarily focus constructing an attack that can enlarge this disparity. However, they also demonstrate the possibility of a defense by showing that the accuracy of class can be improved by manually reweighting class 5 when retraining. Our method CFOL can be seen as automating this process of finding defense by instead adaptively assigning more weight to difficult. classes.
Minimizing the maximum Focused online learning (FOL)
Wexler 2016), takes a
bandit approach similar to work, but instead re-weights the distribution over the N training examples
2

---

Published in Transactions on Machine Learning Research (1/2023)
independent of the class label. This leads to convergence rate in terms of the number of examples N instead of the number of classes k for which usually & < N. We compare in more detail theoretically and empirically in Section 4.2 and Section respectively. Sagawa et al. (2019) instead reweight the gradient over known groups which coincides with the variant considered in Section 4.1 A reweighting scheme has also been considered for data augmentation (Yi et al.] 2021). They obtain a closed form solution under a heuristically driven entropy regularization and full information. In our setting full information would imply full batch updates and would be infeasible.
Interpolations between average and maximum loss have been considered in various other settings: for class imbalanced datasets (Lin et al. 2017). in foderated learning Li ct al. 2019), and more generally the tilted empirical risk minimization (Li et al. 2020 Lee et al. 2020
Distributional robust optimization The accuracy over the worst class be seen as particular re- weighing of the data distribution which adversarially assigns all weights to single class. Worst case pertur- bation of the data distribution have more generally been studied under the framework of distributional robust stochastic optimization (DRO) (Ben-Tall et al.) 2013 Shapiro 2017). Instead of attempting to minimizing the empirical risk on a training distribution Po: this framework considers some uncertainty set around the training distribution (P)) and seeks to minimize the worst case risk within this set, SuPoeu(Po) (e==(((x))
A choice of uncertainty set, which has been given significant attention in the community, is conditional value at risk (CVaR), which aims at minimizing the weighted average of the tail risk (Rockafellar et al. 2000 Levy
et al. 2020 Kawaguchi & Lu. 2020 Fan et al. 2017 Curi et al.) 2019). C'aaR has been specialized to a re- weightttin over class labels, namely labeled conditional value at risk (LCVaR) (Xu et al. 2020). This was originally derived in the context of imbalanced dataset to re-balance the classes. It is stiil applicable in our setting and we thus provide a comparison. The original empirical work of (Xu et al. 2020) only considers
the full-batch sctting. Wc complement this by demonstrating LCVaR in a stochastic sctting.
In Duchi al. (2019); Duchi & Namkoong (2018) they interested in uniform performance over various groups, which is similarly to our setting. However, these groups are assumed to be latent subpopulations, which introduce significant complications. The paper is thus concerned with a different setting, an example being training on a dataset implicitly consisting of multiple text corpora.
CFOL can also be formulated in the framework of DRO by choosing an uncortainty set that can re-weight the class-conditional risks. The precise definition is given in Appendix B.1] We further establish a direct connection between the uncertainty sets of CFOL and CVaR that we make precise in Appendix B. which
also contains summary of the most relevant related methods in Table 5 BB
3 Problem formulation and preliminaries
Notation The data distribution is denoted by 'D with examples E Rd and classes y [R]] A given iteration is characterized by E [T], while de indicates the yhh index of the tth iterate. The indicator function is denoted with 1 (boolcan) and unif(n) indicates the uniform distribution over n elements. An overview of the notation is provided in Appendix A
In classification, we are normally interested minimizing the population risk E(z,) [E(0, x, v)] over our model parameters 0 E RP, where & is some loss function of 0 and example x E Rd with an associated class y E [k]] Madry et al. (2017) formalized the objective of adversarial training by replacing each example with an adversarially perturbed variant. That is, we want to find a parameterization 0 of our predictive model which solves the following optimization problem:
min L(0) == E( max ((0, x + 6, 6.v)] jES
(1)
8
where each is now perturbed by adversarial noise 8 E S S Rd. Common choices of S include norm-ball constraints (Madry al. (017) or bounding some notion of perceptual distance (Laidlaw al.) 2020). When the distribution over classes is uniform this is implicitly minimizing the average loss over all class. This does
3

---

Published in Transactions on Machine Learning Research (1/2023)
Class sampling distribution
0.15
0.10
0.05
0.00
airplane automabile
bird
cat
deer
dog
frog
horse
ship
truck
Figure 2: Contrary to ERM, which samples the examples uniformly, CFOL samples from adaptive distribution. The learned adversarial distribution is non-uniform over the classes in CIFARIO when using adversarial training. As expected, the hardest classes are also most frequently sampled.
not guarantee high accuracy for the worst class as illustrated in Figure I since we only know with certainty that max 2 avg.
Instead, we will focus on different objective, namely minimizing the worst class-cordilioned risk:
min max yE[k] [0)(0)= : Exmpp((l)) max E(0, x + 6, }.m)
(2)
5ES
This follows the philosophy that "a chain is only as strong as weakest link." In safety critical application, such as autonomous driving, modeling even just single class wrong can still have catastrophic consequences. Imagine for instance strect sign rocognition system. Even if the system has 99% average accuracy the system might never label 'stop'-sign correctly, thus preventing the car from stopping at crucial point.
As the maximum in Equation (2) is a discrete maximization problem its treatment requires more care. We will lake a common approach and construct convex relaxation by lifting the problem in Section
4 Method
Since we do not have access to the true distribution D, we will instead minimize over the provided empirical distribution. Let Ng be the set of data point indices for class y such that the total number examples is N Σ (yyl- Then, are interested in minimizing the marimum empirical class- conditoend risk,
max (0) DEA
Σ max E(0, 8.9).
(3)
dES
i,,
We relax this discrete problem to continuous problem over the simplex Ak.
max (,)
max Σ po (.)
(4)
(k) P
y=1
Note that equality is attained when p is a dirac on the argmax over classes.
Equation leaves us with a two-player zero-sum game between the model parameters 0 and the class distribution p. A principled way of solving min-max formulation through the use of no-regret algorithms. From the perspective of p, the objective simply linear under simplex constraints, albeit adversarially picked
4

---

Published in Transactions on Machine Learning Research (1/2023)
by the model. This immediately makes the no-regret algorithm Hedge applicable (Freund & Schapire 1997
w w - -n
(Hedge)
Py exp (wt)) Σ
Eexp Exp
To show convergence for (Hedge) the loss neods satisty certain assumptions. our case of classification, the loss is the zero-one loss E1 v) (2) + w), where ha(-) is the predictive model. Hence, the loss is bounded, which is sufficient requirement.
Note that Hedgeb relies on zero-order information of the loss, which we indeed have available. However, in the current form, (Hedge) requires so called full information over the dimensional loss veclor. In other words, we need to compute (0) for all y E [K], which would require a full pass over the dataset for every iteration.
Following the seminal work of Auer et al. (2002) we instead construct an unbiased estimator the k dimensional loss vector L(0*)= : (0"( (O) based on sampled class y¹ from some distribution y' p². This stochastic formulation further lets us estimate the class conditioned risk (0') with an unbiased sample ~ unif([AG:))) l). This leaves us with the following estimator,
Σi (
I 10,.0000//
y y=
(5)
otherwise
where by (0) maxses (0,x; x; + 8, v). ). It easy to verify that this estimator is unbiased,
(,(0') Lo).
Byou 1 (1 (1- 1 pt) 0 -
(6)
For ease of presentation the estimator only uses a single sample but this can trivially be extended to mini-batch where classes are drawn iidd. from p².
We could pick p' to be the learned adversarial distribution, but it is well known that this can lead to unbounded regret if some Pt is small (see Appendix A for definition of regret). Auer et al. (002) resolve this problem with Exp3, which instead learns a distribution d', then mixes g² with a uniform distribution, to eventually sample from pi, = + (1-7)gt, where y (0, 1). Intuitively, this enforces exploration. In the general case y needs to be picked carefully and small enough, but we show in Theorem [ that larger ? is possible in our setting. We explore the effect of different choices of y empirically in Table
Our algorithm thus updates g²+1 with (Hedge) using the estimator L' with sample drawn from p' and subsequently computes p CFOL in conjunction with the simultaneous update of the minimization player can be found in Algorithm 1 with an example of a learned distribution p in Figure! 2
Practically, the scheme bears negligible computational overhead over ERM since the softmax required to sample is of the same dimensionality as the softmax used in the forward pass through the model. This computation negligible in comparison with backpropagating through the entire model. For further details the implementation refer to Appendix C.3
4.1 Reweighted variant of CFOL
Algorithm D samples from the adversarial distribution p. Alternatively one can sample data points uniformly and instead reweight the gradients for the model using p. In expectation, update of these two schemes are equivalent. To see why, observe that in CFOL the model has access to the gradient Voly. We can obtain an unbiased estimator by instead reweighting uniformly sampled class, i.c Ey-p. (VoLy. = (y [Ky,, Vo (0)] With classes sampled uniformly the unbiased estimator for the adversary becomes Ly = (yr Lyk Vy Thus, one update of CFOL and the reweighted variant are equivalent expectation. However, note that additionally depended on the internals of the model's update rule and that the
5

---

Published in Transactions on Machine Learning Research (1/2023)
Algorithm 1: Class focused online learning (CFOL)
Algorithm parameters: a step rule ModelUpdate for the model satisfying Assumption adversarial step-size 17 > O, uniform mixing parameter ? = 1/2, and the loss (0). Initialization: Set w such that qº and p" are uniform.
foreach in 0..I' do
y' -pp i ~ unif((NV++1) 1)
// sample class // sample uniformly from class // update model parameters // construct estimator // update the adv. class distribution
ge+1 = ModelUpdate(ff" (f
y I
(09)/2/ Vy
w y wi) -nLI Vy
a = exp (u(t1) Eg exp (w fy;
Py +1 > + (1 - -749 Vy:
end foreach
immediate equivalence we get is only in expectation. These modifications recovers the update used in Sagawa et al. (2019). See Table 9 in the supplementary material for experimental results regarding the reweighted variant.
4.2 Convergence rate
To understand what kind result we can expect, it is worth entertaining hypothetical worst case scenario. Imagine a classification problem where one class is much harder to model than the remaining classes. We would expect the learning algorithm to require exposure to examples from the hard class in order to model that class appropriately ottherittely-ooheriisse the classes would not be distinct. From this one can see why ERM might be slow. The algorithm would naively pass over the entire dataset in order to improve the hard class using only the fraction of examples belonging to that class. In contrast, if we can adaptively focus on the difficult class, we can avoid spending time on classes that are already improved sufficiently. As long as we can adapt fast enough, as expressed through the regret of the adversary, we should able to improve on the convergence rate for the worst class.
We will now make this intuition precise by establishing high probability convergence guarantee for the worst class loss analogue to that of FOL. For this we will assume that the model parameterized by 0 enjoys so called mistake bound of C (Shalev-Shwartz et al. 2011 p. 288). The proof is deferred to Appendix A A Assumption 1. For any sequence of classes (y",...,'") vz) E [+] and class conditioned sample indices (i²,..., with if EN the model enjoys the following bound for some C' < oc and C = max (klog log k, C"),
I Σ
Ly (0' SC
Remark The requirement C 2 & log & will be ncoded to satisfy the mild stop-size roquirement 17 S 2h Lemma [ In most settings the smallest. C' is some fraction of the number of iterations T, which in turn is much larger than the number of classes k, C=
With this at hand we are ready to state the convergence of the worst class-conditioned empirical risk.
Theorem 1. If Algorithm is run on bounded rewards Ly (00))) [0, 1] At with step-size 1j = /log k/(4CC), mizing parameter 1/2 and the model satisfies Assumption then after T iterations with probability at least 1 8,
1 L. (A]
6C
+ / 4k log(222//0) (k/0)
(1 + 2k) log (2k(())
Ly (0') 2 T
max
VT
37
J=1
(8)
/ 2 log (2k/8)
2 log (2k)))
Vni
3n

---

Published in Transactions on Machine Learning Research (1/2023)
Table 1: Accuracy on CIFAR10. For both clean test accuracy (aCCelean) and robust test accuracy (acCrob) we report the average, 20% worst classes and the worst class. We compare our method (CFOL-AT) with stan- dard adversarial training (ERM-AT) and two baselines (LCVaR-AI and FOL-AT). CFOL-AT significantly improves the robust accuracy for both the worst class and the 20% tail.
ERM-AT CFOL-AT
LCVaR-AT
FOL-AT
Average
0.8244
0.8308
0.8259
0.8280
aCCelean
20% tail
0.6590
0.7120
0.6540
0.6635
Worst class
0.6330
0.6830
0.6340
0.6210
Average
0.5138
0.5014
0.5169
0.5106
acCrob
20% tail
0.2400
0.3300
0.2675
0.2480
Worst class
0.2350
0.3200
0.2440
0.2370
ERM-AT
CFOL-AT
Robust test accuracy
Robust tast accuracy
0.6
0.6
0.2
0.0
airplane
automobile
bird
cat
dear
dog
frog
horse
ship
truck
airplane
automobie
bird
deer
dog
frog
horse
ship
truck
Figure 3: The robust test accuracy for CFOL-AT and ERM-AT over classes. The horizontal black and red line depicts the average and worst class accuracy over the classes respectively. The improvement in the minimum accuracy is notable when using CFOL-AT, while there is only marginal difference in the average accuracy. The evolution of accuracies for all methods can be found in Figure 7
for ensemble of size where ty (T) for E [n)]
To contextualize Thcorem let consider the simple case of lincar binary classification mentioned in Shalev- Shwartz & Wexler (2016). In this setting SGD needs O(CN) iterations to oblain consistent hypothesis. In contrast, the iteration requirement of FOL decomposes into a sum Ö(C+ N) which much smaller since both C and N are large. When CFOL. can converge as Ó(C+ + k) where usually & & C. Connecting this back to our motivational example in the beginning of this section, this sum decomposition exactly captures our intuition. That is, adaptively focusing the class distribution can avoid the learning algorithm from needlessly going over all k classes in order to improve just one of them.
only concerned with the convergence of the worst class show that
From Thcorem we also see that we need an ensemble of size 22( (og(kk/)/ee)) which has only mild dependency on the number of classes k. If wanted to drive the worst class error e to zero the dependency would be problematic. However, for adversarial training, even in the best case of the CIFAR10 dataset, the average error is almost 1/2. We can expect the worst class error to be even worse, so that only a small n is required. In practice, a single model turns out to suffice.
Note that the maximum upper bounds the average, so by minimizing this upper bound as in Theorem g we are implicitly still minimizing the usual average loss. In addition, Theorem [hsows that mixing parameter of ? = 1/2 sufficient for minimizing the worst class loss. Effectively, the average loss is still directly being minimized, but only through half of the sampled examples.

---

Published in Transactions on Machine Learning Research (1/2023)
Table 2: Clean test accuracy (aCCelean) and robust test accuracy (accrab) on CLEAR100 and STL10. We
compare our method (CFOL-AT) with standard adversarial training (ERM-AT) and two basclines (LCVaR- AT and FOL-AT). CFOL-AT consistently improves the worst class accuracy as well the 20% worst tail.
ERM-AT
CFOL-AT
LCVaR-AT
FOL-AT
Average
0.5625
0.5593
0.5614
0.5605
aCCclean
20% tail
0.2710
0.3550
0.2960
0.2815
Worst class
0.0600
0.2500
0.0600
0.1100
CIFAR100
Average
0.2816
0.2519
0.2866
0,2718
acCrub
20% tail
0.0645
0.0770
0.0690
0.0630
Worst class
0.0000
0.0400
0.0000
0.0000
Average
0.7023
0.6826
0.6696
0.6890
aCColean
20% tail
0.4119
0.4594
0.3600
0.3837
Worst class
0.3725
0.4475
0.3462
0.3562
STL10
Average
0.3689
0.3755
0.3864
0.3736
acCrob
20% tail
0.0900
0.1388
0.0944
0.0981
Worst class
0.0587
0.1225
0.0650
0.0737
Table 3: Interpolation property of y on CIFARIO. By increasing the mixing parameter 7, CFOL-AT can closely match the average robust accuracy ERM-AT while still improving the worst class accuracy. Stan- dard deviations are computed over independent runs.
0.9
0.7
0.5
Average
0.5108 + 0.0015
0.5091 = 0.0037
0.5012 + 0.0008
20% lail
0.2883 + 0.0278
0.3079 + 0.0215
0.3393 + 0.0398
Worst class
0.2717 + 0.0331
0.2965 = 0.0141
0.3083 + 0.0515
5 Experiments
We consider the adversarial setting where the constraint set of the attacker S is an -bunded attack, which is the strongest norm-based attack and has a natural interpretation in the pixel domain. We test on three datasets with different dimensionality, number of examples per class and number of classes. Specifically, we consider CIFAR10, CIFAR100 and STL10 (Krizhevsky et al. 2009 Coates et al. 2011 (sce Appendix C.2 for further details).
Hyper-parameters Unless otherwise noted, we use the standard adversarial training setup of a ResNet-18 network (He et al. 2016) with learning rate 0.1, momentum of 0.9, weight decay of 5 10-4, batch size of 128 with piece-wise constant weight decay of 0.1 at epoch 100 and 150 for total of 200 epochs according to Madry et al. (2017). For the attack we similarly adopt the common attack radius of 8/255 using 7 steps of projected gradient descent (PGD) with a step-size of 2/255 (Madry et al., (017). For evaluation we use the stronger attack of 20 step throughout, except for Table 7 where we show robustness against AutoAttack (Croce & Hein 2020
Baselines With this setup we compare our proposed method, CFOL, against empirical risk minimization (ERM), labeled conditional value at risk (LCVaR) (Xu et al.) 2020 and focused online learning (FOL) Shalev-Shwartz & Wexler 2016). We add the suffix "AT" to all methods to indicate that the training examples are adversarially perturbed according to adversarial training of Madry et al. (2017). We consider ERM-AT as the core baseline, while we also implement FOL-AT and LCVaR-AT as alternative methods that can improve the worst performing class. For fair comparison, and to match existing literature, we do early stopping based on the average robust accuracy on a hold-out set. The additional hyperparameters for CFOL- AT, FOL-AT and LCVaR-AT are chosen optimize the robust accuracy for the worst class on CIFAR10. This parameter choice then used as the basis for the subsequent. datasets. More details hyperparameters
8

---

Published in Transactions on Machine Learning Research (1/2023)
Table 4: Comparison with larger pretrained models from the literature. ERM-AT refers to the training scheme of Madry et al. (2017) and uses the shared weights of the larger model from Engstrom et all (2019). We refer to the pretrained Wide ResNet-34-10 from (Zhang et al. 2019 as TRADES. For fair comparison we apply CFOL-AT to Wide ResNet-34-10 (with 7 0.8), while using the test set the hold-out set similarly to the pretrained models. We observe that CFOL-AT improves the worst class and 20% tail for both clean and robust accuracy.
TRADES
ERM-AT
CFOL-AT
Average 20% tail
0.8492
0,8703
0.8743
aCCclenn
0.6845
0.7220
0.7595
Worst class
0.6700
0.6920
0.7500
Average
0.5686
0.5490
0.5519
acCrob
20% tail
0.3445
0.3065
0.3575
Worst class 0.2810
0.2590
0.3280
and implementation can be found in Appendix C.1 and Appendix CK respectively. In Table we additionally provide experiments for variant of CFOL-AT which instead reweighs the gradients.
Metrics We report the average accuracy, the worst class accuracy and the accuracy across the 20% worst classes (referred to as the 20% lail) for both clean (acCrlean) and robust accuracy (accroh). We note that the aim is not to provide sf accuracy but rather provide a fair comparison between the methods.
The first core experiment is conducted on CIFAR10. In Table D the quantitative results are reported with the accuracy per class illustrated in Figure The results reveal that all methods other than ERM-AT improve the worst performing class with CFOL-AT obtaining higher accuracy in the weakest class than all methods in both the clean and the robust case. For the robust accuracy the 20% worst tail is improved from 24.0% (ERM-AT) to 26.7% using existing method in the literature (LCVaR-AT). CFOL-AT increases the former accuracy to 33.0% which is approximately a 40% increase over ERM-AT. The reason behind the improvement is apparent from Figure S which shows how the robust accuracy of the classes cat and deer are now comparable with bird and dog. A simple baseline which runs training twice is not able improve the worst class robust accuracy (Table 11
The results for the remaining datascts be found in Table 2 The results exhibit similar patterns to the experiment on CIFAR10, where CFOL-AT improves both the worst performing class and the 20% tail. For ERM-AT on STT11,, the relative gap between the robust accuracy for the worst class (5.9%) and the average robust accuracy (36.9%) is even more pronouneed. Interostingly, in this case CFOL-AT improves both the accuracy of the worst class to 12.3% and the average accuracy to 37.6%. In CIFAR100 we observe that both ERM-AT, T.CV and FOL-AT have 0% robust accuracy for the worst class, which CFOL-AT increases the accuracy to 4%.
In the case of CIFAR10, CFOL-AT leads to minor drop in the average accuracy when compared against ERM-AT. For applications where average accuracy is also important, we note that the mixing parameter yy in CFOL-AT interpolates between focusing on the average accuracy and the accuracy of the worst class. We illustrate this interpolation property on CIFAR10 in Table 3 where we observe that CFOL-AT can match the average accuracy of ERM-AT closely with 51% while still improving the worst class accuracy to 30%.
In the supplementary, we additionally assess the performance in the challenging case of test time attack that differs from the attack used for training (see Table 7 C for evaluation AutoAttack). The difference between CFOL-AT and ERM-AT becomes even more pronounced. The worst class accuracy for ERM-AT drops to as low as 12.6% on CIFARIO, while CFOL-AT improves on this accuracy by more than 60% to an accuracy of 20.7%. We also carry out experiments under a larger attack radius (see Table 8 SC Similarly, we observe an accuracy drop of the worst class on CIFAR10 to 9.3% for ERM-AT, which CFOL-AT improves to 17%. We also conduct experiments with the VGG-16 architocture, on Tiny ImageNet, Imagenette and under class imbalanced (Tables 12 to 15 C In all experiments CFOL-AT consistently improves the accuracy with respect to both the worst class and the 20% tail. We additionally provide results for early
9

---

Published in Transactions on Machine Learning Research (1/2023)
stopped models using the best worst class accuracy from the validation set (see Table 6 $C) Using the worst class for early stopping improves the worst class accuracy expected, but is not sufficient to make ERM-AT consistently competitive with CFOL-AT.
Even when compared with larger pretrained models from the literature we observe that CFOL-AT on the smaller ResNet-18 model has a better worst class robust accuracy (see Table 4 for the larger models). For fair comparison we also apply CFOL-AT to the larger Wide ResNet-34-10 model, while using the test set as the hold-out set similarly to the pretrained models. We observe that CFOL-AT improves both the worst class and 20% tail when compared with both pretrained ERM-AT and TRADES (Zhang et al. 2019). Whereas pretrained ERM-AT achieves 30.65% accuracy for the 20% tail, CFOL-AT achieves 35.75%. While the average robust accuracy of TRADES is higher, this is achieved by trading off clean accuracy.
Discussion We find that CFOL-AT consistently improves the accuracy for both the worst class and the 20% tail across the three datasets. When the mixing paramcter y = 0.5, this improvement usual comes at a cost of the average accuracy. However, we show that CFOL-AT can closely match the average accuracy of ERM-AT by picking larger, while maintaining significant improvement for the worst class.
Overall, our experimental results validate our intuitions that (a) the worst class performance is often overlooked and this might be a substantial flaw for safety-critical applications and (b) that algorithmically focusing on the worst class can alleviate this weakness. Concretely, CFOL provides a simple method for improving the worst class, which can easily be added on top of existing training setups.
6 Conclusion
In this work, we have introduced a method for class focused online learning (CFOL), which samples from adversarially learned distribution over classes. We establish high probability convergence guarantees for the worst class using& specialized regret analysis. In the context of adversarial examples, we consider an adversarial threat model in which the attacker can choose the class to evaluate in addition to the perturbation (as shown Equation ) This formulation allows us to develop a training algorithm which focuses on being robust across all the classes as oppose to only providing robustness on average. We conduct a thorough
empirical validation on five datasets, and the results consistently show improvements adversarial training over the worst performing classes.
7 Acknowledgements
This project has received funding from the European Research Council (ERC) under the European Union's 's Horizon 2020 rescarch and innovation programme (grant agreement n 725594 time-data). This work was supported by Zeiss. This work was supported by Hasler Foundation Program: Hasler Responsible Al (project number 21043).
References
Jean-Yves Audibert,, Sébastien Bubeck, and Rémi Munos. Bandit view on noisy optimization. Optimization for Machine Learning, chapter 1. MIT Press, optimization for machine learning edition, January 2010.
Peler Auer, Nicolò Cesa-Bianchi, Yoav Freund, and Robert E. Schapire. The nonstochastic multiarmed bandit problem. SIAM Journal on Computing, 32(1):48 77, January 2002. ISSN 0097-5397, 1095-7111. doi: 00.1137/500975397113983777
Aharon Ben-Tal, Dick Den Hertog, Anja De Waegenaere, Bertrand Melenberg, and Gijs Rennen. Robust solutions of optimization problems affected by uncertain probabilities. Monagement Science, 59(2):341- 357, 2013.
Nicholas Carlini. Is ami (attacks meet interpretability) robust to adversarial examples? orXiv preprint arXiv:1902. 02322, 2019.
10

---

Published in Transactions on Machine Learning Research (1/2023)
Nicholas Carlini and David Wagner. Defensive distillation is not robust to adversarial examples. arXiv preprint Xiv: (0o:1607.do 04 2016.
Adam Coates, Andrew Ng, and Honglak Lee. An analysis of single-layer networks in unsupervised feature learning. In Proceedings of the fourteenth international conference on artificial intelligence and statistics, pp. 215 223. JMLR Workshop and Conference Proceedings, 2011.
Francesco Croce and Matthias Hein. Reliable evaluation of adversarial robustness with an ensemble of diverse parameler-free atlacks. In International conference on machine learning, pp. 2206 2216. PMLR, 2020.
Sebastian Curi, Kfir Levy, Stefanie Jegelka, Andreas Krause, et al. Adaptive sampling for stochastic risk- averse learning. arXiv preprint 1rXf::1110.22511, 2019.
John Duchi and Hongseok Namkoong. Learning models with uniform performance via distributionally robust optimization. arXiv preprint arXiv: 08750, 2018.
John C Duchi, Tatsunori Hashimoto, and Hongsook Namkoong. Distributionally robust losses against mix- ture covariate shifts. Under review, 2019.
Logan Engstrom, Andrew llyas, and Anish Athalye. Evaluating and understanding the robustness of adver- sarial logit pairing. arXie preprint urXi: 10272, 2018.
Logan Engstrom, Andrew lyas, Hadi Salman, Shibani Santurkar, and Dimitris Tsipras. Robustuess (python library), 2019. URL /ttps://github httpp://ggggh......//ttttaaaaaae com//Maarrryallbroosuttssss
Yanbo Fan, Siwei Lyu, Yiming Ying, and Bao-Gang Hu. Learning with average top-k loss. ar preprinl arXiv:1705.00 08826,, 2017.
Yoav Freund and Robert E Schapire. A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences, 55(1):119 139, 1997.
Ian J Goodfellow, Jonathon Shlens, and Christian Szegody. Explaining and harnessing adversarial examples. arXiv preprinl ar Xvv: 14t..1122000 6572, 2014.
Chuan Guo, Mayank Rana, Moustapha Cisse, and Laurens Van Der Maaten. Countering adversarial images using input transformations. XrXiv preprint ar 171::711.000 2017.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. Tn Proceedings of the IEEE conference computer vision and pattern recognition, pp. 770 708,778, 2016.
Kenji Kawaguchi and Haihao La. Ordered sgd: A new stochastic optimization framework for empirical risk minimization. In International Conference on Artificial Intelligence and Statistics, pp. 669 679. PMLR, 2020.
Alex Krishevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. 2009.
Cassidy Laidlaw, Sahil Singla, and Soheil Feizi. Perceptual adversarial robustness: Defense against unseen threat models. arXis preprint Xrv:200000 12655, 2020.
Jaeho Lee, Sejun Park, and Jinwoo Shin. Learning bounds for risk-sensitive learning. ar preprinl 2:Xi:2200608190 2020.
Daniel Levy, Yair Carmon, John C Duchi, and Aaron Sidford. Large-scale methods for distributionally robust optimization. arXio preprint arXiv:2010. 05893, 2020.
Tian Li, Maziar Sanjabi, Ahmad Beirami, and Virginia Smith. Fair resource allocation in federated lcarning. arXiv preprinl ar Xr:::: 10497, 2019.
Tian Li, Ahmad Beirami, Maziar Sanjabi, and Virginia Smith. Tilted empirical risk minimization. arXiv preprint or orXii:2007.01100 2020.
11

---

Published in Transactions on Machine Learning Research (1/2023)
Tsung-Y Yi Lin, Priya Goyal, Ross Girshick, Kaiming He, and Piotr Dollár. Focal loss for dense object detection. In Proceedings of the IEEE international conference computer vision, pp. 2980 2980-2988, 2017.
Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. arXiv preprint (00e100660083, 2017.
Nicolas Papernot, Patrick McDaniel, Xi Wu, Somesh Jha, and hram Swami. Distillation as a defense to adversarial perturbations against deep neural networks. In 2016 IEEE symposium on security and privacy (SP), pp. 582-597. IEEE, 2016.
Aditi Raghunathan, Jacob Steinhardt, and Percy Liang. Certified defenses against adversarial examples. arXiv preprinl Xie: NX11:080100944 2018.
R Tyrrell Rockafellar, Stanislav Uryasev, et al. Optimization of conditional value-at-risk. Journal of risk, 2: 21- 21-42, 2000.
Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, Alexander C. Berg, and Li Fei-Fei. ImageNet Large Scale Visual Recognition Challenge. International Journal of Computer Vision (IJCV), 115(3):211-25) 2015.
doi:
Shiori Sagawa, Pang Wei Koh, Tatsunori Hashimoto, and Percy Liang, Distributionally robust neural networks for group shifts: On the importance of regularization for worst-case generalization. arXiv preprint arXi::1911.08991, 2019.
Shai Shalev-Shwartz and Yonatan Wexler. Minimizing the maximal loss: How and why. In International Conference on Machine Learning, pp. 793 801. PMLR, 2016.
Shai Shalev-Shwartz et al. Online learning and online convex optimization. Foundations and trends in Machine Learning, 4(2):107 194, 2011.
Alexander Shapiro. Distributionally robust stochastic programming. SIAM Journal on Optimization, 27(4): 2258-2275, 2017.
Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recognition. arXiv preprint 14o::1409.1556 2014.
Christian Szegedy, Wojeiech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus. Intriguing properties of neural networks. arXio preprint orXix:1812.6199 2013.
Qi Tian, Kun Kuang, Kelu Jiang, Fei Wu, and Yisen Wang. Analysis and applications of class-wise robustness in adversarial training. arXiv preprint ar arXiv:2105. 14240, 2021.
Ziyu Xu, Chen Dan, Justin Khim, and Pradeep Ravikumar. Classtweighted classification: Trade-offs and robust approaches. In International Conference on Machine Learning, pp. 10544 10554. PMLR, 2020.
Mingyang Yi, Lu Hon, Lifeng Shang, Xin Jiang, Qun Liu, and Zhi-Ming Ma. Reweighting augmented samples by minimizing the maximal expected loss. arXiv preprint arXiw:2133.0800 2021.
Hongyang Zhang, Yaodong Yu, Jiantao Jiao, Eric Xing, Laurent El Ghaoui, and Michael Jordan. Theo- retically principled trade-off between robustness and accuracy. In International Conference on Machine Learning, pp. 7472-7482. PMLR, 2019.
12

---

Published in Transactions on Machine Learning Research (1/2023)
A Convergence analysis
A.1 Preliminary and notation
Consider the abstract online learning problem where for every t E [2] the player chooses an action æ' and subsequently the environment reveals the loss function ((s./)) As traditional in the online learning literature we will measure performance in terms of regret, which compares our sequence of choices {a Te=1 with fixed stratogy in hindsight u,
RT(u)= Σ (
min Σ (u.
(9)
If we, instead of minimizing over losses I, maximize over rewards we define regret as,
T
Rí (u) = max Er(u, Σ r
(10)
t=1
When we allow randomized strategy, as in the bandit sctting, Ry becomes a random variable that we wish to upper bound with high probability. For convenience we include overview over the notation defined and used in Section X and Section f below.
Number of classes
T
Number of iterations
py
The nth index of the (hh iterate
1 [boolcan)
The indicator function
unif(n)
Discrete uniform distribution over n elements
Ny
Set of data point indices for class y E [k]
N Ey= Wgy)
The size of the data set Loss on particular example
Tyg (0) = maxses e (0, vi + 5,y)
(0) = Wel EiEN, Ly (0)
The empirical class-conditioned risk
[() = (I, (0) ...., Lg (8))
The vector of all empirical class-conditioned risks
1 = Ly. (0*)
Class-conditioned estimator itcration with unif((A)))
A.2 Convergence results
We restate Algorithm 1 while leaving out the details of the classifier for convenience. Initialize w such that q and pº are uniform. Then Exp3 proceeds for every t E [2'] as follows:
Draw class y pd
2. Observe scalar reward L
3. Construct estimator Σn L (y= Vy
Update distribution w++ n² I
9t exp (w++1) E
Ey= exp (w Vy
p = 7o + (1 - Vy
We can bound the regret of Exp3 Auer al. 2002) in our setting, even when the mixing parameter y not small herwwie usually required, by following similar argument as Shalev-Shwartz & Wexler (2016
13

---

Published in Transactions on Machine Learning Research (1/2023)
For this we will use the relationship between p and throughout. From Py (1 - -20gg it can easily be
verified that,
k
and qy
1 1 for all y.
(11)
Py Y
Py -
Lemma 1. If Exp3 is run bounded rewards Liy= E [0, 1] Vt with 17 k/k then
T
log
nk
T
RT (w)
Σ
(u,
1 Σ 4 19
Σ (1 - 7)7
L
(12)
Proof. We write the estimator vector as L'
_ Ey¹ where eye is canonical basis vector. By assumption p.
have T. 00 1). Combining this with the bound on 1/p, in Equation 111 we have Tú, s 8/s for all y- This lets us invoke Shalev-Shwards et al. 2011 Thm 2.22) as long as the step-size rj is small enough. Specifically,
we have that if S 1/2; 7/K
T
(u,
Σ
(d'
log
(Tn)² y=1
(13)
77
By expanding b and applying the inequalities from Equation and p.
we can get rid of the dependency on q
T
log k
log h
17
+ 17
(p) ( y.)))
17
[=1
1=1
T
log l
+ 11
(1 ?) (Ti
(14)
(=1
log k
ni
Light
1)
7)7
t=
where the last line uses the boundedness assumption I, E [0, Vy.
It apparent that we need to control the sum of losses. We will do so by assuming that the model admits mistake bound as Shalev-Shwartz & Wexler (2016): Shalev-Shwartz et al. (2011).
Assumption 1. For any sequence of classes (y²...,,y") y²') E [K] and class conditioned sample indices (6", (i',...,?") :T with it E Ny. the model enjoys the following bound for some C" < so and C = max log k. C"),
T Σ
L SC
(15)
Recall that we need to bound D,, = (09), where y are picked adversarially and it are sampled uniformly, the above is sufficient.
Lemma 2. If the model satisfies Assumption 1 and Algorithm is run on bounded rewards Ly x (0') [0,
Vt with step-size Vlog k/(AK()) and mixing coefficient 1/2 then
T
Σ
(u, b') < 6C.
(16)
14

---

Published in Transactions on Machine Learning Research (1/2023)
Proof. By expanding and rearranging Lemma
T
(u,
log
+
n/
+
Let pzy
Σ
I
4.
(17)
(1
71
1=1
as long as S 7/k. Then by using our favorite inequality in Equation
11
and under Assumption |
log + ( nk
(1- Ty
Σ
T
by
- log k
+
(11
1yK
1- C2
1 1
)
(18)
7h pyy
t=1
To maximize (1 (})) we pick 7= 1/2 that,
log k
( nk
)
log k + (4nk + 2) c
(19)
7 (1 (1-7) 1 1-
"
For y, notice that it appears two the terms. To minimize the bound respect to 7) we pick `j = Vlog k4KC))) such that 4nkC, which leaves with,
log + (4nk + 2]C s 2 4kCllgg 2C'
(20)
From the first term it is clear that since C 2 ls log k by assumption then the original step-size requirement of v S 1/2k is satisfies. In this case we can additionally simplify further,
2 4kClogg + 2C S 6C.
(21)
This still only gives us a bound on a stochastic object. We will now relate it to the empirical class conditional risk (0) by using standard concentration bounds. To be more precise, we want to show that by picking by in (u, L') we concentrate to (0). Following Shalev-Shwartz & Wexler (2016) adopt their use of Bernstein's type inequality.
Lemma 3 (e.g. Audibert et al. 2010 Thm. 1.2)). Let Ai..., Ar be martingale difference sequence with ......,,)
respect to a Markovian sequence Bi,... By and assume A < V and IE [A] BI...., B₂] S s. Then for any E (0, 1),
2/ log (1/)
Vlog (1/8) 3T
A
2 1 -
(22)
T
VT
Lemma 4. If Algorithm 1 is run on bounded rewards Ly (0') mizing coefficient 7 = 1/2 and the model satisfies Assumption following bound with probability at least 6/R,
E [0, 1] Vt with step-size nj 1 Vlg (4kC') then for any E [kt] we obtain have the
60
/ 4k log (4/8)
(1+ 2-)kk) log (k/5)
T Σ Ly (09) <
(23)
VT
37
Proof. Pick any y [k] and let = in (u, L.). By construction the following defines martingale difference sequence,
A A = E. - (ey. T.a) (0' (ey. py,
1 Leg. (or)))))
(24)
In particular note that it uniformly sampled. To apply the Bernstein's type inequality we just need bound |Ae and [A] q², 0t). For |As] we can crudely bound as,
k 7
|Ael < (0') )| + Ke py.
Ly (0 (0))) < 11
(25)
15

---

Published in Transactions on Machine Learning Research (1/2023)
To bound the variance observe that we have the following:
Py E (
E (ey. I-²² 1 08
100) (ey)s
(with i ~ unif(( byA
(pl,)
Inns (0t)² 10
Pd
k
[ S
(by Equation
Pyy
?
It follows that E [A]] q², 0*] - k/7
Invoking Lemma 3 we get the following bound with probability at least 1 - 8/ke,
2 log (k/6)
(1+ )) log (+/b)
Ly (0))
Σ ce Ed
(26)
VT
3T
By bounding the first term on the right hand side with Lemma 2 and taking 7 = 1/2 we obtain,
T
6C
V log (1/8)
(1 + 2k) log (4/6)
T Ly (0))
Σ t=1
T VT +
+
(27)
3T
This completes the proof.
We are now ready to state the main theorem.
Theorem 1. Tf Algorithh is run on bounded rewards Ly a (8') E [0, 1] Vt with step-size 1j Vlg k/ (4kC), mizing parameter = 1/2 and the model satisfies Assumption then after iterationn with probability at lenst 8,
6C' +
V (gg(///)//) + (1+ + 2k) (og(2k/8)
2 log(2k/8)
2 log(2k/0)
max DEIN
Ly (B0)
+
(28)
T
VT
3T
vn
3n
j=1
for an ensemble of size n where 1j sid
~ UT) for j E [n].
Proof. If we fix and let tj iid un) then the following is martingale difference sequence,
A, E.y (84)
Σ T (0*)
(29)
for which it is casy to see that |A,1 < 2 and A3 < 1 given boundedness of the loss. This readily let us apply Lemma 3 with high probability 1 8/22... Combining this with the bound of Lemma 4 with probability 1 6/2k by using union bound completes the proof.
16

---

Published in Transactions on Machine Learning Research (1/2023)
B CVaR
Conditional value at risk (CVaR) is the expected loss conditioned on being larger than the (1- a) quantille This has a distributional robust interpretation which for a discrete distribution can be written as,
m
C.aaa (0, Po) = sup PE dom
Σ (0, (0) s.t. Il I
(30)
am
The optimal p of the above problem places uniform mass on the tail. Practically we can compute this best response by sorting the losses {8(0, x1)} in descending order and assigning am saturation.
mass per index until
Primal CVaR When m large stochastic variant necessary. To obtain stochastic subgradient for the model parameter, the naive approach is to compute stochastic best response over mini-batch. This has been studied in detail in (Levy et al. 2020).
Dual CVaR An alternative formulation relies on strong duality of CVaR originally showed in (Rockafellar et al. 2000
m
CVaR., (0)
inf A am Σ (1(0,
- 1)+
(31)
XER
In practice, one approach is to jointly minimize À and the model parameters by computing stochastic gradients done in (Curi et al. 2019). Alternatively, we can find close form solution for under mini-batch (see e.g. (Xu et al. 2020 Appendix I.1)).
In comparison CFOL acts directly on the probability distribution similarly to primal CVaR. However, instead of finding a best response on a uniformly sampled mini-batch it updates the weights iteratively and samples accordingly (see Algorithm 1 Despite this difference, it is interesting that a direct connection can be established between the uncertainty sets of the methods. The following section is dedicated to this.
To be pedantic it is worth pointing out minor discrepancy when applying CVaR in the context of deep learn- ing. Common architectures such as ResNets incorporates batch normalization which updates the running mean and variance the forward pass. The implication is that the entire mini-bated is used to update these statistics despite the gradient computation only relying on the worst subset. This makes implementation in this setting slightly more convoluted.
B.1 Relationship with CVaR
Exp3 is run with uniform mixing to enforce exploration. This turns out to imply the necessary and sufficient condition for CVaR. We make this precise in the following lemma:
Lemma 5. Let p be a uniform mixing p = ? + (1- with an arbitrary distribution E Am and e [0, 1] such that the distribution has lower bound on each element ps 2 7 1. Then an upper bound is implicitly implied on each element, ||p|loo := max;=|m] < (m-1))
Proof. Consider Pt for any i. At least (m- 1m-1)] 1) 2 of the mass must be on other components so p; S 1 (m--))
The result. follows.
Corollary 1. Consider the uncertainty set of Eap3,
Exp3 (Po) = p 1m
1
Vi
(32)
Given that the above lower bound is only introduced for practical reasons, we might as well consider an instantiation of CVaR which turns out to be a proper relaration,
{ 1p P Am I1ellc S am
CVaR (Po)
(33)
17

---

Published in Transactions on Machine Learning Research (1/2023)
Table 5: Summary of methods where N is the number of samples and k the number of classes. The discrete distribution p either governs the distribution over all N samples or over the ht classes depending on the algorithm (see columns).
Uncertainty set 4xxp3 (Po) {pe Am ucVaR (Po) {pE Am
Over data point (m- NN
Over class labels (m
pi 2 E m
Vi) am
FOL (Shalev-Shwartz & Wexler 2016
CFOL (ours)
p
CVaR
Levy et al. 2020
LCVaR [Xu et al, 2020
with (1-3)m+y- This leaves with the following primal formulation,
CVaR., Po) = sup
Σ (0, 8.1. I| S
(34)
PEdm
am
Proof. From Lemma and since CVaR requires
1|p|,x S am
we have,
(m 1)y
(35)
m
am
By simple algebra we have,
a= - (1-7)m +7
(36)
which completes the proof.
So if the starting point of the uncertainty set is the simplex and the uniform mixing in Exp3 is therefore only for tractability reasons, then might as well minimize the CVaR objective instead. This could even potentially lead to more robust solution as the uncertainty set is larger (since the upper bound in u UVaR does not imply the lower bound in Exp3).
There are two things to keep in mind though. First, a should not be too small since the optimization problem gets harder. It is informative to consider the case where 7 1/2 such that a m+1 From this it becomes clear that the recasting as CVaR only works for small m. Secondly, despite the uncertainty being related, the training dynamics, and thus the obtained solution, can be drastically different, observe experimentally in Sention 5 For instance CVaR is known for having high variance (Curi et al. 2019) while the uniform mixing in CFOL prevents this. It is worth noting that despite this uniform mixing WC are still able to show convergence for CFOL in terms of the worst class in Section 4.2
2
also
In Table we provide an overview of the different methods induced by the choice of uncertainty set.
18

---

Published in Transactions on Machine Learning Research (1/2023)
C
Experiments
(a)
|0 1/255
(b) || = 2/255
1.0
1.0
Robust a accuracy
Robust accuracy
0.8
0.8
0.6
0.6
airplane
automobile
bird
deer
frog
horse
ship
truck
airolane
automoble
bird
car
deer
dog
frog
home
ship
truck
Figure 4: Robust test accuracy under different norm-ball sizes after clean training on CIFAR10. The non- uniform distribution over class accuracies, even after clean training, indicates that the inferior performance on some classes is not a consequence of adversarial training. Rather, the inhomogeneity after perturbation seems to be an inherent feature of the dataset.
Clean training
Adversarial training
1.0
1.0
test accuracy
0.8
Robust test test-accurac accuracy
0.8
0.6
0.6
0.4
0.0
0.0
Classes
Classes
Clean training
Adversarial training
1.0
1.0
accuracy
Robusttest accuracy
0.8
0.8
0.6
0.6
0.4
0.4
0.2
0.2
airplane
bird
car
cat
deer
dog
horse
monkey
ship
truck
airplane
bird
cat.
deer
dog
horse
monkey
ship
truck
Figure Clean training and adversarial training on CIFAR100 (top) and STL10 (bottom) using ERM- AT with PGD-7 attacks train time and PGD-20 attacks test time for the robust test accuracy. The CIFAR100 classes sorted for convenience. Notice that CIFAR100 has class with robust accuracy with adversarial training.
19

---

Published in Transactions on Machine Learning Research (1/2023)
ERM-AT
CFOL-AT
0.7
0.7
Robust accuracy
0.6
Robust tast accuracy
0.6
0.5
0.4
0.3
0.3
0.2
0.2
0.1
0.1
0.0
0.0
ERM-AT
CFOL-AT
test accuracy
0.6
Robust accuracy
0.6
0.4
0.4
0.0
0.0
airpiane bird
car
deer
dog
hurse
monkey
ship
trick
airplame
bird
car
deer
dog
horsa
monkey
ship
truck
Figure 6: Robust class accuracy for CIFAR100 and STL10 respectively. Vertical black error bars indicate one standard deviation. The red and black horizontal line indicates the minimum and average respectively. The classes on both ERM-AT and CFOL-AT are ordered according to the accuracy on FRM-AT to make comparison easier.
Table 6: For fair comparison we also consider early stopping based the worst class accuracy on the hold-out set. As can be observed the results for CFOL-AT do not differ significantly from early stopping using the average robust accuracy, standard training setups do not have to be modified further.
ERM-AT
CFOL-AT
LCVaR-AT
FOL-AT
Average
0.8348
0.8336
0.8298
0.8370
aCCclean
20% tail
0.6920
0.7460
0.6640
0.7045
Worst class
0.6710
0.7390
0.6310
0.7000
CIFAR10
Average
0.4907
0.4939
0.5125
0.4982
acCrob
20% tail
0.2895
0.3335
0.2480
0.2840
Worst class
0.2860
0.3260
0.2450
0.2780
Average
0.5824
0.5537
0.5800
0.5785
aCColean
20% tail
0.3115
0.3550
0.3065
0.3105
Worst class
0.1500
0.2200
0.1500
0,1700
CIFAR100
Average
0.2622
0.2483
0.2629
0.2552
acCrob
20% tail
0.0540
0.0790
0.0605
0.0595
Worst class
0.0200
0.0400
0.0100
0.0200
20

---

Published in Transactions on Machine Learning Research (1/2023)
Table 7: Model performance on CIFARIO (left) and Imagenette (right) under AutoAttack (Croce & Hein 2020p. The models still uses steps of PGD training time with a & constraint of $/255. Only at test time is the attack exchanged with AutoAttack under the same constraint. CFOL-AT is robust AutoAttack in the sense that the worst class performance is still improved. However, expected, the performance is worse for both methods in comparison with 20-step PGD based attacks.
ERM-AT
CFOL-AT
ERM-AT
CFOL-AT
Average
0.8244
0.8342
Average
0.8638
0.8650
aCCclean
20% tail
0.6590
0.7510
aCCclean
20% tail
0.7687
0.7890
Worst class
0.6330
0.7390
Worst class
0.7150
0.7709
Average
0.4635
0.4440
Average
0.5911
0.5838
aCCrob
20% tail
0.1465
0.2215
aCCrob
20% tail
0.3576
0.4087
Worst class
0.1260
0.2070
Worst class
0.2254
0.3187
Table 8: Comparison between different sizes bbll attacks on CIFARIO. The same constraint used both training and test time. When the attack size is incrcased beyond the usual 8/255 constraint we still observe that CFOL-AT increases the robust accuracy for the weakest classes while taking minor drop in the average robust accuracy. Interestingly, the gap between ERM-AT and CFOL-AT seems enlarge. See Appendix C.2 for more detail on the attack hyperparameters.
ERM-AT
CFOL-AT
Average
0.8244
0.8308
aCCelean
20% tail
0.6590
0.7120
Worst class
0.6330
0.6830
|10||cc S 8//255
Average
0.5138
0.5014
acCrob
20% tail
0.2400
0.3300
Worst class
0,2350
0.3200
Average
0.7507
0.7594
aCCelean
20% tail
0.4565
0.6060
Worst class
0.3850
0.5730
||0 & 12/255
Average
0.4054
0.3873
acC,ob
20% tail
0.1180
0.2095
Worst class
0.0930
0.1700
Table 9: We test the reweighted variant of CFOL-AT (described in Section 4.1 on CIFAR10 and observe similar results as for CFOL-AT. The experimental setup described in Section
aCCelean
aCCroh 20% tail
Average
20% tail
Worst class
Average
Worst class
CFOL-AT (reweighted)
0.8286
0.7465
0.7370
0.4960
0.3290
0.3100
21

---

Published in Transactions on Machine Learning Research (1/2023)
Table 10: Mean and standard deviation computed over 3 independent executions using different random sceds for both ERM-AT and CFOL-AT on CIFAR1O.
ERM-AT
CFOL-AT
Average
0.8324 = 0.0069
0.8308 + 0.0054
aCCclean
20% tail
0.6467 + 0.0273
0.7192 + 0.0436
Worst class
0.5747 + 0.0506
0.6843 + 0.0121
Average
0.5158 + 0.0033
0.5012 + 0.0008
acCrob
20% tail
0.2540 0.0428
0.3393
0.0398
Worst class
0.2067 0.0254
0.3083
0.0515
ERM-AT
CFOL-TT
Robust test accuracy
Robust test accuracy
0.4
0.4
0.2
0.2
0.0
0.0
airplone
automobile
bird
cat
deer
dog
frog
horse
ship
truck
airplane
automobile bird
cat
deer
dog
frog
horse
ship
truck
Table 11: A naive baseline improves the 20%-tail (marginally) over ERM-AT but is not able to improve the worst class. The baseline first runs ERM-AT to obtain the robust class accuracies on the test set. The negative accurcies are pushed through a softmax to obtain a distribution and ERM-AT is rerun with classes sampled from = 7 + (1 - 7)4.
0.5
0.9
Average
0.8384
0.8277
aCCelean
20% tail
0.6810
0.6470
Worst class
0.6340
0.6430
Average
0.5070
0.5094
acCrob
20% tail
0.2760
0.2530
Worst class
0.2180
0.2330
Table 12: We investigate the effect of using a diffent architecture (VGG-16 (Simonyan Zisserman, 2014)) on CIFAR10. The worst classes for CFOL-AT remains improved over ERM-AT. Unsurprisingly the older network performs worse for both methods when compared with their ResNet-18 counterpart.
ERM-AT CFOL-AT
Average 20% tail
0.7805
0.7904
aCCclean
0.5385
0.6550
Worst class
0.5020
0.6220
Average
0.4799
0.4606
acCrob
20% tail
0.2165
0.3075
Worst class
0.1940
0.3070
22

---

Published in Transactions on Machine Learning Research (1/2023)
Experiments on Tiny ImageNet (Russakovsky et al. 2015) which consists 100,000 images across 200 classes. CFOL-AT improves the average accuracy for the 123 worst classes without optimizing the hyperparameters (the hyperparameters remains the same as for CIFARIO, CIFAR100 and STL10).
Table 13:
atiieatve robust average accuracy
ERM-AT
ERM-AT
CFOL-AT
CFOL-AT
Average
0.4784
0.4606
0.15
aCCelean
20% tail
0.1725
0.2685
Worst class
0.0200
0.1600
0.10
Average
0.2349
0.2023
Cuinulative
0.05
&CCrab
20% tail
0.0445
0.0735
Worst class
0.0000
0.0200
0.00
0
25
50
75
100
125
150
175
200
ascending accuracy)
Table 14: Imbalanced CIFAR10 with imbalance factor 10 such that the majority class and the minority class has 5000 and 500 training samples respectively. We carly stop based on the uniformly distributed test set. Three variants of CFOL are considered. Both the mixing distribution ion and initialization the adversary do can be either uniform over classes (U) or according to the empirical training distribution (E), For instance if qº follows the empirical distribution (E) and the mixing distribution is uniform (U), then we suffix CFOL- AT with 'EU". We observe that CFOL-AT EE in particular improves the worst class accuracies, while the average accuracy incurs understand the seeming tradeoff between clean accuracy, average robust accuracy and robust worst class accuracy cause by different instantiations of CFOL.
smaller drop than under uniform CIFAR10 (Table [ It is interesting to
ERM-AT CFOL-AT UU CFOL-AT EU
CFOL-AT EE
Average
0,7300
0.7772
0.7708
0.7586
aCCelean
20% tail
0.5770
0.6780
0.6450
0.6420
Worst class
0.5410
0.6730
0.6360
0.6000
Average
0.4065
0.3988
0.4098
0.3991
acCrob
20% tail
0.2200
0.2550
0.2625
0.2625
Worst class
0.2140
0.2220
0.2240
0.2590
Cumulative robust average accuracy
0.40
ERM-AT
CFOL-AT
0.35
CFOL-AT EE
0.30
0.25
8
Classes (in ascending accuracy)
23

---

Published in Transactions on Machine Learning Research (1/2023)
CFOL
0.8
ERM
0.8
5
0.7
5-
0.7
0.6
0.6
25
25
0.5
0.5
Epoch
50
Epoch
50
95
95
0.3
0.3
100
100
102
0.1
103
0.1
2
0.0
0.0
Classes
FOL
0.8
LCVaR
0.8
0.7
5-
0.6
0.6
Epoch
Epoch
0.4
0.4
0.2
0.2
103
0.1
0.1
2
0.0
0.0
Classes
Classes
Figure 7: The evolution of the robust validation accuracies across epochs. The final epoch is when the model was early stopped based on the average robust accuracy. There is a clear pattern in what classes are harder (class 2-5). Both CFOL and FOL have a curious non-monotonic evolution for the class conditional accuracy, Notice for instance that both methods drop in accuracy for the Oth class the stepsize change at epoch 100.
Table 15: Imagenette classification using ResNet-18.
ERM-AT
CFOL-AT
Average
0.8638
0.8650
aCCelean
20% tail
0.7687
0.7890
Worst class
0.7150
0.7709
Average
0.6285
0.6181
acCrob
20% tail
0.4026
0.4534
Worst class
0.2850
0.3912
C.1 Hyperparameters
In this section we provide additional details to the hyperparameters spocified in Section
For CFOL-AT we the same parameters for ERM-AT described in Section We set. 7 1/2 and the adversarial step-size 17 2 1-100 across all experiments, if not otherwise noted. One exception is Imagenette were number of iterations are roughly 5 times fewer due to the smaller datasct. Thus, 17 is picked 2.5 times larger as suggested by theory through 1) Õ(1v/T) in Theorem [ CFOL-AT seems to be reasonable robust to step-size choice as seen in Table 16 Similarly for FOL-AT we use the same parameters as for ERM-AT and set 77 = 10-7 after optimizing based on the worst class robust accuracy on CIFAR10.
The adversarial step-size picked for both FOL-AT and CFOL-AT smaller in practice than theory suggests. This suggests that the mistake bound in Assumption not satisfied for any sequence. Instead rely
24

---

Published in Transactions on Machine Learning Research (1/2023)
Table 16: Hyperparameter exploration for CFOL-AT, LCVaR-AT and FOL-AT on CIFARIO.
Parameters
aCCrob
aCCelean
7
a
Average
Worst class
Average
Worst class
1
10
0.5076
0.2700
0.8372
0.7110
2 10-6
0.5014
0.3200
0.8308
0.6830
CFOL-AT
5
10-6
0.4963
0.3000
0.8342
0.7390
x 10-5
0.4927
0.2850
0.8314
0.7160
0.1
0,4878
0.1590
0.7909
0.3120
0.2
0.5116
0.1250
0.8274
0.4710
ACVaR-AT
0.5
0.5104
0.1890
0.8177
0.5550
0.8
0.5169
0.2440
0.8259
0.6340
0.9
0.5249
0.1760
0.8233
0.5170
X 10-
0.5106
0.2370
0.8280
0.6210
5 X 10-7
0.5038
0.2180
0.8372
0.6100
FOL-AT
10 10-6
0.4885
0.2190
0.8237
0.5880
5 10
0.4283
0.1560
0.8043
0.5380
the sampling process to only mildly adversarial initially as implicitly enforced by the small adversarial step-size. It an interesting future direction to incorporate this implicit tempering directly into the mixing parameter y instead.
For I.CaR--T optimized over the size of the uncertainty set by adjusting the parameter CIFAR1O. This leads to the choice of 0.8 which use across all subsequent datasets. The hyperparameter exploration can be found in Table 16
C.2 Experimental setup
In this section we provide additional details for the experimental setup specified in Section 5 GPU on an internal cluster. The experiments are conducted on the following five datasets:
We use one
CIFAR10 inchudes 50,000 training examples 32 x 32 dimensional images and 10 classes.
CIFAR100 includes 50,000 training examples of 32 32 dimensional images and 100 classes.
STL10 includes 5000 training examples of 96 x 96 dimensional images and 10 classes.
TinyImageNet 100,000 training examples of 64 x 64 dimensional images and 200 classes.
Imagenette 9469 training examples of 160 x 160 dimensional images and 10 classes.
For all experiment we use data augmentation in the form of random cropping, random horizontal flip, color jitter and degrees random rotations. Prior to cropping a black padding added with the exception of Imagenette. For Imagenette we follow the standard for ImageNet evaluation and center crop the validation and test images.
Early stopping As noted in Section 5 we use the average robust accuracy to early stop the model. contrast with common practice though, we use a validation set instead the test set to avoid overfitting to the test set. The class accuracies across the remaining two datasets, CIFAR100 and STL10, can be found in Figure 0. We also include results when the model is early stopped based the worst class accuracy the validation set in Table 0
http:::/ggtaub..cm/fftttst/////
25

---

Published in Transactions on Machine Learning Research (1/2023)
Attack parameters A radius of 6/255 is used for the fo--corstrii attack unless otherwise noted. For training we use 7 steps of PGD and step-size of 2/255. At test time we use 20 steps of PGD with step- size of 2.5 8/206 For 13/25s-bunnded attacks in Table scale the training step-size and test step-size proportionally.
Larger models For the larger model experiments in Table we train Wide ResNet-34-10 using CFOL- AT (with 0.8). Similarly to the pretrained weights from the literature, we early stop based on the test-set. We compare against TRADES (Zhang et al.) 2019 (Wide ResNet-34-10) and the training setup of Madry et al. (2017), which we throughout have denoted as ERM-AT, using shared weights from Engstrom et al. (018) (ResNel-50).
Mcan & standard deviation In Table 10 we apply CFOL-AT and ERM-AT CIFAR10 for multiple different random seeds and verify that the observations made in Section S remains unchanged.
C.3 Implementation
We provide pytorch pseudo code for how CFOT, can be integrated into existing training setups in Listing 1 FOL is similar in structure, but additionally requires associating unique index with each training example. This allows the sampling method to re-weight each example individually.
For L.VaR we use the implementation of (Xu et al. 2020). which uses the dual CVaR formulation described in Appendix B More specifically, LCVaR uses the variant which finds a closed form solution for X, since this was observed to be both faster and more stable in their work.
Listing Pseudo code for CFOL.
from torch. utils data import DataLoader from class_ _ampler import ClassSampler
sampler = ClassSampler (dataset, gamma=0.5)
dataloader = DataLoader ( dalasee, .... sampler=sampler)
# Training loop:
for img,y in iter (dtaaloderr):
# allack img
# compute gradients and update model # compute logils
loss logits argmax (dim=-1) != y sampler. batch update (y, eta adv loss)
26


Reviews:
[{"review_id": "lQwdYp5jWb", "reviewer": "Reviewer_Tj9o", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "-The model is well-motivated, in some applications, such an imbalance could be problematic.\nThis is also supported by an example that shows a larger variance in losses between classes than in the standard setting.\n(The perspective of adversarial examples where the adversary chooses what class to evaluate on in addition\nto the perturbation, seems unrealistic to me. However, I believe that problem still models some interesting scenarios).\n\n-The suggested method is intuitive and seems effective in rebalancing the loss over different classes.\n\n-The technical novelty of the convergence guarantee is only marginally novel and follows mainly from known analysis (if I'm not missing something), but it is good to include it.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The setting:\nThe setting is robust learning to perturbations at test time (adversarial examples), where instead of minimizing the average loss, the goal is to minimize the loss of the class with the lowest accuracy. \nSo instead of a min-max problem objective, the resulting objective consists of another maximization component over the classes, and is named \"worst class-conditioned risk\".\n\nContributions:\nThe authors suggest a method from online learning - exponential weight in the bandit setting (that is - Exp3).\n\nMoreover, they provide a convergence guarantee with high probability.\n\nFinally, the effectiveness of the suggested method is tested empirically, on CIFAR100 and STL10. \n\n", "claims_and_evidence_yn": "", "claims_explanation": "", "audience_yn": "", "audience_explanation": "", "requested_changes": "No request for changes.\n\n", "broader_impact_concerns": "-", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "UdnYtkjKHt", "reviewer": "Reviewer_ujcb", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "**Strength**\n\n* The paper is clearly written and mathematically sound. \n* The claims are backed up both in theoretically and empirically\n* The experimental results consistently demonstrate the effectiveness over the baselines\n\n\n**Weakness**\n\n* The empirical significance of the proposed method could be questionable, especially in CIFAR-100 which has more number of classes and thus more realistic, e.g., there is only 0.00 → 0.04 of accuracy gain in worst-case robustness\n* The paper specifies that the major evaluation metric (those reported in the main tables) is from PGD-20, although it also reports AutoAttack results in Appendix. I think the stronger attack of AutoAttack should be the major evaluation protocol to report, given that the paper proposes a modification on adversarial training that is sometimes witnessed to show a false defense claim without a thorough evaluations.\n* Any discussion on the (practical) training overhead compared to standard AT would help for practitioners to use the proposed method.\n\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper considers a class-conditional form of adversarial training where an adversary is further allowed to choose the worst-performing class $y$ in the minimax optimization - which is related to some safety critical applications. The paper relaxes the given formulation in discrete optimization of $y$ with a continuous online learning algorithm and proposes techniques to improve convergence rate and unbiasedness of the algorithm, as well as performing a convergence rate analysis. Experimental results on CIFAR-10/100 and STL show that the proposed method successfully recovers a proper distribution of $y$ suitable for the worst-class adversarial training, improving the worst-class robust accuracy of adversarial training compared to baselines.", "claims_and_evidence_yn": "", "claims_explanation": "", "audience_yn": "", "audience_explanation": "", "requested_changes": "See the Weakness above.", "broader_impact_concerns": "None that I am aware of.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "VcEJntQgGs", "reviewer": "Reviewer_oVMu", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "The main strengths of the submission are: \n+ The approach appears to be theoretically grounded, with a convergence rate guarantee. In practice, this allows the algorithm to only need to optimize over the worst class at each iteration instead of all classes at once. Though I am less of an expert on the online learning literature and cannot fully vouch for its technical correctness. \n+ Experimentally, the results consistently lead to sizeable improvements for the worst-case class. The comparison points in the adversarial robustness space are reasonable (standard adversarial training & trades), which understandably perform worse as they do not optimize for the worst case class. \n+ The method has only a small amount of overhead,\n\nOne weaker aspect of this paper is that the experimental results are on 3 fairly similar datasets (CIFAR10/CIFAR100/STL10). It would be nice to see results on substantially different datasets to demonstrate the broader applicability of this method, i.e. on Imagenet or other non-32x32 pixel classification tasks. ", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper studies the problem of adversarial training where the desired metric is to ensure the robustness of the worst-case class as opposed to the average-case class. In contrast to standard training, in adversarial training there is an even larger gap between the best and the worst performing classes (with respect to robust accuracy). \n\n+ The authors propose an online-learning type algorithm that adjust the adversarial training algorithm to optimize for the worst-case class. \n+ The authors prove a convergence rate for the algorithm\n+ The authors evaluate their approach on CIFAR10, CIFAR100, and STL10 and compare to various other adversarial training and robust optimization approaches. ", "claims_and_evidence_yn": "", "claims_explanation": "", "audience_yn": "", "audience_explanation": "", "requested_changes": "I would suggest the authors run a similar comparison on ImageNet (or a restricted/smaller version thereof). Other different settings could also work as well---any additional setting would certainly help argue for a broader argument that the approach works on more than just classification of 32x32 images. \n\nWhile I think this would be nice to have, the paper includes other non-empirical results such as convergence guarantees, and thus this would simply strengthen the work in my view and is not critical. ", "broader_impact_concerns": "N/A", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: All reviewers were generally satisfied with the authors' response to their review and recommended acceptance of the paper.
Decision: Accept (poster)