August 7, 2022
DRAFT

Statistical Learning under Adversarial
Distribution Shift
Chen Dan
November 2021

School of Computer Science
Carnegie Mellon University
Pittsburgh, PA 15213
Thesis Committee:
Pradeep Ravikumar, Chair
Zico Kolter
Zachary Lipton
Avrim Blum (Toyota Technological Institute in Chicago)
Yuting Wei (University of Pennsylvania)
Submitted in partial fulfillment of the requirements
for the degree of Doctor of Philosophy.

Copyright © 2021 Chen Dan

August 7, 2022
DRAFT

Keywords: Statistical Learning, Distribution Shift, Adversarial Examples, Class ImbalanceAugust 7, 2022
DRAFT

Abstract
One of the most fundamental assumptions in statistical machine learning is thattraining and testing data should be sampled independently from the same distribution.
However, modern real world applications require that the learning algorithm should
perform robustly even when this assumption is no longer valid. Specifically, thetraining and testing distributions may shift slightly (yet adversarially) within a smallneighborhood of each other. This formulation encompasses many new challenges inmachine learning, including adversarial examples, outlier contaminated data, groupfairness and label imbalance.
In this thesis, we seek to understand the statistical optimality and provide better
algorithms under the aforementioned adversarial distribution shift. Our contributions
include (1) the first near optimal minimax lower bound for the sample complexity ofadversarially robust classification in a Gaussian setting. (2) introducing the frameworkof distributional and outlier robust optimization, which allowed us to apply distributionally robust optimization to large scale experiments with deep neural networks andoutperformed existing methods in sub-population shift tasks. (3) margin sensitive
group risk, a principled way of improving distributional robust generalization via
group-asymmetric margin maximization.

August 7, 2022
DRAFT

iv

August 7, 2022
DRAFT

Contents
1

Introduction
1.1 Organization of this thesis . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

1
3

2

Statistical Minimax Guarantees for Adversarially Robust Classification
2.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
2.1.1 Our contributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
2.1.2 Other related works . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
2.1.3 Notations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
2.2 Preliminaries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
2.3 A Coputationally Efficient Estimator and Risk Upper Bound . . . . . . . . . . .
2.4 Minimax Lower Bounds . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
2.5 Comparing Adversarial and Standard Rates . . . . . . . . . . . . . . . . . . . .
2.6 Proofs and further details . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
2.6.1 Proof of Theorem 2.3.1 . . . . . . . . . . . . . . . . . . . . . . . . . . .
2.6.2 Proof of Lemma 2.4.1 . . . . . . . . . . . . . . . . . . . . . . . . . . .
2.7 Proof of Theorem 2.2.1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
2.8 Proof of Proposition 2.5.1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
2.9 Improved analysis when Σ is known . . . . . . . . . . . . . . . . . . . . . . . .
2.10 Proof of Lemma 2.6.3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

5
5
6
6
7
7
11
13
15
16
16
18
21
23
24
25

3

Distributional and Outlier Robust Optimization
3.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3.2 Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3.2.1 Subpopulation Shift . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3.2.2 Distributionally Robust Optimization (DRO) . . . . . . . . . . . . . . .
3.3 DRO is Sensitive to Outliers . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3.4 DORO . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3.5 Theoretical Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3.6 Experiments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3.6.1 Setup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3.6.2 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3.6.3 Effect of Hyperparameters . . . . . . . . . . . . . . . . . . . . . . . . .
3.7 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3.8 Proofs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

27
27
29
29
30
31
33
34
36
36
37
38
38
39

v

August 7, 2022
DRAFT

3.9

3.8.1 Proof of Proposition 1 . . . . . . . . . . . . . . . . . . . . . . . . . . .
3.8.2 Proof of Corollary 2 . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3.8.3 Proposition 3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3.8.4 Proofs of Results in Section 3.5 . . . . . . . . . . . . . . . . . . . . . .
Experiment Details . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3.9.1 Domain Definition . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3.9.2 Model Selection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3.9.3 Training Hyperparameters . . . . . . . . . . . . . . . . . . . . . . . . .

39
40
40
42
51
51
52
54

4

High Dimensional Imbalanced Classification
59
4.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60
4.1.1 Binary classification . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60
4.1.2 Surprises in high-dimensional imbalanced classification . . . . . . . . . 61
4.1.3 Our contributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63
4.1.4 Other related works . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63
4.2 Analysis of re-weighted M-estimators . . . . . . . . . . . . . . . . . . . . . . . 65
4.2.1 A warm-up example: Diagnosis of LDA . . . . . . . . . . . . . . . . . . 65
4.2.2 Main results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65
4.2.3 Bias correction for M-estimators . . . . . . . . . . . . . . . . . . . . . . 68
4.3 Sharp non-asymptotic analysis of Deev’s estimator . . . . . . . . . . . . . . . . 70
4.4 Minimax lower bounds . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74
4.4.1 The Bayesian connection to minimax risk . . . . . . . . . . . . . . . . . 75
4.4.2 Imbalanced lower bounds . . . . . . . . . . . . . . . . . . . . . . . . . 76
4.4.3 The importance of a carefully selected prior . . . . . . . . . . . . . . . . 87
4.4.4 Additional lemmas . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87
4.5 Numerical experiments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 88
4.6 Proof of Theorem 4.2.2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92
4.7 Comparison with Related Works . . . . . . . . . . . . . . . . . . . . . . . . . . 95
4.7.1 Comparison with [62] . . . . . . . . . . . . . . . . . . . . . . . . . . . 95
4.8 Technical Lemmas . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96
4.8.1 Moreau Envelope . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96
4.8.2 Solving the asymptotic version of AO numerically . . . . . . . . . . . . 97
4.8.3 Lemmas for Lower Bounds . . . . . . . . . . . . . . . . . . . . . . . . 99
4.8.4 Exact Asymptotic Minimax Risk in Balanced Setting . . . . . . . . . . . 101
4.8.5 Other Technical Lemmas . . . . . . . . . . . . . . . . . . . . . . . . . 102
4.9 LDA and Square Loss Minimization . . . . . . . . . . . . . . . . . . . . . . . . 102

5Interpolation in Distributionally Robust Optimization
105
5.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105
5.1.1 Related work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 106
5.2 Preliminaries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107
5.2.1 Reweighting Algorithms . . . . . . . . . . . . . . . . . . . . . . . . . . 107
5.2.2 Reweighting algorithms can easily overfit . . . . . . . . . . . . . . . . . 108
5.3 Implicit biases of reweighting algorithms . . . . . . . . . . . . . . . . . . . . . 108
viAugust 7, 2022
DRAFT

5.4

5.5
5.6
5.7

5.8

5.9

6

5.3.1 Linear models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109
5.3.2 Linearized neural networks . . . . . . . . . . . . . . . . . . . . . . . . . 110
5.3.3 Wide fully-connected neural networks . . . . . . . . . . . . . . . . . . . 111
Does regularization really help? . . . . . . . . . . . . . . . . . . . . . . . . . . 113
5.4.1 Theoretical analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113
5.4.2 Empirical study . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 114
Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 116
Other reweighting algorithms . . . . . . . . . . . . . . . . . . . . . . . . . . . . 117
Proofs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 118
5.7.1 Proof of Theorem 11 . . . . . . . . . . . . . . . . . . . . . . . . . . . . 118
5.7.2 Proof of Theorem 14 . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121
5.7.3 Proof of Theorem 15 . . . . . . . . . . . . . . . . . . . . . . . . . . . . 122
5.7.4 Proof of Lemma 21 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 128
5.7.5 Proof of Theorem 16 . . . . . . . . . . . . . . . . . . . . . . . . . . . . 131
5.7.6 Proof of Theorem 17 . . . . . . . . . . . . . . . . . . . . . . . . . . . . 131
5.7.7 Proof of Theorem 18 . . . . . . . . . . . . . . . . . . . . . . . . . . . . 132
A note on the proofs in Lee et al. [82] . . . . . . . . . . . . . . . . . . . . . . . 135
5.8.1 Their problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 135
5.8.2 Our fixes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 138
Experiment details and additional experiments . . . . . . . . . . . . . . . . . . . 139
5.9.1 Experiment details . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 139
5.9.2 Sample weights converge in Group DRO . . . . . . . . . . . . . . . . . 139

MSG: Margin Sensitive Group Risk
141
6.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 141
6.2 Preliminaries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 142
6.2.1 Problem Formulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 142
6.2.2 Domain-incomplete Setting and Post-hoc Weight Normalization . . . . . 143
6.2.3 Generalized Logit Adjustment (GLA) . . . . . . . . . . . . . . . . . . . 144
6.3 MSG: Margin Sensitive Group Risk . . . . . . . . . . . . . . . . . . . . . . . . 145
6.3.1 Derivation of the MSG-Risk . . . . . . . . . . . . . . . . . . . . . . . . 145
6.3.2 Alternating Minimization for the Domain-incomplete Setting . . . . . . . 147
6.3.3 End-to-end Training with Stochastic Gradient Descent for the Domainaware Setting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 148
6.4 Experiments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 148
6.4.1 Setup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 148
6.4.2 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 149
6.5 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 151

vii

August 7, 2022
DRAFT

viii

August 7, 2022
DRAFT

List of Tables
3.1
3.2
3.3

3.4
3.5

CVaR and χ2 -DRO. α is the ratio between the size of the smallest domain and thesize of the population. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
The average and worst-case test accuracies of the best models achieved by different methods. (%) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37
Standard deviations of average/worst-case test accuracies during training on
CelebA. (α = 0.1 for CVaR/CVaR-DORO; α = 0.3 for χ2 -DRO/χ2 -DORO.
 = 0.01) (%) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37
Number of training instances in each domain of CelebA and CivilComments-Wilds. 52
The average and worst-case test accuracies of the best models selected by different
strategies. (%) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53

5.1

Mean average training accuracy and worst-group test accuracy (%) of the last 10
training epochs of ERM, IW and Group DRO under different levels of weightdecay (WD). Each entry is Average training accuracy / Worst-group test accuracy.
Blue entries are mean accuracies of epochs 11-20 with no weight decay. Eachexperiment is repeated five times with different random seeds. . . . . . . . . . . 115

6.1

Results for CelebA and CivilComments-Wilds. Each experiment is run with
5 different random seeds, and the mean and std. dev. of the worst-group test
accuracies (%) are reported. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 150
Results for CIFAR-10 with Long-Tail or Step class imbalance (“-100” means thatthe size of the largest class is 100 times that of the smallest). Each experiment is
run with 5 different random seeds, and the mean and std. dev. of the balanced
average test accuracies (%) are reported. . . . . . . . . . . . . . . . . . . . . . . 150
The optimal δk (= ρ−1k ) found by post-hoc weight normalization with alternating
minimization. Each experiment is run 5 times. . . . . . . . . . . . . . . . . . . . 151

6.2

6.3

ix

August 7, 2022
DRAFT

x

August 7, 2022
DRAFT

Chapter 1
Introduction
One of the most fundamental assumptions in statistical machine learning is that training and testingdata should be sampled independently from the same distribution. Mathematically speaking, ina supervised learning problem where we have feature vector x ∈ Rd and label y ∈ R, it was
assumed that there is an underlying distribution P, such that the learner has access to n i.i.d.
training samples(x1 , y1 ), (x2 , y2 ), · · · , (xn , yn ) ∼i.i.d. Pn ,(1.1)
and the learner’s goal is to find a classifier fˆn : Rd → R based on observed training samples{(xi , yi )}ni=1 , which makes the test error Ltest as small as possible:
Ltest (fˆn ; P) := E(x,y)∼P l(fˆn (x), y).

(1.2)

Here, l(y 0 , y) can be chosen as square loss (y 0 − y)2 for regression problems, or zero-one loss
1[y 0 6= y] for classification problems.
Here, we can see that both training and testing samples are sampled from the same underlying
distribution P. However, in recent years, modern real world applications require that the learningalgorithm should perform robustly even when this assumption is no longer valid.
Here are some typical applications where the training and testing distributions differ fromeach other:Adversarial Examples While deep learning algorithms have achieved tremendous success in avariety of different domains such as image classification, natural language processing and strategy
games (e.g. Bahdanau et al. [8], Krizhevsky et al. [74], Silver et al. [117]), a crucial weakness,
i.e. adversarial examples, has been observed by recent works Szegedy et al. [121] (among otherse.g. Goodfellow et al. [46], Papernot et al. [100]). Namely, deep learning models often achieve
extremely accurate performances yet are susceptible to small perturbations of the inputs, i.e. one
can add small (nearly imperceptible) perturbation δ to image x , which cause neural networkclassifiers to make wrong predictions fˆn (x) far from ground truth y, with very high confidence.
Outlier Contaminated Data Outlier robust estimation is a classic problem in statistics startingwith the pioneering works of [58, 129]. The classic Huber’s ε-contamination model assumes thatat most ε fraction of training data can be contaminated i.e. sampled from any arbitrary distribution.
1August 7, 2022
DRAFT
Efficient algorithms for very basic tasks, e.g. mean estimation, remains unsolved until late 2010s[35, 36, 78, 103].
Label Imbalance Datasets with class imbalance — that is, the number of samples of one
class far exceeds the number of data of another class — are prevalent in cutting-edge data
science applications [26, 53]. Take COVID-19 testing data for example: a dominant fractionof data samples often come from the negative class (i.e., non-targeted people who have notcontracted the virus). The evaluation criterion in reality, however, might place equal, or evenhigher, emphasis on the minority class (e.g., the infected people in the COVID-19 case). The
ability to generalize favorably in both majority and minority classes plays a pivotal role in criticalscientific and societal issues (e.g., fairness/equity in machine learning, discovery of rare disease,transferability of knowledge to sample-starved tasks). It has been widely recognized, however,that the imbalanced availability of data can cause severe issues to modern data-limited learningalgorithms including neural networks (e.g., [20, 53, 131]), particularly when reasoning about theunderrepresented class.
Subpopulation Shift Another common type of distributional shift studied in this thesis is
subpopulation shift, where the training and testing distributions are both a mixture of the samegroup of subpopulations, while the mixing weights can be different. Mathematically, we assume
there are K subpopulations P1 , P2 , · · · , Pk , and training/testing distributions can be written as
Ptrain = w1 P1 + w2 P2 + · · · wK PK

(1.3)

0
Ptest = w10 P1 + w20 P2 + · · · wK
PK

(1.4)

It is commonly assumed that w and w0 are similar, yet not exactly the same, from each other.
Subpopulation shift is closely related to algorithmic fairness and class imbalance. The setting of
subpopulation shift is most closely related to the algorithmic fairness notion of Rawlsian Max-Min
fairness [51, 109].
In this thesis, we seek to understand the statistical optimality and provide better algorithms
under aforementioned types of distrbution shift. In fact, we can formulate all of these applications
as certain realization of adversarial distribution shift, defined in detail below.
Under the framework of adversarial distribution shift, we assume that Ptrain and Ptest can be
perturbed slightly, yet adversarially, from the underlying distribution P, and the performance ofthe classifier is evaluated as the worst possible outcome from such perturbations. Mathematically,
we assume there exists a collection of distributions Btrain (P ), Btest (P ), both close to P in certain
sense, and Ptrain ∈ Btrain (P ), Ptest ∈ Btest (P ). The learner has access to n i.i.d. training samples(x1 , y1 ), (x2 , y2 ), · · · , (xn , yn ) ∼i.i.d. P≈rain n ,(1.5)

and the learner’s goal is to find a classifier fˆn : Rd → R based on observed training samples{(xi , yi )}ni=1 , which makes the worst-case test error Ltest as small as possible:
sup

Ltest (fˆn ; Ptest ) := E(x,y)∼Ptest l(fˆn (x), y).

Ptrain ∈Btrain (P ),Ptest ∈Btest (P )

2

(1.6)

August 7, 2022
DRAFT
We can see that all of the four applications: adversarial examples, outlier contaminated data,label imbalance and sub-population shift, are realizations of this framework. (For the mathematicaldetails, we refer the readers to the corresponding chapters.)
Adversarial Examples: The testing distribution Ptest can be written as a sum of a vectorx ∈ P and a perturbation vector δ where kδk≤ ε.
Outlier contaminated data: The training distribution Ptrain can be written as a mixture
Ptrain = (1 − ε)P + εP 0 , where P 0 can be arbitrary.
Label Imbalance: Denote the class conditional distributions as P0 := P (X|Y = 0) andP1 := P (X|Y = 1). We assume that the testing distribution is a balanced mixture of P0 and P1 ,while the training distribution can be imbalanced.
P
PK
0
Subpopulation Shift: Assume that Ptrain = K
j=1 wj Pj and Ptest =j=1 wj Pj . We assume
that the testing distribution has the weight w0 satisfying a bounded divergence condition:
Df (w0 ||w) =

K
X
j=1

1.1

wj f (

wj0) ≤ t.
wj(1.7)

Organization of this thesis• In Chapter 2, we provide the first minimax lower bounds for adversarially robust classifica-tion in a Gaussian setting, along with an algorithm that achieves this lower bound. This isbased on our paper published in ICML 2020.
• In Chapter 3, we introduced the framework of distributional and outlier robust optimization(DORO), which allowed us to apply distributionally robust optimization to large scale experiments with deep neural networks and outperformed existing methods in sub-population
shift tasks. This is based on our paper published in ICML 2021.
• (Proposed Work) In chapter 4, we study the problem of imbalanced classification in ahigh dimensional Gaussian setting, where the number of samples n scale linearly withdimension d. This is a regime where classical theory breaks down (due to the requirementof n >> d). Our analysis reveals a surprising phenomenon: more samples can hurt theperformance of M-estimators, even when popular heuristic of re-weighting is applied. We
also derived a new lower bound which remains tight in the d = Θ(n) regime, showing thata bias-correcting estimator first proposed in Deev, 1970 is optimal.
• (Proposed Work) In chapter 5, we analyze the implicit bias of Distributionally Robust

Optimization methods for overparameterized and interpolating models. We show thatthe implicit bias of gradient-based DRO leads to the convergence to the same solution of
ERM in many settings, including linear regression and heavily overparameterized neuralnetworks.

3

August 7, 2022
DRAFT

4

August 7, 2022
DRAFT

Chapter 2
Statistical Minimax Guarantees for
Adversarially Robust Classification
2.1

Introduction

Recent years, machine learning algorithms have revolutionized our life due to their tremendous
success in a variety of different domains such as image classification, natural language processing
and strategy games (e.g. Bahdanau et al. [8], Krizhevsky et al. [74], Silver et al. [117]). Thesealgorithms often achieve extremely accurate performances yet are susceptible to small perturbations of the inputs. In particular, Szegedy et al. [121] (among others e.g. Goodfellow et al.
[46], Papernot et al. [100]) noticed that small perturbations (nearly imperceptible) to images
could cause neural network classifiers to make wrong predictions with high confidence. While a
growing amount of effort has been made in order to empirically improve the robustness of theselearning algorithms against adversarial attacks, the problems of assessing statistical optimality,
understanding generalization and statistical significance are important but far less understood. In
this paper, we take a step towards this end.
In this work, we consider the adversarially robust classification problem under the Gaussian
mixture model proposed by Schmidt et al. [113]. While the classification for mixture of Gaussian
distributions — which is also referred to as discriminant analysis — has now been standard in
statistics and computer science literature (see, e.g. McLachlan and Peel [89]), it is only until
recently that researchers start to consider what can go wrong in the adversarial scenarios for thissimple problem. It turns out (and as is shown in the sequel) that this simple yet instructive model
demonstrates clear tradeoffs between adversarially robustness and the statistical complexities, andat the same time, capturing some of the features one would encounter in real applications.
Under minimal assumptions of the adversarial perturbations, we provide optimal minimax
lower bounds, and show that a natural computationally efficient estimator achieves these minimax
lower bounds in terms of the adversarial signal to noise ratio. Putting these together gives a sharp
characterization of the intrinsic hardness of this problem in terms of how far one can push towards
a robust estimator without any essential loss of statistical accuracy. These optimal lower andupper bounds are useful since that they provide a comprehensive view of the adversarially robust
sample complexity of the conditional Gaussian model, which could then be contrasted with that5August 7, 2022
DRAFT
of the rates of the classical conditional Gaussian model.
Despite of an extensive line of work considering this problem, Schmidt et al. [113] and Bhagoji
et al. [12] lie most closely to this paper. In order to obtain tight statistical characterizations of
the risk, they made a number of simplifications, which thus do not directly provide answers to
the minimax sample complexity of the original problem. As one main contrast, they consider
the Bayesian setting where the means of the conditional Gaussians have as prior an independent
standard Gaussian distribution. For other simplifications, Schmidt et al. [113] considered thespherical models so that the covariance is identity and also made additional simplifications such aslarge separation between two Gaussians and an upper bound on the noise level. These additional
assumptions made it hard to compare with that of the adversary-free scenario. More detailed
comparisons and discussions are provided after our main results.

2.1.1

Our contributions

The main contributions of this paper are summarized below, all of which are built upon a carefulanalysis of the classification error for linear classifiers.
• We develop the first minimax lower bounds for the classification excess risk in the conditional Gaussian model, stated in Theorem 2.4.1. In terms of the Adversarial Signal-to-Noise
Ratio (AdvSNR), this excess risk scales as ΩP (exp(−( 18 + o(1))r2 ) nd ) for AdvSNR = r,dimension d and sample size n.
• We construct a computationally efficient estimator based on the solution of a constrained

quadratic optimization problem that has excess risk of order OP (exp(−( 18 + o(1))r2 ) nd ).
This result is given in Theorem 2.3.1. Hence, the upper bound is nearly tight (up to lower
order terms in r) with the minimax lower bound in our regime of interest in terms of
AdvSNR r, dimension d and sample size n .
• The recipe provided herein, works for a wide range of adversarial perturbations, generalizing

the result by Schmidt et al. [113] who focus only on the `∞ -type perturbations.
• Finally, our results are built upon minimum set of assumptions, without assuming strong

separations between two classes, allowing for unknown and arbitrary covariance structure
and the rates are naturally adaptive to the true signal.
Our findings unveil new insights into the adversarially robust sample complexity of theconditional Gaussian model which goes beyond of what the current theory has to offer.

2.1.2

Other related works

The conditional Gaussian models or mixture of Gaussians has been studied a lot in statistics andcomputer science literature. An incomplete and more recent list includes Azizyan et al. [7], Cai
and Zhang [22], Kim et al. [67], Li et al. [84, 85]. In the context of adversarial robustness, since
the seminal work of [113], there are several other papers that studied the sample complexity issue
in conditional Gaussian models. Bhagoji et al. [12] also provided a slightly improved bound in
the same setting. Carmon et al. [24], Stanforth et al. [118], Zhai et al. [154] showed that with thehelp of unlabeled data, it is possible to achieve high robust accuracy with the same number oflabeled data required for standard learning.
6August 7, 2022
DRAFT
Another line of research study the sample complexity of adversarially robust learning under the
PAC framework, using extensions of Rademacher complexity or VC dimension, including Attias
et al. [3], Khim and Loh [66], Yin et al. [151], Cullina et al. [29], Montasser et al. [95], Awasthi
et al. [6]. The tradeoff in standard and robust accuracy has been theoretically and empiricallystudied in Zhang et al. [158], Suggala et al. [119], Tsipras et al. [128], Raghunathan et al. [107]and Javanmard et al. [62].
Several previous works analyzed the robustness of specific family of classifiers. The early
work of Xu et al. [145, 146] estabilished the connections between robust optimization for linear
models and certain types of regularization in classification and regression settings. Subsequently,
Xu and Mannor [147] also showed that under certain notion of robustness, robust algorithms cangeneralize well. Wang et al. [139] studied the robustness of nearest neighbor classifiers.
From the aspect of computational complexity, some recent works showed that learning a robustmodel or even verifying robustness of a given model can be computationally hard, including[18, 19] and [5, 140].

2.1.3

Notations

For the reader’s convenience, we list here our notational√conventions.
For positive semi-definite matrix A, we use kxkA := xT Ax. Let Φ(·) be the CDF of standard
Gaussian distribution N (0, 1) and Φ̄(x) := 1 − Φ(x). The notation f (n, d) = O (g(n, d)) meansthat there exists a universal constant c > 0 that does not depend on the problem parameters such
as n, d etc, such that |f (n, d)|≤ c|g(n, d)|. Similarly, we define f (n, d) = Ω (g(n, d)) when there
exist constants c1 , c2 > 0 such that c1 |g(n, d)|≤ |f (n, d)|≤ c2 |g(n, d)|. Notation OP , ΩP areused if the corresponding relations happen with probability
converges to 1 as n → ∞ (see e.g.
Pd
p 1/p
Chapter 2 of [130]). We define the `p norm kxkp = ( i=1 xi ) and the corresponding `p -ball as{x ∈ Rd |kxkp ≤ 1}.

2.2

Preliminaries

This section is devoted to setting up the adversarial robust classification problem that is consideredin this paper. Along the way, we introduce necessary background and state several preliminary
results for future comparisons.
Conditional Gaussian Model We consider the binary classification problem with data pair(x, y) generated from the mixture of two Gaussian distributions Pµ,Σ ,11p(y = 1) = , p(y = −1) = ,22p(x|y) = N (x; yµ, Σ).
Here µ ∈ Rd , Σ ∈ Rd×d , Σ  0 denote the mean and covariance of the Gaussian distribution.
Given n training samples (xi , yi ) ∼i.i.d. Pµ,Σ for 1 ≤ i ≤ n, the goal is to learn a classifier fˆ(x)for predicting the class of a future data point that is drawn from the same distribution Pµ,Σ .
7August 7, 2022
DRAFT
Adversarially Robust Classification In the standard setting of classification, the optimalclassifier is defined as the one that which minimizes the population classification error
stdRµ,Σ(f ) := E(x,y)∼Pµ,Σ [I(f (x) 6= y)] .

which we refer to the standard error throughout. In this paper, we consider the classification
problem under conditional Gaussian generative model in presence of an adversary — which is to
say — at the testing stage, an adversary is allowed to add any perturbation δ to the input x, thathas bounded magnitude kδkB ≤ ε. The norm defined here is the standard Minkowski functionalthat associated with a convex set [124]. Formally, given a closed and origin-symmetric convex set
B, the Minkowski functional is defined as
kxkB := inf{λ ∈ R>0 : x ∈ λB}.
For instance, when B is the `p unit ball, then kxkB boils down to the classical `p norm of x. In
practice, the most widely considered norm for the adversary are `∞ and `2 norms.
In the adversarially robust setting, a mapping f : Rd → {−1, +1} classifies a sample (x, y)
correctly, if and only if the prediction agrees with the true label for all possible perturbations of
the adversary. To put it in mathematical form,`B,ε (f ; x, y) := I (∃δ : kδkB ≤ ε, f (x + δ) 6= y) .
Our goal is to obtain a classifier with minimal expected robust classification error, i.e. finding
mapping f that minimizes
B,εRµ,Σ(f ) = E(x,y)∼Pµ,Σ [`B,ε (f ; x, y)]= E(x,y)∼Pµ,Σ [I (∃kδkB ≤ ε, f (x + δ) 6= y)].

(2.1)

The optimal risk is then defined as the classification error regarding the optimal classifier, namely
B,ε
B,ε
Rµ,Σ
∗ := Rµ,Σ
(f∗ ),(2.2)

and accordingly, we define the excess risk of any classifier f as
B,ε
B,ε
Rµ,Σ
(f ) − Rµ,Σ
∗,

(2.3)

which by definition is always non-negative.
Robust Bayes Optimal Classifier To motivate the robust optimal classifiers, we start our
discussion with the optimal risk and optimal classifier in the conditional Gaussian Model. We notethat when ε = 0, i.e. there is no adversary, the classification problem reduces to the well-known
Fisher’s Linear Discriminant Analysis problem, where the Bayes optimal classifier is a simple
linear classifier
fBayes (x) = sign(µT x),
8August 7, 2022
DRAFT
known as Fisher’s linear discriminant rule (see, e.g. Johnson et al. [64]). The Bayes optimalclassifier minimizes the misclassification rate. However, the classifier that minimizes the robust
classification error is not known until recently, where [12] provided a tight lower bound on theminimal robust classification error via optimal transport techniques. It is also proved that the
optimal risk can be written as the optimal value of a convex program, and the oracle optimalclassifier is a linear classifier that has a closed form given the solution of the convex program.
We find it is useful to first simplify and restate this result in order to set the stage for our main
result.
Theorem 2.2.1 (Restated and simplified from Bhagoji et al. [12]). Let zΣ (µ) be the solution of
the following convex program:zΣ (µ) = argminkµ − zk2Σ−1 ,(2.4)

kzkB ≤ε√
where kxkA = xT Ax. 1 Then, the optimal robust classifier for Pµ,Σ is a linear classifier f∗ (x) =sign(w0T x), wherew0 := Σ−1 (µ − zΣ (µ)),(2.5)
and the optimal robust classification error is
B,εRµ,Σ∗ := Φ̄(kw0 kΣ ) = Φ̄(kµ − zΣ (µ)kΣ−1 ).

We remark that the above mentioned classifier is indeed an oracle classifier since it is constructedusing the unknown parameters µ and Σ.
Adversarial Signal-To-Noise Ratio (AdvSNR). In the context of standard classification in
the conditional Gaussian model, the notion of Signal-To-Noise Ratio was introduced to measure
the effective separation which is defined as the Mahalanobis distance between the means of two
conditional distributions.
Definition 2.2.1 (Standard Signal-To-Noise Ratio). The Standard Signal-To-Noise Ratio (StdSNR)
of conditional Gaussian model Pµ,Σ is defined as
StdSNR(µ, Σ) := 2kµkΣ−1 .
Here, the constant 2 is introduced to be consistent with the literature in Fisher’s LDA, e.g. [22],where SNR is defined as the Mahalanobis distance between means of two mixture components. We
make the note that the StdSNR measures the difficulty of standard classification in the conditional
Gaussian model, since the minimal misclassification error equals to Φ̄( 12 StdSNR(µ, Σ)) [22]. In
fact, the misclassification error decreases exponentially as the StdSNR increases.
When it comes to the adversarial setting, StdSNR, however, is no longer a proper metric forthe classification difficulty. Specifically, conditional Gaussian models with the same StdSNR canhave very different levels of hardness in the adversarially robust classification problem. In order
to illustrate this, we demonstrate a simple example.
1

Note that this notation is different with [12], where in their notation kxkA =9√xT A−1 x.

August 7, 2022
DRAFT
Example 2.2.1. Consider an adversary which is allowed to perturb the input with budget ε = √6d
in terms the `∞ norm. Set the covariance Σ to be the identity matrix Id . We examine two
conditional Gaussian models, Pµ1 ,Σ and Pµ2 ,Σ with different means µ1 and µ2 , where6µ1 = √ · (1, 1, 1, · · · , 1)T ,dµ2 = (6, 0, 0, · · · , 0)T .

It is easily seen that kµ1 kΣ−1 = kµ2 kΣ−1 = 6, therefore Pµ1 ,Σ and Pµ2 ,Σ have the same StdSNR.
However, by Theorem 2.2.1, these two distributions actually exhibit completely different minimalrobust classification error, indeed,1RµB,ε= Φ̄(0) = ,
1 ,Σ
2

6
RµB,ε
= Φ̄(6 − √ ).
2 ,ΣdWhen the dimension d is sufficiently large, the optimal risk RµB,εapproaches Φ̄(6) ≈ 10−8 , which
2 ,Σ
means there exists a very good robust classifier for Pµ2 ,Σ . In contrast, the optimal risk RµB,ε= 12 ,1 ,Σ
i.e. no classifier can achieve a robust accuracy better than a uninformative predictor that classifies
everything as the same class. From this simple example, it is safe to conclude that StdSNR is not
an ideal measurement for the difficulty in the adversarially robust classification problem.
To address the above issue, one need a proper definition of the signal-to-noise-ratio that issuitable for the adversarial robust setting. Therefore we introduce the Adversarial Signal-To-Noise
Ratio (AdvSNR) for any (B, ε) adversary.
Definition 2.2.2 (Adversarial Signal-To-Noise Ratio). Define the (B, ε) Adversarial Signal-ToNoise Ratio (AdvSNR) of conditional Gaussian model Pµ,Σ as
AdvSNRB,ε (µ, Σ) := 2kµ − zΣ (µ)kΣ−1 = 2kw0 kΣ ,where w0 is defined in (2.5).
As a consequence of Theorem 2.2.1, the minimal robust classification error satisfies


1
B,ε
AdvSNR(µ, Σ) .
Rµ,Σ ∗ = Φ̄
2

(2.6)

Consequently, the AdvSNR fully characterizes the difficulty for the adversarially robust setting
as the StdSNR in the standard setting. We also note that when ε = 0, i.e. there is no adversary,
the AdvSNR reduces to the traditional definition of the StdSNR. Thus, AdvSNR is a reasonablegeneralization for StdSNR.
Naturally, for every r > 0, one can consider a class of distributions where each of them has
the same (B, ε)-AdvSNR equal to r. Within each class, they should enjoy the same hardness ofthe classification problem. Formally, let us define the class DB,ε (r).
Definition 2.2.3. The family of conditional Gaussian models with (B, ε)-AdvSNR value of r, isdefined as:
DB,ε (r) := {(µ, Σ)|AdvSNRB,ε (µ, Σ) = r}.
In the sequel, we develop our minimax lower bounds over these classes of distributions. To
assist our analysis, we also define the family of conditional Gaussian models with a standard SNRvalue of r similarly.
10August 7, 2022
DRAFT
Definition 2.2.4. The family of conditional Gaussian models with a standard SNR value of r, isdefined as:
Dstd (r) := {(µ, Σ)|StdSNR(µ, Σ) = r}.
In the derivations of our upper bounds and minimax lower bounds, we make the assumptionthat the AdvSNR r is strictly bounded away from zero by a universal constant 2 , otherwise as aresult of Theorem 2.2.1, no classifier can achieve accuracy much better than 12 , the robust risk of aconstant classifier f (x) ≡ 1.

2.3

A Coputationally Efficient Estimator and Risk Upper Bound

Thus far, we introduce the notion of AdvSNR which is known to characterize the minimal robust
classification error as in expression (2.6). However, whether there exists a computation-efficientclassifier that behaves similarly to the oracle best classifier is still unclear.
This section, we aim to answer this question in the affirmative by constructing such a classifier.
For the classifier that we shall define in the sequel, we give an exact characterization of its excess
robust classification error compared with the oracle best classifier. Motivated by the fact thatthe optimal robust classifier has the form of (2.5), we design a ”plug-in” estimator for w0 . The
estimator is described in the following algorithm.
Algorithm 1 A plug-in estimator of w0
Input: Data pairs {(xi , yi )}ni=1 .
Output: w.
bb asStep 1: Define µb and Σn1Xµ
b :=yi xi ,n i=1nXb := 1
Σxi xTi − µ
bµ
bT .
n i=1

Step 2: Solve for zb in the followingµ) = argminkbµ − zk2Σb −1 .
zb := zΣb (bkzkB ≤εb −1 (bStep 3: Define w
b := Σ
µ − zb).
The main theorem of this section is to characterize the excess risk bound of the classifier
induced by w.
b
Theorem 2.3.1. For the (k·kB , ε) adversary, suppose the adversarial signal-to-noise ratio
AdvSNRB,ε (µ, Σ) = r, then the excess risk of fwb is upper bounded by
B,ε
B,ε
Rµ,Σ
(fwb ) − Rµ,Σ∗ ≤ OP2

for instance, r ≥ 10−911− 18 r2

e

d
·r·
.
n

August 7, 2022
DRAFT
We take a moment to make several remarks. First recall that the AdvSNR is defined as ameasurement for the hardness of the classification problem. Indeed, as the above result shows, theexcess risk vanishes exponentially with the AdvSNR. Moreover, our estimator is adaptive in thesense that it does not require knowing any information about the value of r, but the theoreticalguarantee improves automaticallywith larger AdvSNRs. We also note that the dependency with
1sample
size
n
is
O
,
which
is
the
same as the rate of Fisher’s LDA, but faster than the typical
n
 
O

√1
n

rate.

Comparisons to [113] We note that our result generalizes the one showed in [113] in manydifferent aspects:
1. In terms of the perturbations, Schmidt et al. [113] considered perturbations in `∞ balls,
while ours allow for any convex, closed and origin-symmentric perturbaion set B, including
all `p balls for p ≥ 1.
2. Our upper and lower bounds hold for both spherical and non-spherical Gaussians, without
the knowledge of the population covariance structure.
3. We impose no restrictions on the separation between Gaussian distributions. Schmidt et al.
[113] studied a very specific regime, where the budget of `∞ adversary is bounded by 14 ,√the separation between the means of two Gaussians is d, and the spherical covariance
1 1/4
matrix Σ = σ 2 I satisfies σ ≤ 32
d . This regime is low-noise by design, while our analysis
applies to any regime whenever there exists a classifier with robust accuracy slightly better
than 12 .
4. Our estimator is consistent, i.e. the excess risk converges to zero as sample size n → ∞.
The classifier used in Schmidt et al. [113] is actually sign(µ̂T x). While this classifierachieve near-optimal classification error in the regime of their interst (the low noise regime
mentioned above with Gaussian prior on µ), the excess risk does not converge to zero ingeneral. This is due to the fact that the large-sample limit of their classifier is actuallysign(µT x), i.e. the Bayes optimal classifier for the standard setting. As we can see from
Theorem 2.2.1 and a simple simulation in Figure 2.1, the excess risk of their algorithm
saturates at a level above zero, which is very different from the behavior of Algorithm 1.
Proof Sketch:
Section 2.6.

Here we provide a brief sketch of the proof. More details can be found in the

Step 1: First order approximation of the risk. Since both the learned fŵ and the optimalrobust classifier f∗ are linear classifiers, we can calculate the robust excess risk in closed form
using Lemma 2.6.2 (also shown in [12]): Tw
b µ − εkwk
b B∗
1
B,ε
B,ε
Rµ,Σ (fwb ) − Rµ,Σ ∗ = Φ̄− Φ̄( r).
kwkb Σ
2
By the Taylor expansion of Φ̄(·), we have
 T

1 2
w
b µ − εkwk
b B∗
1
1
Φ̄
− Φ̄( r) ≈ √ e− 8 r δn ,
kwk
b Σ
2
2π
12

August 7, 2022
DRAFT

Figure 2.1: A simple simulation on the performance of Algorithm 1 and the algorithm proposedin [113] is shown here with different values of AdvSNR r. Here we consider a 50-dimensional
example under `∞ adversary with ε = 0.1. The covariance matrix is fixed to be Σ = I, andthe mean parameter µ is set as µ = (r + ε, ε, ε, · · · , ε) for r ∈ {0.5, 1.0, 2.0}. We evaluate theB,εB,εexcess risk Rµ,Σ(fwb ) − Rµ,Σ∗ returned by the two algorithms using n i.i.d. training data pairs,
where n ∈ {50, 100, 200, 400, 800, 1600, 3200, 6400, 12800}. For each combination of (n, r), theaveraged excess risk over 10 random repetitions is reported respectively.
where
1
w
bT µ − εkwkb B∗
δn = r −
.
2
kwk
b Σ
Therefore, it is sufficient to show that δn = OP (r · nd ).
Step 2: Controlling δn . To give an upper bound of δn , we will use the fact that sample mean µbband sample covariaceq Σ converge to µ and Σ respectively. Furthermore, the convergence rate iswell known as OP ( nd ).
From a high level, the upper bound of δn is estabished (see Lemma 2.6.3) by carefully
decomposing δn into four terms and each term is in the form of the differences between populationb µb we are able
and sample quantities like Σ vs Σ,b vs µ. Invoking the convergence rates of µb and Σ,to bound each of these terms and complete the proof.

2.4

Minimax Lower Bounds

This section is dedicated to developing minimax excess risk lower bounds for the adversariallyrobust classification with conditional Gaussian models.
13

August 7, 2022
DRAFT
As is mentioned above, we consider a class of distributions DB,ε (r) which have the same
AdvSNRB,ε = r, as in Definition 2.2.3. As quantity AdvSNR characterizes the minimal robust
classification error, this class of distributions DB,ε (r) all share the same adversarially robust
classification error. Therefore, our lower bounds here measure the fundamental informationtheoretic limit of this problem, namely, no estimator can achieve an essential improvement interms of the adversarial classification error.
Theorem 2.4.1. Let fb be any estimator based on n samples (x1 , y1 ), · · · , (xn , yn ) ∼i.i.d. Pµ,Σ . We
have the following lower bound on the minimax excess risk:
B,ε b
B,ε
min
max [Rµ,Σ
(f ) − Rµ,Σ∗] ≥ ΩPbf (µ,Σ)∈DB,ε (r)−( 81 +o(1))r2 d

e

n



.

Putting together with the upper bound in Theorem 2.3.1, this lower bound matches almost
exactly with the upper bound in the regime of interest, therefore they are both optimal up to lower
order terms.
The main technique we used for this lower bound is with a flavor of black-box reduction. In
particular, we show that the minimax robust excess risk in DB,ε (r) cannot be smaller than theminimax standard excess risk in Dstd (r). In other words,
Lemma 2.4.1. The minimax excess error satisfies
B,ε b
B,ε
[Rµ,Σ
(f ) − Rµ,Σ∗] ≥ min

min

max

fb

(µ,Σ)∈DB,ε (r)

fb

max

std
b
[Rµstd0 ,Σ (f ) − Rµ0 ,Σ ∗].

(µ0 ,Σ)∈Dstd (r)The right hand side of (2.7), i.e. the minimax rate for standard classification, is well-studied
in the existing literature of Fisher’s LDA. For example, [85] proved the following lower bound:
Theorem 2.4.2 (Theorem 1 of [85]). Suppose the covariance matrix satisfies Σ = I and is known
to the learner, then we have the minimax lower bound


std
std
− 81 r2 1 db
min 0 max [Rµ0 ,Σ (f ) − Rµ0 ,Σ ∗] ≥ ΩP e
· ·
.
r n
fb (µ ,I)∈Dstd (r)Since the parameterin [85] is a subset of Dstd (r), we have (2.7) is also lower
 1 space considered− 8 r2 1 dbounded by ΩP e
· r · n , therefore proves Theorem 2.4.1.
Comparisons to [113] and [12] To the best of our knowlege, Theorem 2.4.1 is the first minimaxtype lower bound in adversarially robust classification. Existing works [113] and [12] also studied
the sample complexity of robust learning in conditional Gaussian model. However, both of themsimplified the problem and considered the case when µ follows from a prior distribution N (0, I).
This assumption is crucial to their analysis, otherwise the posterior distribution of µ given training
data is intractable. Hence, the technical tool used in prior works is not sufficient for developing
such a minimax lower bound of our interest.
Proof Sketch: Here we also provide a proof sketch to Lemma 2.4.1. More details can be foundin the Section 2.6.
14

August 7, 2022
DRAFT
Step 1: Connecting standard and robust risks In Lemma 2.6.4, we prove that for any classifier
f and a perturbed distribution Pµ0 ,Σ , where kµ0 − µkB ≤ ε, the robust risk of f on Pµ,Σ is always
lower bounded by the standard risk on Pµ0 ,Σ .
As a consequence, in Corollary 2.6.1 we show that if we choose µ0 = µ − zΣ (µ), then therobust excess risk of f on Pµ,Σ is always lower bounded by the standard excess risk on Pµ0 ,Σ .

Step 2: A mapping fron Dstd (r) to DB,ε (r) To prove Lemma 2.4.1, we only need to answer
the following question: for any (µ0 , Σ) ∈ Dstd (r), can we find a (µ, Σ) ∈ DB,ε (r), so that therobust excess risk on Pµ,Σ is always lower bounded by the standard excess risk on Pµ0 ,Σ ? We givean affirmative answer to this question. The proof in a combination of Corollary 2.6.1 showed in
Step 1 and an examination of optimality condition in the optimization problem 2.4.

2.5

Comparing Adversarial and Standard Rates

Putting the upper and lower bounds together provides a comprehensive view of the statisticalaspect of the adversarially robust classification. A key question to ask is that: How much does theclassification error blows up as the price of being adversarially robust?
To answer this question, it is sufficient to compare the optimal risks in both cases. Informally,
one can write the logarithm ratio between two rates aslog

AdvRate
StdRate≈1kµ − zΣ (µ)k2Σ−1 −kµk2Σ−1 .
2

(2.7)

From the definition of zΣ (µ) in (2.4), we can see that kµ − zΣ (µ)k2Σ−1 ≤ kµk2Σ−1 , hence
adversarial rate is always slower.
To analyze this difference quantitively and interpretably, we consider the special case whereΣ = I and the adversary is `2 bounded. Similar results hold for other adversaries as well. The key
observation is that depending on the different scale of kµk2 and the budget of perturbation ε, thisdifference can be as small as O(1), or as large as Ω(exp(d)).
Proposition 2.5.1. When Σ = I and the adversarial perturbation satisfies kδk2 ≤ ε, then1
• When ε ≤ O( kµk), the adversarial rate is at most O(1) times slower than the standard
2

rate.
log d
• When kµk2 ≥ Ω(log d) and ε ≥ Ω( kµk), the adversarial rate can be slower than the2standard rate by a poly(d) factor.
√d• When kµk2 ≥ Ω( d) and ε ≥ Ω( kµk), the adversarial rate can be slower than the standard
2
rate by an exp(d) factor.
In general, the difference is more significant when ε or kµk2 is larger. This example demonstrates a clear tradeoff between being adversarial robust and obtaining the optimal accuracy, inparticular in the case of large perturbations.
15August 7, 2022
DRAFT

2.6

Proofs and further details

In this section, we provide detailed proofs for our main results. The proof details of some lemmas
are deferred to our supplementary file.

2.6.1

Proof of Theorem 2.3.1

Before presenting our analysis, we first state a standard lemma about the convergence of empiricalmean and covariance.
Lemma 2.6.1 (Convergence of the empirical mean and covariance (see, e.g. Wainwright [134])).
b to the corresponding ground truth satisfy
The convergence rates of the empirical mean µb and Σr !
d,kb
µ − µkΣ−1 = OP
n
and
− 21

kΣ

− 12

b
ΣΣ

− Ikop = OP

r !
d
.
n

The following lemma about the classification error of linear classifiers will also be useful forus.
Lemma 2.6.2 (Robust classification error of linear classifier, (see e.g. in [12], Appendix B.3)).
For a linear classifier fw (x) = sign(wT x), the robust classification error with a B, ε adversary is T
w µ − εkwkB∗B,εRµ,Σ (fw ) = Φ̄
.
kwkΣ
B,ε
B,ε
Here, k·kB∗ is the dual norm of k·kB . We use Rµ,Σ(w) as a shorthand for Rµ,Σ(fw ) when themeaning is clear from context.

Proof of Theorem 2.3.1. By Lemma 2.6.2 and Taylor expansion of Φ̄(t) around t = 12 r = kw0 kΣ ,the excess risk can be written as: Twb µ − εkwk
b B∗
B,ε
B,ε
Rµ,Σ (w)b − Rµ,Σ ∗ = Φ̄− Φ̄(kw0 kΣ )
kwk
b Σ
1 2
1
= √ e− 8 r δn + O(δn2 ),
2π
where
w
bT µ − εkwkb B∗
δn = kw0 kΣ −
.
kwk
b Σ
Therefore, to analyze the convergence rate of the excess risk, we only need to analyze theconvergence rate of δn . We would like to prove thatd.
δn = OP r ·n
The following lemma is the key of our analysis: it decomposes δn into four terms, each in theb µform of the difference between population and sample quantities like Σ vs Σ,b vs µ.
16

August 7, 2022
DRAFT
Lemma 2.6.3. We have the following decomposition for δn :
1
1
1
b w
kwk
b Σ δn = − (kw0 kΣ −kwkb Σ )2 +w0T (bz − zΣ (µ)k2Σ−1 + k(Σ − Σ)b + (bµ − µ)k2Σ−1 .
z − zΣ (µ)) − kb
|
{z
}
2
2
2
|
|
{z
}
{z
}|
{z
}
T2
T1

T3

T4

where ẑ is the shorthand for ẑ = zΣb (bµ).
The proof of Lemma 2.6.3 is provided in Appendix 2.10. Based on this decomposition, ourgoal is to establish the following relations.
2dT1 ≤ 0, T2 ≤ 0, T3 ≤ 0, T4 ≤ OP r.
n
It is obvious that T1 ≤ 0, T3 ≤ 0. For the second term T2 , consider φ(z) = kµ − zk2Σ−1 . SincezΣ (µ) = argminkzkB ≤ε kµ − zk2Σ−1 = argminkzkB ≤ε φ(z) , by the first order optimality condition,
we have (z 0 − zΣ (µ))T ∇φ(zΣ (µ)) ≤ 0 holds for any kz 0 kB ≤ ε. Choosing z 0 = ẑ gives:(µ − zΣ (µ))T Σ−1 (bz − zΣ (µ)) ≤ 0 ⇔ w0T (bz − zΣ (µ)) ≤ 0.
Therefore, T2 ≤ 0 as we desired.

The remaining work is to prove that T4 ≤ OP (1 + r)2 nd . By triangle’s inequality,b w
b wkk(Σ − Σ)b + (bµ − µ)kΣ−1 ≤ k(Σ − Σ)b Σ−1 +kb
µ − µkΣ−1 .
Both terms can be controled using covergence of sample mean and covariance. By Lemma 2.6.1,
one hasr !
d,kbµ − µkΣ−1 ≤ OPnand direct calculations give1
b wk
b − 21 )(Σ 12 w)kk(Σ − Σ)b Σ−1 = k(I − Σ− 2 ΣΣ
b 2
1
1
1
b − 2 kop kΣ 2 wk
≤ kI − Σ− 2 ΣΣ
b 2
!
r
d
= OP
kwk
b Σ.
n

Combined pieces together, triangle’s inequality further guarantees thatrb wk(Σ − Σ)b + (bµ − µ)kΣ−1 ≤ OP
17

!
d
(kwk
b Σ +1) .
nAugust 7, 2022
DRAFT
b → Σ, we have w
Since µb → µ, Σb → w0 , therefore kwk
b Σ = (1 + o(1))kw0 kΣ = ( 12 + o(1))r,hence,1b w
T4 = k(Σ − Σ)b + (bµ − µ)k2Σ−1
2
!2
r
1
d
≤
OP (
)(kwk
b Σ +1)
2
n


2 d
.
= OP r ·n
Putting things together and recall that r = Ω(1), we havedδn = OP r ·.
nTherefore we have completed the proof.

2.6.2

Proof of Lemma 2.4.1

To prove Lemma 2.4.1, we start with a simple observation: for any classifier f , its standard
error on any perturbed distribution Pµ0 ,Σ is always a lower bound on robust error of the original
distribution Pµ,Σ , as long as the perturbation has bounded B-norm kµ0 − µkB ≤ ε:Lemma 2.6.4. For any classifier f : Rd → {−1, +1} and any µ0 ∈ Rd , kµ0 − µkB ≤ εB,εRµ,Σ(f ) ≥ Rµstd
0 ,Σ (f ).

Proof. By the definition of robust classification error (2.1), we can decompose the error into twoparts: the error on positive class (y = 1) and negative class (y = −1), namely,
B,εRµ,Σ(f ) =E(x,y)∼Pµ,Σ [I (∃kδkB ≤ ε, f (x + δ) 6= y)]
1
1= Ex∼N (µ,Σ) [I (∃kδkB ≤ ε, f (x + δ) 6= 1)] + Ex∼N (−µ,Σ) [I (∃kδkB ≤ ε, f (x + δ) 6= −1)].
2
2
(2.8)

By choosing the adversarial perurbation as δ = µ0 − µ, we have the error on positive class is lower
bounded by:
Ex∼N (µ,Σ) [I (∃kδkB ≤ ε, f (x + δ) 6= 1)] ≥Ex∼N (µ,Σ) [I (f (x − µ + µ0 ) 6= 1)]=Ex0 ∼N (µ0 ,Σ) [I (f (x0 ) 6= 1)].

(2.9)

Similarly, by choosing δ = µ − µ0 , we have the error on negative class is lower bounded by:
Ex∼N (−µ,Σ) [I (∃kδkB ≤ ε, f (x + δ) 6= 1)] ≥ Ex0 ∼N (−µ0 ,Σ) [I (f (x0 ) 6= −1)].
Hence, combining (2.8) , (2.9) and (2.10), we get
1
1
B,ε
Rµ,Σ
(f ) ≥ Ex0 ∼N (µ0 ,Σ) [I (f (x0 ) 6= 1)] + Ex0 ∼N (−µ0 ,Σ) [I (f (x0 ) 6= −1)]22
std
=Rµ0 ,Σ (f ),18

August 7, 2022
DRAFT
where the last step is by the definition of standard error (2.2). Therefore we have completed theproof.
Next, we show more connections between robust and standard classification. Namely, therobust Bayes classifier of Pµ,Σ coincides with the standard Bayes classifier of Pµ−zΣ (µ),Σ , as statedin the following Lemma:
Lemma 2.6.5. Let zΣ (µ) be the solution of (2.4), then the robust Bayes classifier of Pµ,Σ ,f∗ (x) = sign(w0T x), satisfies the following conditions:
B,εstd
1. Rµ,Σ
(f∗ ) = Rµ−z(f∗ ).
Σ (µ),Σ
2. f∗ is the standard Bayes Optimal Classifier of Pµ−zΣ (µ),Σ .
Proof. Note that by setting ε = 0 in Theorem 2.2.1, we get the characterization of the standard
Bayes error and Bayes optimal classifier for conditional Gaussian models. Applying this result
for the distribution Pµ−zΣ (µ),Σ , we have
1. The standard Bayes Optimal Classifier of Pµ−zΣ (µ),Σ is sign((µ − zΣ (µ))T Σ−1 x), which isexactly f∗ (x).
p
2. The standard Bayes error of of Pµ−zΣ (µ),Σ is Φ̄( (µ − zΣ (µ))T Σ−1 (µ − zΣ (µ))), which isB,εexactly Rµ,Σ∗.
Hence we have completed the proof.
As a direct consequence of Lemma 2.6.4 and Lemma 2.6.5, we have the robust excess risk
under Pµ,Σ is lower bounded by the standard excess risk under Pµ−zΣ (µ),Σ :Corollary 2.6.1. For any classifier f : Rd → {−1, +1},
B,ε
B,ε
std
std
Rµ,Σ
(f ) − Rµ,Σ∗ ≥ Rµ−z(f ) − Rµ−z(f∗ )Σ (µ),ΣΣ (µ),Σ
std
std= Rµ−z(f ) − Rµ−z∗,
Σ (µ),ΣΣ (µ),Σ

where
std
Rµstd
0 ,Σ ∗ = inf Rµ0 ,Σ (g)gis the optimal standard risk.
The last piece of tool needed for proving Lemma 2.4.1 is a mapping from Dstd (r) to DB,ε (r)that keeps the excess risk non-decreasing. This is established via the following lemma:
Lemma 2.6.6. For any (µ0 , Σ) ∈ Dstd (r), there exists (µ, Σ) ∈ DB,ε (r), such that µ−zΣ (µ) = µ0 ,here zΣ (µ) is the optimal solution of (2.4).
Proof. The proof is constructive: we choose µ = µ0 + zeΣ (µ0 ), where zeΣ (µ0 ) is the maximizer ofthe following convex program (which is maximizing a linear function over a convex set):zeΣ (µ0 ) = argmax µ0 · Σ−1 z.

(2.10)

kzkB ≤εWe want to prove that µ − zΣ (µ) = µ0 . By our choice of µ, we also have µ = µ0 + zeΣ (µ0 ). Hence,
we only need to prove thatzeΣ (µ0 ) = zΣ (µ).
19August 7, 2022
DRAFT
In other words, we only need to show that zeΣ (µ0 ) is the minimizer of (2.4).
Since (2.4) is a convex program with a strongly convex objective, it suffices to prove thefollowing first order optimality condition holds for any ∀kz 0 kB ≤ ε:(µ − zeΣ (µ0 ))T Σ−1 (z 0 − zeΣ (µ0 )) ≤ 0.
Since µ − zeΣ (µ0 ) = µ0 , the inequality is equivalent to:µ0 · Σ−1 z 0 ≤ µ0 · Σ−1 zeΣ (µ0 ),which is correct by the definition of zeΣ (µ0 ). Hence we have completed the proof.
Equipped with Lemma 2.6.6, now we can prove the important lemma:
Proof of Lemma 2.4.1. By Lemma 2.6.6, for any (µ0 , Σ) ∈ Dstd (r), there exists (µ, Σ) ∈ DB,ε (r),such that µ − zΣ (µ) = µ0 , where zΣ (µ) is the optimal solution of (2.4). By Corollary 2.6.1, we
have the following inequality holds for any fixed fb:
B,ε
std
b
b
Rµstd
0 ,Σ (f ) − Rµ0 ,Σ ∗ ≤ Rµ,Σ (f ) − Rµ,Σ ∗ .

Therefore,
std
b
Rµstd
0 ,Σ (f ) − Rµ0 ,Σ ∗ ≤

max

B,ε
[Rµ,Σ (fb) − Rµ,Σ∗].

(µ,Σ)∈DB,ε (r)holds for all (µ, Σ) ∈ DB,ε (r), which meansmax(µ,Σ)∈DB,ε (r)

B,ε
B,ε b
(f ) − Rµ,Σ
∗] ≥
[Rµ,Σ

max

(µ0 ,Σ)∈Dstd (r)stdb[Rµstd0 ,Σ (f ) − Rµ0 ,Σ ∗].

Then, taking minimum over fb on both sides proves the theorem.

Acknowledgements
Y.W. is supported in part by the NSF grant DMS-2015447 and CCF-2007911. C.D. and P.R. aresupported by DARPA via HR00112020006, and NSF via IIS1909816.
The authors would also like to thank Kaizheng Wang for many helpful discussions, Tianle Cai
and Justin Khim for pointing us toward the work of [12, 22], and annonymous reviewer for manysuggestions about improving the presentation of the paper.

20August 7, 2022
DRAFT

2.7

Proof of Theorem 2.2.1

For completeness, in this section, we present the proof of Theorem 2.2.1. This result follows
from combining Theorem 1, Theorem 2 and Lemma 1 in [12]. The proof is mainly a simplified
presentation of their proofs (e.g. without using the language of optimal transport) which makesome of their results explicit to interpret for our case (e.g. they did not provide the expression for
optimal linear classifier, which is useful to our algorithmic results).
Σ−1 (µ−zΣ (µ))
To start with, let us define w1 := kww00kΣ = kµ−z
be the normalized version of w0 soΣ (µ)kΣ−1that kw1 kΣ = 1. The following lemma is implicit in [12]:Lemma 2.7.1. Suppose we defineG(z, w) = wT (µ − z),then (zΣ (µ), w1 ) is solution of the following minimax optimization problem:
min max G(z, w).

kzkB ≤ε kwkΣ ≤1(2.11)

Proof. We first show that the optimal value of the inner maximization problem can be written as:
max wT (µ − z) = kµ − zkΣ−1 ,kwkΣ ≤1(2.12)and the maximum is achieved whenw=

Σ−1 (µ − z).
kµ − zkΣ−1(2.13)

In fact, for any w such that kwkΣ ≤ 1, Cauchy-Schwarz inequality giveswT (µ − z) = (Σ1/2 w)T Σ−1/2 (µ − z) ≤ kΣ1/2 wk2 kΣ−1/2 (µ − z)k2
= kwkΣ kµ − zkΣ−1≤ kµ − zkΣ−1 .
−1Σ (µ−z)Furthermore, it is easy to check that the choice w = kµ−zk
directly yields wT (µ − z) =Σ−1
kµ − zkΣ−1 achieving the equality. Therefore we have proved (2.12) and (2.13).
Using (2.12), the minimax problem (2.11) therefore simplifies to:

min kµ − zkΣ−1 .

kzkB ≤εRecall that we define zΣ (µ) (cf. (2.4)) aszΣ (µ) = argminkµ − zk2Σ−1 ,kzkB ≤εwhich is the optimal solution to this outer minimization problem. Combining with the optimality
condition for the inner maximization (2.13), we conclude that (zΣ (µ), w1 ) is solution of theminimax problem (2.11) and complete the proof.
21

August 7, 2022
DRAFT
Corollary 2.7.1. The following relation is satisfied for quantities w1 and zΣ (µ):
w1T µ − εkw1 kB∗ = kµ − zΣ (µ)kΣ−1 .
Proof. Since G(z, w) is linear in both z and w and both constraint sets {kzkB ≤ ε} and {kwkΣ ≤1} are convex, the minimax problem (2.11) satisfies strong duality by Von Neumann’s Minimax
Theorem. In other words, we can switch the order of the min and max, namely,
min max G(z, w) = max min G(z, w),

kzkB ≤ε kwkΣ ≤1

kwkΣ ≤1 kzkB ≤εand (zΣ (µ), w1 ) is the solution to both sides. By the stationary condition of the minimax problem,zΣ (µ) = argmin G(z, w1 ).
kzkB ≤εBy the definition of dual norm, we also havemin G(z, w1 ) = min w1T (µ − z) = w1T µ − εkw1 kB∗ .

kzkB ≤εkzkB ≤εHence,
kµ − zΣ (µ)kΣ−1 = G(zΣ (µ), w1 ) = min G(z, w1 ) = w1T µ − εkw1 kB∗ .
kzkB ≤εThus we completed the proof.
Now we are ready to prove Theorem 2.2.1.
Proof of Theorem 2.2.1. The proof can be divided into two parts:
B,ε1. Show that fw0 has robust risk Rµ,Σ(fw0 ) = Φ̄(kµ − zΣ (µ)kΣ−1 ).
2. Show that no classifier can achieve robust risk smaller than Φ̄(kµ − zΣ (µ)kΣ−1 ).

The first part is a consequence of Corollary 2.7.1. In order to see this, we first note that since w1
is a rescaling of w0 , the induced linear classifiers are the same, hence,
B,ε
B,ε
Rµ,Σ
(fw0 ) = Rµ,Σ(fw1 ).

By Lemma 2.6.2, the robust risk of fw1 is
B,εRµ,Σ
(fw1 ) = Φ̄(

w1T µ − εkw1 kB∗) = Φ̄(w1T µ − εkw1 kB∗ ).
kw1 kΣ

By Corollary 2.7.1,
Φ̄(w1T µ − εkw1 kB∗ ) = Φ̄(kµ − zΣ (µ)kΣ−1 ).
Therefore, we have proved the first part.
For the second part, we invoke Lemma 2.6.4. By setting µ0 = µ − zΣ (µ) in Lemma 2.6.4, wehave that for any classifier f ,
B,ε
std
Rµ,Σ
(f ) ≥ Rµ−z(f ).
Σ (µ),Σ
22

August 7, 2022
DRAFT
We also know that no classifier can achieve standard risk smaller than the Bayes Risk in Pµ−zΣ (µ),Σ .
Recall that for a conditional Gaussian kmodel Pµ0 ,Σ , the standard Bayes Risk is Φ̄(kµ0 kΣ−1 ). In
other words, for any classifier f , we havestdRµ−z(f ) ≥ Φ̄(kµ − zΣ (µ)kΣ−1 ).
Σ (µ),Σ

Combining the two inequalities, we conclude thatB,εRµ,Σ(f ) ≥ Φ̄(kµ − zΣ (µ)kΣ−1 )(2.14)

holds for all classifiers f . Therefore, we prove the second part and thus complete the proof.

2.8

Proof of Proposition 2.5.1

Proof of Proposition 2.5.1. Recall that the setting of interest here is Σ = I and k·kB corresponds
to the `2 norm. In this setting, we show that zΣ (µ) has a simplified form. In fact, directly invokingzΣ (µ) = argminkµ − zk2Σ−1 = argminkµ − zk22 ,kzkB ≤εkzk2 ≤εµgives zΣ (µ) = min(ε, kµk2 ) kµk, and2µ − zΣ (µ) = max(0,kµk2 −ε)µ.
kµk2

From this expression, we can see that when ε > kµk2 , the Adversarial Signal-to-Noise Ratio of
Pµ,Σ is 2kµ − zΣ (µ)k2 = 0. Hence, no classifier can achieve accuracy better than 12 . Below weonly consider the case when ε < kµk2 .
Recall that we want to compare the minimax rate in adversarial and standard setting. As
we showed earlier, the minimax rates are O(exp(− 12 kµ − zΣ (µ)k22 ) nd ) and O(exp(− 12 kµk22 ) nd )respectively. The ratio between the two quantities equals to:exp(− 12 kµ − zΣ (µ)k22 ) nd
1
1
= exp( ((kµk2 −ε)2 − kµk22 )) = exp(εkµk2 − ε2 ).
1
2 d
2
2
exp(− 2 kµ − zΣ (µ)k2 ) n(2.15)

Since 0 ≤ ε < kµk2 , we have


1 2
1
1
εkµk2 − ε = ε(kµk2 − ε) ∈ εkµk2 , εkµk2 .
222
Equipped with the above relation, we are in the position of establishing Proposition 2.5.1.
1• When ε ≤ O( kµk), one has21
εkµk2 − ε2 ≤ εkµk2 ≤ O(1),2thereby, the adversarial rate is at most exp(O(1)) = O(1) times slower than the standard
rate.
23

August 7, 2022
DRAFT
log d
• When kµk2 ≥ Ω(log d) and ε ≥ Ω( kµk), we conclude2

1
1
εkµk2 − ε2 ≥ εkµk2 ≥ Ω(log d),22
the adversarial rate can be slower than the standard rate by an Ω(exp(log d)) = Ω(poly(d))
factor.
√d• When kµk2 ≥ Ω( d) and ε ≥ Ω( kµk), it is guaranteed that211
εkµk2 − ε2 ≥ εkµk2 ≥ Ω(d),22therefore, the adversarial rate can be slower than the standard rate by an Ω(exp(d)) factor.

2.9

Improved analysis when Σ is known

Meticulous
bound may find a tiny gap between our bounds: the upper in Theorem 2.3.1 is 1 readers122
OP e− 8 r · r · nd , while the lower bound above gives ΩP e− 8 r · 1r · nd . Since the dominant
1 2

factor is e− 8 r and r = Ω(1), this difference is only in a lower order term. This gap is due to thefact that [85] assumed the covariance matrix Σ is known to the learner. In this section, we will
prove that under the same assumption, there is a modified version of Algorithm 1 that achieves
the truly optimal rate which matches the lower bound even with lower order term in r.
The only modification we made in Algorithm 1 is to replace the sample covariance matrix by
the true covariance Σ. The modified algorithm is presented below in Algorithm 3.
Algorithm 3 An improved estimator for w0 when Σ is known
Input: Data pairs {(xi , yi )}ni=1 .
Output: w.
bb asStep 1: Define µb and Σ
n

µ
b :=

1X
yi xi ,n i=1

b := Σ.
Σ

Step 2: Solve for zb in the following
zb := zΣb (bµ) = argminkbµ − zk2Σb −1 .
kzkB ≤εb −1 (bStep 3: Define w
b := Σ
µ − zb).

24

August 7, 2022
DRAFT
Theorem 2.9.1. For the (k·kB , ε) adversary, suppose the adversarial signal-to-noise ratio
AdvSNRB,ε (µ, Σ) = r, then the excess risk of fwb defined in Algorithm 3 is upper bounded by
B,ε
B,ε
Rµ,Σ
(fwb ) − Rµ,Σ∗ ≤ OP− 18 r2 1 d
e
.
· ·
r n

This improved rate can be proved by some simple modification to the proof of Theorem 2.3.1.
Proof. We demonstrate that in this setting, there is a stronger upper bound δn = OP 1r · nd andthe rest of proof follows the same as that of Theorem 2.3.1. To this end, let us recall that by
Lemma 2.6.3 and one has the decomposition,
1
1
1
b w
b Σ )2 +w0T (bz − zΣ (µ)k2Σ−1 + k(Σ − Σ)b + (bµ − µ)k2Σ−1 .
kwkb Σ δn = − (kw0 kΣ −kwkz − zΣ (µ)) − kb
{z
}| 2
| 2
{z
}|
{z
}| 2
{z
}
T2
T1

T3

T4

Similar to the proof of Theorem 2.3.1, we shall establish that d.
T1 ≤ 0, T2 ≤ 0, T3 ≤ 0, T4 ≤ OPnNote that the only difference
here is that we can now give a tighter upper bound for T4 : OP ndinstead of OP r2 nd .
Since Σ = Σ̂, by Lemma 2.6.1, we have
 
1
1
d
2
2
b w
T4 = k(Σ − Σ)b + (bµ − µ)kΣ−1 = k(bµ − µ)kΣ−1 = OP.
(2.16)
22nHence, we have proved that T4 = OP

d
n


, and


δn = OP

1 d
·
r n


.

Therefore we have completed the proof.

2.10

Proof of Lemma 2.6.3

Proof of Lemma 2.6.3. Recall that our goal is to establishkwkb Σ δn = kwkb Σ kw0 kΣ − w
bT µ − εkwk
b B∗
1
1
1
b w
b Σ )2 +w0T (bz − zΣ (µ)) − kbz − zΣ (µ)k2Σ−1 + k(Σ − Σ)b + (bµ − µ)k2Σ−1 .
= − (kw0 kΣ −kwk
{z
}| 2
| 2
{z
}|
{z
}| 2
{z
}
T1

T2

T3

T4

(2.17)
25

August 7, 2022
DRAFT
b −1 (bSince wb=Σµ − zΣb (bµ)), by Theorem 2.2.1, fwb is the optimal robust classifier for Pµb,Σb ,therefore, one can observewbT µb − εkwkb B∗= kwkb Σb .
kwkb Σb
Hence, direct calculations yieldbT (µ − µb)kwkb Σ δn = kw0 kΣ kwkb Σ −kwk
b 2Σb −w
T b −1b −1 (b= kw0 kΣ kwkb Σ −(bµ − zb) Σ (bµ − zb) + (bµ − zb)T Σµ − µ)= kw0 kΣ kwk
b Σ +wbT (bz − µ).
Now by use of the relation µ = Σw0 + zΣ (µ), we can further obtainkwkb Σ δn = kw0 kΣ kwk
b Σ +wbT (bz − Σw0 − zΣ (µ))= kw0 kΣ kwkb Σ −w
bT Σw0 + w
bT (bz − zΣ (µ))
111
= − (kw0 kΣ −kwkb Σ )2 + kw0 k2Σ + kwk
b 2Σ −w
bT Σw0 + w
bT (bz − zΣ (µ))
2221
= T1 + (wb − w0 )T Σ(wb − w0 ) + w0T (bz − zΣ (µ)) + (wb − w0 )T (bz − zΣ (µ))
21b − w0 )T Σ(wb − w0 ) + T2 + (wb − w0 )T (bz − zΣ (µ)),= T1 + (w2where the last equality invokes the definitions in expression (2.17). To finish the proof, we makethe observation about Σ(wb − w0 ) in the following
b w
bw
Σ(w
b − w0 ) = (Σ − Σ)b + (Σ
b − Σw0 )b w= (Σ − Σ)b + (bµ − µ) − (bz − zΣ (µ)) := U1 + U2 − U3 .
| {z } | {z } |
{z
}
U1

U2

U3

Therefore, putting everything together and rearranging terms, it is guaranteed that1b − w0 )T Σ(wb − w0 ) + ( w
b − w0 )T (bz − zΣ (µ))
kwkb Σ δn = T1 + T2 + (w21
= T1 + T2 + (Σ(wb − w0 ))T Σ−1 (Σ(wb − w0 )) + (Σ(wb − w0 ))T Σ−1 (bz − zΣ (µ))
21
= T1 + T2 + (U1 + U2 − U3 )T Σ−1 (U1 + U2 − U3 ) + (U1 + U2 − U3 )Σ−1 U3
2
1= T1 + T2 + (U1 + U2 − U3 )T Σ−1 (U1 + U2 + U3 )2
1
1
= T1 + T2 − U3T Σ−1 U3 + (U1 + U2 )T Σ−1 (U1 + U2 )2
2
= T1 + T2 + T3 + T4 .
Thus we have finished the proof.

26

August 7, 2022
DRAFT

Chapter 3
Distributional and Outlier Robust
Optimization
3.1

Introduction

Many machine learning tasks require models to perform well under distributional shift, where
the training and the testing data distributions are different. One type of distributional shift that
arouses great research interest is subpopulation shift, where the testing distribution is a specific or
the worst-case subpopulation of the training distribution. A wide range of tasks can be modeled as
subpopulation shift problems, such as learning for algorithmic fairness [10, 41] where we want totest model’s performance on key demographic subpopulations, and learning with class imbalance[45, 61] where we train a classifier on an imbalanced dataset with some minority classes having
much fewer samples than the others, and we want to maximize the classifier’s accuracy on theminority classes instead of its overall average accuracy.
Distributionally robust optimization (DRO) [40, 96] refers to a family of learning algorithms
that minimize the model’s loss over the worst-case distribution in a neighborhood of the observed
training distribution. Generally speaking, DRO trains the model on the worst-off subpopulation,
and when the subpopulation membership is unknown, it focuses on the worst-off training instances,that is, the tail performance of the model. Previous work has shown effectiveness of DRO in
subpopulation shift settings, such as algorithmic fairness [51] and class imbalance [148].
However, in our empirical investigations, when we apply DRO to real tasks on modern datasets,
we observe that DRO suffers from poor performance and severe instability during training. The
issue that DRO is sensitive to outliers has been raised by several previous papers [51, 56, 159] .
In this paper, we study the cause of these problems with DRO, and develop approaches to addressthem.
In particular, we identify and study one key factor that we find directly leads to DRO’s suboptimal behavior: DRO’s sensitivity to outliers that widely exist in modern datasets. In general,
DRO maximizes a model’s tail performance by putting more weights on the “harder” instances,i.e. those which incur higher losses during training. On the one hand, this allows DRO to focus
its attention on worst-off sub-populations. But on the other hand, since outliers are intuitively“hard” instances that incur higher losses than inliers, DRO is prone to assign large weights to27

Density

August 7, 2022
DRAFT

Outliers

Loss

DRO
DORO

Figure 3.1: DORO avoids overfitting to outliers.
outliers, resulting in both a drop in performance, and training instability. To provide empiricalinsights into how outliers affect DRO, in Section 3.3 we conducted experiments examining how
the performance of DRO changes as we removed or added outliers to the dataset. The results ofthese experiments indicate that outliers bring about the observed bad performance of DRO. Thus,
it is crucial to first enhance the robustness of DRO to outliers before applying it to real-world
applications.
To this end, we propose DORO, an outlier robust refinement of DRO which takes inspirationfrom robust statistics. At the core of this approach is a refined risk function which prevents DRO
from overfitting to potential outliers. Intuitively speaking, the new risk function adaptively filters
out a small fraction of data with high risk during training, which is potentially caused by outliers.
Figure 3.1 illustrates the difference between DRO and DORO. In Section 3.4 we implement
DORO for the Cressie-Read family of Rényi divergence, and for our theoretical and empirical
study we primarily focus on CVaR-DORO and χ2 -DORO. In Section 3.5 we provide theoreticalresults guaranteeing that DORO can effectively handle subpopulation shift in the presence ofoutliers. Then, in Section 3.6 we empirically demonstrate that DORO improves the performanceand stability of DRO. We conduct large-scale experiments on three datasets: the tabular dataset
COMPAS, the vision dataset CelebA, and the language dataset CivilComments-Wilds.
Contributions Our contributions are summarized below:• We demonstrate that the sensitivity of DRO to outliers is a direct cause of the irregularbehavior of DRO with some intriguing experimental results in Section 3.3.
• We propose and implement DORO as an outlier robust refinement of DRO in Section 3.4.

Then, in Section 3.5 we provide theoretical guarantees for DORO.
• We conduct large-scale experiments in Section 3.6 and empirically show that DORO im-

proves the performance and stability of DRO. We also analyze the effect of hyperparameters
on DRO and DORO.
Related Work Distributional shift naturally arises in many machine learning applications andhas been widely studied in statistics, applied probability and optimization [13, 57, 106, 116].
One common type of distributional shift is domain generalization where the training and testing
distributions consist of distinct domains, and relevant topics include domain adaptation [101, 138]and transfer learning [99, 122]. Another common type of distributional shift studied in thispaper is subpopulation shift, where the two distributions consist of the same group of domains.
28August 7, 2022
DRAFT
Subpopulation shift is closely related to algorithmic fairness and class imbalance. For algorithmic
fairness, a number of fairness notions have been proposed, such as individual fairness [41, 153],
group fairness [50, 152], counterfactual fairness [76] and Rawlsian Max-Min fairness [51, 109].
The setting of subpopulation shift is most closely related to the Rawlsian Max-Min fairness notion.
Several recent papers [51, 97, 148] proposed using DRO to deal with subpopulation shift, but
it was also observed that DRO was prone to overfit in practice [110, 112]. [51] raised the open
question whether it is possible to design algorithms both fair to unknown latent subpopulations
and robust to outliers, and this work answers this question positively.
Outlier robust estimation is a classic problem in statistics starting with the pioneering works of[58, 129]. Recent works in statistics and machine learning [35, 36, 78, 103] provided efficiently
computable outlier-robust estimators for high-dimensional mean estimation with corresponding
error guarantees. Outliers have a greater effect on the performance of DRO than ERM [56], due
to its focus on the tail performance, so removing this negative impact of outliers is crucial for the
success of DRO in its real-world applications. One closely related recent work is [81], and DORO
can be viewed as a combination of risk-averse and risk-seeking methods discussed in this paper.

3.2

Background

This section provides the necessary background of subpopulation shift and DRO.

3.2.1

Subpopulation Shift

A machine learning task with subpopulation shift requires a model that performs well on thedata distribution of each subpopulation. Let the input space be X and the label space be Y.
We are given a training set containing m samples i.i.d. sampled from some data distribution P
over X × Y. There are K predefined domains (subpopulations) D1 , · · · , DK , each of which is asubset of X × Y. For example, in an algorithmic fairness task, domains are demographic groups
defined by a number of protected features such as race and sex. Let Pk (z) = P (z|z ∈ Dk ) be
the conditional training distribution over Dk , where z = (x, y). The goal is to train a modelfθ : X → Y parameterized by θ ∈ Θ that performs well over every Pk . Denote the expectedrisk over P by R(θ; P ) = EZ∼P [`(θ; Z)] where `(θ; z) is a measurable loss function. Then the
expected risk over Pk is Rk (θ; P ) = EZ∼Pk [`(θ; Z)]. The objective is to minimize the worst-caserisk defined as
Rmax (θ; P ) = max Rk (θ; P )(3.1)
k=1,···,K

Several different settings were studied by previous work:
Overlapping vs Non-overlapping The overlapping setting allows the domains to overlap witheach other while non-overlapping does not. For example, suppose we have two protected features:race (White and Others) and sex (Male and Female). Under either setting we will have fourdomains. Under the overlapping setting we will have White, Others, Male and Female, while
under the non-overlapping setting we will have White Male, White Female, Others Male and29August 7, 2022
DRAFT
Others Female. All the experiments in this work are conducted under the overlapping setting.
Each instance may belong to zero, one or more domains.
Domain-Aware vs Domain-Oblivious Some previous work has assumed that domain memberships of instances are known at least during training. This is called the domain-aware setting.
However, [51] argue that in many real applications, domain memberships are unknown during
training, either because it is hard to extract the domain information from the input, or because
it is hard to identify all protected features. Thus, a line of recent work [51, 77] studies thedomain-oblivious setting, in which the training algorithm does not know the domain membershipof any instance (even the number of domains K is unknown). In this work, we focus on thedomain-oblivious setting.

3.2.2

Distributionally Robust Optimization (DRO)

Under the domain-oblivious setting, we cannot compute the worst-case risk since we have noaccess to D1 , · · · , DK . In this case, the framework of DRO instead maximizes the performanceover the worst-off subpopulation in general. Specifically, given some divergence D between
distributions, DRO aims to minimize the expected risk over the worst-case distribution Q (thatis absolutely continuous with respect to training distribution P , so that Q  P ) in a ball w.r.t.
divergence D around the training distribution P .
Thus, while empirical risk minimization (ERM) algorithm minimizes the expected riskR(θ; P ), DRO minimizes the expected DRO risk defined as:RD,ρ (θ; P ) = sup {EQ [`(θ; Z)] : D(Q k P ) ≤ ρ}

(3.2)

QP

for some ρ > 0. Different divergence functions D derive different DRO risks. In this work, we
focus on the Cressie-Read family of Rényi divergence [28] formulated as:
Z
dQ
(3.3)
Dβ (Q k P ) = fβ ( )dP
dP
where β > 1, and fβ (t) is defined as:fβ (t) =
1tβ − βt + β − 1β(β − 1)

(3.4)

An advantage of the Cressie-Read family is that it has the following convenient dual characterization (see Lemma 1 of [40] for the proof):no1RDβ ,ρ (θ; P ) = inf cβ (ρ)EP [(l(θ; Z) − η)β+∗ ] β∗ + η
(3.5)
η∈R

1

β
where β∗ = β−1, and cβ (ρ) = (1 + β(β − 1)ρ) β .
The following proposition shows that DRO can handle subpopulation shift under the domainoblivious setting. The only information DRO needs during training is α, the ratio between thesize of the smallest domain and the size of the population. See the proof in Appendix 3.8.1.

30

August 7, 2022
DRAFT
Proposition 1. Let α = mink=1,···,K P (Dk ) ≤ exp(−1) ≈ 36.8% be the minimal group size, anddefine ρ = fβ ( α1 ), then we have
Rmax (θ; P ) ≤ RDβ ,ρ (θ; P ).

(3.6)

While the Cressie-Read formulation only defines the f -divergence for finite β ∈ (1, +∞), it
can be shown that the dual characterization is valid for β = ∞ as well, for which the DORO risk
becomes the well-known conditional value-at-risk (CVaR) (See e.g. [40], Example 3). In ourtheoretical analysis and experiments, we delve into two most widely-used sepecial cases of the
Cressie-Read family: (i) β = ∞, which corresponds to CVaR; (ii) β = 2, which corresponds toχ2 -DRO risk used in [51]. Table 3.1 summarizes the relevant quatities in these two special cases.
Table 3.1: CVaR and χ2 -DRO. α is the ratio between the size of the smallest domain and the sizeof the population.
CVaR
β
β∗
ρ

∞
1
− log(α)

cβ (ρ)α−1

Dβ (Q k P )sup log dQdP

DRO Risk/home/chen/Dropbox/App/Overleaf /ICM L0 21 : DRORobusttoOutliers/mathc ommands.texα (For example, the dual form of CVaR is
CVaRα (θ; P ) = inf {α−1 EP [(`(θ; Z) − η)+ ] + η}
η∈R

(3.7)

It is easy to see that the optimal η of (3.7) is the α-quantile of l(θ; Z) defined asqθ (α) = inf {PZ∼P (`(θ; Z) > q) ≤ α}
q

(3.8)

The dual form (3.7) shows that CVaR in effect minimizes the expected risk on the worst α
portion of the training data.
The following corollary of Proposition 1 shows that both CVaRα (θ; P ) and RDχ2 ,ρ (θ; P ) are
upper bounds of Rmax (θ; P ), so that minimizing either of them guarantees a small worst-case risk(see the proof in Appendix 3.8.2):
Corollary 2. Let α = mink=1,···,K P (Dk ) be the minimal group size, and ρ = 12 ( α1 − 1)2 . Then
Rmax (θ; P ) ≤ CVaRα (θ; P ) ≤ RDχ2 ,ρ (θ; P )

3.3

(3.9)

DRO is Sensitive to Outliers

Although the construction of DRO aims to be effective against subpopulation shift as detailed
in the previous section, when applied to real tasks DRO is found to have poor and unstable31August 7, 2022
DRAFT
performance. After some examination, we pinpoint one direct cause of this phenomenon: thevulnerablity of DRO to outliers that widely exist in modern datasets. In this section, we will
provide some intriguing experimental results to show that:
1. DRO methods have poor and unstable performances.
2. Sensitivity to outliers is a direct cause of DRO’s poor performance. To support thisargument, we show that DRO becomes good and stable on a “clean” dataset constructedby removing the outliers from the original dataset, and new outliers added to this “clean”
dataset compromise DRO’s performance and stability.
We conduct experiments on COMPAS [79], a recidivism prediction dataset with 5049 training
instances (after preprocessing and train-test splitting). We select two features as protected features:race and sex. The two protected features define four overlapping demographic groups: White,
Others, Male and Female. A two-layer feed-forward neural network with ReLU activations isused as the classification model. We train three models on this dataset with ERM, CVaR andχ2 -DRO. Then we remove the outliers from the training set using the following procedure: We
first train a model with ERM, and then remove 200 training instances that incur the highest loss
on this model, as outliers are likely to have poorer fit. Then we reinitialize the model, train it on
the new training set with ERM, and remove 200 more instances with the highest loss from the
new training set. This process is repeated 5 times, so that 1000 training instances are removed
and we get a new training set with 4049 instances. Note that this procedure is not guaranteed toremove all outliers and retain all inliers, but is sufficient for the purposes of our demonstration.
We then run the three algorithms again on this same “clean” training set.
We plot the test accuracies (average and worst across four demographic groups) of the models
achieved by the three methods in Figure 3.2. The first row shows the results on the original dataset,and the second row shows the results on the “clean” dataset with the outliers removed. We can seethat in the first row, for both average and worst-case test accuracies, the DRO curves are below
the ERM curves and jumping up and down, which implies that DRO has lower performance
than ERM and is very unstable on the original dataset. However, the third row shows that DRO
becomes good and stable after the outliers are removed. For comparison, in the second row weplot the train/test loss on the original dataset of the three methods (for ERM we plot the ERM loss,
and for DRO we plot the corresponding DRO loss). The train and test losses of DRO descend
steadily while the average and worst-case accuracies jump up and down, which indicates that the
instability is not an optimization issue, but rather stems from the existence of outliers. It should
also be emphasized that these outliers naturally exist in the original dataset since no outliers havebeen manually added yet.
To further substantiate our conclusion, we consider another common source of outliers:
incorrect labels. We randomly flip 20% of the labels of the “clean” COMPAS dataset withthe outliers removed, and run the three training methods again. The results are plotted in the
fourth row of Figure 3.2, which shows that while the label noise just slightly influences ERM, it
significantly downgrades the performance and stability of the two DRO methods.
Likewise, [56] also found in their experiments that DRO had even lower performance thanERM (see their Table 1). Essentially, DRO methods minimize the expected risk on the worst
portion of the training data, which contains a higher density of outliers than the whole population.
Training on these instances naturally result in the observed bad performance of DRO.
32

August 7, 2022
DRAFT
In the next section we will propose DORO as a solution to the problem revealed by theexperiments in this section. We plot the performances of the two DORO algorithms we implementin the last row of Figure 3.2, which compared to the first row shows that DORO improves theperformance and stability of DRO on the original dataset.

3.4

DORO

Problem Setting The goal is to train a model on a dataset with outliers to achieve high tailperformance on the clean underlying data distribution P . Denote the observed contaminatedtraining distribution by ptrain . We formulate ptrain with Huber’s -contamination model [58], inwhich the training instances are i.i.d. sampled fromptrain = (1 − )P + P̃

(3.10)

where P̃ is an arbitrary outlier distribution, and 0 <  < 12 is the noise level. The objective is tominimize Rmax (θ; P ), the worst-case risk over the clean distribution P .
DORO Risk We propose to minimize the following expected -DORO risk:
RD,ρ, (θ; ptrain ) =inf0 {RD,ρ (θ; P 0 ) : ∃P̃ 0 s.t. ptrain = (1 − )P 0 + P̃ 0 }(3.11)P

The DORO risk is motivated by the following intuition: we would like the algorithm to avoid
the “hardest” instances that are likely to be outliers, and the optimal P 0 of (3.11) consists of
the “easiest” (1 − )-portion of the training set given the current model parameters θ. The 
in DORO is a hyperparameter selected by the user since the real noise level of the dataset is
unknown. Let the real noise level of ptrain be 0 . For any  ≥ 0 , there exist P̃0 and P̃ such thatptrain = (1 − 0 )P + 0 P̃0 = (1 − )P + P̃ , so we only need to make sure that  is not less than
the real noise level.
The following proposition provides the formula for computing the DORO risk for the CressieRead family (See the proof in Appendix 3.8.3):
Proposition 3. Let ` be a continuous non-negative loss function, and suppose ptrain is a continuous
distribution. Then the formula for computing the DORO risk with Dβ is
RDβ ,ρ, (θ; ptrain ) =inf {cβ (ρ)EZ∼ptrain [(`(θ; Z) − η)β+∗ |(3.12)

η

PZ 0 ∼ptrain (`(θ; Z 0 ) > `(θ; Z)) ≥ ]

1
β∗

+ η}

Remark In Proposition 3, we assume the continuity of ptrain to keep the formula simple. For
an arbitrary distribution ptrain , we can obtain a similar formula, but the formula is much more
complex than (3.12). The general formula can be found in Appendix 3.8.3.
With this formula, we develop Algorithm 4. In the algorithm, we first order the batch samples
according to their training losses, then find the optimal η ∗ using some numerical method (we use
33

August 7, 2022
DRAFT
Algorithm 4 DORO with Dβ Divergence
Input: Batch size n, outlier fraction , minimal group size α
for each iteration do
Sample a batch z1 , · · · , zn ∼ ptrain
Compute losses: `i = `(θ, zi ) for i = 1, · · · , n
Sort the losses: `i1 ≥ · · · ≥ `inPnβ∗ 1/β∗1+
Find η ∗ = argminη F (θ, η) where F (θ, η) = cβ (ρ)· [ n−bncj=bnc+1 (`(θ; zij ) − η)+ ]ηUpdate θ by one step to minimize `(θ) = F (θ, η ∗ ) with some gradient method
end forBrent’s method [17] in our implementation), and finally update θ with some gradient method.
Note that generally it is difficult to find the minimizer of the DORO risk for neural networks, andour algorithm is inspired by the ITLM algorithm [115], in which they proved that the optimization
converges to ground truth for a few simple problems. Particularly, using the quantities listed
in Table 3.1, we can implement CVaR-DORO and χ2 -DORO. In the sections that follow, we
will focus on the performances of CVaR-DORO and χ2 -DORO in particular. We denote the
CVaR-DORO risk by CVaRα, (θ; ptrain ), and the χ2 -DORO risk by RDχ2 ,ρ, (θ; ptrain ).

3.5

Theoretical Analysis

Having the DORO algorithms implemented, in this section we prove that DORO can effectively
handle subpopulation shift in the presence of outliers. The proofs to the results in this section canbe found in Appendix 3.8.4. We summarize our theoretical results as follows:
1. The minimizer of DORO over the contaminated distribution ptrain achieves a DRO risk close
to the minimum over the clean distribution P (Theorem 5). We complement our analysis
with information-theoretical lower bounds (Theorem 6) implying that the optimality gaps
given by Theorem 5 are optimal.
2. The worst-case risk Rmax over P is upper bounded by the DORO risk over ptrain times aconstant factor (Theorem 7). This result parallels Corollary 2 in the uncontaminated setting
and guarantees that minimizing the DORO risk over ptrain effectively minimizes Rmax overP.
Our results are based on the following lemma which lower bounds the DORO risk over ptrain
by the infimum of the original RDRO risk in a TV-ball centered at P :Lemma 4. Let TV(P, Q) = 12 X ×Y |P (z) − Q(z)|dz be the total variation, and ptrain be definedby (3.10). Then the DORO risk can be lower bounded by:RD,ρ, (θ; ptrain ) ≥inf00 {RD,ρ (θ; P 00 ) : TV(P, P 00 ) ≤
P


}
1−

(3.13)The main results we are about to present only require very mild assumptions. For the first
result, we assume that ` has a bounded (2k)-th moment on P , a standard assumption in the robust34August 7, 2022
DRAFT
statistics literature:
Theorem 5. Let ptrain be defined by (3.10). Denote the minimizer of the DORO risk by θ̂. If ` is2k
non-negative, and `(θ̂; Z) has a bounded (2k)-th moment: EZ∼P [l(θ̂; Z)2k ] = σ2k< +∞, thenwe have:1
CVaRα (θ̂; P ) − inf CVaRα (θ; P ) ≤ Oα,k (1)σ2k 1− 2k(3.14)
θand if k > 1, then we have:1

1

RDχ2 ,ρ (θ̂; P ) − inf RDχ2 ,ρ (θ; P ) ≤ Oρ,k (1)σ2k ( 2 − 2k )(3.15)

θ

Furthermore, the above optimality gaps are optimal:
Theorem 6. There exists a pair of (P, ptrain ) where ptrain = (1 − )P + P 0 and P has uniformly
2kbounded 2k-th moment: ∀θ ∈ Θ, EP [l(θ, Z)2k ] ≤ σ2ksuch that for any learner with only accessto ptrain , the best achievable error in DRO over P is lower bounded by
1

CVaRα (θ̂; P ) − inf CVaRα (θ; P ) ≥ Ωα,k (1)σ2k 1− 2k(3.16)

θ∈Θ

1

1

RDχ2 ,ρ (θ̂; P ) − inf RDχ2 ,ρ (θ; P ) ≥ Ωρ,k (1)σ2k ( 2 − 2k )(3.17)

θ∈Θ

1

1

1

We make a few remarks on these theoretical results. The O(1− 2k ) and O( 2 − 2k ) rates
resemble the existing works on robust mean/moment estimation, see e.g. [73, 102]. The robustmean estimation problem can be seen as a special case of CVaR when α = 1, where CVaR ofany θ is just the mean of l(θ, Z). On the other hand, the connection between CVaR and robust
moment estimation can be built with the dual characterization (3.5): for any fixed dual variableη, evaluating the dual is nothing but a robust (β∗ -th) moment estimation of the random variable(l(θ, Z) − η)+ . However, the problem we are trying to tackle in the above theorems is morechallenging, in the sense that (1) DRO risk involves taking infimum over all η ∈ R, but the
moments of (l(θ, Z) − η)+ are not uniformly bounded for all possible η’s; and (2) the optimal dual
variable η ∗ can be very different even for distributions extremely close in total-variation distance.
In Appendix 3.8.4 we discuss how to overcome these difficulties in detail.
Our second result is a robust analogue to Corollary 2: we show that the worst-case risk Rmax
can be upper bounded by a constant factor times the DORO risk CVaRα, , under the very mild
assumption that ` has a uniformly bounded second moment on P and Rmax is not exceedingly
small:
Theorem 7. Let ptrain be defined by (3.10). Let α = mink=1,···,K P (Dk ), and ρ = 12 ( α1 −1)2 . If `(θ; Z) is a non-negative loss function with a uniformly bounded second moment:
EZ∼P [`(θ; Z)2 ] ≤ σ 2 for all θ, then we have:r
−1
Rmax (θ; P ) ≤ max{3CVaRα, (θ; ptrain ), 3α σ}
1−r
(3.18)
−1≤ max{3Dχ2 ,ρ, (θ; ptrain ), 3α σ}1−Note that a similar result can be derived under the bounded 2k-th moment condition withdifferent constants.
35

August 7, 2022
DRAFT

3.6

Experiments

In this section, we conduct large-scale experiments on modern datasets. Our results showthat DORO improves the performance and stability of DRO. We also analyze the effect ofhyperparameters on DRO and DORO.

3.6.1

Setup

Datasets Our goal is to apply DRO to real tasks with subpopulation shift on modern datasets.
While many previous work used small tabular datasets such as COMPAS, these datasets are
insufficient for our purpose. Therefore, apart from COMPAS, we use two large datasets: CelebA[87] and CivilComments-Wilds [16, 71]. CelebA is a widely used vision dataset with 162,770
training instances, and CivilComments-Wilds is a recently released language dataset with 269,038
training instances. Both datasets are captured in the wild and labeled by potentially biased humans,so they can reveal many challenges we need to face in practice.
We summarize the datasets we use as follows: (i) COMPAS: recidivism prediction, where thetarget is whether the person will reoffend in two years; (ii) CelebA: human face recognition, where
the target is whether the person has blond hair; (iii) CivilComments-Wilds: toxicity identification,
where the target is whether the user comment contains toxic contents. All targets are binary. For
COMPAS, we randomly sample 70% of the instances to be the training data (with a fixed randomseed) and the rest is the validation/testing data. Both CelebA and CivilComments-Wilds haveofficial train-validation-test splits, so we use them directly.
Domain Definition On COMPAS we define 4 domains (subpopulations), and on CelebA and
CivilComments-Wilds we define 16 domains for each. Our domain definitions cover several typesof subpopulation shift, such as different demographic groups, class imbalance, labeling biases,
confounding variables, etc. See Appendix 3.9.1 for details.
Training We use a two-layer feed-forward neural network activated by ReLU on COMPAS,
a ResNet18 [54] on CelebA, and a BERT-base-uncased model [34] on CivilComments-Wilds.
On each dataset, we run ERM, CVaR, χ2 -DRO, CVaR-DORO and χ2 -DORO. Each algorithm isrun 300 epochs on COMPAS, 30 epochs on CelebA and 5 epochs on CivilComments-Wilds. For
each method we collect the model achieved at the end of every epoch, and select the best modelthrough validation. (On CivilComments-Wilds we collect 5 models each epoch, one for every∼20% of the training instances.)
Model Selection To select the best model, we assume that the domain membership of each
instance is available in the validation set, and select the model with the highest worst-casevalidation accuracy. This is an oracle strategy since it requires a domain-aware validation set.
Over the course of our experiments, we have realized that model selection with no group labels
during validation is a very hard problem. On the other hand, model selection has a huge impact
on the performance of the final model. We include some preliminary discussions on this issue in36

August 7, 2022
DRAFT
Table 3.2: The average and worst-case test accuracies of the best models achieved by different
methods. (%)
DatasetMethodAverage Accuracy

Worst-case Accuracy

COMPAS

ERM
CVaR
CVaR-DOROχ2 -DROχ2 -DORO

69.31 ± 0.19
68.52 ± 0.31
69.38 ± 0.10
67.93 ± 0.40
69.62 ± 0.16

68.83 ± 0.18
68.22 ± 0.30
69.11 ± 0.05
67.32 ± 0.60
69.22 ± 0.11

CelebAERM
CVaR
CVaR-DOROχ2 -DROχ2 -DORO

95.01 ± 0.38
82.83 ± 1.33
92.91 ± 0.48
83.85 ± 1.42
82.18 ± 1.17

53.94 ± 2.02
66.44 ± 2.34
72.17 ± 3.14
67.76 ± 3.22
68.33 ± 1.79

CivilComments-Wilds

ERM
CVaR
CVaR-DOROχ2 -DROχ2 -DORO

92.04 ± 0.24
89.11 ± 0.76
90.45 ± 0.70
90.08 ± 0.92
90.11 ± 1.09

64.62 ± 2.48
63.90 ± 4.42
68.00 ± 2.10
65.55 ± 1.51
67.19 ± 2.51

Table 3.3: Standard deviations of average/worst-case test accuracies during training on CelebA.
(α = 0.1 for CVaR/CVaR-DORO; α = 0.3 for χ2 -DRO/χ2 -DORO.  = 0.01) (%)
MethodAverage

Worst-case

ERM
CVaR
CVaR-DOROχ2 -DROχ2 -DORO

0.73 ± 0.06
11.53 ± 1.72
4.03 ± 1.57
8.88 ± 2.98
1.60 ± 0.34

8.59 ± 0.90
21.47 ± 0.71
16.84 ± 0.91
19.06 ± 1.18
13.01 ± 1.40

Appendix 3.9.2. Since model selection is not the main focus of this paper, we pose it as an open
question.

3.6.2

Results

The 95% confidence intervals of the mean test accuracies on each dataset are reported in Table 3.2.
For every DRO and DORO method, we do a grid search to pick the best α and  that achieve thebest worst-case accuracy (see the optimal hyperparameters in Appendix 3.9.3). Each experiment isrepeated 10 times on COMPAS and CelebA, and 5 times on CivilComments-Wilds with different
random seeds. Table 3.2 clearly shows that on all datasets, DORO consistently improves theaverage and worst-case accuracies of DRO.
Next, we analyze the stability of the algorithms on the CelebA dataset. We use the α that
achieves the optimal DRO performance for each of CVaR and χ2 -DRO, and compare them to
DORO with the same value of α and  = 0.01. χ2 -DRO achieves its optimal performance with a37

August 7, 2022
DRAFT
bigger α than CVaR because it is less stable. To quantitatively compare the stability, we computethe standard deviations of the test accuracies across epochs and report the results in Table 3.3.
To further visualize the training dynamics, we run all algorithms with one fixed random seed,
and plot the test accuracies during training in Figures 3.3 and 3.4. Table 3.3 shows that thestandard deviation of the test accuracy of DORO is smaller and in Figures 3.3a and 3.4a the
DORO curves are flatter than the DRO curves, which implies that DORO improves the stability of
DRO. Although it is hard to tell whether DORO has a more stable worst-case accuracy from thefigures, our quantitative results in Table 3.3 confirm that DORO has more stable worst-case test
accuracies.

3.6.3

Effect of Hyperparameters

In this part, we study how α and  affect the test accuracies of DORO with two experiments on
CelebA, providing insight into how to select the optimal hyperparameters.
In the first experiment, we fix α = 0.2, and run the two DORO algorithms with different
values of . The results are plotted in Figure 3.5. We can see that for both methods, as  increases,
the average accuracy slightly decreases, while the worst-case accuracy first rises and then drops.
Both average and worst-case accuracies will drop if  is too big. Moreover, both methods achieve
the optimal worst-case accuracy at  = 0.005. We conjecture that the real noise level of the
CelebA dataset is around 0.005, and that the optimal  should be close to the real noise level.
In the second experiment, we run DRO and DORO ( = 0.01) with different values of α.
The results are plotted in Figure 3.6. First, we observe that for all methods, the optimal α ismuch bigger than the real α of the dataset. The real α of the CelebA dataset is around 0.008 (seeAppendix 3.9.1, Table 3.4), much smaller than those achieving the highest worst-case accuracies
in the figures. Second, in all four figures the overall trend of the average accuracy is that it grows
with α. Third, both CVaR-DORO and χ2 -DORO achieve the optimal worst-case accuracy at
α = 0.25, but the worst-case accuracy drops as α goes to 0.3.

3.7

Discussion

In this work we pinpointed one direct cause of the performance drop and instability of DRO: thesensitivity of DRO to outliers in the dataset. We proposed DORO as an outlier robust refinementof DRO, and implemented DORO for the Cressie-Read family of Rényi divergence. We made apositive response to the open question raised by [51] by demonstrating the effectiveness of DORO
both theoretically and empirically.
One alternative approach to making DRO robust to outliers is removing the outliers from thedataset via preprocessing. In Section 3.3 we used a simple version of iterative trimming [115] toremove outliers from the training set. Compared to iterative trimming, DORO does not require
retraining the model and does not throw away any data. In addition, preprocessing methods suchas iterative trimming cannot cope with online data (where new instances are received sequentially),
but DORO is still feasible.
The high-level idea of DORO can be extended to other algorithms that deal with subpopulation
shift, such as static reweighting [116], adversarial reweighting [56, 77] and group DRO [110].
38

August 7, 2022
DRAFT
The implementations might be different, but the basic ideas are the same: to prevent the algorithm
from overfitting to potential outliers. We leave the design of such algorithms to future work.
There is one large open question from this work. In our experiments, we found that model
selection without domain information in the validation set is very hard. In Appendix 3.9.2 westudy several strategies, such as selecting the model with the lowest CVaR risk or the lowest CVaRDORO risk, but none of them is satisfactory. A recent paper [92] proposed two selection methods
Minmax and Greedy-Minmax, but their performances are still much lower than the oracle’s(see their Table 2a). [48] also pointed out the difficulty of model selection in domain-oblivious
distributional shift tasks. Thus, we believe this question to be fairly non-trivial.

3.8

Proofs3.8.1

Proof of Proposition 1

We have the following observation: when α ≤ exp(−1), we have:11∀t ∈ [0, ], fβ (t) ≤ fβ ( ).
αα(3.19)1(tβ−1 − 1)
β−1(3.20)

Notice thatfβ0 (t) =Hence, fβ0 (t) is decreasing when t ∈ [0, 1] and increasing when t ∈ [1, α1 ]. therefore, fβ (t) ≤max(fβ (0), fβ ( α1 )). We can further verify that


1
β
1
1
−
fβ ( ) − fβ (0) =(3.21)αβ(β − 1) αβ α1which is nonnegaative whenever α ≤ β − β−1 . Since when β > 1, one always have exp(−1) ≤1
β − β−1 and we have proved equation 3.19.
Since Pk is a mixture component of P with probability mass at least α, we can see that
dPk
1
≤
dP
α

(3.22)

thus, by equation 3.19,
Z

dPk
)dP
dP
Z
1
≤ fβ ( )dPα1= fβ ( )αDβ (Pk ||P ) =fβ (by the definition of β-DRO risk, we have completed the proof.
39(3.23)(3.24)(3.25)

August 7, 2022
DRAFT

3.8.2

Proof of Corollary 2

For any k, let pk = P (Dk ), then P (z) = pk P (z|Dk ) + (1 − pk )P (z|Dk ) holds for all x. Letk −αk
Q = Pk and Q0 (z) = p1−α
P (z|Dk ) + 1−p
P (z|Dk ). Then P = αQ + (1 − α)Q0 , which implies
1−αthat EPk [`(θ; Z)] ≤ CVaRα (θ; P ). Thus, Rmax (θ; P ) ≤ CVaRα (θ; P ). On the other hand, for any(z) ≤ α1 a.e., so that
Q such that there exists Q0 satisfying P = αQ + (1 − α)Q0 , we have dQ
dP
1 1
2Dχ2 (Q k P ) ≤ 2 ( α − 1) = ρ. Thus, CVaRα (θ; P ) ≤ RDχ2 ,ρ (θ; P ).

3.8.3

Proposition 3

Proof of Proposition 3
By (3.6) and (3.11) we havenoRDβ ,ρ, (θ; ptrain ) = inf0 RDβ ,ρ (θ; P 0 ) : ∃P̃ 0 s.t. ptrain = (1 − )P 0 + P̃ 0
P
o
n
β∗ β1
∗
0
= inf
cβ (ρ)EP [(`(θ; Z) − η)+ ] + η
P 0 ,η
Z
1
β∗0= inf cβ (ρ) inf0 {[P ((`(θ; Z) − η)+ > u)du] β∗ } + ηηP(3.26)

R+

By ptrain = (1 − )P 0 + P̃ 0 we have for all `0 ,
1
0P (`(θ; Z) ≤ `0 ) ≤ min 1,
ptrain (`(θ; Z) ≤ `0 )1−(3.27)

and we can also show that there exists a P ∗ = P 0 such that the equality is achieved in (3.27) forall `0 : Since both ` and ptrain are continuous, ptrain (`(θ; z)) is a continuous function of z for any
fixed θ, so there exists an `∗ such that ptrain (`(θ; Z) > `∗ ) = . Define 1p(z) , `(θ; z) ≤ `∗∗1− train
P (z) =
(3.28)0, `(θ; z) > `∗R
R
1
1
For (3.28), we have X ×Y P ∗ (z)dz = 1−
p(z)dz = 1−ptrain (`(θ; Z) < `∗ ) = 1`(θ;z)<`∗ train
because ptrain (`(θ; Z) = `∗ ) = 0, so (3.28) is a distribution function.
 1
1
Let v = u β∗ . Plugging P ∗ (`(θ; Z) ≤ `0 ) = min 1, 1−ptrain (`(θ; Z) ≤ `0 ) into (3.26)
produces
)
(
Z
 β1
∗
β
RDβ ,ρ, (θ; ptrain ) = inf cβ (ρ)[1 − P ∗ ((`(θ; Z) − η)+∗ ≤ v β∗ )]dv β∗
+η
η

R+

(
= inf
η

= inf
η

Z

cβ (ρ)


1
ptrain (`(θ; Z) ≤ η + v)]+ dv β∗
[1 −
1
−

R+

"Z
cβ (ρ)
0(`∗ −η)+

 β1

)

∗

+η

1
[(1 − ) − ptrain (`(θ; Z) ≤ η + v)]+ dv β∗
1−

# β1

∗

+η


(3.29)40August 7, 2022
DRAFT
On the other hand, we have
Eptrain [(` − η)β+∗ | PZ 0 ∼ptrain (`(θ; Z 0 ) > `(θ; Z)) ≥ ]Z `∗1=(u − η)β+∗ d(ptrain (` ≤ u))
1− 0

h
i`∗ Z `∗
1β∗β∗ptrain (` ≤ u)d((u − η)+ )(u − η)+ ptrain (` ≤ u) −
=
1−
0
0


Z `∗
1
β∗
β∗
∗
ptrain (` ≤ u)d((u − η)+ )(` − η)+ (1 − ) −
=
1−
0
)
(Z ∗
Z (`∗ −η)+(` −η)+
1=
ptrain (` ≤ η + w)dwβ∗(1 − )dv β∗ −1−00(3.30)

where w = (u − η)+ . Thus, (3.29) is equal to the right-hand side of (3.12).
Extension to Arbitrary ptrain
For any distribution ptrain , we can obtain a similar but more complex formula (3.33). For any
ptrain , there exists an `∗ such that ptrain (`(θ; Z) > `∗ ) ≤  and ptrain (`(θ; Z) < `∗ ) ≤ 1 − . If
ptrain (`(θ; Z) = `∗ ) = 0, then the proof above is still correct, so the formula is still (3.12).
Now assume that ptrain (`(θ; Z) = `∗ ) > 0. Similar to (3.28), define∗

P (z) = , `(θ; z) < `∗/ptrain (`(θ; Z) = ` ) , `(θ; z) = `∗, `(θ; z) > `∗1p(z)
1− train ∗1ptrain (`(θ; Z) < ` )1 − 1−0

∗(3.31)

 1
Then we still have P ∗ (`(θ; Z) ≤ `0 ) = min 1, 1−ptrain (`(θ; Z) ≤ `0 ) , so (3.29) still holds.
On the other hand, we have
Eptrain [(` − η)β+∗ | PZ 0 ∼ptrain (`(θ; Z 0 ) > `(θ; Z)) > ](Z ∗)Z (`∗ −η)+(` −η)+
1=(1 − )dv β∗ −ptrain (` ≤ η + w)dwβ∗
ptrain (`(θ; Z) < `∗ )00(3.32)

Thus, the formula becomes
ptrain (` < `∗ )EZ [(`(θ; Z) − η)β+∗ | PZ 0 (`(θ; Z 0 ) > `(θ; Z)) > ]
η1−1
1 − ptrain (` < `∗ ) ∗+(` − η)β+∗ ) β∗ + η}
1−
(3.33)RDβ ,ρ, (θ; ptrain ) = inf {cβ (ρ)(

41

August 7, 2022
DRAFT

3.8.4

Proofs of Results in Section 3.5

A Key Technical Lemma
The following lemma will be useful in the analysis of CVaR-DORO and χ2 -DORO: it controls thedifference of dual objective in two distributions P, P 0 by their total variation distance, with theassumption that loss function l has bounded 2k-th moment under P .
Lemma 8. For any distributions P, P 0 , non-negative loss function l(·, Z) and 1 ≤ β∗ < 2k, suchthat EP [l(θ, Z)2k ] < ∞, we have111
10 ( β1∗ − 2k
) β − 2k1 ·2kEP [(`−η)β+∗ ] β∗ ≤ EP 0 [(`−η)β+∗ ] β∗ +EP [(l(θ, Z)−η)2k∗+ ] TV(P, P )



 β1
∗
2k
2k − β∗(3.34)

Proof. By the definition of total variation distance, we haveP (`(θ; Z) > u) − P 0 (`(θ; Z 0 ) > u) ≤ TV(P, P 0 )(3.35)

holds for any u ≥ 0.
By Markov’s Inequality and the non-negativity of `, we have for any η ≥ 0,E[(` − η)2k
s2k+]:= ( )2k
P (` − η > u) ≤2kuu

(3.36)

1

2k Using integration by parts, we can see
where we introduced the shorthand s2k := E[(` − η)2k
+]
that:
Z ∞
β∗
EP [(` − η)+ ] =(3.37)β∗ (t − η)(β∗ −1) P (` ≥ t)dtηZ ∞=β∗ u(β∗ −1) P (` − η ≥ u)du
(3.38)0Thus,
EP [(` − η)β+∗ ] − EP 0 [(` − η)β+∗ ] =Z ∞β∗ u(β∗ −1) (P (` − η ≥ u) − P 0 (` − η ≥ u)) du0Z M Z ∞ =+β∗ u(β∗ −1) (P (` − η ≥ u) − P 0 (` − η ≥ u)) du0M(3.39)
Here, M is a positive parameter whose value will be determined later. Next, we will upper boundeach of the two integrals separately. By equation 3.39,
Z M
β∗ u

(β∗ −1)Z M

0(P (` − η ≥ u) − P (` − η ≥ u)) du ≤0β∗ u(β∗ −1) TV(P, P 0 )du(3.40)0= M β∗ TV(P, P 0 ),42

(3.41)

August 7, 2022
DRAFT
which gives an upper bound for the first integral. For the second integral, notice that P 0 (` − η ≥ u)
is non-negative and use equation 3.36, we have:Z ∞Z ∞(β∗ −1)0β∗ u(P (` − η ≥ u) − P (` − η ≥ u)) du ≤β∗ u(β∗ −1) P (` − η ≥ u)du (3.42)
M
M
Z ∞
 2k
(β∗ −1) s2k≤
β∗ u(3.43)uM1
s2k
2k· 2k−β∗(3.44)=
2k − β∗ M
Therefore, by setting M = s2k (TV(P, P 0 )β∗ )−1/2k which minimizes the sum of two terms, we
have


β∗
s2k
2k
1
− β∗
β∗
β∗
2k
β∗
0
EP [(`−η)+ ]−EP 0 [(`−η)+ ] ≤ inf M TV(P, P ) +· 2k−β∗ = sβ2k∗ TV(P, P 0 )1− 2k β∗ 2k ·M >0
2k − β∗ M
2k − β∗(3.45)
1
1
1
Using the inequality (A + B) β∗ ≤ A β∗ + B β∗ when β∗ ≥ 1, we have:
1
1
1
1
− 1
EP [(` − η)β+∗ ] β∗ ≤ EP 0 [(` − η)β+∗ ] β∗ + s2k TV(P, P 0 )( β∗ − 2k ) β∗ 2k ·

2k
2k − β∗ β1∗(3.46)

Proof of Lemma 4
For any P 0 such that ptrain = (1 − )P 0 + P̃ 0 for some P̃ 0 , let U = P ∧ P 0 , i.e. U (z) =min{P (z), P 0 (z)} for any z ∈ X × Y. We have
(1 − )U (z) + P̃ (z) + P̃ 0 (z) ≥ ptrain (z)for any z ∈ X × Y

because both P̃ (z) and P̃ 0 (z) are non-negative. Integrating both sides producesZ
1 − 2
U (z)dz ≥1−
X ×Y(3.47)(3.48)


which implies TV(P, P 0 ) ≤ 1−
. Thus,RD,ρ (θ; P 0 ) ≥ inf00 {RD,ρ (θ, P 00 ) : TV(P, P 00 ) ≤
P


}
1−

(3.49)

which together with (3.11) proves (3.13).
Proof of Theorem 5, Analysis of CVaR-DORO
Proof of Theorem 5, CVaR-DORO. For any θ, by Lemma 4 we have
CVaRα, (θ; ptrain ) ≥ CVaRα, (θ̂; ptrain ) ≥ inf0 {CVaRα (θ̂; P 0 ) : TV(P, P 0 ) ≤
P

43


}
1−

(3.50)

August 7, 2022
DRAFT

By Lemma 8, we have for any η ≥ 0 and TV(P, P 0 ) ≤ 1−
,



1
1
1
0 1− 2k
2k
EP [(` − η)+ ] − EP 0 [(` − η)+ ] ≤ 1 +EP [(` − η)2k
+ ] TV(P, P )2k − 1
1
1≤ 1+
σ2k TV(P, P 0 )1− 2k
2k − 1(3.51)

2k
Here, we used the fact that 0 ≤ (` − η)2k
Moreover, for any η < 0, EP [(` − η)+ ] − EP 0 [(` −+ ≤ `η)+ ] = EP [(` − 0)+ ] − EP 0 [(` − 0)+ ] because ` is non-negative. So (3.51) holds for all η ∈ R.
Thus, by (3.7) we have for any η ∈ R,(
1− 2k1 )1CVaRα (θ̂; P ) ≤ α−1 EP [(`−η)+ ]+η ≤ α−1 EP 0 [(` − η)+ ] + 1 ++η
2k − 11−(3.52)and taking the infimum over η, we have the following inequality holds for any θ:0CVaRα (θ̂; P ) ≤ CVaRα (θ̂; P ) + 1 +1
2k − 1

α−1 σ2k1−

1− 2k1(3.53)By (3.11) we have CVaRα, (θ; ptrain ) ≤ CVaRα (θ; P ). Thus, by (5.69), taking the infimum
over P 0 yields1− 2k11CVaRα (θ̂; P ) ≤ CVaRα, (θ; ptrain ) + 1 +α−1 σ2k
2k − 11−
1− 2k11
−1≤ CVaRα (θ; P ) + 1 +α σ2k2k − 11−(3.54)(3.55)Taking the infimum over θ completes the proof.
Proof of Theorem 5, Analysis of χ2 -DORO
We begin with a structral lemma about the optimal dual variable η in the dual formulation
equation 3.5. Recall that β = β∗ = 2 for χ2 divergence.
Lemma 9. Let η ∗ (P ) be the minimizer of equation 3.5. We have the following characterization
about η ∗ (P ):P [l(θ,Z)]
1. When ρ ≤ Var, we have η ∗ ≤ 0;
2E[l(θ,Z)]2
Furthermore, the DRO risk and optimal dual variable η ∗ can be formulated as:pRDχ2 ,ρ (θ; P ) = EP [l(θ, Z)] + 2ρVarP [l(θ, Z)](3.56)s
VarP [l(θ, Z)]η ∗ = EP [l(θ, Z)] −(3.57)
2ρ
P [l(θ,Z)]
2. When ρ ≥ Var, we have η ∗ ≥ 0.
2E[l(θ,Z)]244August 7, 2022
DRAFT
Proof. (1) We will prove that for any ρ > 0,RDχ2 ,ρ (θ; P ) ≤ EP [l(θ, Z)] +p
2ρVarP [l(θ, Z)](3.58)

EQ [l(θ, Z)](3.59)

P [l(θ,Z)]and the equality is achievable when ρ ≤ Var.
2E[l(θ,Z)]22
By the definition of χ -DRO risk,RDχ2 ,ρ (θ; P ) =sup
Q:Dχ2 (Q||P )≤ρ

Let µ := EP [l(θ, Z)], notice that
EQ [l(θ, Z)] = EP [l(θ, Z)

dQ
]
dP

(3.60)dQ= EP [l(θ, Z)] + EP [l(θ, Z)
−1 ]
dP


dQ
= µ + EP [(l(θ, Z) − µ)−1 ]dP
where in the last step we used the fact that EP dQ
= 1.
dP
2
By the definition of χ divergence,
2dQ− 1 ] = 2Dχ2 (Q||P ) ≤ 2ρ,EP [dP(3.61)(3.62)(3.63)

Therefore, by Cauchy-Schwarz inequality,EP [(l(θ, Z) − µ)
2 !1/2
dQ
dQ
1/2− 1 ] ≤ (EP [(l(θ, Z) − µ)])
EP [
−1 ]
dP
dP
p
≤ VarP [l(θ, Z)] · 2ρ

(3.64)(3.65)

Plug in this upper bound to equation 3.60 completes the proof of equation 3.58.
P [l(θ,Z)]
To see that the equality can be achieved when ρ ≤ Var, we only need to verify that
E[l(θ,Z)]2pη = η ∗ gives the same dual objective EP [l(θ, Z)] + 2ρVarP [l(θ, Z)]. Since η ∗ < 0, we have
EP [(l(θ, Z) − η ∗ )2+ ]=EP [(l(θ, Z) − η ∗ )2 ](3.66)(3.67)r

1
VarP [l(θ, Z)])2 ](3.68)2ρ
r
1
1=EP [(l(θ, Z) − EP [l(θ, Z)])2 ] + 2
VarP [l(θ, Z)]E[(l(θ, Z) − EP [l(θ, Z)])] + VarP [l(θ, Z)]2ρ
2ρ
(3.69)
1
1 + 2ρ
=VarP [l(θ, Z)] + 0 + + VarP [l(θ, Z)] =VarP [l(θ, Z)](3.70)2ρ
2ρ
=EP [(l(θ, Z) − EP [l(θ, Z)] +45

August 7, 2022
DRAFT
Therefore,p1/2
1 + 2ρ p
1 p
1 + 2ρ EP [(l(θ, Z) − η ∗ )2+ ]+ η∗ = √VarP [l(θ, Z)] + EP [l(θ, Z)] − √VarP [l(θ, Z)]2ρ
2ρ
(3.71)p= EP [l(θ, Z)] + 2ρVarP [l(θ, Z)](3.72)
and we have completed the proof.
1√(2) Let g(η, P ) = 1 + 2ρ EP [(l(θ, Z) − η)2+ ] 2 + η and recall that RDχ2 ,ρ (θ; P ) =inf η∈R g(η, P ). To show that η ∗ ≤ 0, we only need to prove that g(η) ≤ g(0) whenever η < 0,which is equivalent to:
p
1 p
1
1 + 2ρ EP [(l(θ, Z) − η)2 ] 2 ≥ 1 + 2ρ EP [l(θ, Z)2 ] 2 − η(3.73)

Since both sides are non-negative, this inequality is equivalent to:(1 + 2ρ)EP [(l(θ, Z) − η)2 ≥ (1 + 2ρ)EP [l(θ, Z)2 + η 2 − 2ηp

1
1 + 2ρ EP [l(θ, Z)2 ] 2 (3.74)

After re-organizing terms, it remains to prove2ρη 2 − 2(1 + 2ρ)ηEP [l(θ, Z)] + 2ηp
1
1 + 2ρ EP [l(θ, Z)2 ] 2 ≥ 0(3.75)2E[l(θ,Z) ]P [l(θ,Z)]
Since ρ ≥ Var, we have (1 + 2ρ) ≥ E[l(θ,Z)]
2 . Therefore,
2E[l(θ,Z)]2

1
1 + 2ρ EP [l(θ, Z)2 ] 2 − 2(1 + 2ρ)ηEP [l(θ, Z)]


p
1 p
= 2η 1 + 2ρ EP [l(θ, Z)2 ] 2 − 1 + 2ρEP [l(θ, Z)](3.77)≥0(3.78)

LHS ≥ 2ηp

(3.76)

where in the last step we used the assumption that η ≤ 0. Therefore we have completed the proof.
Having prepared with Lemma 9, we are now ready to prove the χ2 -DORO part of Theorem 5.
Proof of Theorem 5, χ2 -DORO. We will first show thatRDχ2 ,ρ (θ̂; P ) ≤ RDχ2 ,ρ (θ̂; P 0 )+p

1 + 2ρ(1+Cρ )σ2k TV(P, P 0 )(

1
1
− 2k
2

)2

1
− 2k



k
·
k−1

 21
(3.79)

This inequality will be proved by combining two different strategies: when η ∗ (P 0 ) is relatively
large, we will use an argument based on Lemma 8, similar to what we did in the analysis of
CVaR-DORO. Otherwise, when η ∗ (P 0 ) is small, we need a different proof which builds upon the
structral result Lemma
9.
√
1+2ρ
Define Cρ = 2ρ . Below we discuss two cases: η ∗ (P 0 ) < −Cρ σ2k and η ∗ (P 0 ) ≥ −Cρ σ2k .
46

August 7, 2022
DRAFT
Case 1: η ∗ (P 0 ) < −Cρ σ2k .

When η ∗ (P 0 ) < −Cρ σ2k , by Lemma 9, we haveq0RDχ2 ,ρ (θ̂; P ) = EP 0 [l(θ̂, Z)] + 2ρVarP 0 [l(θ̂, Z)]s
VarP 0 [l(θ̂, Z)]< −Cρ σ2kη ∗ (P 0 ) = EP 0 [l(θ̂, Z)] −2ρq
Therefore, we can lower bound VarP 0 [l(θ̂, Z)] as:qppp
VarP 0 [l(θ̂, Z)] ≥ 2ρEP 0 [l(θ̂, Z)] + 2ρCρ σ2k ≥ 2ρCρ σ2k ,(3.80)(3.81)(3.82)and consequently, we have a lower bound for RDχ2 ,ρ (θ; P 0 ):q
0RDχ2 ,ρ (θ̂; P ) = EP 0 [l(θ̂, Z)] + 2ρVarP 0 [l(θ̂, Z)]qp≥ 2ρVarP 0 [l(θ̂, Z)] ≥ 2ρCρ σ2k = 1 + 2ρσ2k(3.83)(3.84)On the other hand, by setting the dual variable η = 0, we have a simple upper bound forRDχ2 ,ρ (θ̂; P ):pp
RDχ2 ,ρ (θ̂; P ) ≤ 1 + 2ρEP [l(θ̂, Z)2 ]1/2 ≤ 1 + 2ρσ2k(3.85)
Combining equation 3.83 and equation 3.85, we conclude that RDχ2 ,ρ (θ̂; P 0 ) ≥ RDχ2 ,ρ (θ̂; P ) andthe inequality is trivially true.
Case 2: η ∗ (P 0 ) ≥ −Cρ σ2k

By Lemma 8, we have111
1
0 ( 21 − 2k
) 2− 2k1 ·2kEP [(`−η)2+ ] 2 ≤ EP 0 [(`−η)2+ ] 2 +EZ [(l(θ, Z)−η)2k+ ] TV(P, P )



k
k−1

 21
(3.86)

holds for any η ∈ R. Since η ∗ (P 0 ) ≥ −Cρ σ2k , we can upper bound the 2k-th moment
1
2k as:EZ [(l(θ, Z) − η ∗ (P 0 ))2k
+]
1

1

2k
2k ≤ E [(l(θ, Z) + C σ
EZ [(l(θ, Z) − η ∗ (P 0 ))2k
Z
ρ 2k )+ ] 2k+]

≤ EZ [(l(θ, Z)]

12k(3.87)+ Cρ σ2k = (1 + Cρ )σ2k

(3.88)

Hence,
RDχ2 ,ρ (θ̂; P ) ≤p
1
1 + 2ρEP [(` − η ∗ (P 0 ))2+ ] 2 + η ∗ (P 0 )(3.89)pp
1
1
1
1≤ 1 + 2ρEP 0 [(` − η ∗ (P 0 ))2+ ] 2 + η ∗ (P 0 ) + 1 + 2ρ(1 + Cρ )σ2k TV(P, P 0 )( 2 − 2k ) 2− 2k ·(3.90)= RDχ2 ,ρ (θ̂; P 0 ) +p
1
1
1
1 + 2ρ(1 + Cρ )σ2k TV(P, P 0 )( 2 − 2k ) 2− 2k ·



k
k−1

 12
(3.91)

47



k
k−1

August 7, 2022
DRAFT
Hence, we have proved the inequality equation 3.79. The rest of proof mimics CVaR-DORO.
For any θ, by Lemma 4 we haveRDχ2 ,ρ,ε (θ; ptrain ) ≥ RDχ2 ,ρ,ε ≥ inf0 {RDχ2 ,ρ (θ̂; P 0 ) : TV(P, P 0 ) ≤
P


}
1−

(3.92)By (3.11) we have RDχ2 ,ρ,ε (θ; ptrain ) ≤ RDχ2 ,ρ (θ; P ). Thus, by (5.69), taking the infimum overP 0 yields
( 12 − 2k1 )
 12

1

k
− 2k
2
·
1−
k−1
(3.93)111

( 2 − 2k )
2

p
1

k
− 2k
≤ RDβ ,ρ (θ; P ) + 1 + 2ρ(1 + Cρ )σ2k
2
·
1−
k−1
(3.94)

pRDχ2 ,ρ (θ̂; P ) ≤ RDχ2 ,ρ, (θ; ptrain ) + 1 + 2ρ(1 + Cρ )σ2k

Taking the infimum over θ completes the proof.
Proof of Theorem 6
We consider an optimization problem with the parameter space restricted to only two possible
values Θ = {θ0 , θ1 }. Our proof is constructive, which relies on the following distribution PM,∆,ε :l(θ0 , Z) = 0, l(θ1 , Z) = ∆ w.p. (1 − ε)l(θ0 , Z) = M, l(θ1 , Z) = ∆ w.p. ε(3.95)(3.96)

here M, ∆ are some non-negative parameters whose value to be determined later and the probability is taken over the randomness of Z.
We have the following characterization of CVaR and χ2 -DRO risk:
Lemma 10 (DRO Risk of PM,∆,ε ). Assume that α ≥ ε and 1 + 2ρ ≤ 1ε , we have the following
closed-form expressions for CVaR and χ2 -DRO risk:Mεα
CVaRα (θ1 ; PM,∆,ε ) = ∆CVaRα (θ0 ; PM,∆,ε ) =(3.97)(3.98)

andRDχ2 ,ρ (θ0 ; PM,∆,ε ) = M ε + M
RDχ2 ,ρ (θ1 ; PM,∆,ε ) = ∆p2ρε(1 − ε)(3.99)(3.100)

Proof. Since l(θ1 , Z) is always a constant ∆, it’s immediate to see CVaRα (θ1 ; PM,∆,ε ) =RDχ2 ,ρ (θ1 ; PM,∆,ε ) = ∆. Hence we only need to focus on θ0 .
48

August 7, 2022
DRAFT
By the dual formulation of DRO risk, we have CVaRα (θ0 ; PM,∆,ε ) = inf η∈R h(η) andRDχ2 ,ρ (θ0 ; PM,∆,ε ) = inf η∈R g(η), where we use the shorthand g(η) and h(η) for1
1 + 2ρ EP [(l(θ, Z) − η)2+ ] 2 + ηg(η) :=

ph(η) =

1
EP [(l(θ, Z) − η)+ ] + η
α(3.102)

Direct calculation gives:
√
p
2
2

p1 + 2ρ (η − M ) + ε(1 − ε)M + η, for η < 0g(η) =ε(1 + 2ρ)(M − η) + η,
for 0 ≤ η ≤ Mη,for η > Mand(3.101)M ε−ηfor η < 0
 α + η,ε(M −η)h(η) =+ η, for 0 ≤ η ≤ Mαη,for η > M

(3.103)(3.104)

Therefore, when α ≥ ε and 1 + 2ρ ≤ 1ε , we have
CVaRα (θ0 ; PM,∆,ε ) = inf h(η)= h(0) =η∈R

RDχ2 ,ρ (θ0 ; PM,∆,ε ) = inf g(η) = g(εM −

M

Mε
α

(3.105)pη∈Rpε(1 − ε)√) = M ε + M 2ρε(1 − ε) (3.106)2ρ

and we have completed the proof.
Equipped with Lemma 10, we are now ready to prove the main lower bound Theorem 6.
Proof of Theorem 6. Consider ptrain = PM,∆,ε . We have two different ways to decompose ptrain
into mixture of two distributions:ptrain = PM,∆,ε = (1 − ε)PM,∆,ε + εPM,∆,ε = (1 − ε)P0,∆,0 + εPM,∆,0(3.107)

In other words, with only access to ptrain = PM,∆,ε , the learner cannot distinguish the following
two possibilities:• (a) The clean distribution is P = PM,∆,ε , and the outlier distribution is P 0 = PM,∆,ε .
• (b) The clean distribution is Q = P0,∆,1 , and the outlier distribution is Q0 = PM,∆,1 .
1Furthermore, as long as M ≤ σ2k ε− 2k and ∆ ≤ σ2k , both P and Q satisfy the bounded 2k-th
2kmoment condition E[l(θ, Z)2k ] ≤ σ2k. With our construction below, we can ensure that θ1 isΘ(∆)-suboptimal under P , while θ0 is Θ(∆)-suboptimal under Q. Therefore, in the worst casescenario, it’s impossible for the learner to find a solution with O(∆) sub-optimality gap underboth P and Q.
49

August 7, 2022
DRAFT
1

1

1− 1

CVaR lower bound Assume that α ≥ 12 ε1− 2k . Let M = σ2k ε− 2k , ∆ = σ2k ε 2α2k ≤ σ2k . Recall
that P = PM,∆,ε , by Lemma 10, we have:σ2k 1− 1Mε=ε 2k = 2∆αα
CVaRα (θ1 ; P ) = ∆CVaRα (θ0 ; P ) =(3.108)(3.109)Therefore,
11CVaRα (θ0 ; P ) − inf CVaRα (θ; P ) = ∆ = Ω( σ2k ε1− 2k )θ∈Θα(3.110)For Q = P0,∆,1 , both l(θ0 , Z) and l(θ1 , Z) are constants, and hence
CVaRα (θ0 ; Q) = 0
CVaRα (θ1 ; Q) = ∆(3.111)(3.112)1
1CVaRα (θ1 ; Q) − inf CVaRα (θ; Q) = ∆ = Ω( σ2k ε1− 2k )θ∈Θα(3.113)andCombining equation 3.110 and equation 3.113 completes the proof.
11

χ2 -DROlower boundAssume that ρ = O(ε k −1 ).
Let M = σ2k ε− 2k , ∆pM
ε + 2ρε(1 − ε) ≤ σ2k . Recall that P = PM,∆,ε , by Lemma 10, we have:2
RDχ2 ,ρ (θ0 ; P ) = M ε + Mp2ρε(1 − ε) = 2∆

RDχ2 ,ρ (θ1 ; P ) = ∆Therefore,=(3.114)(3.115)√ 1 1
RDχ2 ,ρ (θ0 ; P ) − inf RDχ2 ,ρ (θ; P ) = ∆ = Ω(σ2k ρε 2 − 2k )θ∈Θ

(3.116)For Q = P0,∆,1 , both l(θ0 , Z) and l(θ1 , Z) are constants, and hence

and

CVaRα (θ0 ; Q) = 0
CVaRα (θ1 ; Q) = ∆(3.117)(3.118)√ 1 1
CVaRα (θ0 ; Q) − inf CVaRα (θ; Q) = ∆ = Ω(σ2k ρε 2 − 2k )(3.119)θ∈ΘCombining equation 3.116 and equation 3.119 completes the proof.

50

August 7, 2022
DRAFT
Proof of Theorem 7
By Lemma 8, for any P 0 such that TV(P, P 0 ) ≤ 1−
,

0

−1

CVaRα (θ; P ) − CVaRα (θ; P ) ≤ 2α σ
By Proposition 2, if Rmax (θ; P ) > 3α−1 σimplies that

p 
1−

r


1−

, then CVaRα (θ; P ) > 3α−1 σ(3.120)p p 
2α−1 σ 1−CVaRα (θ; P 0 )CVaRα (θ; P 0 )
δ
1
≥
=1−
≥ 1 − −1 p  =Rmax (θ; P )CVaRα (θ; P )CVaRα (θ; P )33α σ 1−

1−

, which(3.121)holds for any P 0 such that TV(P, P 0 ) ≤ 1−
. By Lemma 4, taking the infimum over P 0 yields
the first inequality of (3.18). Moreover, by Proposition 2, for any θ and P 0 , Dχ2 ,ρ (θ; P 0 ) ≥CVaRα (θ; P 0 ), which together with (3.121) yields the second inequality of (3.18).

3.9

Experiment Details

3.9.1

Domain Definition

One important decision we need to make when we design a task with subpopulation shift is how
to define the domains (subpopulations). We refer our readers to the Wilds paper [71], which
discusses in detail the desiderata and considerations of domain definition, and defines 16 domains
on the CivilComments-Wilds dataset which we use directly. The authors selected 8 features suchas race, sex and religion, and crossed them with the two classes to define the 16 domains. Such a
definition naturally covers class imbalance. There is no official domain definition on CelebA, so
we define the domains on our own. Following their approach, on CelebA we also select 8 featuresand cross them with the two classes to compose the 16 domains. Our definition is inspired by[110], but we cover more types of subpopulation shift apart from demographic differences.
We select 8 features on CelebA: Male, Female, Young, Old, Attractive, Not-attractive, Straighthair and Wavy-hair. We explain why we select these features as follows:• The first four features cover sex and age, two protected features widely used in algorithmic
fairness papers.
• We select the next two features in order to cover labeling biases, biases induced by thelabelers into the dataset. Among the 40 features provided by CelebA, the Attractive featureis the most subjective one. Table 3.4 shows that among the people with blond hair, morethan half are labeled Attractive; while among the other people, more than half are labeled
Not-attractive. It might be that the labelers consider blond more attractive than other haircolors, or it might be that the labelers consider females more attractive than males, and itturns out that more females have blond hair than males in this dataset. Although the reason
behind is unknown, we believe that these two features well represent the labeling biases inthis dataset, and should be taken into consideration.
51

August 7, 2022
DRAFT
Table 3.4: Number of training instances in each domain of CelebA and CivilComments-Wilds.
CelebA

Blond

Others

CivilComments-Wilds

Toxic

Non-toxic

Male
FemaleYoung
Old
Attractive
Not-attractive
Straight-hair
Wavy-hair

1387
22880
20230
4037
17008
7259
5178
11342

66874
71629
106558
31945
66595
71908
28769
40640

Male
FemaleLGBTQ
ChristianMuslimOther Religions
Black
White

4437
4962
2265
2446
3125
1003
3111
4682

25373
31282
6155
24292
10829
5541
6785
12016

Total

162770Total

269038• We select the last two features in order to cover confounding variables, features the model

uses to do classification that should have no correlation with the target by prior knowledge.
Since the target is the hair color, a convolutional network trained on this dataset would focus
on the hair of the person, so we conjecture that the output of the convolutional network ishighly correlated with the hair style. In our experiments, we find that models trained with
ERM misclassify about 20% of the test instances with blond straight hair, much more thanthe other three combinations.
Table 3.4 lists the number of training instances in each domain of each dataset. Each instance
may belong to zero, one or more domains. In CivilComments-Wilds, the aggregated group size ofthe 16 groups is less than the total number 269,038, because most online comments do not containsensitive words.

3.9.2

Model Selection

In Section 3.6 we assume access to a domain-aware validation set, which is not available in realdomain-oblivious tasks. In this part we study several domain-oblivious model selection strategies,
and discuss why model selection is hard.
We study the following model selection strategies:• Max Average Accuracy: The model with the highest average accuracy in validation.
• Min CVaR: The model with the lowest CVaR risk (α = 0.2) over the validation set.
• Min CVaR-DORO: The model with the lowest CVaR-DORO risk (α = 0.2,  = 0.005) overthe validation set.
Note that selecting the model that achieves the highest average accuracy over the worst α
portion of the data is almost equivalent to the Max Average Accuracy strategy because the modelwith the highest average accuracy over the population also achieves the highest accuracy on theworst α portion (see e.g. [56], Theorem 1).
We conduct experiments on CelebA and report the results in Table 3.5. From the table wedraw the following conclusions:52August 7, 2022
DRAFT
Table 3.5: The average and worst-case test accuracies of the best models selected by different
strategies. (%)

Training Algorithm

Model Selection

Average Accuracy

Worst-case Accuracy

ERM

OracleMax Avg Acc
Min CVaR
Min CVaR-DORO

95.01 ± 0.38
95.65 ± 0.05
95.68 ± 0.04
95.69 ± 0.04

53.94 ± 2.02
45.83 ± 1.87
44.83 ± 2.74
44.50 ± 2.72

CVaR
(α = 0.2)

OracleMax Avg Acc
Min CVaR
Min CVaR-DORO

95.52 ± 0.08
95.74 ± 0.06
95.79 ± 0.05
95.81 ± 0.05

49.94 ± 3.36
39.28 ± 3.58
38.67 ± 2.06
38.83 ± 2.05

CVaR-DORO(α = 0.2,  = 0.005)

OracleMax Avg Acc
Min CVaR
Min CVaR-DORO

92.91 ± 0.48
95.60 ± 0.05
95.58 ± 0.06
95.56 ± 0.07

72.17 ± 3.14
45.39 ± 3.22
39.83 ± 2.37
41.28 ± 3.26

OracleMax Avg Acc
Min CVaR
Min CVaR-DORO

82.44 ± 1.22
90.70 ± 0.26
87.28 ± 2.05
89.16 ± 1.41

63.36 ± 2.51
20.67 ± 3.86
21.44 ± 11.13
25.50 ± 9.14

OracleMax Avg Acc
Min CVaR
Min CVaR-DORO

80.73 ± 1.41
90.06 ± 0.57
84.37 ± 4.08
88.76 ± 0.81

65.36 ± 1.02
22.06 ± 5.82
29.83 ± 12.10
23.61 ± 7.45

χ2 -DRO(α = 0.2)

χ2 -DORO(α = 0.2,  = 0.005)

1. For every training algorithm, the oracle strategy achieves a much higher worst-case test
accuracy than the other three strategies, and the gap between the oracle and the non-oracle
strategies for DRO and DORO is larger than ERM. While it is expected that the oracle
achieves a higher worst-case accuracy, the large gap indicates that there is still huge room
for improvement.
2. For χ2 -DRO/DORO, Min CVaR and Min CVaR-DORO work better than Max Average
Accuracy. However, for the other three algorithms, Max Average Accuracy is better. Thisshows that model selection based on CVaR and selection based on CVaR-DORO are notgood strategies.
3. With the three non-oracle strategies, ERM achieves the highest worst-case test accuracy.
This does not mean that DRO and DORO are not as good as ERM, but suggests that weneed other model selection strategies that work better with DRO and DORO.
The reason why Min CVaR is not a good strategy is that CVaR does not decrease monotonicallywith Rmax . Corollary 2 only guarantees that CVaR is an upper bound of Rmax , but the θ that
achieves the minimum CVaR does not necessarily have the smallest Rmax . For the same reason,
53

August 7, 2022
DRAFT
Min CVaR-DORO is not a good strategy either.
Model selection under the domain oblivious setting is a very difficult task. In fact, Theorem
1 of [56] implies that no strategy can be provably better than Max Average Accuracy under thedomain-oblivious setting, i.e. for any model selection strategy, there always exist D1 , · · · , DK
such that the model it selects is not better than the model selected by the Max Average Accuracy
strategy. Thus, to design a provably model selection strategy, prior knowledge or reasonableassumptions on the domains are necessary.

3.9.3

Training Hyperparameters

On the COMPAS dataset, we use a two-layer feed-forward neural network activated by ReLU as
the classification model. For optimization we use ASGD with learning rate 0.01. The batch size is128. The hyperparameters we used in Table 3.2 were: α = 0.5 for CVaR; α = 0.5,  = 0.2 for
CVaR-DORO; α = 0.5 for χ2 -DRO; α = 0.5,  = 0.2 for χ2 -DORO.
On the CelebA dataset, we use a standard ResNet18 as the classification model. For optimization we use momentum SGD with learning rate 0.001, momentum 0.9 and weight decay
0.001. The batch size is 400. The hyperparameters we used in Table 3.2 were: α = 0.1 for CVaR;
α = 0.2,  = 0.005 for CVaR-DORO; α = 0.25 for χ2 -DRO; α = 0.25,  = 0.01 for χ2 -DORO.
On the CivilComments-Wilds dataset, we use a pretrained BERT-base-uncased model as theclassification model. For optimization, we use AdamW with learning rate 0.00001 and weightdecay 0.01. The batch size is 128. The hyperparameters we used in Table 3.2 were: α = 0.1
for CVaR; α = 0.1,  = 0.01 for CVaR-DORO; α = 0.2 for χ2 -DRO; α = 0.2,  = 0.01 forχ2 -DORO.

54

August 7, 2022
DRAFT(a) Average (Original)(b) Worst (Original)(c) Train Loss (Original)(d) Test Loss (Original)(e) Average (Outliers removed)(f) Worst (Outliers removed)(g) Average (Labels flipped)(h) Worst (Labels flipped)

55

August 7, 2022
DRAFT

Accuracy

Accuracy

1
0.8
ERM
CVaR
CVaR-DORO

0.6
0.4

0

10

20

0.6
0.4
0.2
0

30

0

10

Epochs20

30

Epochs(a) Average Accuracy(b) Worst-case Accuracy

Figure 3.3: Test accuracies of CVaR and CVaR-DORO on CelebA (α = 0.1,  = 0.01).

0.8

Accuracy

Accuracy

1

ERM

0.6
0

10

2

-DRO

2

-DORO

20

0.6
0.4
0.2
0

30

0

10

Epochs20

30

Epochs(a) Average Accuracy(b) Worst-case Accuracy

Figure 3.4: Test accuracies of χ2 -DRO and χ2 -DORO on CelebA (α = 0.3,  = 0.01).

Accuracy

Accuracy

1
0.8
0.6
0.4

Avg Acc
Worst Acc

0

0.5

1

1.5

(

0.01)

0.8
0.7
0.6

2

2.5

0

0.5

1

1.5

(

0.01)

2

2.5

(b) χ2 -DORO(a) CVaR-DORO

Figure 3.5: Effect of  on the test accuracies of CVaR/χ2 -DORO on CelebA (α = 0.2). DORO
with  = 0 is equivalent to DRO.

56

August 7, 2022
DRAFT

Accuracy

Accuracy1
Avg Acc
Worst Acc

0.8
0.6

0.6
0.4

0.4
0.05 0.1 0.15 0.2 0.25 0.3

0.2
0.05 0.1 0.15 0.2 0.25 0.3(a) CVaR(b) χ2 -DRO

1

0.8

Accuracy1

Accuracy

0.8

Avg Acc
Worst Acc

0.8
0.6

0.6
0.05 0.1 0.15 0.2 0.25 0.3

0.05 0.1 0.15 0.2 0.25 0.3(d) χ2 -DORO(c) CVaR-DORO

Figure 3.6: Effect of α on the test accuracies of DRO and DORO on CelebA ( = 0.01).

57

August 7, 2022
DRAFT

58

August 7, 2022
DRAFT

Chapter 4High Dimensional Imbalanced
Classification

59

August 7, 2022
DRAFT

4.1

Introduction

Datasets with class imbalance — that is, the number of samples of one class far exceeds thenumber of data of another class — are prevalent in cutting-edge data science applications [26, 53].
Take COVID-19 testing data for example: a dominant fraction of data samples often come fromthe negative class (i.e., non-targeted people who have not contracted the virus). The evaluation
criterion in reality, however, might place equal, or even higher, emphasis on the minority class(e.g., the infected people in the COVID-19 case). The ability to generalize favorably in bothmajority and minority classes plays a pivotal role in critical scientific and societal issues (e.g.,fairness/equity in machine learning, discovery of rare disease, transferability of knowledge tosample-starved tasks). It has been widely recognized, however, that the imbalanced availability ofdata can cause severe issues to modern data-limited learning algorithms including neural networks(e.g., [20, 53, 131]), particularly when reasoning about the underrepresented class.
For concreteness, let us discuss a puzzling phenomenon that arises in a classical binaryclassification problem. Imagine there are two classes: the majority class has n0 samples andthe minority class has n1 samples, where n1  n0 . We take the generalization error to be themisclassification error when averaged over two classes (with equal weight). Prior statistical theory
−1/2
−1/2
typically suggested a generalization bound that scales as O(n0+ n1 ) as long as the sample
sizes for both classes tend to infinity [23, 148]; if this were true, then one would predict thatthe sample size of the minority class plays a dominant role in imbalanced classification, while
adding more data to either class helps improve generalization. However, an intriguing empiricalphenomenon seems to contradicts this theory: in data-hungry settings, adding more data in the
majority class might sometimes even hurt generalization [133, 149].
In this paper, we aim to take steps towards theoretically understanding the detrimental roleof majority data for M-estimators, including the popular learning algorithms like Fisher linear
discriminant analysis (LDA), logistic regression and SVM among others. We pursue a comprehensive understanding of these classifiers in the proportional regime — where the number ofparameters d scales linearly with the number of linear equations n, with their ratio d/n held
fixed to be a constant δ ∈ (0, ∞). It is demonstrated that more data from majority class canprovably hurt the performance of these algorithms, using recent techniques from high dimensional
statistics like random matrix theory or convex Gaussian minimax theorem (CGMT). Finally, we
also develop effective schemes to correct the biases brought by having imbalanced classes.

4.1.1

Binary classification

Consider classifying a mixture of two d-dimensional Gaussian components, with n0 (resp. n1 )samples for the majority (resp. minority) class. In each sample (xi , yi ), the binary variableyi ∈ {±1} denotes the class label, while xi = [1, xei ] = [1, xei,1 , · · · , xei,d ]> ∈ Rd+1 comprises
the (augmented) feature variables, with the first coordinate encoding the intercept. The problem
is this: based on n independent observations, can we hope to identify a classifier such that theclassification error on a new sample xnew is as small as possible?
Formally, suppose we have acquired n independent observations {(xi , yi )}ni=1 , with n0(resp. n1 ) samples drawn from the majority class with yi = −1 (resp. minority class with60

August 7, 2022
DRAFT
yi = 1). The Gaussian mixture model assumes thate | Y = +1 ∼ P1 := N (µ1 , Σ)X
e | Y = −1 ∼ P0 := N (µ0 , Σ)
X

(4.1)

for different mean vectors µ1 , µ0 ∈ Rd and shared covariance structure Σ.
Despite the imbalanced availability of training data, we would like to treat both classes equallyin the generalization error. Specifically, given any classifier f that maps a feature vector x ∈ Rd+1
to {+1, −1}, the generalization error is defined w.r.t. a balanced mixture of P0 and P1 as followso1n
Risk(f ) :=Ex∼P1 [1{f (x) 6= 1}] + Ex∼P0 [1{f (x) 6= −1}] .
(4.2)
2
In other words, R(f ) characterizes the out-of-same test error of classifier f at a new sample drawn
from Ptest := 12 P0 + 21 P1 . The focus is the challenging proportional growth regime
n1 /d = α1 ,n0 /d = α0 ,(n0 , n1 , d) → ∞,(4.3)

where both α0 and α1 are fixed constants.

4.1.2

Surprises in high-dimensional imbalanced classification

An illustrative example: Fisher’s LDA in high dimensions. In order to determine whether
an observed vector x has been drawn from distribution P1 or distribution P0 , arguably the mostwidely used procedure is Fisher’s linear discriminant analysis (LDA). In its simplest form, when
the covariance matrix is known a priori as identity, Fisher’s LDA is given by


1
T
2
2
b
f (x) = sign (bµ1 − µb0 ) x − (kbµ1 k2 −kb
µ0 k 2 ) ,
(4.4)
2
P
P
where µb1 := n11 i:yi =1 xi and µb0 := n10 i:yi =−1 xi , corresponding to the sample average ofeach class. Most classical analyses on LDA have focused on the asymptotic regime where thenumber of samples n largely overwhelms the feature dimension d.In this regime, LDA is known
to achieve classification error approaching Bayes risk Φ(−r/2) as n grows to infinity1 , wherer := kµ1 − µ0 k2 denotes the `2 -norm separation between the means. In addition, LDA is also
known to be minimax-optimal among certain natural family of distributions.
However, in the regime where n and d grows proportionally to infinity, the performance of
LDA is no longer characterized by the classical analyses (see [31] and a nice survey paper by [108]and references therein). As is also explained in [134, Chapter 1.2.1], assuming n1 /d = n0 /d → α,[31] showed that the classification error of fb converges in probability to a fixed number — inparticular,


r
r
(4.5)
Risk(fb) → Φ − q
6= Φ(− ),2
2 1 + r22α
1

Throughout, the cumulative distribution function (CDF) of the standard normal distribution is denoted as Φ(·).

61

August 7, 2022
DRAFT

Figure 4.1: Asymptotic risk versus the sample ratio α0 of the majority class for different choicesof α1 . The `2 separation in population means is set to be r = kµ1 − µ0 k2 = 6.
which is strictly larger than the Bayes risk for non-zero α. In addition, [31] also considered an
imbalanced variant of this result, which is of particular interest: under the imbalanced proportional
regime equation 4.3 the classification risk of fb converges to:
r2 + α0−1 − α1−1  1 
r2 − α0−1 + α1−1 
1 pRisk(fb) → Φ − p
+
Φ
−
.
2
2
r2 + α0−1 + α1−1
r2 + α0−1 + α1−1

(4.6)

Risk (non)-monotonicity. To give the readers a sense of how Risk(fb) behaves, we present here
an illustrative example. Fixing the value of α1 (the proportion of samples from the minority class),
let α0 vary from 0 to ∞ and we can plot the risk curve Risk(fb) against different values of α0 . The
asymptotics risk provided in expression equation 4.6 is shown in Figure 4.1.
When α1 = 0.1 or 0.2, a surprising non-monotonic risk behavior arises (shown in theblue/orange curve): as one increases the number of samples from the majority class, the test error
actually increases. In other words,
More data from the majority class can hurt the generalization of Fisher’s LDA.
It contradicts the common intuition that more data always help. This non-monotonic or U-shape
behavior, however, does not always appear in this plot. In fact, it disappears after α1 , the numberof samples from minority class, grows above a certain threshold; for example, the red curve —corresponding to r = 6 and α1 = 0.4 — is monotonically decreasing.
It turns out that the phenomenon aforementioned is not merely restrictive to Fisher’s LDA.
This paper also considers a spectrum of other standard classifiers such as support vector machine(SVM), logistic regression with or without regularization and other type of M-estimators, allof which suffer from the same sample inefficiency regarding the majority class. See also an
illustrative plot in Figure 4.2.
62

August 7, 2022
DRAFT

Figure 4.2: Non-monotonicity of the asymptotic risk behavior: Test errors of logistic regression
and SVM are shown for α1 = 0.1, r = 6, Σ = I. We generate the data and fit the models 20 times
and average over their generalization error (on a new data sample).

4.1.3

Our contributions

A natural question that arises immediately is whether the observed sample-inefficiency could be
an outcome of biases caused by imbalanced data and whether one can design a classifier that
automatically correct these biases, instead of throwing away data. Indeed, [31] proposed a simple
modification of Fisher’s LDA that takes care of the imbalance. In addition, a characterization of
the risk obtained for this modified estimator is also provided, as shall be explained in Section 4.3.
• analysis of M-estimator• propose bias correction• analysis of Deev’s debiasing Estimator [Chen: this part was mostly known in the prior

works, if I recall correctly. I think I added this part for the sole purpose of comparing rates
of debias/no-debias algorithms, as requested by Pradeep.]• develop minimax lower bounds – no loss of constants

4.1.4

Other related works

Classifying Gaussian mixtures. Classifying mixture of Gaussians has been one of the mostwidely studied prototypes for binary classification due to its simplicity and applicability. While its
study can be dated back to Fisher in 1930s, this stylized model has been used as a starting point foranalysis of various new challenges in machine learning, such as adversarial robustness [30, 113],
self-training algorithms [24, 75], non-convex optimization [9, 143], and domain adaptation [75].
Another evolving line of recent work studies this model in the high dimensional proportional
regime where d/n = Θ(1), see for example [37, 42, 62, 135]. Some earlier works also consideredsparse variant of LDA.
63

August 7, 2022
DRAFT
Imbalanced classification. Class-imbalanced classification has received a great amount ofattention recently due to its pervasiveness in modern data applications (see surveys [26, 53]).
The standard approach to correct the bias that introduced by class imbalance is by direct classre-weighting [104, 105]. It also motivates the study of designing proper loss functions to accommodate class variability, see for example [23, 38, 69]. Another line of research considers
another kind of re-weighting framework by sub-sampling the major class (see [43, 114, 136] andreferences therein). [98] provided interesting asymptotic characterization of logistic regression
for infinitely imbalanced datasets.
Exact high-dimensional asymptotics. The framework that adopted in the first part of this paper
is the exact high-dimensional asymptotics of convex optimization based estimators. In order toprovide sharp characterization of these estimators in the proportional regime, this framework
has recently been considered in various problems in statistics and machine learning such as theanalysis of LASSO estimators [25? ], logistic regression [33, 120], double-descent phenomena[11, 90] and adversarial training [62], just to name a few. Most related to our work are the recent
work [33, 69] which study the binary and multi-class classification error respectively, without
explicitly modeling the role of class imbalance. The classifiers analyzed in this paper are also
closely related to the M-estimators used prominently for high dimensional regression. The efforts
of characterizing the risk and distribution of robust regression estimators are initiated in [? ? ? ]and also considered in [37, 52, 125? ].
Fairness/Distributionally Robust Generalization The algorithmic bias of machine predictionsin the face of biased data collection has been a pressing issue that raises serious concerns
across various communities [59]. Our results concerning data-imbalanced classification and bias
correction should be integrated broadly with the growing efforts in equitable learning and fairness.
See [44, 50, 142, 152] and references therein.

64August 7, 2022
DRAFT

4.2

Analysis of re-weighted M-estimators

4.2.1

A warm-up example: Diagnosis of LDA

Here we provide a brief analysis of why Fisher LDA is sub-optimal in the high dimensional
imbalanced setting. For simiplicity, in this section we assume that µ1 = µ, µ0 = −µ be symmetricaround the origin.
In this setting, the Bayes classifier is fBayes = sign(µT x). Notice that the constant term is 0
in the Bayes classifier.
However, if we look at the LDA estimator closely - the constant term is 12 kµ̂0 k22 − 21 kµ̂1 k22 .
Using basic properties of Gaussian distribution, we can show thatdn0dE[kµ̂1 k22 ] = kµk22 +n1

E[kµ̂0 k22 ] = kµk22 +In the classical regime where n0 , n1 >> d, the O(1/n) terms are negligible and the constantterm in LDA does converge to 0 as n grows to infinity. Or alternatively, in the high dimensionalbalanced regime where n0 = n1 , the O(1/n) terms cancel out with each other, and the constantterm is unbiased too. But when the dataset is both imbalanced and high-dimensional, thisestimation causes trouble: the difference becomes 2α1 0 − 2α1 1 , i.e. the bias is non-zero!
In light of this observation, [31] proposed a simple improved version of LDA in this regime:
He uses a shifted linear predictor, which corrects bias in the intercept term:


1
d
1 d
T
2
2
ˆ
fshif ted (x) = sign (µ̂1 − µ̂0 ) x − (kµ̂1 k2 −kµ̂0 k2 ) + ( − )2
2 n0 n1(4.7)

Deev showed that the risk of this bias-corrected version of LDA converges to2
Φ − √ r−1 −1 , which not only always outperform LDA, but also always monotone in2

r+α0 +α1

both α0 and α1 , as shown in the figure 4.3. An interesting interpretation of Deev’s result is that0 n1the ”effective” sample size seem to be twice of the harmonic mean of n0 and n1 , i.e. n4n0 +n. Since1
harmonic mean is always upper bounded the arithmetic mean (with the equality when n0 = n1 ),this indeed suggest that imbalanced classification is always harder than balanced case.

4.2.2

Main results

Without loss of generality, we assume α1 < α0 , i.e. class 1 is the minority class. We also
shuffle the training data so that the first n0 samples are from class 0. In other words, the samples
from class 0 are (x1 , y1 = 0), · · · , (xn0 , yn0 = 0), and samples from class 1 are (xn0 +1 , yn0 +1 =1), · · · , (xn0 +n1 , yn0 +n1 = 1).
65August 7, 2022
DRAFT

Figure 4.3: Comparing LDA and Bias-corrected LDA.
Class-weighted M-estimator: Throughout, we consider a class of classifiers that referred to asclass-weighted M-estimators. Specifically, let l(·) : R → [0, ∞) be a convex classification loss
function. For every linear classifier f (x; β) = sign(β T x), define its risk function as
n0
n1
1 X
1 X
T
l(yi β T xi ) + λkβk22 .
R̂emp (β; l, λ) =l(yi β xi ) +2n0 i=1
2n1 i=n +1

(4.8)

0

Here the loss from class 0 (resp. class 1) is scaled by n10 (resp.
M-estimator is given byfˆλ (x) := sign(β̂ T x)

1
) and the correspondingn1β̂ := argmin R̂emp (β; l, λ).

(4.9)

β∈Rd+1

Our goal is to understand the performance of fˆλ (x) in terms of population risk equation 4.2. Whenf is a linear classifier f (x; β), we write R(β) as a shorthand for R(f (·; β)) when the meaning isclear from context.
Here l(t) denotes any convex classification loss function. For example, l(t) := log(1 + e−t )corresponds to the logistic loss, whereas l(t) := max{1 − t, 0} is the hinge loss. For notational
convenience, we adopt the conventional Moreau envelope definition asel (x; τ ) := min {u1(x − u)2 + l(u)}.
2τ(4.10)In this section, we study the high dimensional asymptotic behavior of the convexified classweighted ERM estimator. The main technical tool used in the analysis is Convex Gaussian
Minimax Theorem (CGMT), stated below.
We consider two settings, where the linear predictor is either homogeneous (i.e. f (x) =sign(β T x)) or non-homogeneous (i.e. f (x) = sign(β T x + c)). The analysis for both settings are
similar and we will only provide the proof for non-homogeneous setting since it’s more general.
66August 7, 2022
DRAFT
Theorem 4.2.1 (Asymptotic Classification Error of Homogeneous Linear Classifiers). Consider
the class-weighted convexified ERM estimator βb which minimizes the normalized logistic loss:
n0
n0 +n11 X
1 X
TR̂emp (β) =l(yi β xi ) +l(yi β T xi ) + λkβk22 .
2n0 i=1
2n1 i=n +1(4.11)

0

Assuming the Gaussian Mixture model, ndi → αi for i ∈ {0, 1}. Then, the asymptotic classification
error can be characterized as:
 ∗

r
1
b →P Φ − kµk2 ,R(β)(4.12)
R∗
where el is the Moreau envelope of l(·) defined in equation 4.10, and (r1∗ , R∗ , θ∗ ) is the optimal
solution of the following convex-concave scalar optimization problem:
 



α0 + α1
α0 + α1
θ(R2 − r12 )1+ el RZ + r1 kµk2 ;−+λR2
min max EZ∼N (0,1) el RZ + r1 kµk2 ;r1 ,R:|r1 |≤R θ
2
2α0 θ
2α1 θ
2(α0 + α1 )(4.13)
Theorem 4.2.2 (Asymptotic Classification Error of Non-Homogeneous Linear Classifiers). Consider the class-weighted convexified ERM estimator β which minimizes the weighted logistic
loss:
n0
n0 +n1
q0 X
q1 X
TR̂emp (β; q0 , q1 ) =l(yi (β xi + c)) +l(yi (β T xi + c)) + λkβk22 .
2n0 i=1
2n1 i=n +1(4.14)

0

Assuming the Gaussian Mixture model, ndi → αi for i ∈ {0, 1}. Then, the asymptotic classification
error can be characterized as:
 ∗
 ∗


r1 kµk2 +c∗
r1 kµk2 −c∗
1
1
+ Φ −
,
(4.15)R(β) →P Φ −
2
R∗
2
R∗
where el is the Moreau envelope of l(·) defined in equation 4.10, and (r1∗ , R∗ , θ∗ ) is the optimal
solution of the following convex-concave scalar optimization problem:
min

max
θ





1
(α0 + α1 )q1
θ(R2 − r12 )(α0 + α1 )q0
EZ∼N (0,1) q0 el RZ + r1 kµk2 −c;+ q1 el RZ + r1 kµk2 +c;
−
+
2
2α0 θ
2α1 θ
2(α0 + α1 )r1 ,R,c:|r1 |≤R

Proof techniques.
This theorem is established by exploiting tools from modern highdimensional probability, particularly the convex Gaussian minimax theorem (CGMT) [47] that has
recently proven to be effective in enabling fine-grained characterization for both high-dimensional
asymptotics and over-parameterized problems [4, 62, 126? ? ]. As it turns out, this theorem allows
one to pin down the asymptotic class-balanced classification error for a broad family of `(·). The
theoretical prediction can be numerically computed, matching the empirical behavior of, say, the67August 7, 2022
DRAFT
Figure 4.4: Comparing empirical and theoretical test error of logistic regression . In this figure,
we set λ = 0.2 and nd1 = α1 = 0.1, and d = 5000, T = 3 times average of indpendent generationof dataset for the empirical curve.

LDA classifier. Let G ∈ Rn×d , g ∈ Rn , h ∈ Rd have i.i.d. Gaussian entries, Sw ⊂ Rd , Su ⊂ Rn
be bounded, compact, convex sets. Define two Gaussian processes:
Xw,u = uT Gw + ψ(w, u)
Yw,u = kwk2 g T u + kuk2 hT w + ψ(w, u)
Define two optimization problems, primary optimization (PO) and auxilary optimization (AO):
P O(G) = min max Xw,u(4.16)

AO(g, h) = min max Yw,u

(4.17)

w∈Sw u∈Su

w∈Sw u∈Su

Assuming ψ(·, ·) is convex-concave on Sw × Su , then for any ν ∈ R, t > 0, we have:
Pr[|P O(G) − ν|> t] ≤ 2 Pr[|AO(g, h) − ν|> t](4.18)

where the randomness is from G, g, h. In other words,
Concentration of OP T (AO) ⇒ Concentration of OP T (P O)
The main benefit of CGMT is that it reduces the analysis of PO to that of AO, which is typicallyeasier since it does not involve the Gaussian random matrix G and only depends on two Gaussian
vectors g, h. Furthermore, it’s often possible to simplify AO, which is a O(n)-dimensional
problem, to an equivalent convex-concave program which involves only a small (constant) numberof scalar variables, where the scalar variables also encode the quantities of our interest, e.g. thedistance to the population optimal solution, the limit correlation with ground truth parameter, toname a few.

4.2.3

Bias correction for M-estimators

In Section 4.1.2, we showed that applying bias-correction on Fisher’s LDA leads to a better
test accuracy and monotonic risk. A natural question is whether it’s possible to apply a similarbias-correction procedure to other classifiers, like logistic regression or SVMs. In this section, we
answer this question in affirmative and provide two different ways of de-biasing high dimensional
logistic regression in imbalanced dataset.
68August 7, 2022
DRAFT
Figure 4.5: Comparing test error of logistic regression before and after bias-correction. In thisfigure, we set λ = 0.2 and nd1 = α1 = 0.1

Method 1: De-biasing with high dimensional asymptotics
Recall that in section 4.2, we provided sharp high dimensional asymptotics for weighted logistic
regression. In particular, we showed that the solution (β, c) satisfies
1
β T (µ1 − µ0 ) → r1∗ kµ1 − µ0 k2 , kβk2 → R∗ , − β T (µ1 + µ0 ) + c → c∗ ,2

(4.19)

where (r1∗ , R∗ , c∗ ) is the solution to a convex-concave program. By inspecting the asymptotic
behavior, we have the following observation:• The correlation between β and β ∗ is increasing with α0 , the sample size of majority class.
• The bias of c∗ is also growing with α0 , and eventually becomes the dominating factor inclassification error.
Motivated by these observations, we propose the following de-biasing procedure.
First, let c0 = 12 (β ∗ )T (µ1 + µ0 ) be the constant term of Bayes-Optimal classifier. We use theweighted logistic loss to learn a linear classifier β T x + c, then replace c with ccorrected = c − c∗ .
This step makes c0 asymptotically unbiased. This procedure can be viewed as a generalization to
Deev’s bias correction for LDA, for which he also derived the asymptotic formula of the bias termand substracted it from the estimator.
As shown in figure ??, this correction improves the classification accuracy of logistic regression
and eliminates the non-monotonicity of risk, although still being worse than the bias-corrected
LDA.
Method 2: De-biasing with validation Set
While Method 1 is effective under the high-dimensional asymptotic setting we considered, it
has a few crucial drawbacks. First of all, the validity of this procedure depends heavily on theGaussian distributional assumption. Without such assumptions, it is very difficult to derive theasymptotic formula for the bias. Furthermore, even if we know the data is indeed Gaussian,
this asymptotic bias varies from different Signal-to-noise ratios (i.e. the separation between the
mixture components), which is unlikely to be known apriori. These drawbacks motivate us to
explore alternative, practical way of de-biasing the estimator.
Another popular way of de-biasing is using a hold-out validation set, and choose the bestparameter based on the accuracy therein. This method is often not statistically efficient, especiallyin the proportional regime d/n = Θ(1). This is because it often requires a constant fraction ofdata for validation, which changes the ratio of d/n and leads to an inferior statistical accuracy.
However, a key observation here is that we only need to tune a single thereshold parameter, forwhich we only use a sublinear number of samples for validation.
More specifically, suppose we use O( logn n ) samples from each class for validation. We firstuse a weighted logistic loss to learn a linear classifier β T x + c. Then, we replace c with cvalidation
which minimizes the validation error of the shifted linear classifier.
69August 7, 2022
DRAFT
Since tuning c is equivalent to (agnostic) learning a threshold function over real line, by a VC
dimension argument, we can show that:1
R(β, cvalidation ) ≥ inf R(β, c) + Õ( √ ),cn

(4.20)

because the generalization error scales like Õ( √n 1), and nvalidation = O( logn n ) = Õ(n). Notevalidationthat this inequality holds without any distributional assumptions. In fact, we (should be) able toprove that this gap can be improved to O( n1 ) for Gaussian mixtures.
Since all of our previous analysis only requires n0 /d → α0 , n1 /d → α1 and removing a
sublinear number of samples does not change the limiting behavior, the homogeneous part of thelinear√ predictor inherits all of the theoretical properties we derived in the previous section. Since
1/ n → 0 in the asymptotic regime, we can achieve the bias-corrected asymptotic error with thisvalidation-based approach.
TODOs:
1. Interpolating regime
2. Lower Bounds
3. Other re-weighting?

4.3

Sharp non-asymptotic analysis of Deev’s estimator

In this section, we provide a sharp, non-asymptotic analysis to Deev’s Estimator:


1
d
1 d
T
2
2
ˆ
fDeev (x) = sign (µ̂1 − µ̂0 ) x − (kµ̂1 k2 −kµ̂0 k2 ) + ( − )2
2 n0 n1(4.21)

Theorem 4.3.1. (Informal) Assuming ∆ = Ω(1), n0 ≥ n1 = Ω(d), we have:


2



∆

 + Õ
R(fˆDeev ) = Φ − qdd2
2 ∆ + n0 + n1

√ !
d
n1

(4.22)

In contrast, as long as nd1 − nd0 = Ω(1), we have:


2



∆
 + Ω̃
R(fˆLDA ) = Φ − qdd2
2 ∆ + n0 + n1



d
n1


(4.23)

We first equivalently re-formulate Deev’s Estimator in the following form:


µ
+
µ
1
1
d
d
1
0
T
T
fˆDeev (x) = sign (µ̂1 − µ̂0 ) (x −) − (µ̂1 − µ̂0 ) (µ̂1 − µ1 + µ̂0 − µ0 ) + ( − )22
2 n0 n1(4.24)
70

August 7, 2022
DRAFT
Let µ1 − µ0 = 2v, µ̂1 − µ1 = z1 ∼ N (0, n11 Id ), µ̂0 − µ0 = −z0 ∼ N (0, n10 Id ), we can further
simplify the above equation to:µ1 + µ0TˆfDeev (x) = sign w (x −) + b)(4.25)
2
where:
w = 2v + z1 + z0

(4.26)1d1db = −v T (z1 − z0 ) + (kz1 k2 − ) − (kz0 k2 − )
2
n1
2
n0

(4.27)

Recall that the test error can be written as:




1
−wT v − b
1
−wT v + b
R(f ) = Φ
+ Φ
2
kwk2
2
kwk2

(4.28)We have the following simple lemma:
Lemma 4.3.1.
s2
1
11
Φ (−s + t) + Φ (−s − t) = Φ(s) + √ e− 2 st2 + O(t4 )
2
2
2 2π

(4.29)

Therefore,
1
R(f ) = Φ
2



−wT v
kwk2




+O

b2
kwk22


.

(4.30)

Next, we will analyze the two terms separately.
Lemma 4.3.2. With probability 1 − 2δ ,
wT v
≥ ...
kwk2
Lemma 4.3.3. With probability 1 − 6δ2 − exp(−0.04 · d),




2
1
20
1 2
1
b2
1
1
1
1
+
+
≤5
log( ) +d log( ) + log( )
kwk22
n0 n1
δ2
kvk22
δ2
δ2
n0 n1

(4.31)(4.32)

Proof of Lemma 4.3.2. First, we notice that w ∼ N (2v, ( n10 + n11 )Id ) can be decomposed asr
w = 2v +11v+ (zv ·+ z⊥ ),
n0 n1
kvk2(4.33)

1T
where zv ∼ N (0, 1) and z⊥ ∼ N (0, I − kvk
2 vv ) are independent Gaussian variables. Note that2v T z⊥ = 0, hence we have

kwk22 =



r
2 

1
1
1
1
2kvk2 ++ zv ++kz⊥ k22
n0 n1
n0 n1
71

(4.34)

August 7, 2022
DRAFT
and

r
w

T

1
1
+ zv kvk2
n0 n1

v = 2kvk22 +(4.35)

therefore,
q

wT v
= 
kwk2

2kvk22 +1+ n11 zv kvk2
n0

1/2
2 

q
1
1
1
1
2
2kvk2 + n0 + n1 zv + n0 + n1 kz⊥ k2(4.36)

By the Gaussian/chi-square tail bounds ([80], Lemma 1), we have with probability 1 − 2δ4 ,r
1
zv ≥ − log( ),(4.37)
δ4andkz⊥ k22 ≤ (d − 1) + 2

r1
1(d − 1) log( ) + 2 log( )δ4δ4(4.38)

Let
s


1
1
1
T4 =
+
log( )(4.39)
n0 n1
δ4
 r


1
1
1
1
T5 = 2 (d − 1) log( ) + 2 log( )+(4.40)
δ4
δ4
n0 n1


n0 n1
1
Assuming T4 ≤ kvk2 (which is equivalent to δ4 ≥ exp − kvk= exp (−Θ(n1 ))), we
2 n0 +n1
have:
wT v≥ kvk2 kwk2=

2kvk2 −T4
(2kvk2 −T4 )2 + d−1
+ d−1
+ T5
n0
n1
2kvk22

4kvk22 + d−1
+ d−1
n0
n1

kvk2
 21 −

(4.41)

1/2


T4 + kvk2 T5+ O(T42 + T52 ) (4.42)
 32
4kvk22 + d−1
+ d−1
n0
n1
d−1
+ d−1
n0
n1



PluginProof of Lemma 4.3.3. b consists of three terms:
1d1db = −v T (z1 − z0 ) + (kz1 k2 − ) − (kz0 k2 − )
|
{z
} 2
n1
2
n0
|
{z
}|
{z
}
T1
T2

We will bound each term separately.
72

T3(4.43)

August 7, 2022
DRAFT
Upper Bounding T1 Notice that z1 − z0 ∼ N (0, ( n10 + n11 )Id ), therefore, T1 ∼ N (0, ( n10 +2
1)kvk22 ). Consequently, by the normal tail bound P r[N (0, 1) ≥ t] ≤ exp(− t2 ), we haven1s"
Pr |T1 |≤

1
1
+
n0 n1



#
2
log( )kvk2 ≥ 1 − δ1δ1Notice that n1 kz1 k2 ∼ χ2 (d), by Lemma 1 of [80],ih√2
Pr n1 kz1 k2 −d ≥ 2 dt + 2t ≤ exp(−t)
h√ iPr n1 kz1 k22 −d ≤ −2 dt ≤ exp(−t)(4.44)

Upper Bounding T2 and T3(4.45)(4.46)

Therefore, w.p (1 − 2δ2 ),−q
d log( δ12 )n1≤ T2 ≤

qd log( δ12 )n1+log( δ12 )+log( δ12 )n1(4.47)

Similarly, w.p (1 − 2δ2 ),−q
d log( δ12 )n0≤ T3 ≤

qd log( δ12 )n0

n0Therefore, w.p (1 − 4δ2 ), we haverlog( δ12 )1
1
1|T2 + T3 |≤ d log( )
+
+
δ2
n0 n1
min(n0 , n1 )Putting things together, we have
s
r



1
1
1
1
1
2
1
|b|≤
+
+
log( )kvk2 +d log( ) + log( )
n0 n1
δ1
δ2
δ2
n0 n1
with probability 1 − δ1 − 4δ2 . Thus, by Cauchy-Schwarz,




2
1
1
2
1
1 2
1
1
2
2
|b| ≤ 4kvk2+
log( ) + 16 d log( ) + log( )
+
n0 n1
δ1
δ2
δ2
n0 n1

(4.48)(4.49)(4.50)(4.51)

n1
Roughly speaking, this implies that b = Õ( nd2 + n1∗ ), where n∗ = nn00+n1∗

Lower Bounding kwk2 Recall that w = 2v + z1 + z0 ∼ N (2v, ( n10 + n11 )Id ), we have ( n10 +1 −1) kwk22 follows non-central χ2 distribution χ2d (λ), where λ = 4kvk22 ( n10 + n11 )−1 . By [14], we
n1
have:
h
i
p
Pr χ2d (λ) ≤ d + λ − 2 (d + 2λ)t ≤ exp(−t)(4.52)
73

August 7, 2022
DRAFT
as a result, as long as t ≤ d+2λ, we have25p
3d + λd + λ − 2 (d + 2λ)t ≥,5(4.53)

and



1
1 −1
3d + λ2Pr ( + ) kwk2 ≤≤ exp(−t),
n0 n15thus, we have with probability 1 − exp(−Ω(d)),
3 d
d
4
4
kwk22 ≥ kvk22 + ( + ) ≥ kvk22
5
5 n0 n1
5(4.54)(4.55)

Finally, choosing δ1 = 2δ2 and apply union bound, we have with probability at least 1 − 6δ2 −exp(−0.04 · d),




2
b2
1
1
1
20
1
1 2
1
1
≤5
+
log( ) +d log( ) + log( )+(4.56)
kwk22
n0 n1
δ2
kvk22
δ2
δ2
n0 n1

4.4

Minimax lower bounds

For any classifier fˆ(xn+1 ; x1 , · · · , xn ),n = n0 + n1 , the expected classification error is defined as:R(fˆ) = Pr[fˆ(xn+1 ; x1 , · · · , xn ) 6= yn+1 ](4.57)

where the probability is taken over both training data {x1 , · · · , xn } and test data (xn+1 , yn+1 )drawn from the following distribution:
x1 , · · · , xn0 ∼ N (µ0 , I)
xn0 +1 , · · · , xn0 +n1 ∼ N (µ1 , I)
yn+1 ∼ Uniform{+1, −1}x|y = 1 ∼ N (µ1 , I)
x|y = −1 ∼ N (µ0 , I)

(4.58)(4.59)(4.60)(4.61)(4.62)

We focus on the parameter space where (µ0 , µ1 ) are ∆-separated:
Pd (∆) = {µ0 , µ1 ∈ Rd , kµ0 − µ1 k2 ≥ ∆}(4.63)

We will sometimes use P as a shorthand for Pd (∆) when the meaning is clear from context.
Now, let us define the minimax classification error:
M (n0 , n1 , P) =

inf

sup

fˆ(xn+1 ;x1 ,···,xn ) (µ0 ,µ1 )∈P

R(fˆ)(4.64)

And the quantity of interest to us is the asymptotic minimax error when n0 /d = α0 , n1 /d =α1 , ∆ are fixed constants and n0 , n1 , d → ∞. To be precise,∗Rasymp(∆, α0 , α1 ) =lim supn0 /d=α0 ,n1 /d=α1 ,d→∞74

M (n0 , n1 , Pd (∆))

(4.65)

August 7, 2022
DRAFT

4.4.1

The Bayesian connection to minimax riskThis part is analogous to Larry’s Lecture Notes, chapter 36.8.
Let Q(µ0 , µ1 ) be a prior distribution over (µ0 , µ1 ) ∈ P. The Q-risk of fˆ is defined as:
BQ (fˆ) = E(µ0 ,µ1 )∼Q R(fˆ)(4.66)

The classifier that minimizes the Q-risk is the MAP classifier:fQ (x; x1 , · · · , xn ) = argmax Pr[Y = y|X = x, x1 , · · · , xn ],y∈{+1,−1} Q(4.67)

for which we denote its Q-risk as B(n0 , n1 , Q). Clearly, the minimax risk is always lower
bounded by the Q-risk:
M (n0 , n1 , P) ≥ B(n0 , n1 , Q).
(4.68)
Utilizing this Bayesian connection, [85] provided a minimax lower bound on classification
error when the classes are balanced (n0 = n1 ). In particular, they chose the prior distribution
as uniform over a d-dimensional sphere: µ1 = −µ0 = µ ∼ 12 ∆ · Uniform(Sd−1 ), and used adelicate argument to show that the MAP classifier is a linear classifier. However, we weren’t able
to generalize their proof to imbalanced setting, mostly due to the fact that posterior distributionp(yn+1 |xn+1 , x1 , · · · , xn ) is nearly intractable without the balancedness assumption. Motivatedby this, we introduce a notion of ”improper prior”, where the prior distribution Q(µ) may havesmall probability mass outside of the parameter space P. This will enable more flexible choice ofprior distributions, e.g. Gaussian priors, which in turn help us circumvent the intractable posterior
issue.
Definition 4.4.1 (Improper Prior). A prior distribution Q(µ) is a δ-improper prior over P, if
Pr[µ ∈ P] = 1 − δQ(4.69)

The following lemma shows that we can also link the minimax risk with the bayes risk of an
improper prior:
Lemma 4.4.1. Let Q(µ) be a δ-improper prior over P. Then, the minimax risk over P is lower
bounded by:
M (n0 , n1 , P) ≥ B(n0 , n1 , Q) − δ.
(4.70)
Proof. By definition of minimax risk, for any ε > 0, there exists an estimator ĝ, such thatsupR(ĝ) ≤ M (n0 , n1 , P) + ε(4.71)(µ0 ,µ1 )∈P

By definition of Q-risk, we have
BQ (ĝ) ≥ B(n0 , n1 , Q)75

(4.72)

August 7, 2022
DRAFT
The rest of proof is reminiscent of Markov’s inequality. Notice that the 0-1 loss function is R(ĝ)
bounded in [0, 1], we can upper bound BQ (ĝ) as folows:
BQ (ĝ) = E(µ0 ,µ1 )∼Q R(ĝ)(4.73)= E [R(ĝ)|(µ0 , µ1 ) ∈ P] Pr [(µ0 , µ1 ) ∈ P] + E [R(ĝ)|(µ0 , µ1 ) ∈/ P] Pr [(µ0 , µ1 ) ∈
/ P]
Q

Q

(4.74)≤ (M (n0 , n1 , P) + ε)(1 − δ) + 1 · δ< M (n0 , n1 , P) + ε + δ.

(4.75)(4.76)

Therefore,
B(n0 , n1 , Q) < M (n0 , n1 , P) + ε + δ,

(4.77)and set ε → 0+ completes the proof.

4.4.2

Imbalanced lower bounds

In this section, we try to establish a tight lower bound to the high dimensional imbalanced
classification problem. Below we state the main theorem:
Theorem 4.4.1. Assuming n0 = α0 d and n1 = α1 d, the minimax risk M (n0 , n1 , Pd (∆)) is lower
bounded by:
!
!
r
2
∆
log d
M (n0 , n1 , Pd (∆)) ≥ Φ − p(4.78)−O−1−12
d2 ∆ + α0 + α1
As an immediate corollary of this theorem, we can see that Deev’s Bias correction achieves
the information-theoretically optimal risk under the asymptotic regime nd0 → α0 , nd1 →α1 ; n0 , n1 , d → ∞. Furthermore, our lower bound is non-asymptotic, i.e. it holds for any
finite (n0 , n1 , d), which generalizes the exising results even in the balanced setting.
Our proof relies on a careful analysis of Bayesian Q-risk, just like a few recent works which
studied various Bayesian learning settings in high dimensional Gaussian classification [93], [127].
However, since we are interested in the minimax risk, an additional step required here is tochoose an (approximately) least favorable prior over parameters (µ0 , µ1 ). This step is not requiredin the Bayesian learning setting where the prior distribution is fixed, and it is usually a very
difficult task (maybe add some reference here). Nevertheless, our analysis below indicates that it
is approximately tractable when the dimensionality d is very large.
Our choice of prior distribution Q(µ0 , µ1 ) is a generalization of those appeared in recent
works [93] and [127]. In [93], the prior distribution is chosen as symmetric Gaussian Prior2µ1 ∼ N (0, ∆4d I), µ0 = −µ1 . In [127], the prior is chosen as independent Gaussian Prior2µ1 , µ0 ∼ N (0, ∆2d I). While neither of these priors leads to a tight minimax lower bound, a key
observation is that both of these priors can be written in the following form:
 
  2

µ0
(R + r2 )Id (R2 − r2 )Id∼ N 0,,(4.79)µ1(R2 − r2 )Id (R2 + r2 )Id76

August 7, 2022
DRAFT
Another way of interepreting equation 4.79 is as follows: let u, v be independent randomvariables each with N (0, Id ) distribution, then equation 4.79 is equivalent to:µ0 = Ru − rv
µ1 = Ru + rv.

(4.80)(4.81)∆
Indeed, the prior considered in [93] corresponds to R = 0, r = 2√
, and the one in [127]d∆corresponds to R = r = 2√d . As we will see in the analysis, our choice of R is much larger√
than both of them: we require R = Ω( d) for our desired lower bound. From a high level, thischoice of prior is inspired by a classical result in determining the exact minimax risk of Normalmean estimation, where the prior distribution was chosen as N (0, γ 2 Id ) and let the varianceγ 2 → ∞ (See e.g. Example 36.67 of Larry’s Lecture Notes).
√ In our case, the difference of means2µ1 − µ0 ∼ N (0, 4r Id ), which implies kµ1 − µd k2 ≈ 2r d ≈ ∆, while µ1 + µ0 ∼ N (0, 4R2 Id ),which has a very large variance similar to the text-book proof for normal mean estimation.
The main technical lemma is a careful non-asymptotic analysis of the Bayes risk under theprior distribution mentioned above, stated below:
Lemma 4.4.2. Under
Q defined in equation 4.79, with the choice of√ the prior distribution
˜∆parameters R = Ω( d) and r = 2√d , the Q-risk is lower bounded by:



˜2

∆



r

−O
B(n0 , n1 , Q) ≥ Φ − q
−1
−1
2
˜
2 ∆ + α0 + α1

log d
d

!
(4.82)

Theorem 4.4.1 is a consequence of Lemma 4.4.1 and Lemma 4.4.2.
q
˜ = ∆(1+C log d ). By our construction, 1 (µ1 −µ0 ) ∼ N (0, Id ). Therefore,
Proof. We choose ∆
d
2r
1
2
2
kµ1 − µ0 k2 ∼ χ (d). Using the tail bound for chi-square variables [80], we have
4r2



p
1
1
2
√
Pr
kµ
−
µ
k
d
log
d
≤
≤
d
−
C
1
0
2
4r2
d

(4.83)

In other words, with probability 1 − d−1/2 , we havekµ1 − µ0 k22 ≥ 4r2



r

p
log d
2
˜
d − C d log d = ∆
1−C
d

!
≥ ∆2

(4.84)

Thus, Q is a δ-improper prior with parameter δ ≤ d−1/2 . Therefore, by Lemma 4.4.1,
M (n0 , n1 , Pd (∆)) ≥ B(n0 , n1 , Q) − δ


!
r
2
˜
∆
log d
−O
≥ Φ − qd˜ 2 + α0−1 + α1−1
2 ∆77

(4.85)(4.86)

August 7, 2022
DRAFT
Finally, it remains to show that


!
!
r
2
2
˜
∆
∆
log
d
≥Φ − p−OΦ − q2 + α−1 + α−1
d
−1
−1
2
∆
2
˜
0
1
2 ∆ + α0 + α1

(4.87)

This can be proven via a simple Lipschitzness argument. Notice that Φ(·) is √12π -Lipschitz, and by
q
q
t2
32
1
√
√
Lemma 4.4.3, h(t) = − 2 t2 +c is 27 -Lipschitz for any c > 0. Therefore, Φ(h(t)) is 2π · 32
=
q
 27
log d
˜ − ∆|= O√4 -Lipschitz, and we complete the proof by using the fact that |∆.
d
3 3π
The remaining work is to prove the main technical lemma 4.4.2. Before proceeding, it’ll be
useful to establish a closed form of posterior probability p(yn+1 |xn+1 , xn1 , y1n ), as summarized in
the following claim (whose proof is included in the appendix):
Claim 4.4.2. (Posterior probability)
Pr[Ynew = yn+1 |Xnew = xn+1 ; {(xi , yi )}ni=1 ] ∝ exp

n+1

1
1X
− d log Z −kxi k2
2Z 2
2 i=1

T

(4.88)

whereZ = 4R2 r2 |C0 ||C1 |+(R2 + r2 )(|C0 |+|C1 |) + 1(4.89)X
X
X
X
T = (4R2 r2 |C1 |+R2 + r2 )kxi k22 +(4R2 r2 |C0 |+R2 + r2 )kxi k22 +2(R2 − r2 )(xi ) T (
xi )
i∈C0

i∈C1

i∈C0

i∈C1

(4.90)
andC0 = {1, · · · , n0 , n + 1}, C1 = {n0 + 1, · · · , n} if ynew = 0
C0 = {1, · · · , n0 }, C1 = {n0 + 1, · · · , n, n + 1} if ynew = 1(4.91)(4.92)

Equipped with Claim 4.4.2, we are now ready to prove Lemma 4.4.2.
˜ in the proof below.
Proof of Lemma 4.4.2. To avoid notational clutter, we will use ∆ instead of ∆By definition of MAP classifier,fQ (x; x1 , · · · , xn ) = argmax Pr[Ynew = yn+1 |Xnew = xn+1 ; {(xi , yi )}ni=1 ](4.93)y∈{+1,−1}Denotel(y) = P r[Ynew = y|Xnew = xn+1 ; {(xi , yi )}ni=1 ](4.94)The expected classification error of the MAP classifier can be formulated as:
BQ (fQ ) = Pr[fQ (xn+1 ) 6= yn+1 ]11= Pr[l(1) < l(0)|yn+1 = 1] + Pr[l(0) < l(1)|yn+1 = 0]2
2
78

(4.95)(4.96)

August 7, 2022
DRAFT
where the probability is taken over the prior distribution (µ0 , µ1 ) ∼ Q , as well as {(xi , yi )}i∈[n+1]generated according to equation 4.58. By symmetry, we will focus our attention on the first term,
i.e. the probability of l(1) < l(0) when xn+1 ∼ N (µ1 , Id ).
Next, we will compare l(1) and l(0). Define S0 , S1 as the summation of all training data fromtwo classes:
S0 =
S1 =

n0
X

xi

i=1
n
X

(4.97)
xi(4.98)

i=n0 +1

Notice that in Claim 4.4.2, when yn+1 = 1, we have |C0 |= n0 , |C1 |= n1 +1, while when yn+1 = 0,
instead we have |C0 |= n0 + 1, |C1 |= n1 . For notational simplicity, denote
Z0 := 4R2 r2 (n0 + 1)n1 + (R2 + r2 )(n0 + n1 + 1) + 1
Z1 := 4R2 r2 (n1 + 1)n0 + (R2 + r2 )(n0 + n1 + 1) + 1(4.99)(4.100)T0 := (4R2 r2 n1 + R2 + r2 )kS0 + xn+1 k22 +(4R2 r2 (n0 + 1) + R2 + r2 )kS1 k22 +2(R2 − r2 )(S0 + xn+1 )T S1(4.101)T1 := (4R2 r2 (n1 + 1) + R2 + r2 )kS0 k22 +(4R2 r2 n0 + R2 + r2 )kS1 + xn+1 k22 +2(R2 − r2 )S0T (S1 + xn+1 )(4.102)
Then,

log

l(1)
l(0)



 
T0
1
T1
Z0
−
+ d log=
2Z1 2Z0 2Z
1 Z0
T1 Z0 − T0 Z1 1
+ d log=
2Z0 Z12Z1(4.103)(4.104)In order to analyze equation equation 4.104, we will takea closer look at each term appeared
Z0
1
in equation 4.104, namely, T1 Z0 − T0 Z1 , 2Z0 Z1 , and 2 d log Z1 .
∆It is easy to see that when r = 2√
, n0 = α0 d and n1 = α1 d, we haved
1 + O(R−1 + d−1 )
Z1 = R2 d ∆2 α0 α1 + α0 + α1 1 + O(R−1 + d−1 )Z0 = R2 d ∆2 α0 α1 + α0 + α1

(4.105)(4.106)andZ0∆2 (α1 − α0 )= 1 + d−1 · 2+ O(d−2 )Z1∆ α0 α1 + α0 + α1

(4.107)

Therefore,
1
d log
2



Z0
Z1


=

∆2 (α1 − α0 )+ O(d−1 )2 (∆2 α0 α1 + α0 + α1 )79

(4.108)August 7, 2022
DRAFT
and
2Z0 Z1 = 2R4 ∆2 α0 α1 + α0 + α1

2

1 + O(R−1 + d−1 )(4.109)The remaining work,
P 0 also the most challenging part, is to analyze the term T1 Z0 − T0 Z1 .
Since S0 = ni=1
xi where xi ∼i.i.d. N (µ0 , Id ), we have the decomposition
S0 = n0 µ0 +p√
n0 w0 = α0 d(Ru − rv) + α0 dw0 , w0 ∼ N (0, Id ).

(4.110)S 1 = n1 µ1 +p√
n1 w1 = α1 d(Ru + rv) + α1 dw1 , w1 ∼ N (0, Id ).

(4.111)xn+1 = µ1 + wn+1 = (Ru + rv) + wn+1 , wn+1 ∼ N (0, Id ).

(4.112)

Similarly,

We also denote that∆
Putting equation 4.110, equation 4.111, equation 4.112 and r = 2√
into T1 Z0 − T0 Z1 and withdthe help of symbolic computation, we have the following decomposition:

T1 Z0 − T0 Z1 =

4
i−1
X
X

C(i, j)Ri dj/2

(4.113)i=0 j=i−4

Here, C(i, j) , are bi-linear forms of u, v, wn+1 , w0 , w1 , whose coefficients only depend on
α0 , α1 , ∆ but independent with d and R.
Recall that u, v, wn+1 , w0 , w1 are all d-dimensional normal variables ∼ N (0, Id ). For theconvenience of our analysis, we denote the following event as egood :The event egood : For all (γ, ν) ∈ {u, v, wn+1 , w0 , w1 } × {u, v, wn+1 , w0 , w1 }:pkγk22 = d + O( d log d)pγ T ν = O( d log d)(4.114)(4.115)
Since kγk22 ∼ χ2 (d), γ T ν = 12 12 kγ + νk22 − 21 kγ − νk22 and kγ + νk22 , kγ − νk22 are also χ2
distributed, by standard concentration inequalities for chi-squared variables and union bounds
over all (γ, ν) pairs, we have egood happens with probability at least 1 − d−2021 .
Hence, condition on the event egood , all of quadratic forms C(i, j) are bounded by O(d). Since
when i ≤ 3, we have j ≤ 2 and C(i, j)Ri dj/2 at most O(R3 d2 ). Using the same argument, we
3
can see that C(i, j)Ri dj/2 is at most O(R4 d 2 ) when i = 4, j = 0, 1. As we will show below,
these terms are all negligible and we can focus on the C(4, 3) and C(4, 2) terms.
The precise expressions for C(4, 3), C(4, 2) are as follows:√√TC(4, 3) = 2∆2 wn+1
∆2 α0 α1 + α0 + α1 (∆α0 α1 v − α0 α1 w0 + α0 α1 w1 ) ,and80(4.116)

August 7, 2022
DRAFT
C(4, 2) = ∆2 ∆2 α02 α1 ∆2 α1 + 2 kvk22
−α0 α1 ∆2 α0 +2 kw1 k22 + ∆2 α02 α1 −∆2 α0 α12 +α02 −α12 kwn+1 k22 +α0 α1 ∆2 α1 +2 kw0 k22
!

 3

√
√
√
− 2∆v T α02 α1 ∆2 α1 + 2 w0 − α0 α1 (α0 − α1 ) w1 − 2 α0 α1 (α0 − α1 ) w0T w1 .
(4.117)√
Condition on the event egood , we√know that v T w0 , v T w1 , w0T w1 = O( d log d), andkvk2 , kw1 k22 , kw0 k22 , kwn+1 k22 = d + O( d log d). Therefore,
C(4, 2) =d∆2 ∆2 α02 α1 ∆2 α1 + 2 − α0 α1 ∆2 α0 + 2 + ∆2 α02 α1 − ∆2 α0 α12 + α02 − α12 + α0 α1 ∆2 α1 + 2
p(4.118)+ O( d log d)p
=∆2 ∆4 α02 α12 + 2∆2 α02 α1 + α02 − α12 d + O( d log d)(4.119)p(4.120)=∆2 ∆2 α0 α1 + α0 − α1 ∆2 α0 α1 + α0 + α1 d + O( d log d)

√
√
−1/2
Regarding C(4, 3), notice that w̃ := (∆2 α0 α1 d + α0 + α1 )∆α0 α1 v − α0 α1 w0 + α0 α1 w1 ∼T
w̃ →P N (0, 1) when d → ∞. Futhermore, by
N (0, Id ). By central limit theorem, zd := √1d wn+1
Berry-Esseen theorem, there exists an universal constant C such thatC
| Pr{zd ≥ t} − Φ(t)| ≤ √ ,d(4.121)and C(4, 3) can be equivalently re-formulated as: T√√
C(4, 3) = 2∆2 ∆2 α0 α1 + α0 + α1 wn+1(∆α0 α1 v − α0 α1 w0 + α0 α1 w1 )p
3
= 2 dα0 α1 ∆2 ∆2 α0 α1 + α0 + α1 2 zd

(4.122)(4.123)

Substituting equation 4.118 and equation 4.122 into equation 4.113, we get
T1 Z0 − T0 Z1(4.124)
 √
3



=R4 d2 ∆2 2 α0 α1 ∆2 α0 α1 + α0 + α1 2 zd + ∆2 α0 α1 + α0 − α1 ∆2 α0 α1 + α0 + α1
(4.125)
4

3
2

3 2

+ O(R d + R d )(4.126)81

August 7, 2022
DRAFT
Substituting equation 4.109, equation 4.108 and equation 4.124 into equation 4.104, we have

log

l(1)
l(0)



 
T1 Z0 − T0 Z1 1
Z0=
+ d log(4.127)
2Z0 Z1
2
Z1
 √
 21

∆2
2
2
=
2
α
α
∆
α
α
+
α
+
α
z
+
∆
α
α
+
α
−
α
0 1
0 1
0
1
d
0 1
0
1
2 (∆2 α0 α1 + α0 + α1 )(4.128)!
r∆2 (α1 − α0 )log d+
+ O(d−1 )(4.129)+Od
2 (∆2 α0 α1 + α0 + α1 )
!!
r
1

√
∆2
log
d
=
2 α0 α1 ∆2 α0 α1 + α0 + α1 2 zd + ∆2 α0 α1 + O
2 (∆2 α0 α1 + α0 + α1 )d(4.130)!
r√∆2 α0 α1∆2log d 
zd +
=
1 + O

1/2
d
(∆2 α0 α1 + α0 + α1 )2 ∆2 + α0−1 + α1−1 2(4.131)Hence, condition on the event egood , l(1) < l(0) happens if and only ifr∆2

zd ≤ −
2∆2 + α0−1 + α1−1 21 − O

log d
d

!
(4.132)

By Berry-Esseen and union bound, the probability is lower bounded by:Pr[l(1) < l(0)|yn+1 = 1] ≥ Φ −

r

2

∆

1 − O2 ∆2 + α0−1 + α1−1 2!log d 
−O
d



1
√
d


− Pr(egood )(4.133)≥ Φ −∆2r

1  − O−1 2

2 ∆2 + α0−1 + α1

log d
d

!
,

(4.134)

where the last step is due to the √12π -Lipschitzness of Φ(t).
By symmetricity, when yn+1 = 0 , we also have:Pr[l(0) < l(1)|yn+1 = 0] ≥ Φ −

2

∆



1  − O2 ∆2 + α0−1 + α1−1 2Therefore we have completed the proof.

82

r

log d
d

!
.

(4.135)August 7, 2022
DRAFT
Proof of Claim 4.4.2

Proof of Claim 4.4.2: By law of total probability and Bayes’s theorem,
Pr[Ynew = yn+1 |Xnew = xn+1 ; {(xi , yi )}ni=1 ]∝p[Ynew = yn+1 , Xnew = xn+1 ; {(xi , yi )}ni=1 ]=E(µ0 ,µ1 )∼Q p(xn+1|µ0 , µ1 , y1n+1 )p(y1n+1 |µ0 , µ1 )1
1X
1X
∝E(µ0 ,µ1 )∼Q exp(−
kxi − µ0 k22 −kxi − µ1 k22 )
2 i∈C
2 i∈C
0
1
Z
X
X
1
1
= exp(−
kxi − µ0 k22 −kxi − µ1 k22 )dQ(µ0 , µ1 )
2 i∈C
2 i∈C
0

(4.136)(4.137)(4.138)(4.139)(4.140)

1

Z

1X
1X
kxi − µ0 k22 −kxi − µ1 k22 )dQ(µ0 , µ1 )(4.141)
2 i∈C
2 i∈C
0
1
Z
X
X
X
1
|C1 |
|C0 |
= exp(−
kµ0 k22 −kµ1 k22 )dQ(µ0 , µ1 )kxi k22 ) exp(µT0xi + µT1
xi −
2
2
2
i∈C
i∈C
exp(−

i∈[n+1]0

1(4.142)
Substitute µ0 = Ru − rv and µ1 = Ru + rv, we haveZ

exp(µT0X

xi + µT1

i∈C0

Z
∝

X
i∈C1

xi −

|C1 |
|C0 |
kµ0 k22 −kµ1 k22 )dQ(µ0 , µ1 )2
2(4.143)

|C1 |
1
1
|C0 |
kRu − rvk22 −kRu + rvk22 − kuk22 − kvk22
exp (Ru − rv)T
xi + (Ru + rv)T
xi −
2
2
2
2
i∈C
i∈C
X

X

0

1

(4.144)!

Z=

exp R(

X
i∈C0

xi +

X
i∈C1

xi )T u + r(

X

i∈C1

X

xi −

xi )T v ·(4.145)i∈C0R2 (|C0 |+|C1 |) + 1
r2 (|C0 |+|C1 |) + 1
2
T
2
exp −
kuk2 −Rr(|C1 |−|C0 |))u v −kvk2 dudv
22(4.146)
Z
1:= exp(θT b − θT Aθ)dθ(4.147)2s(2π)2d=exp(bT A−1 b)(4.148)det(A)
83!

dAugust 7, 2022
DRAFT
Here,
 
u
θ :=
∈ R2d
v

 P
P
R(P i∈C0 xi + P i∈C1 xi )b :=xi − i∈C0 xi )r(
 2 i∈C1(R (|C0 |+|C1 |) + 1)Id
Rr(|C1 |−|C0 |)Id
A :=Rr(|C1 |−|C0 |)Id
(r2 (|C0 |+|C1 |) + 1)Id

(4.149)(4.150)(4.151)

Direct calculation gives:
Z := 4R2 r2 |C0 ||C1 |+(R2 + r2 )(|C0 |+|C1 |) + 1
ddet A = Z
1 (r2 (|C0 |+|C1 |) + 1)Id
−Rr(|C1 |−|C0 |)Id−1
A =−Rr(|C1 |−|C0 |)Id
(R2 (|C0 |+|C1 |) + 1)IdZXX
1 T −1
1b A b=
R2 (r2 |C0 |+r2 |C1 |+1)kxi +xi k22
2
2Z
i∈C0
i∈C1
X
X
+ r2 (R2 |C0 |+R2 |C1 |+1)kxi −xi k22i∈C0− 2R2 r2 (|C1 |−|C0 |)(kX

(4.153)(4.154)(4.155)(4.156)i∈C1

xi k22 −ki∈C1(4.152)X
xi k22 )(4.157)

i∈C0

X
X
1 
(4R2 r2 |C1 |+R2 + r2 )kxi k22 +(4R2 r2 |C0 |+R2 + r2 )kxi k22 (4.158)
=
2Z
i∈C0
i∈C1
X
X 
+ 2(R2 − r2 )(xi )T (
xi )(4.159)
i∈C0

i∈C1

T
.
:=
2Z
and we have proved the claim.

(4.160)

Full expansion of Equation equation 4.113 In√this section, we provide a full expansion ofequation equation 4.113 in power series of R and d, which is given by the following equation
T1 Z0 − T0 Z1 =

i−1
4
X
X

C(i, j)Ri dj/2

(4.161)i=0 j=i−4

and the coefficients C(i, j) are given by:
 T√√(∆α0 α1 v − α0 α1 w0 + α0 α1 w1 )C(4, 3) = 2∆2 ∆2 α0 α1 + α0 + α1 wn+1C(4, 2) = kvk22 ∆6 α02 α12 + 2∆4 α02 α1


 
3
3
3 
√
+ v T w0 −2∆5 α02 α12 − 4∆3 α02 α1 + w1 2∆3 α02 α1 − 2∆3 α0 α12

3
3


2
4
2
2
T
2 2√
2√
+kw0 k2 ∆ α0 α1 +2∆ α0 α1 +w0 w1 −2∆ α0 α1 +2∆ α0 α12 +kw1 k22 −∆4 α02 α1 −2∆2 α0 α1+ kwn+1 k22 ∆4 α02 α1 − ∆4 α0 α12 + ∆2 α02 − ∆2 α1284

August 7, 2022
DRAFT


3
3

T
C(4, 1) = wn+1
v 2∆5 α02 α1 + 2∆3 α02 + w0 −2∆4 α02 α1 − 2∆2 α023
3 + w1 2∆4 α0 α12 + 2∆2 α12

3
3


C(4, 0) = kvk22 ∆6 α02 α1 + ∆4 α02 + v T w0 −2∆5 α02 α1 − 2∆3 α02 + kw0 k22 ∆4 α0 α1 + ∆2 α0

+ kw1 k22 −∆4 α0 α1 − ∆2 α1T−∆4 α02 α1 u + ∆4 α0 α12 u − ∆2 α02 u + ∆2 α12 uC(3, 2) = wn+1


3

T
5
2
3 2
3
T
4√
2√
2
2 2
C(3, 1) = v ∆ α0 α1 u − ∆ α0 u + 3∆ α0 α1 u + w0 −∆ α0 α1 u + ∆ α0 u − 3∆ α0 α1 u

3 
√
√
+ w1T ∆4 α02 α1 u + 3∆2 α0 α1 u − ∆2 α12 uTC(3, 0) = wn+1
∆2 α0 u − ∆2 α1 u√√C(3, −1) = v T ∆5 α0 α1 u + ∆3 α0 u + w0T −∆4 α0 α1 u − ∆2 α0 u√√
+ w1T ∆4 α0 α1 u + ∆2 α1 u
∆3 α12∆3 α023T
5 2
52
C(2, 1) = wn+1 v ∆ α0 α1 + ∆ α0 α1 ++ 3∆ α0 α1 +
2
2
!
3
√
3
3∆4 α02 α1 ∆4 α0 α12
2 2
2√+ w0 −−− ∆ α0 − 3∆ α0 α1
2
2
!!
3
√
3
∆4 α02 α1 3∆4 α0 α12√+ w1++ 3∆2 α0 α1 + ∆2 α1222C(2, 0) = −∆4 α02 kuk22 ∆4 α12 kuk22+
44


∆6 α02 α1 ∆6 α0 α12 ∆4 α02 3∆4 α0 α1
+
+
+
2
2
2
2
!
3
3
3 !!
√
5√
2
3√
3
3 2
5
3 2
2
3
∆
α
α
3∆
α
α
3∆
α
α
3∆
α
∆
α
α
∆
α
0
0
1
0
1
0
1
0
1
1
+v T w0 −∆5 α02 α1 −
−
−
+w1
+
−
2
2
2
2
2
2
!
3√
3

 4


√
∆4 α02 α1 ∆4 α0 α12∆4 α0 α1
∆ α0 α12
2
T2
2+kw0 k2+∆ α0 +w0 w1−+kw1 k2 −
−∆ α1
2
2
2
2
 4 2

∆ α0 ∆4 α1222
2+ kwn+1 k2−+ ∆ α0 − ∆ α1
4
4
 5 2

∆ α0
3∆3 α0 ∆3 α1T
5C(2, −1) = wn+1 v+ ∆ α0 α1 +
−
2
2
2
!
!!
3
3
4 2
√
√
√
√
∆4 α02∆α
1+ w0 −− ∆4 α0 α1 − ∆2 α0 + w1 ∆4 α0 α1 ++ ∆2 α1222− ∆ α0 kuk22 + ∆2 α1 kuk22 + kvk22

85

August 7, 2022
DRAFT
∆4 α0 kuk22 ∆4 α0 kw0 k22 ∆4 α1 kuk22
+
+
4
4
4

6 2
6
∆4 α1 kw1 k22∆
α∆
α0 α1 ∆4 α00
2−+ kvk2
+
+
4
4
2
2
!
3
 5 √
!
√
√
5
3
3√
5 2
∆
∆
∆
α
∆
α
α
α
α
α
∆
α
0
0
1
0
1
1
0
−
−
+ w1
+
+ v T w0 −
2
2
2
2
2


∆4 α02 u ∆4 α12 uTC(1, 0) = wn+1−+
− ∆2 α0 u + ∆2 α1 u
4
4
!
3
 5

4√
5 2
4 2
∆
α
α
u
√
∆
α
α
u
u
∆
α
∆
α
u
0 1
0 1
0
1
C(1, −1) = v T++ ∆3 α1 u + w0T −−− ∆2 α0 u
4
4
4
4
!
3
√
4
4 2
∆
α
α
u
√
∆
α
u
0
1
1
+
+ ∆ 2 α1 u
+ w1T
4
4


∆4 α0 u ∆4 α1 uTC(1, −2) = wn+1 −
+
4
4
√
4√
T
5
T
∆ α1 u v ∆ α0 u w0 ∆4 α1 uT w1
C(1, −3) =
−
+
4
4
4
 5 2

5
5 2
∆ α0 ∆ α0 α1 ∆ α1 ∆3 α0 ∆3 α1TC(0, −1) = wn+1 v
+
+
+
+
8
4
8
2
2
!
!!
3
3
√
√
4
4 2
∆
α
α
√
√
∆4 α02 ∆4 α0 α1
∆
α
0
1
1
+ w0 −−− ∆2 α0 + w1+
+ ∆ 2 α14444C(2, −2) = −C(0, −2)∆6 α02 ∆6 α0 α1 ∆6 α12 ∆4 α0 ∆4 α1
+
+
+
+
16
8
16
4
4
!
!!
3
3
√
√
√
5
3
5
3√
5 2
5 2
α
α
α
α
α
∆
∆
∆
α
∆
∆
α
∆
α
0 1
0
1
1
0
0
1
+ v T w0 −
−
−
+ w1
+
+
8
8
2
8
8
2
 5


√
√
∆4 α0 w0 ∆4 α1 w1∆ α0 ∆5 α1TC(0, −3) = wn+1 −
+
+v
+
4
4
8
8
 6



5√
5√
6
∆ α0 w 0 ∆ α1 w 1∆ α0 ∆ α1
2
T
C(0, −4) = kvk2
+
+v −
+
16
16
8
8
= kvk22Computer-aided verification of equation 4.161 We use the python symbolic computation
package sympy for generating and verifying the expansion equation 4.161. Notice that we only
need to verify the simpler case where u, v, w0 , w1 , wn+1 are all scalars, since the identity is
separable in different coordinates and we can repeat the same process for all d dimensions ofthese vectors.
The following python script was used to verify equation 4.161 and generate the LaTeX
equation. We only did some minor edits in the auto-generated LaTeX equation, such as replacing
scalar uv with inner product uT v and adding linebreaks.
86August 7, 2022
DRAFT

4.4.3

The importance of a carefully selected prior

We briefly discuss the importance of prior distribution chosen here. In particular, it can be shownthat the prior distributions considered in [93] and [127] lead to strictly weaker (and therefore
sub-optimal) lower bounds than ours in imbalanced classification. The analysis for both cases arevery similar to Theorem 4.4.2, with the only difference being that the dominating terms in 4.113
are no longer C(4, 3)R4 d3/2 and C(4, 2)R4 d. We will provide the results for both cases but only
summarize the main difference in the proofs and omit the details.
The case of R = 0. When R = 0 as considered in [93], all of the terms in 4.161 disappeared except those with i = 0, and among them the dominating terms are C(0, −1)d−1/2 andC(0, −2)d−1 . Using a similar argument as in Lemma 4.4.2, we can show that the Bayes Q-risk inthis setting is
!
r


∆2
log d
Φ −
−O
(4.162)2(∆2 + 4(α0 + α1 )−1 )1/2
dThis is a weaker bound than Lemma 4.4.2, since Φ(t) is monotone and by Cauchy-Schwarz,
1
1
4
+
≥
α0 α1
α0 + α1(4.163)The equality is achieved only when α0 = α1 , which means this lower bound is only optimal when
the classes are balanced.
∆∆
The case of R = 2√
. When R = r = 2√
as considered in [127], the dominating terms
ddbecome linear combinations of the coefficient C(i, j)’s:
d

−1/2

4
X

4
X
∆ i
∆
−1
C(i, i − 1)( ) and dC(i, i − 2)( )i
2
2
i=0
i=0

(4.164)Using a similar argument as in Lemma 4.4.2, we can show that the Bayes Q-risk in this setting is
!
r


∆2
log d
(∆2 + α0−1 + α1−1 )Φ −(4.165)·−Od2(∆2 + α0−1 + α1−1 )1/2 (∆2 + 2α0−1 )1/2 (∆2 + 2α1−1 )1/2
This is, again, a weaker bound than Lemma 4.4.2, since Φ(t) is monotone and by AM-GM
inequality,∆2 +α0−1 +α1−1 =
1(∆2 + 2α0−1 ) + (∆2 + 2α1−1 ) ≥ (∆2 +2α0−1 )1/2 (∆2 +2α1−1 )1/2 . (4.166)
2

The equality is achieved only when α0 = α1 , which means this lower bound is only optimal when
the classes are balanced.

4.4.4

Additional lemmas
2Lemma 4.4.3. Let h(t) = √tt2 +c where c is any positive constant. Then, h(t) is
87

q

32
-Lipschitz.
27

August 7, 2022
DRAFT
Proof. Direct computation of the derivative h0 (t) yields
h0 (t) =t (2c + t2 )3

(4.167)(c + t2 ) 2
By AM-GM inequality,2t2 (2c + t2 )(c + t2 )31 (2t2 ) · (2c + t2 ) · (2c + t2 )=2
(c + t2 )3|h0 (t)|2 =3
1 13 (2t2 + 2c + t2 + 2c + t2 )≤2
(c + t2 )332=
27
Therefore, |h0 (t)|≤4.5

q(4.168)(4.169)(4.170)(4.171)32
and we have completed the proof.
27

Numerical experiments

In the previous section, we revisited the example of [31], showed the non-monotonicity of LDA
classification error and how to fix it. One may ask if this U-shape behavior is limited to LDA,or maybe it’s true with much more generality? In this section, we first show empirically that
this is indeed the case for widely-used algorithm like logistic regression or SVM, even in simple
settings like classifying two Gaussians. Furthermore, we conjecture that the reason for the nonmonotonicity is similar to LDA: the estimation of constant terms are biased in the regime of ourinterest, as illustrated in the experiments below.
Non-monotonicity of Test Error In Figure 4.6, we plot the test error of Logistic Regression
and SVM when α1 = 0.1, r = 6. We choose the regularization parameter λ to be 0 and 0.1 forlogistic regression / 10−6 and 1 for SVM. The extremely small regularization is with the purposeof simulating the behavior of ”interpolating” classifiers, which caught a lot of attention theseyears. As shown in the figure, all of the four classifiers have the U-shape curve just like LDA.
Note that the ”interpolating” logistic classifier seems to be the least affected model among thefour models presented here.
Effect of Minority Sample Size We repeat the above experiment for larger α1 = 0.4, where theU-shape curve is known as disappearing for LDA. In this scenario, the behavior of four models
are more similar to what conventional wisdom suggests: unregularized models performs poorly;
the U-shape curve disappeared for appropriately regularized models (both SVM and Logistic).
88August 7, 2022
DRAFT

Figure 4.6: Non-monotonicity of Test Error: Test Error of Logistic Regression and SVM when
α1 = 0.1, r = 6. The error is estimated with 20-time average of random samples.

Figure 4.7: Effect of Minority Sample Size: Test Error of Logistic Regression and SVM when
α1 = 0.4, r = 6. The error is estimated with 5-time average of random samples.

89August 7, 2022
DRAFT

Figure 4.8: Effect of Bias-correction: Test Error of Logistic Regression when the constant terms
are fixed to be 0. The error is estimated with 20-time average of random samples.

Effect of Correcting the Bias In this experiment, we fix the two gaussians to be originsymmetric, where the population-level optimal constant is known to be zero. The only modification
here is that we do NOT include the constant term in the linear model. The data-generation iscompletely the same as Figure 4.6, where the U-shape curve appears.
As shown in Figure 4.8, the U-shape curve disappears for both regularized and unregularized
logistic regression. This suggests that the bias in estimating constant term may cause the irregularbehavior in imbalanced classification, at least for logistic regression.
Note that the sharp transition in the unregularized model around α0 = 7 is likely due to theinterpolation threshold: the data is linearly separable when α0 < 7 and the unregularized logistic
regression behaves like SVM in this scenario.

Effect of Regularization In this experiment, we examine the effect of regularization by varying
the level of regularization for SVM. As shown in Figure 4.9, very strong (λ = 10) regularization
does somewhat mitigate the majority data effect. However, the U-shape curve seem to be
only flattened down instead of disappearing. It’s also worth noting that adding even stronger
regularization λ = 30 completely breaks the model (sometimes test error goes to 50%), so λ = 10
is nearly an unreasonable choice of regularization.
90August 7, 2022
DRAFT

Figure 4.9: Effect of Regularization: Test Error of SVM with different level of regularization. The
error is estimated with 5-time average of random samples.

91August 7, 2022
DRAFT

4.6

Proof of Theorem 4.2.2

Proof. The convexified class-weighted ERM objective is equivalent to:
min
β,u

n
X

πi l(ui ) + λkβk22

i=1subject to:ui = yi (β T xi + c)q1
q0
if i ∈ {1, · · · , n0 }, and πi = 2n
if i ∈ {n0 + 1, · · · , n0 + n1 }.
where πi = 2n01
We can re-formulate this program in Lagrangian form:

min max

n
X

v

β,u,c


πi l(ui ) + vi yi β T xi + vi yi c − vi ui + λkβk22

i=1

Step 1: Writing the optimization problem in the form of CGMT.
N (0, Id ). Objective becomesmin max
v

β,u,c

n
X

Let xi = yi (µ + zi ), zi ∼
πi l(ui ) + vi yi2 β T (µ + zi ) + vi yi c − vi ui + λkβk22

i=1since yi = +1/−1, this program is equivalent to:min max
v

β,u,c

n
X


πi l(ui ) + vi β T µ + vi β T zi + vi yi c − vi ui + λkβk22

i=1

Let Z = [z1 , · · · , zn ] ∈ Rd×n which has i.i.d. N (0, 1) entries. Now this program can be
written in matrix form:
min max
v

β,u,c

n
X

πi l(ui ) + β T Zv + (1T v)(β T µ) − v T u + cv T y + λkβk22

i=1which is in the form of PO. Using CGMT, we consider the following AO program:min max
β,u,c

v

n
X

πi l(ui ) + kβk2 g T v + kvk2 hT β + (1T v)(β T µ) − v T u + cv T y + λkβk22

i=1

Step 2: Simplify AO so that it only involves a constant number
p of variables. Let β =µT
r1 kµk + r2 µ⊥ , where µ⊥ µ = 0, kµ⊥ k2 = 1, r2 ≥ 0, then kβk2 = r12 + r22 , β T µ = r1 kµk2 , AO
becomes:min

r1 ,r2 ,µ⊥ ,c,u

max
v

n
X

πi l(ui ) +q

r12 + r22 g T v + kvk2 hT (r1

i=1

92

µ
+ r2 µ⊥ ) + r1 kµk2 (1T v) − v T u + cv T yλkβk22
kµk

August 7, 2022
DRAFT
Fix r1 , r2 and minimize over µ⊥ gives the optimal µ⊥ :µ∗⊥ = −P⊥ h,
kP⊥ hk2(4.172)Hence we can simplify AO as:nXqµmin max
πi l(ui ) + r12 + r22 g T v + r1 kvk2 hT− r2 kvk2 kP⊥ hk2 +r1 kµk2 (1T v) − v T u + cv T y + λ(r1 ,r2 ≥0,c,u v
kµk
i=1Re-organizing terms in v:
n
X

T
q
µ
2
2
min max
r1 + r2 g + r1 kµk2 1 + cy − u v + r1 kvk2 hT− r2 kvk2 kP⊥ hk2 +λ(r12 + r
πi l(ui ) +r1 ,r2 ≥0,c,u v
kµk
i=1
Fix kvk2 = r3p
and maximize over the direction ov v, we have the objective is maximized when v
is parallel to r12 + r22 g + r1 kµk2 1 + cy − u, and the objective becomesmin

r1 ≥0,r2 ≥0,c,u∈Rn

max

n
X

r3

q
r12 + r22 g + r1 kµk2 1 + cy − uπi l(ui ) + r3

i=1

p
The k r12 + r22 g + r1 kµk2 1 + cy − u2+ r1 r3 hTµ− r2 r3 kP⊥ hk2 +λ(r12 + rkµk

term is not separable w.r.t. u. We will use the following
2

variational representation as a workaround: kxk2 = inf t>0 2t + 2t1 kxk22 . After switching the orderingof max-min, we get:
min

max
n

r1 ≥0,r2 ≥0,t>0,c,u∈R

r3

n
X
i=1

πi l(ui ) +r3 t r3+22tq

r12 + r22 g + r1 kµk2 1 + cy − u22+ r1 r3 hTµ− r2 r3 kP⊥ hk2
kµk

Recall the Moreau envolope function (for l(·)):
el (x; τ ) = minu1(x − u)2 + l(u)2τ

(4.173)

Then,
n
X

n
X

q

πi t
2
2
=
πi e lr1 + r2 gi + r1 kµk2 +cyi ;r3
2
i=1(4.174)
Therefore, we have completed the scalarization of (AO):

q
n
X
πi t
r3 t
µ
2
2
min max
πi e lr1 + r2 gi + r1 kµk2 +cyi ;+ +r1 r3 hT−r2 r3 kP⊥ hk2 +λ(r12 +r22 ),r1 ,r2 ≥0,c,t>0 r3
r3
2
kµk
i=1
(4.175)which is a convex-concave(?) optimization over four scalar variables. In fact, we can simplify thisa little further by reducing the number of scalar variables from 4 to 3.
r3minπi l(ui )+
u
2t
i=1

q

r12 + r22 g+r1 kµk2 1+cy−u932August 7, 2022
DRAFT
Let θ = nrt 3 , the optimization problem can be re-written as:
n
X

q

nπi
θt2 r1 θt T µ r2 θt
2
2
min
max
πi el
+ +
h
−
kP⊥ hk2 +λ(r12 +r22 )r1 + r2 gi + r1 kµk2 +cyi ;r1 ≥0,r2 ≥0,c,t>0 θ
θ
2n
n
kµk
n
i=1
(4.176)
Now the objective is a convex quadratic form in t, for which we can find the closed-formµminimizer: t∗ = r1 hT kµk+ r2 kP⊥ hk2 . After introducing another change of variable R =p22
r1 + r2 , we finally reached the fully simplified ”scalarized” version of AO:

2
n
q

X
nπi  θT−
r1 h µ − R2 − r12 kP⊥ hk2 +λR2
min max
πi el Rgi + r1 kµk2 +cyi ;r1 ,R,c:|r1 |≤R θθ
2ni=1(4.177)
Step 3: Analyze the high dimensional asymptotics of scalarized AO. Next, we will study thehigh dimensional asymptotics of the objective 2 .
We start with the second term: note that hT µ ∼ N (0, kµk22 ), so √1n hT µ →a.s. 0. Similarly, recall that h is d-dimensional standard Gaussian, we have P⊥ h ∼ N (0, P⊥ P⊥T ) (which1
is informally, a d − 1 dimensional standard Gaussian), hence √d−1
kP⊥ hk2 →a.s. 1, and1−1/2√ kP⊥ hk2 → (α0 + α1 ). Therefore, the second term becomesnθ(R2 − r12 )2(α0 + α1 )(4.178)Now we focus on the first term, which can be decomposed according to the two class ofsamples:n0
n0 +n1
q1 X
q0 X
nq0
nq1
el Rgi + r1 kµk2 −c;+el Rgi + r1 kµk2 +c;(4.179)2n0 i=12n0 θ
2n1 i=n +1
2n1 θ
0

1
1
Notice that nn0 → α0α+α, nn1 → α0α+α, and gi are independent (scalar) standard normal variables,0
1
it’s not difficult to see the above summation converges to a Gaussian expectation:
 

 

q0
(α0 + α1 )q0
q1(α0 + α1 )q1
EZ∼N (0,1) el RZ + r1 kµk2 −c;+ EZ∼N (0,1) el RZ + r1 kµk2 +c;
2
2α0 θ
2
2α1 θ
(4.180)To summarize, the high-dimensional asymptotics of the scalarized AO can be formulated asfollows:





1
(α0 + α1 )q0(α0 + α1 )q1
min
max EZ∼N (0,1) q0 el RZ + r1 kµk2 −c;+ q1 el RZ + r1 kµk2 +c;
R
θ
2
2α0 θ
2α1 θ
r1 ,R:0≤r1 ≤ kµk(4.181)2

Here, for simplicity we are talking about point-wise covergence, i.e. the limit for a fixed choice of (r1 , R, θ) whend, n0 , n1 → ∞. To justify that this point-wise limit indeed characterizes the limiting behavior of the optimization
problem, we need to check several technical conditions provided in [33].

94August 7, 2022
DRAFT
Step 4: Characterizing the Asymptotic Classification Error We pause here and provide
some interpretation of the asymptotic scalarized AO equation 4.181. Recall that in Step 2, we
introduced change of variables:µβ = r1+ r2 µ⊥(4.182)kµk2q
R = r12 + r22(4.183)Tβ µHence, R = kβk2 and r1 = kµk. It was shown in [125] that, under mild technical conditions,2T

β µ→P r1∗ . Since the classification error can be formulated askβk2 →P R∗ and kµk
2
 T

 T

1
β µ+c
1
β µ−c
R(β) = Φ −
+ Φ −
,
2
kβk2
2
kβk2
Therefore,
 ∗

 ∗

1
r1 kµk2 +c∗
1
r1 kµk2 −c∗R(β) →P Φ
−
+
−
,
2
R∗
2
R∗
and we have completed the proof.

4.7

Comparison with Related Works

4.7.1

Comparison with [62](4.184)(4.185)

Sanity Check: Consider the balanced setting as a special case, where we have α0 = α1 . Letα = α0 + α1 , the optimization problem above is equivalent to: θ(R2 − r12 )1+ λR2(4.186)
min max EZ∼N (0,1) el RZ + r1 kµk2 ;−r1 ,R:|r1 |≤R θθ
2αp
Let r2 = R2 − r12 , λ = 0

 q
1
θr2
2
2
min max EZ∼N (0,1) el
r1 + r2 Z + r1 kµk2 ;− 2(4.187)
r1 ,r2 θθ
2α√Let θ = θ0 r2α :
 q

r2
θ 0 r22
2√√
min max
E
e
r
+
r
Z
+
r
kµk
;
−
l
1
2
Z∼N (0,1)
1
2
r1 ,r2 θ0θ0 α
2 α(4.188)This is equivalent to a special case 0 = 0 in [62], corollary 5.1.c: after changing notation(ours → theirs) as r1 → θ, r2 → α, θ0 → β, α → δ, Z → g, kµk2 → σM,2 , the optimization
problem above becomes
 

√
α
αβ
2
2
√
√
min max
E
e
−
α
+
θ
g
+
θkµk
;
(4.189)
l
2
g∼N
(0,1)
θ,α≥0 θ0β δ
2 δwhich is the same as equation (5.8) in [62].
95August 7, 2022
DRAFT

4.8

Technical Lemmas

4.8.1

Moreau Envelope

Recall that for convex loss function l(·), the Monreau envelope is defined as:el (x; τ ) = minu1(x − u)2 + l(u),2τ(4.190)Define1lsquare (u) = (1 − u)22lhinge (u) = max{1 − u, 0}llogistic (u) = ln(1 + e−u ),(4.191)(4.192)(4.193)

We can compute the Moreau envelope as follows.
(1) Square loss:
1(1 − x)22(1 + τ )(4.194)1
1(x − u)2 + (1 − u)22τ
2(4.195)elsquare (x; τ ) =Proof:
elsquare (x; τ ) = minu∂Let ∂u= 0, we have1x+τ(u − x) + u − 1 = 0 ⇒ u =τ
1+τ

(4.196)

Therefore,11(x − u)2 + (1 − u)2
u 2τ
2
1
τ2
1
1
=
·
(x − 1)2 +(x − 1)2
2
2τ (1 + τ )2 (1 + τ )2
1=
(x − 1)2
2(1 + τ )elsquare (x; τ ) = min(4.197)(4.198)(4.199)(2) Hinge Loss:τ
1 − x − 2elhinge (x) = 2τ1 (1 − x)20Proof:
elhinge (x; τ ) = minu∂= 0, we haveLet ∂uif x < 1 − τ
if 1 − τ ≤ x ≤ 1
if x > 1

1(x − u)2 + max(1 − u, 0)2τ

∂1= (u − x) − I[u < 1]∂u
τ96

(4.200)(4.201)(4.202)August 7, 2022
DRAFT
when x < 1 − τ , u = x + τ ; when x ≥ 1, u = x; otherwise, u = 1.
(3) Logistic Loss:ellogistic (x; τ ) does not have a closed form solution, but it can be very efficiently approximatedby newton iterations [32].
Define1(x − u)2 + ln(1 + e−u )(4.203)g(u; τ ) =
2τ
Then,
1
1
(u − x) −τ
1 + euue
1
g 00 (u; τ ) = +τ
(1 + eu )2
g 0 (u; τ ) =The Newton update can be formulated as:
uk+1 = uk −g 0 (uk ; τ )g 00 (uk ; τ )(4.204)

According to [32], it usually takes at most 3 iterations.

4.8.2

Solving the asymptotic version of AO numerically

We need a few useful properties of the Moreau envelope, in particular, the ones about its derivatives:
Lemma 4.8.1 (From Lemma D.1 of [125]). Define∂1el (x, τ ) = (x − proxl (x, τ ))∂x
τ∂1el (x, τ ) = − 2 (x − proxl (x, τ ))2∂τ2τ

(4.205)(4.206)

Below we derive the fixed point iterations for solving asymptotic version of AO.
We will use the following equivalent form of AO:

min max

r1 ,r2 ≥0

θ


q

q

1
(α0 + α1 )q0(α0 + α1 )q1
2
2
2
2
EZ∼N (0,1) q0 el
r1 + r2 Z + r1 kµk2 −c;+ q1 e l
r1 + r2 Z + r1 kµk2 +c;
2
2α0 θ
2α1 θ
2
θr2
−
+ λ(r12 + r22 )2(α0 + α1 )97August 7, 2022
DRAFT
By the first order optimality condition,
∂
=0
∂r1
∂
=0
∂r2
∂
=0
∂θ
∂
=0
∂c

(Here are some details about the calculation)
Introducing a few shorthands: (only works for logistic; otherwise the definition of C0 , C1
needs to be changed. Also requrires differentiability, otherwise won’t be in a nice form likebelow).
Z0 ∼ N (r1 kµk2 −c, r12 + r22 )Z1 ∼ N (r1 kµk2 +c, r12 + r22 )(α0 + α1 )q0
τ0 =
2α0 θ
(α0 + α1 )q1
τ1 =
2α1 θ
0
A0 = E[l (proxl (Z0 , τ0 ))]A1 = E[l0 (proxl (Z1 , τ1 ))]τ0
2
B0 = E[(l0 (proxl (Z0 , τ0 ))) ]θ
τ12
B1 = E[(l0 (proxl (Z1 , τ1 ))) ]θl00 (proxl (Z0 , τ0 ))C0 = E
1 + τ0 l00 (proxl (Z0 , τ0 ))
l00 (proxl (Z1 , τ1 ))C1 = E
1 + τ1 l00 (proxl (Z1 , τ1 ))q0
q1
A = A0 + A1
2
2
q0
q1
B = B0 + B1
2
2
q0
q1
C = C0 + C1
2
2

98August 7, 2022
DRAFT
Then,
∂
= kµk2 A + r1 C + 2λr1
∂r1
∂
θr2
+ 2λr2= r2 C −∂r2
α0 + α1
∂
1
r22
= B−
∂θ
2
2(α0 + α1 )∂q0
q1
= − A0 + A1
∂c
2
2

=0

(4.207)=0(4.208)=0(4.209)=0(4.210)(4.211)

By equation 4.209,r2 =p(α0 + α1 )B(4.212)

By equation 4.208,
θ = (α0 + α1 )(C + 2λ)(4.213)(α0 + α1 )kµk2 A
kµk2 A=C + 2λθ(4.214)

Combining with equation 4.207,
r1 =equation 4.210 does not have a closed form solution for c. Instead, we use one Newton iteration
to find a approximate solution.
∂

ct+1 = ct − ∂∂c2

(ct )

∂c2

4.8.3

(ct )

= ct −− q20 A0 + q21 A1
C(4.215)

Lemmas for Lower Bounds

Lemma 4.8.2. , Let θ ∼ Uniform(S d−1 ), where S d−1 is the unit ball in Rd . Let v be any unit
vector in Rd ,thend−1 d−1,),(4.216)2θT v − 1 ∼ B(2
2
where B(α, β) denotes the Beta distribution.
Lemma 4.8.3. Let X ∼ B(α, β), thenE[exp(tX)] =1 F1 (α, α + β, t),(4.217)

where 1 F1 (α, α + β, t) is the confluent hypergeometric function:
1 F1 (α, α + β, t) =

∞
k−1
X
tk Y

α+m
k! m=0 α + β + m
k=099(4.218)August 7, 2022
DRAFT
Lemma 4.8.4. Let θ ∼ Uniform(S d−1 ), where S d−1 is the unit ball in Rd . Then for any v ∈ Rd ,T

E[exp(θ v)] = exp(−kvk2 )k−1∞Xkvkk Y
2

k=0

k!

(1 +

m=0

m
)
d+1+m

(4.219)which is an increasing function of kvk2
Proof. Monotonicity:
∞
k
∞
k−1
X
X
∂
kvkk2 Ym
kvkk2 Ym
E[exp(θT v)] = exp(−kvk2 )(1 +) − exp(−kvk2 )
(1 +
∂kvk2
k! m=0d+1+m
k! m=0
d+1+m
k=0
k=0

(4.220)= exp(−kvk2 )

∞
X
kvkk

k−1
Y
k
m
(1 +
)
k! d + 1 + k m=0
d+1+m
2

k=0

≥0

(4.221)(4.222)

The following lemma (analogous to Theorem 36.64 in the reference above) provides a sufficientcondition for asymptotic minimax optimality.
Lemma 4.8.5. Let {fd }∞d=1 be a sequence of classifiers, which satisfies the following conditions:
1. For each fd , there exists a prior distribution Qd , so that fd is the MAP classifier under prior
Qd .
2. limd→∞ sup(µ0 ,µ1 )∈Pd (∆) Rn0 ,n1 ,µ0 ,µ1 (fd ) = R∞ exists.
3. There exists a sequence of positive real number {γd }d≥1 , where γd = od (1), such that thefollowing inequality holds uniformly for all (µ0 , µ1 ) ∈ Pd :|Rn0 ,n1 ,µ0 ,µ1 (fd ) − R∞ |≤ γd(4.223)

Then, we can claim that {fd }∞d=1 is asymptotically minimax optimal.
∞
In other words, if {fd }d=1 is a sequence of MAP classifiers with approximately constant risk
among (µ0 , µ1 ) ∈ Pd , then it’s asymptotically minimax optimal.
Proof. By condition (1), since Minimax Risk is lower bounded by Bayes Risk, we have
M (n0 , n1 , Pd ) ≥ Bn0 ,n1 ,Q (fd )(4.224)

By condition (3), since expectation is lower bounded by the minimum, we have
Bn0 ,n1 ,Q (fd ) ≥ R∞ − γd(4.225)

Combining the two steps above and let d → ∞, we have
M (n0 , n1 , Pd ) ≥ R∞

lim sup(4.226)

n0 /d=α0 ,n1 /d=α1 ,d→∞However, by condition (2), we know that the asymptotic risk of {fd }∞d=1 is R∞ , therefore, it’s
asymptotically minimax optimal and we have completed the proof.
100August 7, 2022
DRAFT

4.8.4

Exact Asymptotic Minimax Risk in Balanced Setting

In this section, we discuss the asymptotic optimality of LDA in the regime n0 = n1 = 12 αd, d →∞. We have already showed that the variant of LDA with known covariance has the asymptotic
risk of∆2
RLDA → Φ(− √),(4.227)2 ∆2 + 4α−1
But is this risk asymptotically optimal? The answer is yes, as shown in the following theorem:
Theorem 4.8.1. The asymptotic minimax risk is given by
∆2
1 1
∗
),
Rasymp(∆, α, α) = Φ(− √2 2
2 ∆2 + 4α−1(4.228)which is achieved by LDA with known covariances.
Proof. We already know that LDA achieves the asymptotic risk above, so it’s sufficient to prove
LHS ≥ RHS. In fact, we can show that in a more restricted parameter space∆} ⊆ Pd
(4.229)2
the minimax risk remain the same. The strategy is to use Lemma 4.8.5.
Let Qd be the uniform distribution over the hyper-sphere ∆2 ·S d−1 , and fd be the MAP classifier
under Qd . [85] had the following observation:
Pdsym = {(µ0 , µ1 ) : µ0 = −µ, µ1 = µ, µ =nfd (x; x1 , · · · , xn ) = sign(1X
yi xi )T x)
n i=1(4.230)where we recall that yi = −1 when 1 ≤ i ≤ n0 , and yi = 1 when n0 + 1 ≤ i ≤ n. Below werephrase their proof.
By definition of MAP classifier,fd (x; x1 , · · · , xn ) = argmax Pr[Y = y|X = x; x1 , · · · , xn ](4.231)y∈{+1,−1}Notice that
Pr[Y = y|X = x; x1 , · · · , xn ] ∝ p(Y = y, X = x, x1 , · · · , xn )= Eµ∼Qd p(Y = y, X = x, x1 , · · · , xn |µ)
n
X
1
2 1
∝ Eµ∼Qd exp(− kx − yµk2 −kxi − yi µk22 )22 i=1∝ Eµ∼Qd [exp (yx +nX

yi xi )]

(4.232)(4.233)(4.234)(4.235)i=1

P
By Lemma 4.8.4, this is a monotone function of kyx + ni=1 yi xi k2 . Therefore, fd (x) = 1 if andonly if
n
n
X
X
kx +
yi xi k2 ≥ k−x +yi xi k2
(4.236)
i=1

i=1

101

August 7, 2022
DRAFT
which is equivalent ton1X
fd (x; x1 , · · · , xn ) = sign(
yi xi )T x)
n i=1(4.237)and we have completed
the proof of equation 4.230.
Pn1
Let µ̂ = n i=1 yi xi . Since fd (x; x1 , · · · , xn ) = µ̂T x is a linear classifier, its classification
error condition on any fixed µ can be written as
Φ(−

µ̂T µ
)
kµ̂k2

(4.238)(the calculation below is not very rigorous)
2
Since µ̂ ∼ N (µ, n1 Id ), we have µ̂T µ = kµk22 +od (1) = ∆4 + od (1), and kµ̂k2 =od (1) =qq
kµk22 + nd +∆2
+ α1 + od (1), hence, for any µ,
4∆2Rµ (fd ) = Φ( q) + od (1)2 ∆2 + α4(4.239)which proves condition (2,3) of Lemma 4.8.5. Therefore, {fd }∞d=1 is asymptotically minimaxoptimal and we have completed the proof.

4.8.5

Other Technical Lemmas

Lemma 4.8.6.

√√
θA − θB = ( A − B)2+
θ>0 1 + θsup(4.240)

Proof. TBD.
Lemma 4.8.7.

θθ

)A − θB =? ? ?
α0 +α1 +1
2θ + α0α+α
θ>0 2θ + α01

sup((4.241)

Does not seem to have a nice form - the maximizer θ∗ is a solution to a quartic equation when
α0 6= α1 .
Proof. TBD.

4.9

LDA and Square Loss Minimization

When the loss function is chosen as square loss, i.e. β minimizes
n
X
1

2
i=1

πi (yi − xTi β)2 + λkβk22
102(4.242)August 7, 2022
DRAFT
This setting is also studied in [68], when the classes are balanced. Below we show that their result(when n > d) can be recovered using our analysis.
It is easy to see
!
!−1
n
n
X
X
T
β=
πi xi xi + 2λI
πi yi xi
(4.243)i=1

i=1This is somewhat related to Regularized LDA, TBD.
Using equation 4.194, the formula for the Moreau envelope of square loss elsquare ,1
EZ∼N (0,1) (RZ + r1 kµk2 −1)22(1 + τ )
1=
R2 + (r1 kµk2 −1)2
2τEZ∼N (0,1) [elsquare (RZ + r1 kµk2 ; τ )] =(4.244)(4.245)In the balanced setting where α0 = α1 = 12 α, using theorem ??(main), we have the classificationr∗
error converges to Φ(− √ ∗ 21 ∗ 2 kµk), where r1∗ , r2∗ , θ∗ is the optimal solution of(r1 ) +(r2 )


 q
1
θr2
min max EZ∼N (0,1) el
r12 + r22 Z + r1 kµk2 ;− 2 + λ(r12 + r22 )r1 ,r2 θθ
2α(4.246)

By equation 4.244, this is equivalent tomin max
r1 ,r2θ θr2
θ
r12 + r22 + (r1 kµk2 −1)2 − 2 + λ(r12 + r22 )2(1 + θ)2α(4.247)

By Lemma 4.8.6, this is equivalent to:
2
q
r2
2
2
2
+ λ(r12 + r22 )min
r1 + r2 + (r1 kµk2 −1) − √r1 ,r2α +(4.248)Now we make a few simplifications: let λ = 0 (so there is no regularization) and α > 1 (so n > dand the minimizer of empirical loss is unique). In this setting, we also haveq
r2
r12 + r22 + (r1 kµk2 −1)2 ≥ r2 ≥ √(4.249)α
So the objective can be simplified to:
q
2
r2
2
2
2
min
r1 + r2 + (r1 kµk2 −1) − √r1 ,r2α(4.250)This function is monotone in r12 + (r1 kµk2 −1)2 , which is minimized at
r1∗ =

kµk2
1 + kµk22103(4.251)August 7, 2022
DRAFT
Plug in r1∗ , we now have a univariate optimization problem:
s
min
r2

1
r2
−√
r22 +
2
1 + kµk2αwhich is minimized whenr2∗ = p!2(4.252)

1(4.253)(α − 1)(1 + kµk22 )Therefore, the asymptotic classification error iskµk2
r∗kµk2 ) = Φ(− q)Φ(− p ∗ 1∗
1+kµk−2(r1 )2 + (r2 )22
1+(4.254)α−1which approaches Bayes error as α → ∞. This expression also coincides with Theorem 2.3 in[135] when the classes are balanced.
Note that the asymptotics are much more complicated in the imbalanced setting, where theinner maximization requires solving a quartic equation (see Lemma 4.8.7).

104August 7, 2022
DRAFT

Chapter 5Interpolation in Distributionally Robust
Optimization
5.1

Introduction

It has been well established by prior work that overparameterized models, whose number ofparameters is much larger than the number of training samples, can empirically achieve high test
performance on a variety of tasks, in contrast to the theory that models with too many parameters
could have large generalization error.
This high performance however is on average; a large body of prior work [15, 55, 123]
showed that these models tend to learn spurious features, such as learning the background inimage classification instead of the object, and learning keywords like “not” in language sentiment
analysis instead of really understanding the sentences. Consequently, these models are unfair, i.e.
they fail on certain minority groups (such as positive sentences containing “not”) while still having
high average-case performance. To solve this problem, people have proposed various reweighting
algorithms to improve the model’s worst-group performance, such as upweighting the minority
groups or using distributionally robust optimization (DRO) based methods [40, 51, 110, 116].
While reweighting algorithms in principle can improve the worst-group performance comparedto vanilla empirical risk minimization (ERM), previous work empirically found that when applied
to modern overparameterized models, these methods could overfit very easily, so that they havepoor test worst-group performance. For example, Sagawa et al. [110] studied a reweighting
algorithm called group DRO. They found that compared to ERM, group DRO does improve
the worst-group test accuracy by a large margin at the early stage of training. However, if noregularization is applied, then as training goes on, the worst-group test accuracy of group DRO
will drop significantly and eventually to a level almost the same as ERM. Some previous work
tried to explain why reweighting algorithms can overfit so easily. For instance, Sagawa et al. [112]argued that with these algorithms, an overparameterized model would typically memorize all
training samples in the minority groups while still learning the spurious features from the majority
groups.
In this work, we aim to understand the overfitting phenomenon in reweighting algorithms bystudying their implicit biases. Specifically, we prove for a family of overparameterized neural105

August 7, 2022
DRAFT
networks that for almost all reweighting algorithms, the model always converges to the sameinterpolator that fits all training samples, no matter the reweighting. Since ERM is a special caseof such reweighting algorithms (where each sample receives the same weight), this means thatthe implicit biases of all reweighting algorithms are equivalent to that of ERM. Consequently,
the model trained by any reweighting algorithm always overfits to the ERM interpolator, so wecannot hope for its worst-group test performance to be better than ERM. In short, reweighting
algorithms always overfit.
Given this pessimistic result, we analyze whether regularization can help mitigate overfitting,as proposed by Sagawa et al. [110]. We find that a necessary condition for regularization towork is that it considerably lowers the training performance. Specifically, we prove that if theoverparameterized model trained by a reweighting algorithm with regularization can still perform
almost perfectly on the training set, then overfitting is still inevitable. This explains why inpractice we need very large regularization that prevents the model from achieving nearly zero
training error to avoid overfitting.
Our results have two important consequences for practice: (i) We should always use large
regularization or early stopping when optimizing for worst-group performance; (ii) We should
always try to obtain more training samples, e.g. with strong data augmentation or semi-supervised
learning.

5.1.1

Related work

Group fairness.
Group fairness in machine learning was first studied in Hardt et al. [50]and Zafar et al. [152], where they required the model to perform equally well over all groups.
Later, Hashimoto et al. [51] studied another type of group fairness called Rawlsian max-min
fairness [109], which does not require equal performance but rather requires high performanceon the worst-off group. The problem we study in this paper is most closely related to Rawlsian
max-min fairness. A large body of recent work in machine learning have studied how to improve
this worst-group performance [40, 86, 97, 148, 155]. Recent work however observe that theseapproaches, when used with modern overparameterized models, easily overfit [110, 112]. Apartfrom group fairness, there are also other notions of fairness, such as individual fairness [41, 153]
and counterfactual fairness [76], which we do not study in this work.
Implicit bias under the overparameterized setting. For overparameterized models, there
could be many model parameters which all minimize the training loss. In such cases, it is ofinterest to study the implicit bias of specific optimization algorithms such as gradient descenti.e. to what training loss minimizer the model parameters will converge to [1, 39]. Our results
use the NTK formulation of wide neural networks [60], and specifically we use linearized neuralnetworks to approximate such wide neural networks following Lee et al. [82]. There is somecriticism of this line of work, e.g. Chizat et al. [27] argued that infinitely wide neural networks
fall in the “lazy training” regime and results might not be transferable to general neural networks.
Nonetheless such wide neural networks are being widely studied in recent years, since they
provide considerable insights into the behavior of more general neural networks, which are
typically intractable to analyze otherwise.
106

August 7, 2022
DRAFT

5.2

Preliminaries

Consider a data domain X × Y ⊆ Rd × R that consists of K groups (subdomains)1 , where eachdata point belongs to one of the groups2 . We assume that the input space X is a subset of the unit
ball of Rd , such that any x ∈ X satisfies kxk2 ≤ 1. We are given a training set {(xi , yi )}ni=1 i.i.d.
sampled from some underlying distribution P over X × Y. Let the K groups be D1 , · · · , DK
where each Di is a subset of X ×Y. Let Pk (z) = P (z|z ∈ Dk ) be the conditional data distribution
over Dk , where z = (x, y). Denote X = (x1 , · · · , xn ) ∈ Rd×n , and Y = (y1 , · · · , yn ) ∈ Rn ;for any function g : X 7→ R, we overload notation and use g(X) = (g(x1 ), · · · , g(xn )) . Let
the loss function be ` : Y × Y → [0, 1]. In vanilla training, the goal is to minimize the expectedrisk denoted
R(f ; P ) = Ez∼P [`(f (x), y)], which is done by minimizing the empirical risk
Pbyn
1R̂(f ) = n i=1 `(f (xi ), yi ).
For tasks requiring high worst-group performance, the goal is to train a model f : X → Y that
performs well over every Pk , which can be achieved by minimizing the worst-group risk definedasRmax (f ; P ) = max R(f ; Pk ) = max Ez∼P [`(f (x), y)|z ∈ Dk ]k=1,···,K

5.2.1

k=1,···,K(5.1)

Reweighting Algorithms

Most existing methods that minimize the worst-group risk are reweighting algorithms that assign
each sample with a weight during training and minimize the weighted average risk. At time t, we(t)assign a weight qi to sample zi , and minimize the weighted empirical risk:R̂q(t) (f ) =nX(t)qi `(f (xi ), yi )

(5.2)

i=1
(t)

(t)

(t)

(t)

where q (t) = (q1 , · · · , qn ) and q1 + · · · + qn = 1.
A static reweighting algorithm assigns to each zi = (xi , yi ) a fixed weight qi that does not(t)change during training, i.e. qi ≡ qi . A famous example is Importance Weighting (IW, Shimodaira[116]), in which if zi ∈ Dk and the size of Dk is nk , then qi = (Knk )−1 . Under IW, each group
has the same weight, and the reweighted empirical risk is a simple (unweighted) average of theempirical risk over each group, so that each group has an equal contribution to the overall risk
objective. Note that ERM is also a special case of static reweighting algorithms: by assigning
q1 = · · · = qn = 1/n.
On the other hand, in a dynamic reweighting algorithm, q (t) changes with t. Specifically, it
upweights samples over which the model has a high risk in order to help the model learn “hard”
samples. A popular dynamic reweighting algorithm is Group DRO [110]. Denote the empirical(t)(t)
risk over group k by R̂k (f ), and the model at time t by f (t) . Group DRO sets qi = gk /nk for
We prove our results for Y ⊆ R, but our results can be easily extended to the multi-class scenario Y ⊆ Rm .
This is the non-overlapping setting. There is also the overlapping setting where groups can overlap with eachother. We focus on the non-overlapping setting in this paper.
1

2

107

August 7, 2022
DRAFT
(t)

all zi ∈ Dk where gk is the group weight that is updated by


(t)
(t−1)
gk ∝ gk
exp ν R̂k (f (t−1) )(∀k = 1, · · · , K)
(t)

(5.3)

(t)

for some ν > 0, and then normalized so that q1 + · · · + qn = 1. Sagawa et al. [110] proved aconvergence rate theorem (their Proposition 2) showing that in the convex setting, the worst-grouptraining risk of Group DRO converges to the global minimum with the rate O(t−1/2 ).
There are many other reweighting algorithms. Particularly, all variants of DRO and DRObased methods like CVaR and χ2 -DRO are reweighting algorithms. See Appendix 5.6 for more
examples.

5.2.2

Reweighting algorithms can easily overfit

In this section, we will empirically demonstrate that while IW and Group DRO can achieve higher
worst-group test performances than ERM at the early stage of training, they can easily overfit
after a number of training epochs.
Following Sagawa et al. [110], we conduct the experiment on two datasets: Waterbirds andCelebA. Each dataset contains a binary confounding variable a and a binary target variable y,dividing the dataset into four groups (four combinations of (a, y)). In Waterbirds y is the typeof the bird and a is the background; In CelebA y is whether the person has blond hair and a iswhether the person is male. On each dataset, a model trained by ERM always exhibits a verystrong empirical correlation between y and a, so its performance on one of the groups is extremely
poor. The goal is to make the model perform well on every group. See Appendix C.1 of Sagawa
et al. [110] for detailed information of these datasets.
On each dataset, we use the ResNet18 model as the classifier and optimize it with momentum
SGD. We run each of the three algorithms: ERM, IW and group DRO (GDRO), for 500 epochs on
Waterbirds and 200 epochs on CelebA, and plot the average training/test and worst-group (WG)
training/test accuracy curves throughout training in Figure 5.1. From the plots we can conclude
that:• All algorithms can achieve and maintain high average training/test accuracy throughout
training, i.e. there is almost no overfitting in the average test accuracy.
• Regarding the worst-group test accuracy, while the two reweighting algorithms outperform

ERM by a large margin at the early epochs, they overfit very quickly. On CelebA afterroughly 100 epochs, the worst-group test accuracies of the two reweighting algorithms
become the same as ERM. On Waterbirds, the worst-group test performances of IW andGroup DRO drop significantly after around 30 epochs though they are still better than ERM.

5.3

Implicit biases of reweighting algorithms

In the previous section, we empirically demonstrated that the worst-group test performances ofreweighting algorithms converge to the same level as ERM. To theoretically understand why thishappens in practice, we analyze the implicit biases of reweighting algorithms. Our main theorem
108

0.6
ERM
IW
GDRO

0

100

200

300

400

0.6
ERM
IW
GDRO

0.4

500

0

100

200

300

400

500

0.6
0.4
ERM
IW
GDRO

0.2
0

0

100

200

300

400

500

0.8
0.6
0.4
ERM
IW
GDRO

0.2
0

0

100

200

300

400

500

EpochsEpochs(a) Avg train acc.

(b) Avg test acc.

(c) WG train acc.

(d) WG test acc.

0.95
0.9
ERM
IW
GDRO

0.85

0

50

100

150

0.95

0.9
ERM
IW
GDRO

0.85

200

0

50

Epochs

100

150

200

1
0.8
0.6
0.4
0.2

Epochs(e) Avg train acc.

ERM
IW
GDRO

(f) Avg test acc.

0

0

50

100

150

200

Worst-group Test Accuracy

Epochs

1

0.8

0.8

1

EpochsWorst-group Training Accuracy

0.4

0.8

1

Worst-group Test Accuracy

0.8

Worst-group Training Accuracy

1

Average Test Accuracy

1

Average Test Accuracy

Average Training Accuracy

Average Training Accuracy

August 7, 2022
DRAFT

0.9

ERM
IW
GDRO

0.7
0.5
0.3
0.1

0

50

100

150

200

EpochsEpochs(g) WG train acc.

(h) WG test acc.

Figure 5.1: Performances of ERM, IW and Group DRO. First row: Waterbirds. Second row:
CelebA.
(Theorem 16) states that almost all reweighting algorithms (including ERM) have equivalent
implicit biases, in the sense that they converge to the same interpolator. Meanwhile, it is observedin practice that the ERM interpolator has a poor worst-group test performance. This leads to the
pessimistic result that reweighting algorithms always overfit. All proofs can be found in Appendix
5.7.

5.3.1

Linear models

We first demonstrate this pessimistic result on simple linear models to provide our readers with a
key intuition, and later we will apply this same intuition to neural networks. Let the linear model
be f (x) = hθ, xi, where θ ∈ Rd . In the overparameterized setting, we have d > n. Consider
using the squared loss `(ŷ, y) = 12 (ŷ − y)2 , and minimizing the weighted empirical risk withgradient descent:
n
X
(t)
(t+1)
(t)
θ
=θ −η
qi ∇θ `(f (t) (xi ), yi )
(5.4)
i=1

where η > 0 is the learning rate. For a linear model with the squared loss, the update rule is
θ

(t+1)

=θ

(t)

n
X
(t)
−η
qi xi (f (t) (xi ) − yi )

(5.5)

i=1

It is a well known result that under the overparameterization setting where d > n, if x1 , · · · , xn arelinearly independent, then with a sufficiently small η, a linear model trained by ERM can always
converge to an interpolator which fits all training samples (i.e. θ(t) → θ∗ such that hθ∗ , xi i = yi
for all i). Here the linear independence is necessary, because otherwise in the extreme case wherex1 = x2 but y1 6= y2 , the model cannot fit (x1 , y1 ) and (x2 , y2 ) simultaneously.
In this section, we aim to extend this ERM convergence analysis to general reweighting
algorithms. Our results require the following assumption:
109

August 7, 2022
DRAFT
(t)

Assumption 1. There exist constants q1 , · · · , qn such that for all i, qi → qi as t → ∞. And
mini qi = q ∗ > 0.
(t)This assumption avoids the scenario where there is some i such that qi ≈ 0 for all t, in whichcase the model could never fit zi . Assumption 1 empirically holds for Group DRO on Waterbirds
and CelebA (see Appendix 5.9.2). Under this assumption, we can prove that the model always
converges to an interpolator:
Theorem 11. For any reweighting algorithm satisfying Assumption 1, if x1 , · · · , xn are linearly
independent, then there exists an η0 > 0 such that for any η ≤ η0 , as t → ∞, θ(t) converges tosome interpolator θ∗ such that for all i, hθ∗ , xi i = yi .
We now make the following key observation regarding the update rule (5.5): θ(t+1) − θ(t)is a linear combination of x1 , · · · , xn for all t, and thus θ(t) − θ(0) always lies in the linear
subspace span(x1 , · · · , xn ). Note that this is an n-dimensional linear subspace if x1 , · · · , xn arelinearly independent, and by Cramer’s rule, there is exactly one θ̃ in this subspace such thathθ̃ + θ(0) , xi i = yi for all i, which implies that θ∗ = θ̃ + θ(0) is unique. Together with Theorem
11, this leads to:
Theorem 12. If x1 , · · · , xn are linearly independent, then there exists η0 > 0 such that for any
reweighting algorithm satisfying Assumption 1, and any η ≤ η0 , θ(t) converges to the same(t)interpolator θ∗ that does not depend on qi .
Note that ERM is also a reweighting algorithm satisfying Assumption 1. Therefore, we haveessentially proved the following result: The implicit bias of any reweighting algorithm satisfying
Assumption 1 is equivalent to ERM, so reweighting algorithms always overfit3 .
The key intuition here is that no matter what reweighting algorithm we use, θ(t) − θ(0) always
lies in a low-dimensional subspace, in which the interpolator is unique. Therefore, as long as amodel trained by the algorithm converges to some interpolator, it must converge to that uniqueinterpolator, which means that the implicit bias of the algorithm is equivalent to ERM.

5.3.2

Linearized neural networks

Now we prove the same result for neural networks. Of course it would be very hard to prove
it for all neural networks. However, we can prove the result for a family of overparameterizedneural networks that can be approximated by their linearized counterparts [82]. Denote the neuralnetwork at time t by f (t) (x) = f (x; θ(t) ) which is parameterized by θ(t) ∈ Rp where p is thenumber of parameters. The linearized neural network of f (t) (x) is defined as(t)flin (x) = f (0) (x) + hθ(t) − θ(0) , ∇θ f (0) (x)i

(5.6)
(t)

where we use the shorthand ∇θ f (0) (x) := ∇θ f (x; θ)|θ=θ0 . Consider training flin (x) via gradient descent on the reweighted risk (as in (5.4)) using the squared loss. Givena training set
{(xi , yi )}ni=1 , we can construct a new training set { ∇θ f (0) (xi ), yi − f (0) (xi ) }ni=1 , so that training a linearized neural network on the original training set is equivalent to training a linear modelon the new training set. Based on this observation, we have the following corollary of Theorem
12:3

By overfit, we are saying that the training error of the model trained by the reweighting algorithm will converge
to zero, but the worst-group test performance will converge to the same low level as ERM.

110

August 7, 2022
DRAFT
Corollary 13. If ∇θ f (0) (x1 ), · · · , ∇θ f (0) (xn ) are linearly independent, then there exists η0 > 0such that for any reweighting algorithm satisfying Assumption 1, and any η ≤ η0 , θ(t) converges
to the same interpolator θ∗ that does not depend on qi .
Here we are still using the key intuition:θ(t) − θ(0) always lies in the n-dimensional linear
subspace span ∇θ f (0) (x1 ), · · · , ∇θ f (0) (xn ) . By Cramer’s rule, there is a unique interpolatorθ∗ such that θ∗ − θ(0) ∈ span ∇θ f (0) (x1 ), · · · , ∇θ f (0) (xn ) , and θ(t) always converges to thatθ∗ . Thus, we have essentially proved that for linearized neural networks, reweighting algorithms
always overfit.
Now let us delve deeper into the training dynamics of a linearized neural network. Note that(t)∇θ flin (X) = ∇θ f (0) (X) ∈ Rp×n , so the change in the training function value vector is
(t+1)

flin

(t)

(t)

(X) − flin (X) = −η ∇θ f (0) (X)> ∇θ f (0) (X)Q(t) ∇ŷ `(flin (X), Y )
(t)

(5.7)

(t)

where Q(t) = diag(q1 , · · · , qn ). The function value vector moves along the kernel gradient with(0)
respect to Θq(t) = ∇θ f (0) (X)> ∇θ f (0) (X)Q(t) . Meanwhile, the neural tangent kernel (NTK,[60]) is Θ(0) (x, x0 ) = ∇θ f (0) (x)> ∇θ f (0) (x0 ) , and the Gram matrix is Θ(0) = Θ(0) (X, X), so(0)
Θq(t) = Θ(0) Q(t) . We can thus extend our result for gradient descent on linearized neural networks
to a kernel gradient descent algorithm as above.

5.3.3

Wide fully-connected neural networks

Now we prove the result for sufficiently wide fully-connected neural networks, which can beapproximated by the linearized neural networks. First we define a fully-connected neural networkwith L hidden layers (we always assume L ≥ 1 so there is at least one hidden layer). Let hl andxl be the pre- and post-activation outputs of layer l, and dl be the width of layer l. Let x0 = x
and d0 = d. Define the neural network as
Wl lhl+1 = √x + βbldl(l = 0, · · · , L)
(5.8)

xl+1 = σ(hl+1 )where σ is a non-linear activation function, W l ∈ Rdl+1 ×dl and W L ∈ R1×dL . The parameters θ
consist of W 0 , · · · , W L and b0 , · · · , bL (θ is the concatenation of all flattened weights and biases).
The final output of the neural network is f (x) = hL+1 . And let the neural network be initializedas( l(0)( L(0)
Wi,j ∼ N (0, 1)
Wi,j = 0
(l
=
0,
·
·
·
,
L
−
1)
and
(5.9)
l(0)
L(0)
bj ∼ N (0, 1)
bj ∼ N (0, 1)
We also need the following assumption for our approximation theorem:
Assumption 2. σ is differentiable everywhere, and both σ and σ̇ are Lipschitz.4
4

f is Lipschitz if there exists a constant L > 0 such that for any x1 , x2 , |f (x1 ) − f (x2 )|≤ L kx1 − x2 k2 .

111

August 7, 2022
DRAFT
Difference from [60]. Our initialization (5.9) is different from the original one in [60] in theL(0)last (output) layer. For the output layer, we use the zero initialization Wi,j = 0 instead of theL(0)Gaussian initialization Wi,j ∼ N (0, 1). This modification enables us to accurately approximate
the neural network with its linearized counterpart (5.6), as we notice that the proofs in [82](particularly the proofs of their Theorem 2.1 and their Lemma 1 in Appendix G) are flawed. In
Appendix 5.8 we will explain what goes wrong in their proofs and how we manage to fix theproofs with our modification.
For our new initialization, we still have the following NTK theorem:
Theorem 14. If σ is Lipschitz and dl → ∞ for l = 1, · · · , L sequentially, then Θ(0) (x, x0 )converges in probability to a non-degenerated5 deterministic limiting kernel Θ(x, x0 ).
The kernel Gram matrix Θ = Θ(X, X) ∈ Rn×n is a positive semi-definite symmetric matrix.
Denote its largest and smallest eigenvalues by λmax and λmin . Note that Θ is non-degenerated,
so we assume that λmin > 0 (which holds almost surely in the overparameterized setting wheredL  n). Then, we can prove the following approximation theorem:
Theorem 15 (Approximation Theorem). Let η ∗ = (λmin + λmax )−1 . For a fully-connected neuralnetwork f (t) that satisfies Assumption 2 and is trained by any reweighting algorithm satisfying(t)Assumption 1, let flin be its linearized neural network which is trained by the same reweighting(t)algorithm (i.e. ∀i, t, qi are the same for both networks). If d1 = d2 = · · · = dL = d˜ andλmin > 0, then for any δ > 0, there exists D̃ > 0 and a constant C such that as long as η ≤ η ∗and d˜ ≥ D̃, for any test point x ∈ Rd such that kxk2 ≤ 1, with probability at least 1 − δ overrandom initialization,(t)
sup flin (x) − f (t) (x) ≤ C d˜−1/4(5.10)t≥0Remark 1. We can easily extend this theorem to the case where there exists αl > 0 for each ofl = 2, · · · , L such that dl /d1 → αl and d1 → ∞.
Combining all the above results altogether, we achieve our main theorem:
Theorem 16. Under the conditions of Theorem 15, there exists an η1 > 0 such that if η ≤ η1 and∇θ f (0) (x1 ), · · · , ∇θ f (0) (xn ) are linearly independent, then as d˜ → ∞, for any test point x ∈ Rd
such that kxk2 ≤ 1, with probability close to 1 over random initialization,(t)
lim sup f (t) (x) − fERM (x) = O(d˜−1/4 ) → 0(5.11)t→∞(t)

where f (t) is trained by the reweighting algorithm and fERM is trained by ERM.
The main theorem shows that at any test point x, the gap between the function values of thetwo models converges to an infinitely small term, so the worst-group test performance of thereweighting algorithm will converge to the same level as ERM. Therefore, we have proved thatfor sufficiently wide fully-connected neural networks, reweighting algorithms always overfit.
Our key intuition tells us that the change in the model parameters always lies in an ndimensional subspace. Thus, one possible way to improve the worst-group test performance isto enlarge this subspace by adding more training samples, e.g. via data augmentation or semisupervised learning. However, even if we have more training samples, as long as the model is still5

Non-degenerated means that Θ(x, x0 ) depends on x and x0 and is not a constant.

112

August 7, 2022
DRAFT
overparameterized, and all ∇θ f (0) (xi ) are linearly independent, then our result still says that noreweighting algorithm can do better than ERM in the long run (though the performance of ERM
itself might be improved).
Moreover, our theoretical results can explain the surprising empirical observation in [112]that removing some samples from the majority groups to match the group sizes can sometimes
achieve even higher worst-group test performance than reweighting even though it wastes lotsof data (see their Section 6). When training samples are removed, the model will converge toan interpolator of the smaller training set which is different from the interpolator of the original
training set, so there is a chance that the performance of the new interpolator is actually higher.

5.4

Does regularization really help?

In the previous section, we proved the pessimistic result that reweighting algorithms always
overfit, i.e. in the long run their worst-group test performances always drop to the same level as
ERM. And even if we use strong data augmentation or semi-supervised learning, reweighting
algorithms still cannot outperform ERM if the training set is not sufficiently enlarged.
[110] proposed to tackle the overfitting problem of reweighting algorithms via regularization.
In particular, they empirically demonstrated with experiments that large regularization is required
to prevent reweighting algorithms such as group DRO from overfitting. With a large regularization,
the model can maintain a high test worst-group performance, but it cannot obtain perfect training
accuracy, in contrast to the case where no regularization is applied.
In this section, we study the necessary conditions for regularization to maintain high worstgroup test performance. Specifically, we will show that regularization will not work if it is notlarge enough to prevent the model from obtaining nearly zero training error. In other words,
lowering the training performance is the key to keeping a high worst-group test performance. Notethat the results in this section do not require Assumption 1, so the results hold for all reweighting
algorithms.

5.4.1

Theoretical analysis(t)Consider a reweighting algorithm with sample weights qi . Following [110], we consider adding
L2 penalty to the weighted empirical risk (5.2):
R̂µq(t) (f ) =

n
X
µ
2
(t)
qi `(f (xi ), yi ) +θ − θ(0) 22i=1(5.12)

Given that sufficiently wide neural networks can be approximated by linearized ones, we first
focus on linearized neural networks. We will use the subscript “reg” to refer to a regularized model(t)(which is trained trained by minimizing the regularized risk (5.12)). Let flinreg be a regularized(t)linearized neural network trained by some reweighting algorithm, and flinERM be an unregularized
linearized neural network trained by ERM. As before, we consider training the models withgradient descent under the squared loss `(ŷ, y) = 12 (ŷ − y)2 . The following result shows that(t)these two models are very close if flinreg can achieve low training error:
113

August 7, 2022
DRAFT
Theorem 17. If there is a constant M0 > 0 such that ∇θ f (0) (x) 2 ≤ M0 for all kxk2 ≤ 1,(t)
∇θ f (0) (x1 ), · · · , ∇θ f (0) (xn ) are linearly independent, and the empirical training risk of flinreg
satisfies(t)
lim sup R̂(flinreg ) < ,
(5.13)t→∞for some  > 0, then for any test point x such that kxk2 ≤ 1 we have
√
(t)
(t)
lim sup flinreg (x) − flinERM (x) = O( ).

(5.14)t→∞

The proof of this theorem also follows the key intuition: we can show that even with the L2
penalty added, θ(t) − θ(0) is still limited in a low-dimensional subspace. And although we cannot
prove that θ(t) always converges to the ERM interpolator, we can prove that it can get very close
to that interpolator if its training error is very low, so the resulting model is very close to the ERM
model.
Then, we can extend this result to sufficiently wide fully-connected neural networks:
Theorem 18. If λmin > 0 and µ > 0, then let η ∗ = (µ + λmin + λmax )−1 . For a wide fully(t)connected neural network freg defined by (5.8) and (5.9) and satisfying Assumption 2, and any
˜ η ≤ η ∗ , ∇θ f (0) (x1 ), · · · , ∇θ f (0) (xn ) arereweighting algorithm, if d1 = d2 = · · · = dL = d,(t)linearly independent, and the empirical training risk of freg satisfies(t)
lim sup R̂(freg)<(5.15)t→∞for some  > 0, then as d˜ → ∞, with probability close to 1 over random initialization, for any
test point x such that kxk2 ≤ 1 we have(t)(t)(x) − fERM (x) = O(d˜−1/4 +lim sup freg√√) → O( )(5.16)t→∞

The result shows that a regularized model trained by any reweighting algorithm will get very
close to an unregularized ERM model at any test point x if the training error of the former isnearly zero. Thus, regularization only helps when it is large enough to keep the training error of
the model away from zero by a margin.
Our results explain the empirical observation of [110] that by using large regularization, themodel can maintain a high worst-group test performance, but it cannot achieve perfect training
accuracy. If smaller regularization is applied and the model can achieve nearly perfect training
accuracy, then its worst-group test performance will still significantly drop.

5.4.2

Empirical study

In this section, we validate our theoretical results above with experiments on Waterbirds andCelebA. We run ERM, IW and group DRO under different levels of weight decay for 500 epochs
on Waterbirds and 250 epochs on CelebA. Note that we do not strictly follow our L2 penalty
formulation (5.12), but we study the L2 weight decay regularization which is most widely usedin practice. We repeat each experiment five times with different random seeds and report the114

August 7, 2022
DRAFT
Table 5.1: Mean average training accuracy and worst-group test accuracy (%) of the last 10
training epochs of ERM, IW and Group DRO under different levels of weight decay (WD). Each
entry is Average training accuracy / Worst-group test accuracy. Blue entries are mean accuracies
of epochs 11-20 with no weight decay. Each experiment is repeated five times with different
random seeds.
DatasetWD
ERM
IWGroup DRO
0
100.0 ± 0.0/56.3 ± 1.8 100.0 ± 0.0/67.6 ± 1.1 100.0 ± 0.0/64.5 ± 1.6(11-20) (Early stopping)
92.4 ± 0.4/83.7 ± 0.6 92.9 ± 0.4/79.9 ± 2.1
0.05
100.0 ± 0.0/71.0 ± 1.9 100.0 ± 0.0/63.5 ± 2.6
Waterbirds
0.1
100.0 ± 0.0/67.7 ± 0.7 100.0 ± 0.0/54.7 ± 2.7
0.15
99.0 ± 0.7/53.7 ± 2.7 99.4 ± 0.6/52.5 ± 2.5
0.2
91.6 ± 2.0/35.9 ± 6.9 94.8 ± 0.9/38.0 ± 7.5099.0 ± 0.2/40.2 ± 5.6 99.4 ± 0.1/42.7 ± 1.7 99.4 ± 0.1/49.5 ± 1.9(11-20) (Early stopping)
92.1 ± 0.3/78.2 ± 3.2 90.5 ± 0.5/85.2 ± 1.7
CelebA
0.01
97.9 ± 0.2/50.0 ± 2.8 96.5 ± 0.5/67.2 ± 1.7
0.03
95.0 ± 0.2/62.8 ± 2.4 88.9 ± 1.1/83.1 ± 2.2
0.1
89.4 ± 2.0/76.0 ± 2.4 75.1 ± 9.5/50.6 ± 15.9

95% confidence interval of the mean average training and worst-group test accuracies of thelast 10 training epochs in Table 5.1. To compare with early stopping, we also report the mean
accuracies of epochs 11-20 with no regularization in blue. Moreover, we plot the average training
and worst-group test accuracy curves throughout training for IW and Group DRO with one of therandom seeds in Figure 5.2.
On both datasets, early stopping achieve the best performances. Particularly, on Waterbirds,
there is no clear sign that regularization could help prevent overfitting. When the regularization issmall, the training accuracy is still 100% and the algorithm continues to overfit. However, when
the regularization is large enough to lower the training accuracy, the worst-group test accuracy
drops more because the model cannot learn the samples well under such a large regularization.
Thus, perhaps not surprisingly, a lower training performance is only a necessary condition but not
sufficient.
On CelebA, regularization does help mitigate overfitting, but a useful regularization must belarge enough to lower the training accuracy. We observe that Group DRO overfits more slowly
than IW, as it still has over 70% worst-group test accuracy after 70 epochs. However, as Figure
5.2d clearly shows, its worst-group test accuracy will still drop to the ERM level at 200 epochs.
We also notice that Group DRO requires a smaller regularization than IW: for IW we need theweight decay level to be as large as 0.1 to achieve a similar performance as early stopping, but forGroup DRO it only needs to be 0.01, and using 0.1 is actually harmful.
Overall, we find that early stopping achieves a markedly better performance. On the other
hand, using large regularization could result in training instability, as well as a loss in overall
performance, and there may or may not be a small band for the regularization parameter where
the worst-group test performance is better.
115

0.8
wd = 0wd = 0.01
wd = 0.03
wd = 0.1

0.7
0.6
0

50

100

150

200

250

Epochs(a) IW: Avg Train Acc.

1
0.8
0.6
wd = 0wd = 0.01
wd = 0.03
wd = 0.1

0.4

1
0.8
0.6
0.4

50

0
0

50

100

150

Epochs0.2
0wd = 0wd = 0.01
wd = 0.03
wd = 0.1

0.2

100

150

200

250

200

250

Worst-group Test Accuracy

0.9

Average Training Accuracy

1

Worst-group Test Accuracy

Average Training Accuracy

August 7, 2022
DRAFT
1
0.8
0.6
0.4

wd = 0wd = 0.01
wd = 0.03
wd = 0.1

0.2
0
0

50

100

150

200

250

Epochs(c) GDRO: Avg Train(b) IW: WG Test Acc. Acc.
(d) GDRO: WG Test Acc.
EpochsFigure 5.2: Average training accuracy and worst-group (WG) test accuracy of IW and GroupDRO (GDRO) under different L2 weight decay levels on CelebA.

5.5

Conclusion

In this work, we theoretically studied why reweighting algorithms overfit in practice by analyzing
their implicit biases. Specifically, we proved the pessimistic result that reweighting algorithms
always overfit. Our proof was based on the key intuition that the change in model parameters
always lies in a low-dimensional subspace, so that even with reweighting, the model still converges
to the same unique interpolator. When regularization is applied, we proved that the regularizationmust be large enough to keep the model from achieving nearly zero training error in order toprevent overfitting. We empirically validated our theoretical results on real datasets, and our
results can also explain the empirical observations in previous work. Our results are especially
important for large-scale machine learning tasks, where early stopping is not always possible inorder to achieve high performances. Practitioners shooting for high worst-group performances inthose tasks must be very careful about to what extent overfitting affects reweighting algorithms.
Reproducibility statement
To guarantee the reproducibility of all our empirical results, in all our experiments we use a fixed
set of random seeds, and we run some of the experiments twice with the same random seed tomake sure that the outputs are the same. See Appendix 5.9.1 for experiment details. After thispaper is deanonymized, we will provide a GitHub repository that contains all the codes, datasets,
hyperparameters, random seeds, machine speculations and anaconda environment speculations
that are sufficient to exactly reproduce our empirical results.

116

August 7, 2022
DRAFT

5.6

Other reweighting algorithms

In this section, we will review some other previously proposed reweighting algorithms. First, we
will look at DRO-based methods, where DRO stands for Distributionally Robust Optimization.
DRO is designed for tasks with distributional shift, where the training distribution and thetest distribution are different, and there are some constraints on the distance between these two
distributions (typically described by a divergence function D). Since the real test distribution is
unknown, DRO minimizes the model’s risk over the worst distribution that satisfies the distanceconstraints, which is an upper bound of the model’s real test error. Formally speaking, given a
training distribution P , DRO minimizes the expected risk over the worst-case distribution Q ina ball w.r.t. divergence D around the training distribution P . For group shift problems which
require high worst-group performance, Q also needs to be absolutely continuous with respect to
P , i.e. Q  P . Overall, DRO minimizes the following expected DRO risk:
RD,ρ (θ; P ) = sup {EQ [`(θ; Z)] : D(Q k P ) ≤ ρ}

(5.17)QP

The expected DRO risk is typically minimized in the following way: for each epoch t, wefirst find the worst Q that maximizes EQ [`(θ; Z)] and satisfies D(Q k P ) ≤ ρ, Q  P , and thenminimize the model’s expected risk over this Q with gradient descent. The rationale behind thisalgorithm is the famous Danskin’s Theorem, which says that if F (x) is the maximum of a familyof functions, then its gradient at point x is equal to the gradient of the function that attains the
maximum value at x.
Note that in practice we only have a finite set of training samples {z1 , · · · , zn }, so P is always
chosen as the empirical distribution, i.e. uniform distribution over z1 , · · · , zn . Then, note thatQ  P , which implies that the support of Q must be a subset of the support of P , which is{z1 , · · · , zn }. This means that Q must be a distribution over z1 , · · · , zn , i.e. it is a reweighting
over the training samples. Thus, we have essentially showed that DRO is a reweighting algorithm,
and in fact almost all methods based on DRO are reweighting algorithms.
Two widely used variants of DRO are CVaR (Conditional Value at Risk) and χ2 -DRO. In
CVaR, for a fixed α ∈ (0, 1), we let D(Q k P ) = sup log dQand ρ = − log α. As a result,dP
1
suppose that αn is an integer, then CVaR will assign weight αn to αn training samples that incur
the highest losses, and weight 0 to the rest of the samples, so we can easily see that CVaR is a
reweighting algorithm. χ2 -DRO was first used in Hashimototasks
R et al. [51] to 2deal with fairness
1
1 1
where the group labels are unknown, where D(Q k P ) = 2 (dQ/dP −1) dP and ρ = 2 ( α −1)2 .
χ2 -DRO is also a reweighting algorithm.
There are many other previously proposed methods of maximizing the worst-group performance that are also reweighting algorithms. For instance, Xu et al. [148] studied the imbalanced
class problem where a standard trained model always has high performance over classes withmany training samples and low performance over minority classes. They proposed to balance theclasses with Label CVaR, which is based on DRO and is a reweighting algorithm.
Liu et al. [86] proposed a two-stage training process called JTT: in the first identification stage
they trained a model with ERM to identify training samples that are hard to learn, and in thesecond upweighting stage they trained a new model with the hard samples upweighted, so that the117August 7, 2022
DRAFT
model could learn all samples equally well. As the process itself suggests, JTT is a reweighting
algorithm.
Finally, Zhai et al. [155] argued that DRO-based methods are very sensitive to outliers in the
training set because they upweight training samples with high losses and outliers tend to incur
high losses. They proposed the DORO algorithm which at each iteration removes the samples
with the highest losses, and then performs DRO on the rest of the samples. DORO is a reweighting
algorithm.

5.7

Proofs

Notations. In all of the proofs, for a matrix A, we will use kAk2 to denote its spectral norm
and kAkF to denote its Frobenius norm.

5.7.1

Proof of Theorem 11

To help our readers understand the proof more easily, we will first prove the result for static(t)reweighting algorithms where qi = qi for all t, and then we will prove the result for dynamic(t)reweighting algorithms that satisfy qi → qi as t → ∞.
Static reweighting algorithms
We first prove the result for all static reweighting algorithms such that mini qi = q ∗ > 0.
PnWe will2 use a standard optimization proof technique called smoothness. Denote A =i=1 kxi k2 . The empirical risk of the linear model f (x) = hθ, xi is
F (θ) =nX2
qi (x>i θ − yi )(5.18)

i=1

whose Hessian is∇2θ F (θ) = 2nXqi x i x >i(5.19)

i=1

So for any unit vector v ∈ Rd , we have (since qi ∈ [0, 1])v>∇2θ F (θ)v = 2nX2qi (x>
i v) ≤ 2

i=1

nX

qi kxi k22 ≤ 2A

(5.20)

i=1

which implies that F (θ) is 2A-smooth. Thus, we have the following upper quadratic bound: forany θ1 , θ2 ∈ Rd ,F (θ2 ) ≤ F (θ1 ) + h∇θ F (θ1 ), θ2 − θ1 i + A kθ2 − θ1 k22(5.21)√√√√
Denote g(θ(t) ) = Q(X > θ(t) − Y ) ∈ Rn where Q = diag( q1 , · · · , qn ). We can√2
see that g(θ(t) ) 2 = F (θ(t) ), so that ∇F (θ(t) ) = 2X Qg(θ(t) ). The update rule of a static
118

August 7, 2022
DRAFT
reweighting algorithm with gradient descent and the squared loss is:θ(t+1) = θ(t) − ηnXqi xi (f (t) (xi ) − yi ) = θ(t) − ηXpQg(θ(t) )(5.22)

i=1

Substituting θ1 and θ2 in (5.21) with θ(t) and θ(t+1) yields
F (θ(t+1) ) ≤ F (θ(t) ) − 2ηg(θ(t) )>p > > pp
2
Q X X Qg(θ(t) ) + A ηX Qg(θ(t) )(5.23)

2

Since x1 , · · · , xn are linearly independent, X√> X is a positive definite matrix. p
Denote the
√ ∗
>
min
(t)
(t)
smallest eigenvalue of X X by λ> 0. And
Qg(θ ) 2 ≥ q g(θ ) 2 = q ∗ F (θ(t) ),√√>so we have g(θ(t) )> Q X > X Qg(θ(t) ) ≥ q ∗ λmin F (θ(t) ). Thus,
p 2
2Qg(θ(t) ) 22
p 2
X Q F (θ(t) )F (θ(t+1) ) ≤ F (θ(t) ) − 2ηq ∗ λmin F (θ(t) ) + Aη 2 X≤ F (θ(t) ) − 2ηq ∗ λmin F (θ(t) ) + Aη 2
∗ min

(t)

≤ F (θ ) − 2ηq λ(t)F (θ ) + Aη2

F
kXk2F F (θ(t) )(5.24)= (1 − 2ηq ∗ λmin + A2 η 2 )F (θ(t) )∗ minLet η0 = q Aλ 2 . For any η ≤ η0 , we have F (θ(t+1) ) ≤ (1 − ηq ∗ λmin )F (θ(t) ) for all t,p∗ min pwhich implies that limt→∞ F (θ(t) ) = 0. Moreover, F (θ(t+1) ) ≤ (1 − ηq λ2 ) F (θ(t) ) due to√1 − x ≤ 1 − x/2.
The convergence in F (θ) implies the convergence in θ. This is because2

θ(t+1) − θ(t) 2 = η 2 X
≤η

2

p
p 2
2
2
Qg(θ(t) ) ≤ η 2 X Q
g(θ(t) ) 2

kXk2F

(t)

2
2

2

F

(5.25)

2A∗
q λmin(5.26)(t)g(θ ) 2 = Aη F (θ )which implies that for any η ≤ η0 ,∞X

θ(t+1) − θ(t) 2 ≤

p

t=T

Aη 2

∞ q
X
F (θ(t) ) ≤t=Tq
F (θ(T ) )P(t+1)
Therefore, limT →∞ ∞− θ(t) 2 = 0, which means that θ(t) converges, and it cont=T θverges to some interpolator.
Dynamic reweighting algorithms
Now we prove the result for all dynamic reweighting algorithms satisfying Assumption1. By
Assumption 1, for any  > 0, there exists t such that for all t ≥ t and all i,(t)qi ∈ (qi − , qi + )
119

(5.27)

August 7, 2022
DRAFT
(t)

This is because for all i, there exists ti such that for all t ≥ ti , qi ∈ (qi − , qi + ). Then,
we can define t = max{t1 , · · · , tn }. Denote the largest and smallest eigenvalues of X > X byλmax and λmin , and ∗because
X is full-rank, we have λmin > 0. Select and a fix an  such thatq ∗ (q λmin )2
0 <  < max{ 3 , 12λmax 2 }, and then t is also fixed.
We still denote Q = diag(q1 , · · · , qn ). When t ≥ t , the update rule of a dynamic reweighting
algorithm with gradient descent and the squared loss is:> (t)θ(t+1) = θ(t) − ηXQ(t)−Y)
 (X θ
(t)(5.28)(t)where Q = Q(t) , and we use the subscript  to indicate that Q − Q < . Then, note that
2
q
p
(t)
(t)
(t) √
we can rewrite Q as Q = Q3 · Q for all  < q ∗ /3. This is because qi + < (qi + 3)qipand qi −  > (qi − 3)qi for all  < qi /3, and qi ≥ q ∗ . Thus, we have
θ

(t+1)

=θ

(t)

q
q
p
(t)
(t)
(t)
(t)
− ηX Q3 g(θ ) where Q = Q3 · Q(5.29)

Again, substituting θ1 and θ2 in (5.21) with θ(t) and θ(t+1) yields
F (θ

(t+1)

q
2
p > > q (t) (t)(t)(t)) ≤ F (θ ) − 2ηg(θ )Q X X Q3 g(θ ) + A ηX Q3 g(θ )(5.30)
(t) >

(t)

2

Then, note that
q

p
p > >
(t)
Q X X
Q3 − Q g(θ(t) )
g(θ )
q

p
p > >
2
(t)
Q X X
Q3 − Qg(θ(t) ) 2
≤
2
q
p
p
2
(t)
≤
X >X 2Q
Q3 − Qg(θ(t) ) 2
2
2
√
(t)
max
≤λ
3F (θ )(t) >(5.31)

where the last step comes from the following fact: for all  < qi /3,
p
p
√
√
√
√
qi + 3 − qi ≤ 3 andqi − qi − 3 ≤ 3

(5.32)

And as proved before, we also haveg(θ(t) )>p>

Q X >Xp
Qg(θ(t) ) ≥ q ∗ λmin F (θ(t) )(5.33)∗ min 2Since  ≤ (q12λλmax)2 , we havep > > q (t) (t)
√ 
1
g(θ )
Q X X Q3 g(θ ) ≥ q ∗ λmin − λmax 3 F (θ(t) ) ≥ q ∗ λmin F (θ(t) ) (5.34)
2
(t) >

120

August 7, 2022
DRAFT
Thus,
F (θ

(t+1)

(t)

∗ min

) ≤ F (θ ) − ηq λ(t)F (θ ) + Aη

2

q
X

2
(t)
Q3

2

g(θ(t) ) 2
2≤ (1 − ηq ∗ λmin + A2 η 2 (1 + 3))F (θ(t) )(5.35)≤ (1 − ηq ∗ λmin + 2A2 η 2 )F (θ(t) )
∗ min

λ
(t+1)
for all  < 1/3. Let η0 = q 4A) ≤ (1 − ηq ∗ λmin /2)F (θ(t) ) for
2 . For any η ≤ η0 , we have F (θall t ≥ t , which implies that limt→∞ F (θ(t) ) = 0. As before, we can prove that the convergencein F (θ) implies the convergence in θ. Thus, θ converges to some interpolator.

5.7.2

Proof of Theorem 14

Note that the first l layers (except the output layer) of the original NTK formulation and our new
formulation are the same, so we still have the following proposition:
Proposition 19 (Proposition 1 in Jacot et al. [60]). If σ is Lipschitz and dl → ∞ for l = 1, · · · , Lsequentially, then for all l = 1, · · · , L, the distribution of a single element of hl converges inprobability to a zero-mean Gaussian process of covariance Σl that is defined recursively by:
1 > 0
x x + β2d0Σl (x, x0 ) = Ef [σ(f (x))σ(f (x0 ))] + β 2

Σ1 (x, x0 ) =(5.36)

where f is sampled from a zero-mean Gaussian process of covariance Σ(l−1) .
Now we show that for an infinitely wide neural network with L ≥ 1 hidden layers, Θ(0)converges in probability to the following non-degenerated deterministic limiting kernel
Θ = Ef ∼ΣL [σ(f (x))σ(f (x0 ))] + β 2(5.37)L√ σ(hL ) + βbL . We can see that for any parameter θiConsider the output layer hL+1 = W
d˜

before the output layer,
W L>∇θi hL+1 = diag(σ̇(hL )) √ ∇θi hL = 0
dL(5.38)

And for W L and bL , we have
1
∇W L hL+1 = √ σ(hL )dL

and∇bL hL+1 = βThen we can achieve (5.37) by the law of large numbers.
121(5.39)

August 7, 2022
DRAFT

5.7.3

Proof of Theorem 15

We will use the following short-hand in the proof:
 (t)
(t)

 g(θ ) = f (X) − YJ(θ(t) ) = ∇θ f (X; θ(t) ) ∈ Rp×n
Θ(t) = J(θ(t) )> J(θ(t) )(5.40)(t)

For any  > 0, there exists t such that for all t ≥ tq
and all i, qi ∈ (qi − , qi + ). Like what
(t)
(t) √
we have done in (5.29), we can rewrite Q(t) = Q = Q3 · Q, where Q = diag(q1 , · · · , qn ).
The update rule of a reweighting algorithm with gradient descent and the squared loss for thewide neural network is:θ(t+1) = θ(t) − ηJ(θ(t) )Q(t) g(θ(t) )(5.41)
and for t ≥ t , it can be rewritten as
θ

(t+1)

=θ

(t)

q
hp
i
(t)
Qg(θ(t) )− ηJ(θ ) Q3
(t)(5.42)First, we will prove the following theorem:
Theorem 20. There exist constants M > 0 and 0 > 0 such that for all  ∈ (0, 0 ], η ≤ η ∗ andany δ > 0, there exist R0 > 0, D̃ > 0 and B > 1 such that for any d˜ ≥ D̃, the following (i)and (ii) hold with probability at least (1 − δ) over random initialization when applying gradientdescent with learning rate η:
1. For all t ≤ t , there isg(θ(t) ) 2 ≤ B t R0
tX

(5.43)

θ(j) − θ(j−1) 2 ≤ ηM R0

j=1

t
X
j=1

B j−1 <

M B t R0
B−1(5.44)

2. For all t ≥ t , we havepQg(θ(t) )≤2

tX
j=t +1

θ

(j)

−θ

(j−1)

ηq ∗ λmin
1−
3

t−t

B t R0


j−t
t
X
ηq ∗ λmin≤ η 1 + 3M B R01−
2
3
j=t +1√
3 1 + 3M B t R0<q ∗ λmin√(5.45)

t(5.46)

Proof. The proof is based on the following lemma:
Lemma 21 (Local Lipschitzness of the Jacobian). Under Assumption 2, there is a constantM > 0 such that for any C0 > 0 and any δ > 0, there exists a D̃ such that: If d˜ ≥ D̃, then with122August 7, 2022
DRAFT
probability at least (1 − δ) over random initialization, for any x such that kxk2 ≤ 1,M∇θ f (x; θ) − ∇θ f (x; θ̃) ≤ p
θ − θ̃

4

2
2

˜

d



k∇θ f (x; θ)k2 ≤ M,∀θ, θ̃ ∈ B(θ(0) , C0 )Mθ − θ̃
J(θ) − J(θ̃) ≤ p

4

2
F
˜

d



kJ(θ)kF ≤ M(5.47)

where B(θ(0) , R) = {θ : θ − θ(0) 2 < R}.
The proof can be found in Appendix 5.7.4. Note that for any x, f (0) (x) = βbL where bL issampled from the standard Gaussian distribution. Thus, for any δ > 0, there exists a constant R0such that with probability at least (1 − δ/3) over random initialization,g(θ(0) ) 2 < R0

(5.48)

And by Theorem 14, there exists D2 ≥ 0 such that for any d˜ ≥ D2 , with probability at least(1 − δ/3),q ∗ λmin(5.49)
Θ − Θ(0) F ≤3∗ min 2λ)∗
2Let M bethe constant in Lemma 21. Let 0 = (q108M
4 . Let B = 1 + η M , and C0 =√B t  R0M B t  R0
+ 3 1+3M. By Lemma 21, there exists D1 > 0 such that with probability at leastB−1q ∗ λmin(1 − δ/3), for any d˜ ≥ D1 , (5.47) is true for all θ, θ̃ ∈ B(θ(0) , C0 ).
By union bound, with probability at least (1 − δ), (5.47), (5.48) and (5.49) are all true. Now
we assume that all of them are true, and prove (5.43) and (5.44) by induction. (5.43) is true fort = 0 due to (5.48), and (5.44) is always true for t = 0. Suppose (5.43) and (5.44) are true for t,then for t + 1 we haveθ(t+1) − θ(t) 2 ≤ η J(θ(t) )Q(t) 2 g(θ(t) ) 2 ≤ η J(θ(t) )Q(t) F g(θ(t) ) 2≤ η J(θ(t) ) F g(θ(t) ) 2 ≤ M ηB t R0(5.50)

So (5.44) is also true for t + 1. And we also haveg(θ(t+1) ) 2 = g(θ(t+1) ) − g(θ(t) ) + g(θ(t) ) 2= J(θ̃(t) )> (θ(t+1) − θ(t) ) + g(θ(t) )2= −ηJ(θ̃(t) )> J(θ(t) )Q(t) g(θ(t) ) + g(θ(t) )
2
(t) >

(t)

(t)

(t)

≤ I − ηJ(θ̃ ) J(θ )Qg(θ ) 22
≤ 1 + ηJ(θ̃(t) )> J(θ(t) )Q(t)g(θ(t) ) 22≤ 1 + η J(θ̃(t) )J(θ(t) ) F g(θ(t) ) 2
∗

F
(t)

2

≤ (1 + η M ) g(θ ) 2 ≤ B t+1 R0
123(5.51)

August 7, 2022
DRAFT√
Therefore, (5.43) and (5.44) are true for all t ≤ t , which implies that
Qg(θ(t ) ) 2 ≤g(θ(t ) ) 2 ≤ B t R0 , so (5.45) is true for t = t . And (5.46) is obviously true for t = t . Now, let
us prove (ii) by induction. Note that when t ≥ t , we have the alternative update rule (5.42). If(5.45) and (5.46) are true for t, then for t + 1, there is

θ

(t+1)

−θ

(t)

q
(t)
≤ η J(θ ) Q3
2p
Qg(θ(t) )

(t)

2

2

q
(t)
≤ η J(θ ) Q3

p(t)

Qg(θ(t) )
2

F

t−t
p
√
√
ηq ∗ λmin(t)(t)≤ η 1 + 3 J(θ ) F
Qg(θ ) ≤ M η 1 + 3 1 −B t R0
3
2(5.52)
So (5.46) is true for t + 1. And we also havepppp
Qg(θ(t+1) ) =Qg(θ(t+1) ) − Qg(θ(t) ) + Qg(θ(t) )22pp(t) > (t+1)
(t)
(t)
=
QJ(θ̃ ) (θ− θ ) + Qg(θ )2pp= −η QJ(θ̃(t) )> J(θ(t) )Q(t) g(θ(t) ) + Qg(θ(t) )2(5.53)qpp(t)≤ I − η QJ(θ̃(t) )> J(θ(t) ) Q3
Qg(θ(t) )


2

2

q
p
(t)
(t) >
(t)
≤ I − η QJ(θ̃ ) J(θ ) Q3tηq ∗ λmin
1−
R0
3
2

where θ̃(t) is some linear interpolation between θ(t) and θ(t+1) . Now we prove thatqp
ηq ∗ λmin
(t)
(t) >
(t)
I − η QJ(θ̃ ) J(θ ) Q3 ≤ 1 −3
2(5.54)For any unit vector v ∈ Rn , we haveppppv > (I − η QΘ Q)v = 1 − ηv > QΘ Qv(5.55)√√√√Qv 2 ∈ [ q ∗ , 1], so for any η ≤ η ∗ , v > (I − η QΘ Q)v ∈ [0, 1 − ηλmin q ∗ ], which√√
implies that I − η QΘ Q 2 ≤ 1 − ηλmin q ∗ . Thus,pp
QJ(θ̃(t) )> J(θ(t) ) Q
2
p
p
p
p
p
p
Q(Θ − Θ(0) ) Q + η
Q(J(θ(0) )> J(θ(0) ) − J(θ̃(t) )> J(θ(t) )) Q
≤ I − η QΘ Q + η
2
2
2
p
p
p
p
min ∗
(0)
(0) >
(0)
(t) >
(t)
≤1 − ηλ q + ηQ(Θ − Θ ) Q + η
Q(J(θ ) J(θ ) − J(θ̃ ) J(θ )) Q
I −η

F
min ∗

≤1 − ηλq +η Θ−Θ

F

(0) >

(0)

(0)

(t) >

(t)

+ η J(θ ) J(θ ) − J(θ̃ ) J(θ )
F

∗ min
2 
ηq
λ
ηM
ηq ∗ λmin
(t)
(0)
(t)
(0)
≤1 − ηλmin q ∗ +
+ p
θ
−
θ
+
θ̃
−
θ
≤
1
−
2
4
3
2
2
d˜
F

124

(5.56)

August 7, 2022
DRAFT




4
2C
0
, which implies thatfor all d˜ ≥ max D1 , D2 , 12M
∗
min
q λ
I −η

p

q
(t)
QJ(θ̃ ) J(θ ) Q3
(t) >

∗ min

(t)

2
q

p
p
(t)
(t) >
(t)
+ η QJ(θ̃ ) J(θ )Q3 − Q

ηq λ
2
∗ min
√
ηq λ
ηq ∗ λmin+ ηM 2 3 ≤ 1 −
≤1 −
2
3

≤1 −

(5.57)
2(due to (5.32))

for all  ≤ 0 . Thus, (5.45) is also true for t + 1.In conclusion, (5.45)and (5.46) are true with
4

2C
0
.
probability at least (1 − δ) for all d˜ ≥ D̃ = max D1 , D2 , 12M
∗ min
q λ

Returning back to the proof of Theorem 15. Choose and fix an  such that  <2q ∗ λmin1
min{0 , 3 3λmax +q∗ λmin }, where 0 is defined by Theorem 20. Then, t is also fixed. Thereexists D̃ ≥ 0 such that for any d˜ ≥ D̃, with probability at least (1 − δ), Theorem 20 and Lemma
21 are true andq ∗ λminΘ − Θ(0) F ≤(5.58)
3
which immediately implies thatΘ(0) 2 ≤ kΘk2 + Θ − Θ(0) F ≤ λmax +√tq ∗ λmin3(5.59)

t

B R0
B R0
We still denote B = 1 + η ∗ M 2 and C0 = MB−1
+ 3 1+3M. Theorem 20 ensures thatq ∗ λmin(t)(0)
for all t, θ ∈ B(θ , C0 ). Then we have
p
p
p
p
p
p
Q(Θ − Θ(0) ) Q
I − η QΘ(0) Q ≤ I − η QΘ Q + η2
2
2(5.60)
∗ min
∗ min
ηq
λ
2ηq
λ
≤ 1 − ηλmin q ∗ +=1−
3
3

so it follows that
q
p
(t)
(0)
I − η QΘQ3pp
≤ I − η QΘ(0) Q
2

2


q
p
p
(t)
(0)
+ η QΘ
Q3 − Q22ηq λq ∗ λmin √≤1−+ η(λmax +
) 3
3
3

2
q ∗ λmin, there isThus, for all  < 13 3λmax+q ∗ λmin
∗ min

I −η

p

q
(t)
QΘ
Q3
(0)

≤1−
2

ηq ∗ λmin
3(5.61)(5.62)The update rule of the reweighting algorithm for the linearized neural network is:
(t+1)

θlin

(t)

= θlin − ηJ(θ(0) )Q(t) glin (θ(t) )125(5.63)

August 7, 2022
DRAFT
where we use the subscript “lin” to denote the linearized neural network, and with a slight abuse(t)of notion denote glin (θ(t) ) = g(θlin ).
First, let us consider the training data X. Denote ∆t = glin (θ(t) ) − g(θ(t) ). We have(glin (θ(t+1) ) − glin (θ(t) ) = −ηJ(θ(0) )> J(θ(0) )Q(t) glin (θ(t) )(5.64)g(θ(t+1) ) − g(θ(t) ) = −ηJ(θ̃(t) )> J(θ(t) )Q(t) g(θ(t) )where θ̃(t) is some linear interpolation between θ(t) and θ(t+1) . Thus,
h
i
(t) >
(t)
(0) >
(0)
∆t+1 − ∆t =η J(θ̃ ) J(θ ) − J(θ ) J(θ ) Q(t) g(θ(t) )(5.65)− ηJ(θ(0) )> J(θ(0) )Q(t) ∆t
By Lemma 21, we haveJ(θ̃(t) )> J(θ(t) ) − J(θ(0) )> J(θ(0) )F>≤ J(θ̃(t) ) − J(θ(0) ) J(θ(t) ) + J(θ(0) )> J(θ(t) ) − J(θ(0) ) F(5.66)

F≤2M 2 C0 d˜−1/4which implies that for all t < t ,hik∆t+1 k2 ≤ I − ηJ(θ(0) )> J(θ(0) )Q(t) ∆t 2 + η J(θ̃(t) )> J(θ(t) ) − J(θ(0) )> J(θ(0) ) Q(t) g(θ(t) )
(0) >

(0)

≤ I − ηJ(θ ) J(θ )Q

(t) >

(t)
F

(t)

(0) >

(t)

(0)

k∆t k2 + η J(θ̃ ) J(θ ) − J(θ ) J(θ )F

g(θ ) 2

≤ (1 + ηM 2 ) k∆t k2 + 2ηM 2 C0 B t R0 d˜−1/4≤ B k∆t k + 2ηM 2 C0 B t R0 d˜−1/4
2(5.67)
Therefore, we haveB −(t+1) k∆t+1 k2 ≤ B −t k∆t k2 + 2ηM 2 C0 B −1 R0 d˜−1/4(5.68)

Since ∆0 = 0, it follows that for all t ≤ t ,k∆t k2 ≤ 2tηM 2 C0 B t−1 R0 d˜−1/4
and particularly we have
p
Q∆t

2

≤ k∆t k2 ≤ 2t ηM 2 C0 B t −1 R0 d˜−1/4(5.69)(5.70)For t ≥ t , we have the alternative update rule (5.42). Thus,
iq
hp
i
p
p
p h (t) >
(t)
(t)
(0) >
(0)
Q∆t+1 − Q∆t =η Q J(θ̃ ) J(θ ) − J(θ ) J(θ )Q3
Qg(θ(t) )q(5.71)
hp
i
p
(t)
(0) >
(0)
− η QJ(θ ) J(θ ) Q3
Q∆t
126

2

August 7, 2022
DRAFT
q
q
√
√
(t)
(t)
Let A = I − η QJ(θ(0) )> J(θ(0) ) Q3 = I − η QΘ(0) Q3 . Then, we have
iq
p

p
p
p h (t) >
(t)
(t)
(0) >
(0)
Q∆t+1 = A Q∆t + η Q J(θ̃ ) J(θ ) − J(θ ) J(θ )Q3
Qg(θ(t) )(5.72)
ηq ∗ λmin
Let γ = 1 − 3 < 1. Combining with Theorem 20 and (5.62), the above leads toiqp h (t) >
p
p
p
(t)
(t)
(0) >
(0)
Q∆t+1 ≤ kAk2
Q∆t + ηQ J(θ̃ ) J(θ ) − J(θ ) J(θ )Q3
Qg(θ(t) )
2

2

2

p
√
≤γ
Q∆t + η J(θ̃(t) )> J(θ(t) ) − J(θ(0) )> J(θ(0) )1 + 3γ t−t B t R0
F
2
p
√
≤γ
Q∆t + 2ηM 2 C0 1 + 3γ t−t B t R0 d˜−1/4
2(5.73)
This implies thatpp√
(5.74)
γ −(t+1)
Q∆t+1 ≤ γ −tQ∆t + 2ηM 2 C0 1 + 3γ −1−t B t R0 d˜−1/4
2

2

Combining with (5.70), it implies that for all t ≥ t ,
h
i
p
√
t−t
2
t
−1
−1
Q∆t ≤ 2γ ηM C0 B R0 t B + 1 + 3γ (t − t ) d˜−1/4(5.75)

2(t)

Next, we consider an arbitrary test point x such that kxk2 ≤ 1. Denote δt = flin (x) − f (t) (x).
Then we have
( (t+1)(t)flin (x) − flin (x) = −η∇θ f (x; θ(0) )> J(θ(0) )Q(t) glin (θ(t) )(5.76)f (t+1) (x) − f (t) (x) = −η∇θ f (x; θ̃(t) )> J(θ(t) )Q(t) g(θ(t) )which yieldshiδt+1 − δt =η ∇θ f (x; θ̃(t) )> J(θ(t) ) − ∇θ f (x; θ(0) )> J(θ(0) ) Q(t) g(θ(t) )
(0) >

(0)

(5.77)(t)− η∇θ f (x; θ ) J(θ )Q ∆t
For t ≤ t , we have
t−1 h
i
X
kδt k2 ≤η∇θ f (x; θ̃(s) )> J(θ(s) ) − ∇θ f (x; θ(0) )> J(θ(0) ) Q(s)
s=0

+η
≤η

t−1
X

∇θ f (x; θ̃(s) )> J(θ(s) ) − ∇θ f (x; θ(0) )> J(θ(0) )

s=0
t−1
X

g(θ(s) ) 2∇θ f (x; θ(0) )> J(θ(0) )Q(s) 2 k∆s k2

s=0
t−1
X

+η

2

F

g(θ(s) ) 2∇θ f (x; θ(0) ) 2 J(θ(0) ) F k∆s k2

s=0≤2ηM 2 C0 d˜−1/4

t−1
X
s=0

B s R0 + ηM 2

t−1
X
(2sηM 2 C0 B s−1 R0 d˜−1/4 )
s=0

127

(5.78)

2

August 7, 2022
DRAFT
So we can see that there exists a constant C1 such that kδt k2 ≤ C1 d˜−1/4 . Then, for t > t , we
have
t−1 h
iq
X
(s)
(s) >
(s)
(0) >
(0)
kδt k2 − kδt k2 ≤η∇θ f (x; θ̃ ) J(θ ) − ∇θ f (x; θ ) J(θ )Q3

Qg(θ(s) )
2

2

s=t

+η

p

t−1
X

q
(s)
∇θ f (x; θ ) J(θ ) Q3
(0) >

p

(0)

Q∆s

2

s=t

2

t−1
X
√
≤2ηM 2 C0 d˜−1/4 1 + 3
γ s−t B t R0
s=t

+ ηM

2

√

1 + 3

t−1 
X

2γ

s−t

h
i

√
−1
−1
−1/4
˜
ηM C0 B R0 t B + 1 + 3γ (s − t ) d
2

t

s=t

(5.79)
Note that t=0 tγ is finite as long as γ ∈ (0, 1). Therefore, there is a constant C such thatfor any t, kδt k2 ≤ C d˜−1/4 with probability at least (1 − δ) for any d˜ ≥ D̃.
P∞

5.7.4

t

Proof of Lemma 21

We will use the following theorem regarding the eigenvalues of random Gaussian matrices:
Theorem 22 (Corollary 5.35 in Vershynin [132]). If A ∈ Rp×q is a random matrix whose entries
are independent standard normal random variables, then for every t ≥ 0, with probability at least1 − 2 exp(−t2 /2),
√

p−

√

q − t ≤ λmin (A) ≤ λmax (A) ≤

√

p+

√
q+t

(5.80)

By this theorem, and also note that W L is a vector, we can see that for any δ, there exist D̃ > 0
and M1 > 0 such that if d˜ ≥ D̃, then with probability at least (1 − δ), for all θ ∈ B(θ(0) , C0 ), we
have
p
p
4
L
l
˜
and
W 2 ≤ C0 ≤ 3 d˜
(5.81)
W 2 ≤ 3 d (∀0 ≤ l ≤ L − 1)
as well as
βbl 2 ≤ M1pd˜(∀l = 0, · · · , L)

(5.82)

Now we assume that (5.81) and (5.82) are true. Then, for any x such that kxk2 ≤ 1,p1
1
3
h1 2 = √ W 0 x + βb0 ≤ √W 0 2 kxk2 + βb0 2 ≤ ( √ + M1 ) d˜
d0
d0
d0
2
1
1
hl+1 2 = p W l xl + βbl ≤ p W l 2 xl 2 + βbl 2(∀l ≥ 1)d˜d˜
2pxl 2 = σ(hl ) − σ(0l ) + σ(0l ) 2 ≤ L0 hl 2 + σ(0) d˜(∀l ≥ 1)128

(5.83)

August 7, 2022
DRAFT
lwhere L0 is the Lipschitz constant of σ and, σ(0)) ∈ Rdl . By induction, therep σ(0 ) = (σ(0), · · ·p
exists an M2 > 0 such that xl 2 ≤ M2 d˜ and hl 2 ≤ M2 d˜ for all l = 1, · · · , L.
l>

√ αl+1
Denote αl = ∇hl f (x) = ∇hl hL+1 . For all l = 1, · · · , L, we have αl = diag(σ̇(hl )) Wd˜

where σ̇(x) ≤ L0 for all x ∈ R since σ is L0 -Lipschitz, α

L+1= 1 and

α

L
2

=

L>

diag(σ̇(hL )) W√

3
L0 . Then, we can easily prove by induction that there exists an
≤ p
4
d˜
2
p
4
M3 > 1 such that αl 2 ≤ M3 / d˜ for all l = 1, · · · , L (note that this is not true for L + 1
because αL+1 = 1).
p4
˜For l = 0, ∇W 0 f (x) = √1d0 x0 α1> , so k∇W l f (x)k2 ≤ √1d0 kx0 k2 kα1 k2 ≤ √1d0 M3 / d.
And for any l = 1, · · · , L, ∇W l f (x) = √1 xl αl+1 , so k∇W l f (x)k2 ≤ √1 xl 2 αl+1 2 ≤
d˜

d˜

d˜

≤ M3 ; and since d˜ ≥ 1, there is αl 2 ≤ M3 forM2 M3 . (Note that if M3 > 1, then α2l ≤ L.) Moreover, for l = 0, · · · , L, ∇bl f (x) = βαl+1 , so k∇bl f (x)k2 ≤ βM3 .√Thus, if (5.81)
and (5.82) are true, then there exists an M4 > 0, such that k∇θ f (x)k2 ≤ M4 / n. And since
kxi k2 ≤ 1 for all i, so kJ(θ)kF ≤ M4 .
L+1

Next, we consider the difference in ∇θ f (x) between θ and θ̃. Let f˜, W̃ , b̃, x̃, h̃, α̃ be thefunction and the values corresponding to θ̃. There ish1 − h̃11= √ (W 0 − W̃ 0 )x + β(b0 − b̃0 )
2
d0
2


1
1
0
0
0
0
≤√
W − W̃
kxk2 + β b − b̃
≤ √ +βθ − θ̃
222d0d0hl+1 − h̃l+1

1
1= p W l (xl − x̃l ) + p (W l − W̃ l )x̃l + β(bl − b̃l )2(5.84)d˜
d˜
2
1
1≤ p W l 2 xl − x̃l 2 + p W l − W̃ lx̃l 2 + β bl − b̃l
2
2
d˜
d˜
≤ 3 xl − x̃l 2 + (M2 + β) θ − θ̃

xl − x̃l 2 = σ(hl ) − σ(h̃l )2≤ L0 hl − h̃l(∀l ≥ 1)2(∀l ≥ 1)
2

By induction, there exists an M5 > 0 such that xl − x̃l 2 ≤ M5 θ − θ̃129

for all l.
2August 7, 2022
DRAFT
For αl , we have αL+1 = α̃L+1 = 1, and for all l ≥ 1,
W̃ l>
W l>
αl − α̃l 2 = diag(σ̇(hl )) p αl+1 − diag(σ̇(h̃l )) p α̃l+1
d˜
d˜
2
W l>
≤ diag(σ̇(hl )) p (αl+1 − α̃l+1 )d˜+ diag(σ̇(hl ))2(W l − W̃ l )> l+1
p
α̃
d˜

2

l>

W̃
+ diag((σ̇(hl ) − σ̇(h̃l ))) p α̃l+1
d˜
2


l+1
l+1
−1/2
−1/4
≤ 3L0 α − α̃
+ M3 L0 d˜+ 3M3 M5 L1 d˜
θ − θ̃
2
2(5.85)
L+1
where L1 is the Lipschitz constant of σ̇. Particularly, for l = L, though α̃
= 1, since
L
1/4
l
˜
W̃
≤ 3d , (5.85) is still true. By induction, there exists an M6 > 0 such that α − α̃l 2 ≤2M6pθ − θ̃ for all l ≥ 1 (note that this is also true for l = L + 1).
42
d˜Thus, if (5.81) and (5.82) are true, then for all θ, θ̃ ∈ B(θ(0) , C0 ), any x such that kxk2 ≤ 1,
we have1∇W 0 f (x) − ∇W̃ 0 f˜(x) = √xα1> − xα̃1> 22d01
α1 − α̃1 2≤√(5.86)d0
1 M6≤√ pθ − θ̃
2
d0 4 d˜and for l = 1, · · · , L, we have
∇W l f (x) − ∇W̃ l f˜(x)

2

1
=p

d˜
1
≤p
d˜
≤

xl αl+1> − x̃l α̃l+1> 2
xl 2 αl+1 − α̃l+1 2 + xl − x̃l 2 α̃l+1 2

M2 M6 M5 M3
p
+ p
4
d˜
d˜



(5.87)!
θ − θ̃
2

Moreover, for any l = 0, · · · , L, there is∇bl f (x) − ∇b̃l f˜(x)βM6= β αl+1 − α̃l+1 2 ≤ p
θ − θ̃
422
d˜(5.88)Overall, we can see that there exists a constant M7 > 0 such that ∇θ f (x) − ∇θ̃ f˜(x)
Mp
7
√ 4 ˜
n·
d

θ − θ̃ , so that J(θ) − J(θ̃)
2

F

M7
≤ p
θ − θ̃ .
4
2
d˜

130

≤
2

August 7, 2022
DRAFT

5.7.5

Proof of Theorem 16

Let η1 = min{η0 , η ∗ }, where η0 is defined in Corollary 13 and η ∗ is defined in Theorem 15. Let
(t)
(t)
(t)
flin (x) and flinERM (x) be the linearized neural networks of f (t) (x) and fERM (x), respectively. By
Theorem 15, for any δ > 0, there exists D̃ > 0 and a constant C such that

(t)
(t)

˜−1/4

 sup flin (x) − f (x) ≤ C dt≥0(5.89)
(t)
(t)

−1/4
˜

 sup flinERM (x) − fERM (x) ≤ C dt≥0

By Corollary 13, we have
lim flin (x) − flinERM (x) = 0

(t)

(t)

(5.90)(t)(5.91)

t→∞

Summing the above yields
lim sup f (t) (x) − fERM (x) ≤ 2C d˜−1/4t→∞which is the result we want.

5.7.6

Proof of Theorem 17

To minimize the regularized risk (5.12) with gradient descent, the update rule is
θ

(t+1)

=θ

(t)

n
X
(t)
−η
qi ∇θ `(f (t) (xi ), yi ) − ηµ(θ(t) − θ(0) )(5.92)

i=1

We can see that under the new rule, θ(t) − θ(0) ∈ span(∇θ f (0) (x1 ), · · · , ∇θ f (0) (xn )) is stilltrue for all t. Let θ∗ be the interpolator in span(∇θ f (0) (x1 ), · · · , ∇θ f (0) (xn )), then the empirical
Pn
2
1
1
∗
(0)
(xi )i2 = 2n
∇θ f (0) (X)> (θ − θ∗ ) 2 . Thus, there exists
risk of θ is 2n
i=1 hθ − θ , ∇θ f
T > 0 such that for any t ≥ T ,2∇θ f (0) (X)> (θ(t) − θ∗ ) 2 ≤ 2n

(5.93)

Let the smallest singular value of √1n ∇θ f (0) (X) be smin , and we have smin > 0. Note that thecolumn space of ∇θ f (0) (X) is exactly span(∇θ f (0) (x1 ), · · · , ∇θ f (0) (xn )). Define H ∈ Rp×n
such that its columns form an orthonormal basis of this subspace, then there exists G ∈ Rn×n
such that ∇θ f (0) (X) = HG, and the smallest singular value of √1n G is also smin . Sinceθ√(t) − θ(0) is also in this subspace, there exists v ∈ Rn√such that θ(t) − θ∗ = Hv. Then we have2
, which implies
2n ≥ G> H > Hv 2 = G> v 2 . Thus, kvk2 ≤ smin
θ

(t)

√
2
− θ 2 ≤ min
s
∗

131

(5.94)

August 7, 2022
DRAFT
By Corollary 13, if we minimize the unregularized risk with ERM, then θ always converges tothe interpolator θ∗ . So for any t ≥ T and any test point x such that kxk2 ≤ 1,
√
M
2
0
(t)
(t)
|flinreg (x) − flinERM (x)|= |hθ(t) − θ∗ , ∇θ f (0) (x)i|≤(5.95)
mins
which implies (5.14).

5.7.7

Proof of Theorem 18

First of all, with some simple linear algebra analysis, we can prove the following proposition:
Proposition 23. For any positive definite symmetric matrix H ∈ Rn×n , denote its largest andsmallest eigenvalues by λmax and λmin . Then, for any q ∈ Rn+ and Q = diag(q1 , · · · , qn ), HQ
has n positive eigenvalues that are all in [mini qi · λmin , maxi qi · λmax ].
Proof.
H is a positive definite symmetric matrix, so there exists A ∈ Rn×n such that
H = A> A, and A is full-rank. First, any eigenvalue of AQA> is also an eigenvalue of
A> AQ and vice versa, because for any eigenvalue λ of AQA> we have some v 6= 0 such that
AQA> v = λv. Multiplying both sides by A> on the left yields A> AQ(A> v) = λ(A> v)
which implies that λ is also an eigenvalue of A> AQ because A> v 6= 0 as λv 6= 0. We can prove
the other direction similarly.
Second, by condition we know that the eigenvalues of A> A are all in [λmin , λmax ] where
min
> >
min
λ
> 0, which, λmax ], which is equivalent√ implies√ for any unit vector v,> v >A Av ∈ [λto kAvk2 ∈ [ λmin , λmax ]. Thus, we have v A QAv ∈ [λmin mini qi , λmax maxi qi ], which
implies that the eigenvalues of A> QA are all in [λmin mini qi , λmax maxi qi ].
Thus, the eigenvalues of HQ = A> AQ are all in [λmin mini qi , λmax maxi qi ].
Now return back to the proof of Theorem 18. We still use the shorthand (5.40). With L2
penalty, the update rule of the reweighting algorithm for the neural network is:θ(t+1) = θ(t) − ηJ(θ(t) )Q(t) g(θ(t) ) − ηµ(θ(t) − θ(0) )(5.96)

And the update rule for the linearized neural network is:
(t+1)

θlin

(t)

(t)

= θlin − ηJ(θ(0) )Q(t) glin (θ(t) ) − ηµ(θlin − θ(0) )(5.97)First, we need to prove that there exists D0 such that for all d˜ ≥ D0 , supt≥0 θ(t) − θ(0) 2 isbounded with high probability. Denote at = θ(t) − θ(0) . By (5.96) we haveat+1 =(1 − ηµ)at − η[J(θ(t) ) − J(θ(0) )]Q(t) g(θ(t) )− ηJ(θ(0) )Q(t) [g(θ(t) ) − g(θ(0) )] − ηJ(θ(0) )Q(t) g(θ(0) )(5.98)

which implies
kat+1 k2 ≤ (1 − ηµ)I − ηJ(θ(0) )Q(t) J(θ̃(t) )>2

kat k2+ η J(θ(t) ) − J(θ(0) ) F g(θ(t) ) 2 + η J(θ(0) ) F g(θ(0) ) 2
132(5.99)

August 7, 2022
DRAFT
where θ̃(t) is some linear interpolation between θ(t) and θ(0) . Our choice of η ensures thatηµ < 1. Similar to (5.48), we can show that for any δ > 0, there exists a constant R0 > 0 suchthat with probability at least (1 − δ/3), g(θ(0) ) 2 < R0 . Let M be as defined in Lemma 21.
Denote A = ηM R0 , and let C0 = 4A
in Lemma 216 . By Lemma 21, there exists D1 such that forηµall d˜ ≥ D1 , with probability at least (1 − δ/3), (5.47) is true.
Now we prove by induction that kat k2 < C0 . It is true for t = 0, so we need to prove that if
kat k2 < C0 , then kat+1 k2 < C0 .
For the first term on the right-hand side of (5.99), we have
(1 − ηµ)I − ηJ(θ(0) )Q(t) J(θ̃(t) )>≤(1 − ηµ) I −2ηJ(θ(0) )Q(t) J(θ(0) )>1 − ηµ2 (5.100)+ η J(θ(0) ) F J(θ̃(t) ) − J(θ(0) )F

Like what we have done before, we can show that all non-zero eigenvalues ofJ(θ(0) )Q(t) J(θ(0) )> are eigenvalues of J(θ(0) )> J(θ(0) )Q(t) . This is because for any λ 6= 0,
if J(θ(0) )Q(t) J(θ(0) )> v = λv, then J(θ(0) )> J(θ(0) )Q(t) (J(θ(0) )> v) = λ(J(θ(0) )> v), andJ(θ(0) )> v 6= 0 since λv 6= 0, so λ is also an eigenvalue of J(θ(0) )> J(θ(0) )Q(t) . On the other
hand, by Theorem 14, J(θ(0) )> J(θ(0) )Q(t) converges in probability to ΘQ(t) whose eigenvalues
are all in [0, λmax ] by Proposition 23. So there exists D2 such that for all d˜ ≥ D2 , with probability
at least (1 − δ/3), the eigenvalues of J(θ(0) )Q(t) J(θ(0) )> are all in [0, λmax + λmin ] for all t. Sinceη/(1 − ηµ) ≤ (λmin + λmax )−1 by our choice of η, we haveI−ηJ(θ(0) )Q(t) J(θ(0) )> ≤ 1
1 − ηµ2(5.101)On the other hand, we can use (5.47) since kat k2 < C0 , so J(θ(0) ) F J(θ̃(t) ) − J(θ(0) ) ≤
F
M2
˜
p
C
.
Therefore,
there
exists
D
such
that
for
all
d
≥
D
,
0
3
3
4
d˜
ηµ
(1 − ηµ)I − ηJ(θ(0) )Q(t) J(θ̃(t) )> ≤ 1 −(5.102)2
2
For the second term, we haveg(θ(t) ) 2 ≤ g(θ(t) ) − g(θ(0) ) 2 + g(θ(0) ) 2≤ J(θ̃(t) )2

θ(t) − θ(0) 2 + R0 ≤ M C0 + R0

(5.103)And for the third term, we have
η J(θ(0) ) F g(θ(0) ) 2 ≤ ηM R0 = A(5.104)ηµ 
ηM (M C0 + R0 )p
kat+1 k2 ≤ 1 −kat k2 ++A
4
2
d˜(5.105)Thus, we have

6

Note that Lemma 21 only depends on the network structure and does not depend on the update rule, so we canuse this lemma here.

133

August 7, 2022
DRAFT
So there exists D4 such that for all d˜ ≥ D4 , kat+1 k2 ≤ 1 − ηµkat k2 + 2A. This shows that2
if kat k2 < C0 is true, then kat+1 k2 < C0 will also be true.
In conclusion, by union bound, we have proved that for any δ > 0, with probability at least(1 − δ) for all d˜ ≥ D0 = max{D1 , D2 , D3 , D4 }, θ(t) − θ(0) 2 < C0 is true for all t. This also
implies that for C1 = M C0 + R0 , we have g(θ(t) ) 2 ≤ C1 for all t by (5.103).
(t)Second, let ∆t = θlin − θ(t) . Then we have∆t+1 − ∆t = η(J(θ(t) )Q(t) g(θ(t) ) − J(θ(0) )Q(t) glin (θ(t) ) − µ∆t )(5.106)

which impliesih∆t+1 = (1 − ηµ)I − ηJ(θ(0) )Q(t) J(θ̃(t) )> ∆t + η(J(θ(t) ) − J(θ(0) ))Q(t) g(θ(t) )(5.107)

By (5.102), with probability at least (1 − δ) for all d˜ ≥ D0 , we havek∆t+1 k2 ≤ (1 − ηµ)I − ηJ(θ(0) )Q(t) J(θ̃(t) )> k∆t k2 + η J(θ(t) ) − J(θ(0) ) F g(θ(t) ) 2
2

ηµ 
M
≤ 1−
k∆t k2 + η p
C0 C1
4
2
d˜
(5.108)Again, as ∆0 = 0, we can prove by induction that for all t,k∆t k2 <2M C0 C1 ˜−1/4dµ(5.109)For any test point x such that kxk2 ≤ 1, we have
(t)

(t)

(t)
freg
(x) − flinreg (x) = f (x; θ(t) ) − flin (x; θlin )(t)≤ f (x; θ(t) ) − flin (x; θ(t) ) + flin (x; θ(t) ) − flin (x; θlin )
(t)

(t)

(t)

(t)

(0)

≤ f (x; θ ) − flin (x; θ ) + ∇θ f (x; θ ) 2 θ

(t)

(t)
− θlin(5.110)2≤ f (x; θ ) − flin (x; θ ) + M k∆t k2
For the first term, note that(f (x; θ(t) ) − f (x; θ(0) ) = ∇θ f (x; θ̃(t) )(θ(t) − θ(0) )flin (x; θ(t) ) − flin (x; θ(0) ) = ∇θ f (x; θ(0) )(θ(t) − θ(0) )(5.111)

where θ̃(t) is some linear interpolation between θ(t) and θ(0) . Since f (x; θ(0) ) = flin (x; θ(0) ),f (x; θ(t) ) − flin (x; θ(t) ) ≤ ∇θ f (x; θ̃(t) ) − ∇θ f (x; θ(0) )2

M 2
θ(t) − θ(0) 2 ≤ p
C0 (5.112)
4d˜

Thus, we have shown that for all d˜ ≥ D0 , with probability at least (1 − δ) for all t and all x,
2M 2 C0 C1 ˜−1/4
(t)
(t)
2
freg (x) − flinreg (x) ≤ M C0 +d
= O(d˜−1/4 )(5.113)µ134

August 7, 2022
DRAFT
(t)

Given that R̂(flinreg ) <  for sufficiently large t, this also implies that
√
(t)
(t)
) = O(d˜−1/4  + d˜−1/2 )R̂(flinreg ) − R̂(freg(5.114)

So for a fixed , there exists D > 0 such that for all d ≥ D, for sufficiently large t,(t)(t)) <  ⇒ R̂(flinreg ) < 2R̂(freg(5.115)(t)(t)
sup flinERM (x) − fERM (x) = O(d˜−1/4 )(5.116)

By Theorem 15, we havet≥0Combining Theorem 17 with (5.113) and (5.116) derives(t)(t)(x) − fERM (x) = O(d˜−1/4 +lim sup freg√)

(5.117)t→∞Letting d˜ → ∞ leads to the result we need.

5.8

A note on the proofs in Lee et al. [82]We have mentioned that the proofs in Lee et al. [82], particularly the proofs of their Theorem 2.1
and Lemma 1 in their Appendix G, are flawed. In order to fix their proof, we change the network
initialization to (5.9). In this section, we will demonstrate what goes wrong in the proofs in Lee
et al. [82], and how we manage to fix the proof. For clarity, we are referring to the following
version of the paper: https://arxiv.org/pdf/1902.06720v4.pdf.
To avoid confusion, in this section we will still use the notations used in our paper.

5.8.1

Their problems

Lee et al. [82] claimed in their Theorem 2.1 that under the conditions of our Theorem 15, forany δ > 0, there exist D̃ > 0 and a constant C such that for any d˜ ≥ D̃, with probability at least(1 − δ), the gap between the output of a sufficiently wide fully-connected neural network and theoutput of its linearized neural network at any test point x can be uniformly bounded by(t)
sup f (t) (x) − flin (x) ≤ C d˜−1/2(claimed)(5.118)t≥0where they used the original NTK formulation and initialization in Jacot et al. [60]:( l(0)
Wl l
hl+1 = √Wi,j ∼ N (0, 1)
x + βbldl
and(∀l = 0, · · · , L)
(5.119)l(0)bi ∼ N (0, 1)
xl+1 = σ(hl+1 )135

August 7, 2022
DRAFT
where x0 = x and f (x) = hL+1 . However, in their proof in their Appendix G, they did not
directly prove their result for the NTK formulation, but instead they proved another result for thefollowing formulation which they called the standard formulation:(W l(0) ∼ N (0, 1 )hl+1 = W l xl + βbl
i,j
dl
and
(∀l = 0, · · · , L)
(5.120)l+1
l+1
 l(0)
x = σ(h )bi ∼ N (0, 1)
See their Appendix F for the definition of their standard formulation. In the original formulation, they also included two constants σw and σb for standard deviations, and for simplicity
we omit these constants here. Note that the outputs of the NTK formulation and the standard
formulation at initialization are actually the same. The only difference is that the norm of theweight W l and the gradient of the model output with respect to W l are different for all l.
In their Appendix G, they claimed that if a network with the standard formulation is trained
˜ where η isby minimizing the squared loss with gradient descent and learning rate η 0 = η/d,our learning rate in Theorem 15 and also their learning rate in their Theorem 2.1, then (5.118)
is true for this network, so it is also true for a network with the NTK formulation because thetwo formulations have the same network output. And then they claimed in their equation (S37)that applying learning rate η 0 to the standard formulation is equivalent to applying the following
learning rates
dl
1
l
η
and
ηbl =
η
(5.121)
ηW
=
dmax
dmax
to W l and bl of the NTK formulation, where dmax = max{d0 , · · · , dL }.
To avoid confusion, in the following discussions we will still use the NTK formulation andinitialization if not stated otherwise.
Problem 1. Claim (5.121) is true, but it leads to two problems. The first problem is that
ηbl = O(d−1max ) since η = O(1), while their Theorem 2.1 needs the learning rate to be O(1).
Nevertheless, this problem can be simply fixed by modifying their standard formulation as√l(0)hl+1 = W l xl + β dl bl where bi ∼ N (0, d−1l ). The real problem that is non-trivial to fix is thatd00by (5.121), there is ηW= dmaxη. However, note that d0 is a constant since it is the dimension of
the input space, while dmax goes to infinity. With that being said, in (5.121) they were essentiallyusing a very small learning rate for the first layer W 0 but a normal learning rate for the rest of thelayers, which definitely does not match with their claim in their Theorem 2.1.
Problem 2. Another big problem is that the proof of their Lemma 1 in their Appendix G iserroneous, and consequently their Theorem 2.1 is unsound as it heavily depends on their Lemma
1. In their Lemma 1, they claimed that for some constant M > 0, for any two models with theparameters θ and θ̃ such that θ, θ̃ ∈ B(θ(0) , C0 ) for some constant C0 , there isJ(θ) − J(θ̃)

M≤ p θ − θ̃F2d˜(claimed)(5.122)pNote that the original claim in their paper was J(θ) − J(θ̃) ≤ M d˜ θ − θ̃ . This isF2because they were proving this result for their standard formulation. Compared to the standard136

August 7, 2022
DRAFT
p
p
formulation, in the NTK formulation θ is d˜ times larger, while the Jacobian J(θ) is d˜ times
smaller. This is also why here we have θ, θ̃ ∈ B(θ(0) , C0 ) instead of θ, θ̃ ∈ B(θ(0) , C0 d˜−1/2 )for the NTK formulation. Therefore, equivalently they were claiming (5.122) for the NTK
formulation.
However, their proof of (5.122) in incorrect. Specifically, the right-hand side of their inequality(S86) is incorrect. Using the notations in our Appendix 5.7.4, their (S86) essentially claimed thatM
αl − α̃l 2 ≤ p θ − θ̃2d˜

(claimed)(5.123)

L+1
for any θ, θ̃ ∈ B(θ(0) , C0 ), where αl = ∇hl hp
and α̃l is the same gradient for the second
model. Note that their (S86) does not have the d˜ in p
the denominator which appears in (5.123).
This is because for their standard formulation, θ is d˜ times smaller than the original NTK
formulation, while αl 2 has the same order in the two formulations because all hl are the same.
However, it is actually impossible to prove (5.123). Consider the following counterexample:
Since θ and θ̃ are arbitrarily chosen, we can choose them such that they only differ in bl1 for some1 ≤ l < L. Then, θ − θ̃ = bl1 − b̃l1 . We can see that hl+1 and h̃l+1 only differ in the first2

element, andhl+1− h̃l+1
1
1= β(bl1 − b̃l1 ) . Moreover, we have W l+1 = W̃ l+1 , so there is

W l+1>
W̃ l+1>
αl+1 − α̃l+1 = diag(σ̇(hl+1 )) p αl+2 − diag(σ̇(h̃l+1 )) p α̃l+2d˜d˜hi W l+1>= diag(σ̇(hl+1 )) − diag(σ̇(h̃l+1 )) p αl+2
d˜W l+1>+ diag(σ̇(h̃l+1 )) p (αl+2 − α̃l+2 )d˜(5.124)

Then we can lower bound αl+1 − α̃l+1 2 by
l+1

α

− α̃

l+1
2

≥

h

l+1

diag(σ̇(hl+1)) − diag(σ̇(h̃i W l+1>)) p αl+2d˜2(5.125)l+1>
W− diag(σ̇(h̃l+1 )) p (αl+2 − α̃l+2 )
d˜
2
h
i
p
l+1
l+1
˜ αl+2 i
The first term on the right-hand side is equal to σ̇(hl+1
)
−
σ̇(
h̃
)
hW
/
d,
1
1
1
p 
where W1l+1 is the first row of W l+1 . We know that W1l+1 2 = Θd˜ with high probability
as its elements are sampled from N (0, 1), and in their (S85) they claimed that αl+2 2 = O(1),which is true. In addition, they assumed that σ̇ is Lipschitz. Hence, we can see thathdiag(σ̇(h

l+1

l+1

)) − diag(σ̇(h̃i W l+1>




l+2
l+1
l+1
)) p α= O h1 − h̃1= O θ − θ̃2d˜2
(5.126)137

August 7, 2022
DRAFT
On the other hand, suppose that claim (5.123) is true, then αl+2 − α̃l+2 2 =O d˜−1/2 θ − θ̃
. Then we can see that the second term on the right-hand side is2p
˜ and σ̇(x) is bounded by a constant as σ isO d˜−1/2 θ − θ̃because W l+1 2 = O( d)2˜ the second-term is an infinitely small term compared to the
Lipschitz. Thus, for a very large d,first term, so we can only prove that
αl+1 − α̃l+1 2 = O θ − θ̃
(5.127)2

which is different from (5.123) because it lacks a critical d˜−1/2 and thus leads to a contradiction.
˜−1/2 factor, and consequently we cannot prove (5.122)
Hence, wepcannot prove (5.123) with the dwith the d˜ in the denominator on the right-hand side. As a result, their Lemma 1 and Theorem
2.1 cannot be proved without this critical d˜−1/2 . Similarly, we can also construct a counterexample
where θ and θ̃ only differ in the first row of some W l .

5.8.2

Our fixes

Regarding Problem 1, we can still use an O(1) learning rate for the first layer in the NTK
formulation given that kxk2 ≤ 1. This is because for the first layer, we have1
1
∇W 0 f (x) = √ x0 α1> = √ xα1>
d0
d0

(5.128)For all l ≥ 1, we have xl 2 = O(d˜1/2 ). However, for l = 0, we instead have kx0 k2 = O(1).
Thus, we can prove that the norm of ∇W 0 f (x) has the same order as the gradient with respect toany other layer, so there is no need to use a smaller learning rate for the first layer.
Regarding Problem 2, in our formulation (5.8) and initialization (5.9), the initialization of theL(0)last layer of the NTK formulation is changed from the Gaussian initialization Wi,j ∼ N (0, 1) toL(0)the zero initialization Wi,j = 0. Now we show how this modification solves Problem 2.
The main consequence of changing
that (5.81) becomesp the initialization of the last layer isp
4LL
˜
˜ In fact, for any
different: instead of W 2 ≤ 3 d, we now have W 2 ≤ C0 ≤ 3 d.
˜ In our proof we chooser ∈ (0, 1/2), we can prove that W L 2 ≤ 3d˜r for sufficiently large d.
r = 1/4.
Consequently, instead of αl 2 ≤ M3 , we can now prove that αl 2 ≤ M3 d˜r−1/2 for all


l
l
r−1/2
˜
l ≤ L by induction. So now we can prove α − α̃ 2 = O dθ − θ̃
instead of2O θ − θ̃, because2

αl+1 2 = O(d˜r−1/2 ) instead of O(1), so we can have theadditional d˜r−1/2 factor in the bound.
• For l = L, although αL+1
= 1, note that W L 2 now becomes O(d˜r ) instead of2
O(d˜1/2 ), so again we can decrease the bound by a factor of d˜r−1/2 .
• For l < L, we now have138August 7, 2022
DRAFT
Then, with this critical d˜r−1/2 , we can prove the approximation theorem with the form(t)
sup f (t) (x) − flin (x) ≤ C d˜r−1/2(5.129)t≥0

for any r ∈ (0, 1/2), though we cannot really prove the O(d˜−1/2 ) bound as originally claimed in(5.118). So this is how we solve Problem 2.
One caveat of changing the initialization to zero initialization is whether we can still safely
assume that λmin > 0 where λmin is the smallest eigenvalue of Θ, the kernel matrix of our new
formulation. The answer is yes. In fact, in our Theorem 14 we proved that Θ is non-degenerated(which means that Θ(x, x0 ) still depends on x and x0 ), and under the overparameterized setting
where dL  n, chances are high that Θ is full-rank. Hence, we can still assume that λmin > 0.
As a final remark, one key reason why we need to initialize W L as zero is that the dimensionof the output space (i.e. the dimension of hL+1 ) is finite, and in our case it is 1. Suppose we allow
the dimension of hL+1 to be d˜ which goes to infinity, then using the same proof techniques, for
L+1(t)
the NTK formulation we can prove that supt hL+1(t) − hlin
≤ C, i.e. the gap between two
2
vectors of infinite dimension is always bounded by a finite constant. This is the approximation
theorem we need for the infinite-dimensional output space. However, when the dimension of theL+1(t)
output space is finite, supt hL+1(t) − hlin
≤ C no longer suffices, so we need to decrease2

the order of the norm of W L in order to obtain a smaller bound.

5.9

Experiment details and additional experiments

5.9.1

Experiment details

All experiments are conducted on a Ubuntu 18.04.6 machine with NVIDIA Geforce GTX 1080ti
GPUs. Each model is trained with one GPU. On each of Waterbirds and CelebA, we use a
ResNet18 as the model. The model is trained with SGD with momentum = 0.9. On Waterbirds
the learning rate is 10−4 , and on CelebA it is 10−3 . For Group DRO, ν is selected as 0.01 (see thedefinition of ν in (5.3)). The batch size used for Waterbirds is 128, and for CelebA it is 400. Dataaugmentation including random cropping, random horizontal flip and normalization is performed
on both datasets.

5.9.2

Sample weights converge in Group DRO(t)

The results in Section 5.3 require Assumption 1 which states that each sample weight qi converges
to some positive value as t → ∞. Our readers might wonder how strong this assumption is, andwhether reweighting algorithms satisfy this assumption in practice. In this section we empiricallydemonstrate that for Group DRO, the dynamic reweighting algorithm we experiment on, thisassumption is satisfied on Waterbirds and CelebA.
Recall that in Section 5.2.2 we empirically showed that reweighting algorithms could easily
overfit without regularization. Here using the same experimental settings, we keep track of theweight of each group gk during training, and we plot the group weight curves in Figure 5.3. We
139

August 7, 2022
DRAFT
also train the models longer (1000 epochs on Waterbirds and 300 epochs on CelebA). Clearly we
can see that as the training accuracy converges to 100%, the group weights also converge to an(t)(t)
equilibrium. Note that qi = gk /nk for all zi ∈ Dk , so the sample weights also converge.
0.6

0.4

Group WeightGroup Weight

0.5

0.3
0.2

0.4

0.2

0.1
0

0

200

400

600

800

1000

0

0

100

200

EpochsEpochs(a) Waterbirds.

(b) CelebA.

300

Figure 5.3: Weights of each group in Group DRO on Waterbirds and CelebA. The four curves
correspond to the four groups.

140August 7, 2022
DRAFT

Chapter 6
MSG: Margin Sensitive Group Risk
6.1

Introduction

Modern deep neural networks achieve high performance on a wide range of tasks, but they arealso known to be non-robust to distributional shift where the training and test distributions can bedifferent. A common instance of such distribution shift is subpopulation shift, where the training
and test distributions have the same set of sub-populations but with different proportions. For
example, many existing datasets are biased, in the sense that some underrepresented demographic
groups have sample sizes much smaller than the others, and models trained on such datasets
typically achieve high average performance but poor performances on these minority groups,
which makes such models unfair. Similarly, many existing datasets have highly imbalanced
classes, where some classes have much fewer training samples than others, and models trained on
those datasets, while performing well on average, typically fail to learn these minority classes
well enough.
To solve this problem, researchers have proposed a variety of methods. The most classicalmethod is importance weighting [116], where the training samples are reweighted so that eachgroup has the same weight in the training objective. Another widely used method is Distributionally Robust Optimization (DRO) [40, 51], which assumes that the test distribution belongs to a
family of distributions close to the training distribution called the uncertainty set, and trains themodel over the worst distribution in that set. Many DRO variants have been proposed recently[56, 148, 155, 157]. One of the most popular variants is Group DRO [110], which defines theuncertainty set as the convex hull of the group-conditionals of the training distribution.
Nevertheless, a line of recent work however has empirically and theoretically shown thatthese methods above do not necessarily perform better than standard empirical risk minimization(ERM). On the empirical side, [21] observed that the effect of importance weighting diminishes
over time on CIFAR-10, and the final performance is close to ERM; [110] found that Group DRO
overfits very easily, i.e. its worst-group performance drops to the same low level as ERM as
training proceeds; [48, 72] demonstrated that these methods do not perform better than ERM on avariety of realistic tasks. On the theoretical side, based on several previous papers [49, 63, 144], arecent work [156] proved the surprising fact that under certain mild conditions, a broad familyof algorithms called generalized reweighting (GRW), which includes all the popular methods
mentioned above, has implicit bias equivalent to ERM on both regression and classification tasks,141

August 7, 2022
DRAFT
implying that GRW does not improve distributionally robust generalization (DRG) over ERM.
The sobering takeaway from this work is that these popular methods, while they might seem
intuitive, do not really help with DRG.
A critical open problem facing the community is thus a principled way to improve DRG. Thereis some recent pioneering work towards this. [137] proposed to replace the exponentially-tailed
logistic loss with a polynomially-tailed loss function, and [111, 141] showed that strong data
augmentation, pretraining and semi-supervised learning could help DRG. One line of recent
work [23, 70, 83, 91, 150] focuses on the logit adjustment technique, which applies a linear
transformation to the logits output by a classifier to make it have larger margins on smaller groups.
This line of work starts from [23] which proposed to add an additive adjustment term to the logits
based on the margin theory, but [70] proved that the additive adjustment term only influences
optimization and has no effect on the implicit bias, and thus does not improve DRG. Instead, they
proposed to combine it with a multiplicative term, which they proved does affect the implicit bias.
In their theoretical analysis, they showed that their method leads to robust models on a simple
Gaussian Mixture model, but did not consider more general scenarios.
In this work, we propose to improve DRG by minimizing a margin sensitive group risk
(MSG-risk) which we derive from the margin theory. Specifically, the margin theory provides a
generalization bound for the test worst-group or balanced risk, and the MSG-risk is a surrogateof that generalization bound. Our method has two major improvements over previous work on
logit adjustment: (i) Logit adjustment only considers the balanced risk, while our method can also
handle the worst-group risk; (ii) Previous methods use fixed margins that are solely determinedby the group sizes, and we show that this could be suboptimal. Instead, the MSG-risk takes themargins as trainable parameters, and searches for the optimal margins along with the model
weights. In this way, our method can make the classifier have larger margins on more difficultgroups, not just smaller groups.
The MSG-risk is a non-convex function w.r.t. the margins, and we propose two ways tominimize it: First, we can minimize it with alternating minimization, which can be used in amethod called post-hoc weight normalization under the domain-incomplete setting; Second, we
can directly minimize it with stochastic gradient descent (SGD), which is an increasingly standard
approach to non-convex optimization. In our experiments, we show that our method achieves
state-of-the-art (SOTA) robust test performance on real datasets, and fixes the overfitting problem
which exists in many previous methods. We believe our approach therefore presents a substantial
advance in both our theoretical and practical understanding of how to tune modern classifiers to
improve DRG.

6.2

Preliminaries

6.2.1

Problem Formulation

Consider a classification task where the input space is X ⊆ Rd0 and the output space is Y ={1, −1} (binary) or Y = {1, · · · , C} (multi-class). We are given a training set {(xi , yi )}ni=1 i.i.d.
sampled from an underlying training distribution P over X × Y. Moreover, the input domain X
contains K groups (demographic groups, classes, etc.) where each sample can belong to zero,142August 7, 2022
DRAFT
one or more groups. Denote the conditional distribution of P over group k by Pk .
Balanced risk vs Worst-group risk. In most subpopulation shift problems, the goal is tominimize either the balanced risk Rbal or the worst-group risk Rmax defined as:K

1 X
Rbal (f ; P ) =R(f ; Pk )
K k=1

and

Rmax (f ; P ) = max R(f ; Pk )k=1,···,K(6.1)

where R(f ; P ) is the expected risk of f over P . In practice the expected risk R(f ; P ) is replacedby the empirical risk R̂(f ; P ), and the corresponding empirical balanced and worst-group risks
are denoted by R̂bal (f ; P ) and R̂max (f ; P ).
Domain-aware vs Domain-oblivious. If the group labels (i.e. which groups each sample
belongs to) are known during training, then it is called the domain-aware setting, which is mostwidely studied. However, in many real applications this might not be the case, either because
collecting group labels is expensive, or because we cannot identify all the groups at train time.
For instance, we train a face recognition model that is fair w.r.t. sensitive features like gender,
skin color, age and so on, but after using it for some time we observe that the model performs
much worse for people wearing glasses than people who don’t, so “wearing glasses” is a groupthat we fail to identify during training. Thus, a line of recent work [51, 86, 155] considers thedomain-oblivious setting, where the group labels are unknown during training. The problemof this setting is that it is too pessimistic, and methods based on this setting typically have low
performances in practice.

6.2.2

Domain-incomplete Setting and Post-hoc Weight Normalization

We saw that the domain-aware setting is not realistic in many real applications, while the domainoblivious setting is too pessimistic. Thus, in this work, we study a third setting called thedomain-incomplete setting which is very common and lies in between domain-aware and domainoblivious.
Consider the following scenario: We have a trained model, say a face recognition model, thatis trained by ERM or perhaps some robust training algorithm with some pre-defined groups. Then,at deployment stage we find that this model performs poorly on a certain group, say the group ofpeople wearing glasses, and we need a “hot-fix” to our model. In this situation, we have no or
incomplete group labels during training, but are provided with the complete group information attest time (without additional training samples), and the goal is to make the trained model robust
as efficiently as possible. This is the domain-incomplete setting. Of course, we can still view thissituation as a domain-aware problem and retrain or fine-tune the model with the new group added
to the set of watched groups, but this could be very inefficient.
To deal with the domain-incomplete setting, we will use a method which is called post-hoc
weight normalization in [70]. Suppose our model has the form w ◦ Φ(x), where Φ is a feature
encoder and w is a linear classifier. In post-hoc weight normalization, we keep Φ fixed and finda new w. This method can also be applied to representation learning, where Φ is learnt from143August 7, 2022
DRAFT
some self-supervised task, and we just need to find a w. This method works as long as Φ encodesfeatures that are robust to distributional shift. Since previous work [65] showed that ERM canlearn sufficiently robust features, we can use the encoder trained by ERM as Φ, which is very
effective in our experiments.

6.2.3

Generalized Logit Adjustment (GLA)

Now we introduce logit adjustment (LA) which motivates this work. The core idea of logit
adjustment is to use a new loss function which makes a classifier have larger margins on smallergroups. Intuitively, the statistics of smaller groups are harder to estimate, so a model is prone
to higher test error on these groups. Thus, we want there to be a larger margin between thesegroups and the decision boundary, which acts as a buffer that provides the model with better
generalization. A couple of different losses have been proposed, and here we use a generalformulation called generalized logit adjustment loss (GLA-loss) to cover all of them. Formallyspeaking, in GLA we apply a linear transformation to the logits output by the classifier before
feeding them to the (weighted) original loss function. For example, for the logistic loss used inbinary classification, its GLA-loss is defined as`GLA (f ; x, y) = q(x, y) log(1 + exp [−y · (δ(x, y)f (x) + τ (x, y))])(6.2)

where δ(x, y) is the multiplicative adjustment term, τ (x, y) is the additive adjustment term, andq(x, y) is the sample weight. This general formulation covers all existing losses, including the
LDAM-loss [23], the LA-loss [91], the CDT-loss [150] and the VS-loss [70].
For linear models, we can show that the only term in the GLA-loss that affects DRG is δ(x, y):Theorem 24. Suppose f (x) = hw, xi is a linear model, the data is linearly independent, andq(xi , yi ) and δ(xi , yi ) are positive for all i. If the model is trained by minimizing the average
training GLA-loss under gradient descent with a sufficiently small learning rate, then we havekw(t) k2 → ∞ andw(t)→ ŵδ = argmax min δ(xi , yi ) · yi hw, xi i
as t → ∞
(6.3)
1≤i≤n
kw(t) k2
kwk2 =1
See the proof in Appendix ??. This is an extension of Theorem 1 in [70]. Alternatively, we
can write ŵδ as the direction of the solution to the following cost-sensitive SVM [88]:minimize
w

s.t.

kwk2
yi hw, xi i ≥ 1/δ(xi , yi ),∀i ∈ [n](6.4)

This result shows that losses that only include the additive term and the sample weight, such as
LDAM-loss and LA-loss, do not really improve DRG. However, [70] showed that they do improve
optimization, so they could lead to better models in practice with early stopping.
While GLA seems intuitive, there are still two remaining questions.
First, the GLA-loss is used to minimize the balanced risk, so what about
the worst-group risk? Second, Theorem 24 only shows that δ can affect
DRG, so how to select δ so as to improve DRG?
144
Figure 6.1: Sample
task.

August 7, 2022
DRAFT
In all the losses mentioned above, δ and τ are fixed and only dependon the group sizes. This can be problematic. For example, in Figure 6.1
we have a sample binary classification task where the two classes havethe same size. Suppose the groups are equivalent to the classes (as inclass imbalance tasks), then all δ(xi , yi ) will be the same. By Theorem
24, the resulting linear classifier would be the SVM (black solid line).
However, ideally we would like the decision boundary to be farther away from the blue crosses(red dashed line) because this group has a larger variance than the pink dots, which means that theclassifier is more difficult to generalize on this group and needs a larger margin. Thus, margins
that only depend on the group sizes could be suboptimal, and we need a way to find the optimal
margins for each group. In other words, we not only need the classifier to have larger margins on
smaller groups, but also on more difficult groups.

6.3

MSG: Margin Sensitive Group Risk

In this section, we propose a new objective function called the margin sensitive group risk (MSGrisk), which is motivated by the GLA-loss and is based on margin theory, and thus is a principled
way to improve DRG. First, we derive the MSG-risk from the margin theory. Then we show two
ways to minimize this non-convex objective: (i) Alternating minimization, which we will apply topost-hoc weight normalization under the domain-incomplete setting; (ii) Direct SGD, which wewill use to train a neural network end-to-end under the domain-aware setting.

6.3.1

Derivation of the MSG-Risk

First, we derive the MSG-risk from the margin theory. For binary classification where theprediction is given by the sign of the model output ŷ, define the ρ-margin loss asŷy < 01,ŷy`ρ (ŷ, y) = φρ (ŷy) := 1 − ρ , 0 ≤ ŷy ≤ ρ0,ŷy > ρ(6.5)

A key advantage of this loss is that it is margin sensitive, as well as 1/ρ-Lipschitz, which thus
allows one to derive generalization bounds w.r.t. the margin. Denote the empirical ρ-margin riskof hypothesis h by R̂ρ (h). Denote the expected zero-one loss of h over the underlying distribution
P by R0/1 (h). Then we have the following generalization bound:
Theorem 25 (Theorem 5.9 and 5.10 in [94]). Let Φ : X → Rd be a feature mapping. Let
the hypothesis set be H = {x 7→ hw, Φ(x)i : kwk2 ≤ Λ} and the input space be X ⊆ {x :kΦ(x)k2 ≤ r}. Let M > 0 be fixed. Then for any δ > 0, with probability at least 1 − δ over the
random sampling of the training set, the following holds for all h ∈ H and all ρ ∈ (0, M ]:s
R0/1 (h) ≤ R̂ρ (h) + 3

log 4δ
2n

+

145

4
ρ

r

s
r 2 Λ2n+

log log2 2M
ρ
n

(6.6)

August 7, 2022
DRAFT
0/1

Denote the expected zero-one loss of h over group k by Rk (h), and the empirical ρ-marginrisk over group k by R̂ρ,k (h). Suppose group k has nk training samples. Then by union bound wehave:Corollary 26. Let Φ : X → Rd be a feature mapping. Let the hypothesis set be H = {x 7→hw, Φ(x)i : kwk2 ≤ Λ} and the input space be X ⊆ {x : kΦ(x)k2 ≤ r}. Let M > 0 be fixed.
Then for any δ > 0, with probability at least 1 − δ over the random sampling of the training set,
the following holds for all h ∈ H and all ρ1 , · · · , ρK ∈ (0, M ]:
s
s
s
4K
2
2
log log2 2M
log δ
4 r Λ
ρk
0/1
Rk (h) ≤ R̂ρk ,k (h) + 3++, ∀k ∈ [K]
(6.7)
2nk
ρk
nk
nk
Suppose ρk is bounded below away from zero. Then, the last term on the right hand sideof (6.7) becomes much smaller than the other terms and can be ignored. Thus, we define the
MSG-risk as the following surrogate of the generalization bound:
"

n

k
1
1 X
`˜ρk (h(xk,i ), yk,i ) + √R̂α,β (h; ρ1 , · · · , ρK ) = maxk∈[K] nk
nk
i=1


#
kwk2
α+β
ρk

(6.8)

where α, β are some non-negative constants given by (6.7) (which we view as hyperparameters), and `˜ρk is a surrogate loss function of `ρk such that `ρk (ŷ, y) ≤ `˜ρk (ŷ, y) (we need thisbecause `ρk is not convex). Eqn. (6.8) is a surrogate of the worst-groupP risk, and if we want tominimize the balanced risk, we only need to replace maxk∈[K] with k∈[K] in (6.8) (in whichcase α makes no difference and can be set to 0). Note that in previous GLA-losses, the margins
ρ1 , · · · , ρK are fixed, but the MSG-risk takes them as trainable parameters and optimizes themtoo.
For multi-class classification where the output of h(x) is a logit vector in RC , we have similar
results. Define the ρ-margin loss as`ρ (h(x), y) = φρ (hy (x) − max
hy0 (x))
0
y 6=y

(6.9)

where φρ is given by (6.5). Then we have the following generalization bound:
Theorem 27 (Corollary 9.4 in [94]). Let Φ : X → Rd be a feature mapping. Let H = {x →w ◦ Φ(x) : w ∈ RC×d , kwkF ≤ Λ} and X ⊆ {x : kΦ(x)k2 ≤ r}. Let M > 0 be fixed. Then forany δ > 0, with probability at least 1 − δ over the random sampling of the training set, for allh ∈ H and all ρ ∈ (0, M ] we have:
s
s
r
4
2
2
log log2 2Mlog δ
4C r Λ
ρR(h) ≤ R̂ρ (h) + 3++(6.10)2nρnn
Again, by union bound we can get a generalization bound similar to Corollary 26. Thus, theMSG-risk for multi-class classification is still defined as (6.8), with `˜ρk defined as a surrogate of(6.9).
The main issue with the MSG-risk is that it is non-convex w.r.t. ρ1 , · · · , ρK , which means that
we need to use some non-convex optimization methods to minimize it. In the following sections
we will introduce two approaches that work under different settings.
146

August 7, 2022
DRAFT

6.3.2

Alternating Minimization for the Domain-incomplete Setting

In post-hoc weight normalization, we have a pretrained feature mapping Φ and only need to findan optimal linear classifier h(x) = hw, Φ(x)i that minimizes the worst-group or balanced risk.
First, we formulate the task of minimizing (6.8) as the following optimization problem:
Xminimizes (orsk )w,s,τ,ρ

k∈[K]

τk,i ≥ `˜ρk (hw, Φ(xk,i )i, yk,i ), ∀k ∈ [K], ∀i ∈ [nk ]


nk
1 X
1
kwk2
τk,i + √α+β≤ s (or sk ), ∀k ∈ [K]nk i=1nkρks.t.

ρk ≥ 0,(6.11)∀k ∈ [K]˜ be the hinge loss. For example, for binary classification
In our implementation,we choose `ρ tonothis is `˜ρ (ŷ, y) := 1 − ŷy= max 1 − ŷy , 0 . Let δk = 1 . With this hinge loss, the
ρ

+

ρ

ρk

optimization problem above can then be re-written as:Xminimizes (orsk )w,s,τ,δ

k∈[K]

(
s.t.

Binary: δk hw, yk,i zk,i i ≥ 1 − τk,i , ∀k ∈ [K], ∀i ∈ [nk ]Multi-class: δk hwyk,i − wy0 , zk,i i ≥ 1 − τk,i , ∀y 0 6= yk,i , ∀k, ∀i
nk
X

1
1
τk,i + √ (α + βδk kwk2 ) ≤ s
nk i=1nkδk ≥ 0, τk,i ≥ 0,(or sk ),(6.12)∀k ∈ [K]∀k ∈ [K], ∀i ∈ [nk ]where the first constraint is either one of the two depending on whether it is binary or multi-classclassification, and zk,i = Φ(xk,i ). This is a bilinear non-convex optimization problem, which is
hard to solve in general. Here we solve this problem with alternating minimization, a heuristic
method that is widely used in matrix completion. Generally speaking, we train the model forseveral iterations, and for each iteration, we first fix δ and solve (6.12) w.r.t. w, s, τ , and then fix
w and solve (6.12) w.r.t. s, τ, δ. Note that if α = 0 and β = ∞, then the optimal solution of δk is
1/2
δk ∝ nk , so we can initialize δk in this way. Comparing (6.8) to (6.7), we can see that α and β
depend on r, the maximum norm of the features. Thus, we always normalize the features before
solving this optimization problem.
The algorithm for post-hoc weight normalization is listed in Algorithm 5, and in our implementation Φ is trained by ERM. Since this method works under the domain-incomplete setting whichis in between domain-aware and domain-oblivious, naturally we would expect the performance of
Algorithm 5 to be in between the performances of domain-aware and domain-oblivious methods.
What is surprising, however, is that we find in our experiments that the performance of Algorithm
5 is always close to, and in many occasions even better than, the performances of state-of-the-art
domain-aware methods. Moreover, given a pretrained encoder, our method only takes minutes torun on a CPU compared to domain-aware methods which typically take hours on a GPU.
147

August 7, 2022
DRAFT
Algorithm 5 Post-hoc Weight Normalization with Alternating Minimization
Require: Training data {(xi , yi )}ni=1 , pretrained encoder Φ, hyperparameters α, β, iterations T
1: Build features for the training samples: zi = Φ(xi ) for all iz −µ
2: Input normalization: z̃i = i√ , where µ and σ are the mean and standard deviation ofσ dz1 , · · · , zn , and d is the dimension of the feature space.
1/2
3: Initialization: δk ∝ nk for k ∈ [K]4: for t = 1, · · · , T do
5:
Fix δ and solve (6.12) w.r.t. w, s, τ with {(z̃i , yi )}ni=1 as input
6:
Fix w and solve (6.12) w.r.t. s, τ, δ with {(z̃i , yi )}ni=1 as input
7: end for
Φ(x)−µ√8: return the final model f (x) = w ∗ ·where w∗ is the solution found
σ d

6.3.3

End-to-end Training with Stochastic Gradient Descent for the
Domain-aware Setting

The second approach is directly minimizing the non-convex MSG-risk (6.8) with SGD w.r.t.
h and ρ1 , · · · , ρK jointly, so that we can train a neural network end-to-end under the domainaware setting. Define δk = ρ−1
k . We make δ1 , · · · , δK trainable parameters, so that it can be
optimized together with the weights of h (with different learning rates). More specifically, in our
implementation, we choose the surrogate loss of `ρ to be the logistic loss which is most widelyused in practice:ŷy(6.13)`ρ (ŷ, y) ≤ C log 1 + exp(− ) := C `˜ρ (ŷ, y)ρ
for some constant C which we ignore in the implementation. And the final objective function is
"
#
nk
1
1 X
R̂α,β (h, δ) = max
log (1 + exp(−δk h(xk,i )yk,i )) + √ (α + βδk khk2 )(6.14)k∈[K] nknki=1
where khk2 is defined as the L2 -norm of all its weights when flattened as a 1-dimensional vector.
We update the model weights and δ by minimizing this objective with stochastic gradient descent(SGD). Moreover, after each update, we normalize δ so that δ1 + · · · + δK is fixed.
Previous work [21, 110] showed that existing methods like importance weighting and GroupDRO suffer from overfitting: At the early stage of training the model trained with these methods
can achieve high worst-group or balanced test performance, but the performance gradually drops tothe same low level as ERM as training proceeds. On the contrary, our method fixes the overfitting
problem as we will show in our experiments, so it does improve DRG over existing methods.

6.4

Experiments

6.4.1

Setup

Datasets. For tasks that require maximizing the worst-group performance, we follow [155]and use two datasets: CelebA and CivilComments-Wilds. For tasks that require maximizing the148August 7, 2022
DRAFT
balanced performance, we follow [70] and use class-imbalanced CIFAR-10 with two types of classimbalance: Long-Tail (LT) imbalance and Step imbalance. CelebA and CivilComments-Wilds
have official train-validation-test splits. For CIFAR-10, we randomly split the test set into two
halves, making one the validation set and the other one the test set. See Appendix ?? for moredetails.
Models and baselines. Following prior work, we use a wide ResNet-18 for CelebA, a Bert-baseuncased model for CivilComments-Wilds, and a wide ResNet-32 for the imbalanced CIFAR-10
datasets. For the fairness tasks, we compare our method with two baselines: the SOTA domainoblivious method CVaR and the SOTA domain-aware methods importance weighting (IW) andGroup DRO (GDRO). For the class imbalance tasks, since they cannot be domain-oblivious, weonly compare with ERM and SOTA domain-aware methods (IW, LDAM-DRW, CDT, LA andVS, see [70] for a summary of these methods). For each method except ours, we train the model
for a fixed number of epochs (100 for CelebA, 5 for CivilComments-Wilds and 300 for CIFAR
datasets), and select the one with the highest validation worst-group/balanced accuracy. Notethat here the “domain-oblivious methods” are not 100% domain-oblivious because we use thegroup labels during validation (as [155] pointed out, model selection without group labels is too
hard, and currently no method is better than ERM). We use the original code from previous work
whenever available.
Implementation. In post-hoc weight normalization, we solve convex optimization with MOSEK[2], a commercial optimizer. To train Φ, we run ERM for a fixed number of epochs (as detailed
above), and take the encoder of the checkpoint at the end of training as Φ (so that Φ is completely
domain-oblivious). We only use the training set but not the validation set, while all other methods
use the validation set to select the best model. For end-to-end training with SGD, we combine itwith importance weighting. See Appendix ?? for details on hyperparameters.

6.4.2

Results

First, we evaluate the performance of post-hoc weight normalization with alternating minimization.
In Tables 6.1 and 6.2 we report the worst-group/balanced test accuracy achieved by post-hoc
weight normalization as well as previous methods with different feature space dimensions, which
we control by changing the width of the network. For alternating minimization, we always run 10
iterations. Overall, we observe that using the MSG-risk in post-hoc weight normalization achieves
performances that are close to, and in many occasions even better than, the performances ofstate-of-the-art domain-aware methods. This lends additional credence to the emerging empirical
understanding [65] that ERM learns sufficiently robust feature encoders. Also note that our
method is a very efficient method which only takes minutes to run on a CPU given a pretrained
encoder, whereas retraining from scratch takes hours on a GPU. Regarding the effect of thefeature dimension, we find that increasing the feature dimension does not always provide higher
performances. On CIFAR-10 the performances are higher for all methods when the feature
dimension increases, but not so for CelebA.
149

August 7, 2022
DRAFT
Table 6.1: Results for CelebA and CivilComments-Wilds. Each experiment is run with 5 different
random seeds, and the mean and std. dev. of the worst-group test accuracies (%) are reported.
DatasetFeature Dim

64

CelebA
128

256

CivilComments
784

ERM
CVaR

42.44 ± 4.72
61.29 ± 7.64

46.89 ± 2.96
66.07 ± 3.69

42.78 ± 5.02
71.25 ± 1.96

58.74 ± 2.94
63.90 ± 4.64

IWGroup DRO

87.00 ± 1.45
85.67 ± 1.49

86.89 ± 2.47
85.60 ± 2.48

87.67 ± 1.44
83.67 ± 2.47

68.05 ± 1.12
68.34 ± 2.40

MSG (Alg. 5) 87.12 ± 0.69

87.70 ± 1.59

87.94 ± 0.39

71.67 ± 1.12

Table 6.2: Results for CIFAR-10 with Long-Tail or Step class imbalance (“-100” means that thesize of the largest class is 100 times that of the smallest). Each experiment is run with 5 different
random seeds, and the mean and std. dev. of the balanced average test accuracies (%) are reported.
DatasetFeature Dim

CIFAR-10 (LT-100)
64
128

CIFAR-10 (STEP-100)
64
128

ERM

72.28 ± 0.44

74.62 ± 0.41

66.00 ± 2.00

68.12 ± 1.46

IW
LDAM-DRW
CDT
LA
VS

72.94 ± 0.94
77.30 ± 0.53
79.87 ± 1.04
80.68 ± 0.69
80.48 ± 0.49

73.92 ± 1.29
78.56 ± 0.45
81.64 ± 0.36
82.57 ± 0.53
82.76 ± 0.34

68.33 ± 1.27
78.06 ± 0.69
75.93 ± 0.86
76.47 ± 0.25
79.67 ± 0.64

69.41 ± 1.26
78.65 ± 0.75
77.76 ± 0.31
80.58 ± 0.96
82.06 ± 0.65

MSG (Alg. 5)

80.44 ± 0.50

82.78 ± 0.47

78.62 ± 0.95

80.57 ± 0.72

Second, we study the convergence rate of alternating minimization. We run Algorithm 5 withdifferent numbers of iterations on CelebA and imbalanced CIFAR-10 (LT-100) and plot the resultsin Figure 6.2. We can see that alternating minimization converges very quickly: It reaches the
optimal point after around 8 iterations on CelebA and around 3 iterations on CIFAR-10 (LT-100).
Third, we evaluate the performance of end-to-end training with SGD. We run our method
together with importance weighting and group DRO on CelebA and plot the worst-group test
accuracies achieved during training in Figure 6.3. The plot clearly shows that the performances ofimportance weighting and group DRO are high at the early stage of training, but as the training
proceeds they will start to drop at some point, while the performance of our method continues torise and maintains high, which implies that using the MSG-risk fixes the overfitting problem ofprevious methods.
Finally, we investigate what factors decide the optimal margins found by minimizing the MSGrisk. First, we look at the optimal δk found by post-hoc weight normalization with alternating
minimization and report them in Table 6.3. Note that a smaller δk means a larger margin for thatgroup. On CIFAR-10, we can see that as expected smaller groups have smaller δk , i.e. larger
margins. However, to our surprise, on CelebA the smaller groups have larger δk , i.e. smaller
150August 7, 2022
DRAFT(a) CelebA.

(b) CIFAR-10 (LT-100).

Figure 6.2: The convergence rate of alternating minimization Figure 6.3: End-to-end training
with SGD on CelebA.
in post-hoc weight normalization.
Table 6.3: The optimal δk (= ρ−1k ) found by post-hoc weight normalization with alternating minimization. Each experiment is run 5
times.

nk

CIFAR-10 (LT-100)
δk = ρ−1
Feat. Stdev.
k

5000
2997
83
50

9.11 ± 0.33
6.93 ± 0.40
1.96 ± 0.06
1.52 ± 0.05

2.37 ± 0.08
2.53 ± 0.13
2.73 ± 0.16
3.00 ± 0.24

nk
1387
66874
22880
71629

CelebAδk = ρ−1
Feat. Stdev.
k28.59 ± 1.69
18.59 ± 1.27
26.54 ± 4.52
15.72 ± 1.34

2.58 ± 0.15
2.94 ± 0.07
2.76 ± 0.28
3.08 ± 0.09

Figure 6.4: δk in end-toend training on CelebA.

margins (even though at initialization their δk is smaller). The same phenomenon is observed in
end-to-end training. We plot the change of δk in end-to-end training on CelebA in Figure 6.4,
which shows that the smallest group has the largest δk while the two majority groups have small
δk . We hypothesize that this is because the two majority groups, while larger, are harder to fit. To
verify this, we compute the L2 -norm of the standard deviation of the feature vector Φ(xk,i ) over
i for each group k, and report the results in Table 6.3 (column “Feat. Stdev.”). We can see that
this quantity aligns with the optimal margin perfectly: Groups with larger feature variances havesmaller δk , i.e. larger margins. Therefore, although the feature variance is not a perfect measureof how hard it is to fit a group, our results show that our method does make the classifier havelarger margins on more difficult groups as we desire.

6.5

Conclusion

In this work, we studied the difficult yet important problem of how to improve distributionallyrobust generalization (DRG), and proposed the MSG-risk that is derived from the margin theory.
We used two ways to minimize the MSG-risk: alternating minimization which can be used in
post-hoc weight normalization for the domain-incomplete setting, and direct SGD which can beused to train a neural network end-to-end for the domain-aware setting. Our experiments showedthat the MSG-risk leads to state-of-the-art robust performance, and thus we believe this work
151August 7, 2022
DRAFT
substantially advances our theoretical and practical understanding of how to improve DRG.
There are two remaining questions from this work: (i) In our experiments we find the optimal
α and β via grid search, but can we integrate them into the objective and optimize them jointly?
(ii) The margin theory only considers linear models, and for neural networks we replace kwk2
with khk2 , which lacks a solid theoretical foundation, so can we extend the margin theory toobtain generalization bounds for more complex models? We leave these problems to future work.

152August 7, 2022
DRAFT

Bibliography[1] Allen-Zhu, Z., Li, Y., and Song, Z. (2019). A convergence theory for deep learning via
over-parameterization. In International Conference on Machine Learning, pages 242–252.
PMLR. 5.1.1[2] ApS, M. (2022). MOSEK Optimizer API for Python. Version 9.3.17. 6.4.1[3] Attias, I., Kontorovich, A., and Mansour, Y. (2018). Improved generalization bounds forrobust learning. arXiv preprint arXiv:1810.02180. 2.1.2[4] Aubin, B., Krzakala, F., Lu, Y. M., and Zdeborová, L. (2020). Generalization error in highdimensional perceptrons: Approaching bayes error with convex optimization. arXiv preprint
arXiv:2006.06560. 4.2.2[5] Awasthi, P., Dutta, A., and Vijayaraghavan, A. (2019). On robustness to adversarial examples
and polynomial optimization. In Advances in Neural Information Processing Systems, pages
13760–13770. 2.1.2[6] Awasthi, P., Frank, N., and Mohri, M. (2020). Adversarial learning guarantees for linear
hypotheses and neural networks. arXiv preprint arXiv:2004.13617. 2.1.2[7] Azizyan, M., Singh, A., and Wasserman, L. (2013). Minimax theory for high-dimensional
gaussian mixtures with sparse mean separation. In Advances in Neural Information Processing
Systems, pages 2139–2147. 2.1.2[8] Bahdanau, D., Cho, K., and Bengio, Y. (2014). Neural machine translation by jointly learning
to align and translate. arXiv preprint arXiv:1409.0473. 1, 2.1[9] Balakrishnan, S., Wainwright, M. J., and Yu, B. (2017). Statistical guarantees for the em
algorithm: From population to sample-based analysis. The Annals of Statistics, 45(1):77–120.
4.1.4[10] Barocas, S. and Selbst, A. D. (2016). Big data’s disparate impact. Calif. L. Rev., 104:671.
3.1[11] Bartlett, P. L., Long, P. M., Lugosi, G., and Tsigler, A. (2020). Benign overfitting in linear
regression. Proceedings of the National Academy of Sciences, 117(48):30063–30070. 4.1.4[12] Bhagoji, A. N., Cullina, D., and Mittal, P. (2019). Lower bounds on adversarial robustness
from optimal transport. In Advances in Neural Information Processing Systems, pages 7496–
7508. 2.1, 2.1.2, 2.2, 2.2.1, 1, 2.3, 2.4, 2.6.2, 2.6.2, 2.7[13] Bickel, S., Brückner, M., and Scheffer, T. (2007). Discriminative learning for differing
training and test distributions. In Proceedings of the 24th international conference on Machine
153

August 7, 2022
DRAFT
learning, pages 81–88. 3.1[14] Birgé, L. (2001). An alternative point of view on lepski’s method. Lecture Notes-Monograph
Series, pages 113–133. 4.3[15] Blodgett, S. L., Green, L., and O’Connor, B. (2016). Demographic dialectal variation
in social media: A case study of African-American English. In Proceedings of the 2016
Conference on Empirical Methods in Natural Language Processing, pages 1119–1130, Austin,
Texas. Association for Computational Linguistics. 5.1[16] Borkan, D., Dixon, L., Sorensen, J., Thain, N., and Vasserman, L. (2019). Nuanced metrics
for measuring unintended bias with real data for text classification. In Companion Proceedingsof The 2019 World Wide Web Conference, pages 491–500. 3.6.1[17] Brent, R. P. (1971). An algorithm with guaranteed convergence for finding a zero of afunction. The Computer Journal, 14(4):422–425. 3.4[18] Bubeck, S., Lee, Y. T., Price, E., and Razenshteyn, I. (2018a). Adversarial examples from
cryptographic pseudo-random generators. arXiv preprint arXiv:1811.06418. 2.1.2[19] Bubeck, S., Price, E., and Razenshteyn, I. (2018b). Adversarial examples from computational
constraints. arXiv preprint arXiv:1805.10204. 2.1.2[20] Buda, M., Maki, A., and Mazurowski, M. A. (2018). A systematic study of the classimbalance problem in convolutional neural networks. Neural Networks, 106:249–259. 1, 4.1[21] Byrd, J. and Lipton, Z. (2019). What is the effect of importance weighting in deep learning?
In Chaudhuri, K. and Salakhutdinov, R., editors, Proceedings of the 36th International Conference on Machine Learning, volume 97 of Proceedings of Machine Learning Research, pages
872–881. PMLR. 6.1, 6.3.3[22] Cai, T. and Zhang, L. (2019). High dimensional linear discriminant analysis: optimality,
adaptive algorithm and missing data. Journal of the Royal Statistical Society: Series B(Statistical Methodology), 81(4):675–705. 2.1.2, 2.2, 2.6.2[23] Cao, K., Wei, C., Gaidon, A., Arechiga, N., and Ma, T. (2019). Learning imbalanced datasetswith label-distribution-aware margin loss. In Advances in Neural Information Processing
Systems, pages 1567–1578. 4.1, 4.1.4, 6.1, 6.2.3[24] Carmon, Y., Raghunathan, A., Schmidt, L., Duchi, J. C., and Liang, P. S. (2019). Unlabeleddata improves adversarial robustness. In Advances in Neural Information Processing Systems,
pages 11190–11201. 2.1.2, 4.1.4[25] Celentano, M., Montanari, A., and Wei, Y. (2020). The lasso with general gaussian designs
with applications to hypothesis testing. arXiv preprint arXiv:2007.13716. 4.1.4[26] Chawla, N. V., Japkowicz, N., and Kotcz, A. (2004). Special issue on learning from
imbalanced data sets. ACM SIGKDD explorations newsletter, 6(1):1–6. 1, 4.1, 4.1.4[27] Chizat, L., Oyallon, E., and Bach, F. (2019). On lazy training in differentiable programming.
In Wallach, H., Larochelle, H., Beygelzimer, A., d'Alché-Buc, F., Fox, E., and Garnett, R.,
editors, Advances in Neural Information Processing Systems, volume 32. Curran Associates,
Inc. 5.1.1

154August 7, 2022
DRAFT
[28] Cressie, N. and Read, T. R. (1984). Multinomial goodness-of-fit tests. Journal of the Royal
Statistical Society: Series B (Methodological), 46(3):440–464. 3.2.2[29] Cullina, D., Bhagoji, A. N., and Mittal, P. (2018). Pac-learning in the presence of adversaries.
In Advances in Neural Information Processing Systems, pages 230–241. 2.1.2[30] Dan, C., Wei, Y., and Ravikumar, P. (2020). Sharp statistical guarantees for adversariallyrobust gaussian classification. arXiv preprint arXiv:2006.16384. 4.1.4[31] Deev, A. (1970). Representation of statistics of discriminant analysis, and asymptotic
expansion when space dimensions are comparable with sample size. In Doklady AkademiiNauk, volume 195, pages 759–762. Russian Academy of Sciences. 4.1.2, 4.1.2, 4.1.3, 4.2.1,
4.5[32] Defazio, A. (2016). A simple practical accelerated method for finite sums. In Advances inneural information processing systems, pages 676–684. 4.8.1, 4.8.1[33] Deng, Z., Kammoun, A., and Thrampoulidis, C. (2019). A model of double descent for
high-dimensional binary linear classification. arXiv preprint arXiv:1911.05822. 4.1.4, 2[34] Devlin, J., Chang, M.-W., Lee, K., and Toutanova, K. (2019). BERT: Pre-training of deep
bidirectional transformers for language understanding. In Proceedings of the 2019 Conferenceof the North American Chapter of the Association for Computational Linguistics: Human
Language Technologies, Volume 1 (Long and Short Papers), pages 4171–4186, Minneapolis,
Minnesota. Association for Computational Linguistics. 3.6.1[35] Diakonikolas, I., Kamath, G., Kane, D., Li, J., Moitra, A., and Stewart, A. (2019). Robust
estimators in high-dimensions without the computational intractability. SIAM Journal on
Computing, 48(2):742–864. 1, 3.1[36] Diakonikolas, I., Kamath, G., Kane, D. M., Li, J., Moitra, A., and Stewart, A. (2017). Being
robust (in high dimensions) can be practical. In Precup, D. and Teh, Y. W., editors, Proceedings of the 34th International Conference on Machine Learning, volume 70 of Proceedingsof Machine Learning Research, pages 999–1008, International Convention Centre, Sydney,
Australia. 1, 3.1[37] Dobriban, E., Wager, S., et al. (2018). High-dimensional asymptotics of prediction: Ridge
regression and classification. The Annals of Statistics, 46(1):247–279. 4.1.4, 4.1.4[38] Donini, M., Oneto, L., Ben-David, S., Shawe-Taylor, J., and Pontil, M. (2018). Empiricalrisk minimization under fairness constraints. arXiv preprint arXiv:1802.08626. 4.1.4[39] Du, S., Lee, J., Li, H., Wang, L., and Zhai, X. (2019). Gradient descent finds global minimaof deep neural networks. In International Conference on Machine Learning, pages 1675–1685.
PMLR. 5.1.1[40] Duchi, J. and Namkoong, H. (2018). Learning models with uniform performance via
distributionally robust optimization. arXiv preprint arXiv:1810.08750. 3.1, 3.2.2, 3.2.2, 5.1,
5.1.1, 6.1[41] Dwork, C., Hardt, M., Pitassi, T., Reingold, O., and Zemel, R. (2012). Fairness throughawareness. In Proceedings of the 3rd innovations in theoretical computer science conference,
pages 214–226. 3.1, 3.1, 5.1.1
155August 7, 2022
DRAFT
[42] Elkhalil, K., Kammoun, A., Couillet, R., Al-Naffouri, T. Y., and Alouini, M.-S. (2017).
A large dimensional study of regularized discriminant analysis classifiers. arXiv preprint
arXiv:1711.00382. 4.1.4[43] Fithian, W. and Hastie, T. (2014). Local case-control sampling: Efficient subsampling in
imbalanced data sets. Annals of statistics, 42(5):1693. 4.1.4[44] Friedler, S. A., Scheidegger, C., and Venkatasubramanian, S. (2016). On the (im) possibilityof fairness. arXiv preprint arXiv:1609.07236. 4.1.4[45] Galar, M., Fernandez, A., Barrenechea, E., Bustince, H., and Herrera, F. (2011). A review on
ensembles for the class imbalance problem: bagging-, boosting-, and hybrid-based approaches.
IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews),
42(4):463–484. 3.1[46] Goodfellow, I. J., Shlens, J., and Szegedy, C. (2014). Explaining and harnessing adversarial
examples. arXiv preprint arXiv:1412.6572. 1, 2.1[47] Gordon, Y. (1985). Some inequalities for gaussian processes and applications. Israel Journalof Mathematics, 50(4):265–289. 4.2.2[48] Gulrajani, I. and Lopez-Paz, D. (2021). In search of lost domain generalization. In International Conference on Learning Representations. 3.7, 6.1[49] Gunasekar, S., Lee, J., Soudry, D., and Srebro, N. (2018). Characterizing implicit bias
in terms of optimization geometry. In Dy, J. and Krause, A., editors, Proceedings of the35th International Conference on Machine Learning, volume 80 of Proceedings of Machine
Learning Research, pages 1832–1841. PMLR. 6.1[50] Hardt, M., Price, E., Price, E., and Srebro, N. (2016). Equality of opportunity in supervised
learning. In Lee, D., Sugiyama, M., Luxburg, U., Guyon, I., and Garnett, R., editors, Advancesin Neural Information Processing Systems, volume 29, pages 3315–3323. Curran Associates,
Inc. 3.1, 4.1.4, 5.1.1[51] Hashimoto, T., Srivastava, M., Namkoong, H., and Liang, P. (2018). Fairness without
demographics in repeated loss minimization. In Dy, J. and Krause, A., editors, International
Conference on Machine Learning, volume 80 of Proceedings of Machine Learning Research,
pages 1929–1938, Stockholmsmässan, Stockholm Sweden. PMLR. 1, 3.1, 3.1, 3.2.1, 3.2.2,
3.7, 5.1, 5.1.1, 5.6, 6.1, 6.2.1[52] Hastie, T., Montanari, A., Rosset, S., and Tibshirani, R. J. (2019). Surprises in highdimensional ridgeless least squares interpolation. arXiv preprint arXiv:1903.08560. 4.1.4[53] He, H. and Garcia, E. A. (2009). Learning from imbalanced data. IEEE Transactions on
knowledge and data engineering, 21(9):1263–1284. 1, 4.1, 4.1.4[54] He, K., Zhang, X., Ren, S., and Sun, J. (2016). Deep residual learning for image recognition.
In Proceedings of the IEEE conference on computer vision and pattern recognition, pages
770–778. 3.6.1[55] Hovy, D. and Søgaard, A. (2015). Tagging performance correlates with author age. In
Proceedings of the 53rd annual meeting of the Association for Computational Linguistics andthe 7th international joint conference on natural language processing (volume 2: Short papers),
156

August 7, 2022
DRAFT
pages 483–488. 5.1[56] Hu, W., Niu, G., Sato, I., and Sugiyama, M. (2018). Does distributionally robust supervised
learning give robust classifiers? In International Conference on Machine Learning, pages
2029–2037. PMLR. 3.1, 3.1, 3.3, 3.7, 3.9.2, 3.9.2, 6.1[57] Huang, J., Gretton, A., Borgwardt, K., Schölkopf, B., and Smola, A. (2006). Correcting
sample selection bias by unlabeled data. Advances in neural information processing systems,
19:601–608. 3.1[58] Huber, P. J. (1992). Robust estimation of a location parameter. In Breakthroughs in statistics,
pages 492–518. Springer. 1, 3.1, 3.4[59] Hutchinson, B. and Mitchell, M. (2019). 50 years of test (un) fairness: Lessons for machinelearning. In Proceedings of the Conference on Fairness, Accountability, and Transparency,
pages 49–58. 4.1.4[60] Jacot, A., Gabriel, F., and Hongler, C. (2018). Neural tangent kernel: Convergence andgeneralization in neural networks. In Bengio, S., Wallach, H., Larochelle, H., Grauman, K.,
Cesa-Bianchi, N., and Garnett, R., editors, Advances in Neural Information Processing Systems,
volume 31. Curran Associates, Inc. 5.1.1, 5.3.2, 5.3.3, 19, 5.8.1[61] Japkowicz, N. (2000). The class imbalance problem: Significance and strategies. In Proc. of
the Int’l Conf. on Artificial Intelligence, volume 56. Citeseer. 3.1[62] Javanmard, A., Soltanolkotabi, M., and Hassani, H. (2020). Precise tradeoffs in adversarial
training for linear regression. arXiv preprint arXiv:2002.10477. (document), 2.1.2, 4.1.4, 4.1.4,
4.2.2, 4.7.1, 4.7.1, 4.7.1[63] Ji, Z. and Telgarsky, M. (2018). Risk and parameter convergence of logistic regression.
arXiv preprint arXiv:1803.07300. 6.1[64] Johnson, R. A., Wichern, D. W., et al. (2002). Applied multivariate statistical analysis,volume 5. Prentice hall Upper Saddle River, NJ. 2.2[65] Kang, B., Xie, S., Rohrbach, M., Yan, Z., Gordo, A., Feng, J., and Kalantidis, Y. (2020). Decoupling representation and classifier for long-tailed recognition. In International Conference
on Learning Representations. 6.2.2, 6.4.2[66] Khim, J. and Loh, P.-L. (2018). Adversarial risk bounds for binary classification via function
transformation. arXiv preprint arXiv:1810.09519, 2. 2.1.2[67] Kim, S.-J., Magnani, A., and Boyd, S. (2006). Robust fisher discriminant analysis. In
Advances in neural information processing systems, pages 659–666. 2.1.2[68] Kini, G. and Thrampoulidis, C. (2020). Analytic study of double descent in binary classification: The impact of loss. arXiv preprint arXiv:2001.11572. 4.9[69] Kini, G. R., Paraskevas, O., Oymak, S., and Thrampoulidis, C. (2021a). Labelimbalanced and group-sensitive classification under overparameterization. arXiv preprint
arXiv:2103.01550. 4.1.4, 4.1.4[70] Kini, G. R., Paraskevas, O., Oymak, S., and Thrampoulidis, C. (2021b). Label-imbalanced
and group-sensitive classification under overparameterization. In Thirty-Fifth Conference on

157

August 7, 2022
DRAFT
Neural Information Processing Systems. 6.1, 6.2.2, 6.2.3, 6.2.3, 6.2.3, 6.4.1, 6.4.1[71] Koh, P. W., Sagawa, S., Marklund, H., Xie, S. M., Zhang, M., Balsubramani, A., Hu, W.,
Yasunaga, M., Phillips, R. L., Beery, S., et al. (2020). Wilds: A benchmark of in-the-wild
distribution shifts. arXiv preprint arXiv:2012.07421. 3.6.1, 3.9.1[72] Koh, P. W., Sagawa, S., Marklund, H., Xie, S. M., Zhang, M., Balsubramani, A., Hu, W.,
Yasunaga, M., Phillips, R. L., Gao, I., Lee, T., David, E., Stavness, I., Guo, W., Earnshaw, B.,
Haque, I., Beery, S. M., Leskovec, J., Kundaje, A., Pierson, E., Levine, S., Finn, C., and Liang,
P. (2021). Wilds: A benchmark of in-the-wild distribution shifts. In Meila, M. and Zhang, T.,
editors, Proceedings of the 38th International Conference on Machine Learning, volume 139
of Proceedings of Machine Learning Research, pages 5637–5664. PMLR. 6.1[73] Kothari, P. K., Steinhardt, J., and Steurer, D. (2018). Robust moment estimation andimproved clustering via sum of squares. In Diakonikolas, I., Kempe, D., and Henzinger, M.,
editors, Proceedings of the 50th Annual ACM SIGACT Symposium on Theory of Computing,
STOC 2018, Los Angeles, CA, USA, June 25-29, 2018, pages 1035–1046. ACM. 3.5[74] Krizhevsky, A., Sutskever, I., and Hinton, G. E. (2012). Imagenet classification with deep
convolutional neural networks. In Advances in neural information processing systems, pages
1097–1105. 1, 2.1[75] Kumar, A., Ma, T., and Liang, P. (2020). Understanding self-training for gradual domain
adaptation. arXiv preprint arXiv:2002.11361. 4.1.4[76] Kusner, M. J., Loftus, J., Russell, C., and Silva, R. (2017). Counterfactual fairness. In
Advances in neural information processing systems, pages 4066–4076. 3.1, 5.1.1[77] Lahoti, P., Beutel, A., Chen, J., Lee, K., Prost, F., Thain, N., Wang, X., and Chi, E. (2020).
Fairness without demographics through adversarially reweighted learning. Advances in Neural
Information Processing Systems, 33. 3.2.1, 3.7[78] Lai, K. A., Rao, A. B., and Vempala, S. (2016). Agnostic estimation of mean and covariance.
In 2016 IEEE 57th Annual Symposium on Foundations of Computer Science (FOCS), pages
665–674. IEEE. 1, 3.1[79] Larson, J., Mattu, S., Kirchner, L., and Angwin, J. (2016). How we analyzed the compas
recidivism algorithm. ProPublica (5 2016), 9(1). 3.3[80] Laurent, B. and Massart, P. (2000). Adaptive estimation of a quadratic functional by model
selection. Annals of Statistics, pages 1302–1338. 4.3, 4.3, 4.4.2[81] Lee, J., Park, S., and Shin, J. (2020). Learning bounds for risk-sensitive learning. In
Larochelle, H., Ranzato, M., Hadsell, R., Balcan, M. F., and Lin, H., editors, Advances in
Neural Information Processing Systems, volume 33, pages 13867–13879. Curran Associates,
Inc. 3.1[82] Lee, J., Xiao, L., Schoenholz, S., Bahri, Y., Novak, R., Sohl-Dickstein, J., and Pennington, J.
(2019). Wide neural networks of any depth evolve as linear models under gradient descent.
Advances in neural information processing systems, 32:8572–8583. (document), 5.1.1, 5.3.2,
5.3.3, 5.8, 5.8.1[83] Li, M., Zhang, X., Thrampoulidis, C., Chen, J., and Oymak, S. (2021). Autobalance:
158

August 7, 2022
DRAFT
Optimized loss functions for imbalanced data. In Thirty-Fifth Conference on Neural Information
Processing Systems. 6.1[84] Li, T., Prasad, A., and Ravikumar, P. K. (2015). Fast classification rates for high-dimensional
gaussian generative models. In Advances in Neural Information Processing Systems, pages
1054–1062. 2.1.2[85] Li, T., Yi, X., Carmanis, C., and Ravikumar, P. (2017). Minimax gaussian classification &clustering. In Artificial Intelligence and Statistics, pages 1–9. 2.1.2, 2.4, 2.4.2, 2.4, 2.9, 4.4.1,
4.8.4[86] Liu, E. Z., Haghgoo, B., Chen, A. S., Raghunathan, A., Koh, P. W., Sagawa, S., Liang, P.,
and Finn, C. (2021). Just train twice: Improving group robustness without training group
information. In International Conference on Machine Learning, pages 6781–6792. PMLR.
5.1.1, 5.6, 6.2.1[87] Liu, Z., Luo, P., Wang, X., and Tang, X. (2015). Deep learning face attributes in the wild. In
Proceedings of the IEEE international conference on computer vision, pages 3730–3738. 3.6.1[88] Masnadi-Shirazi, H. and Vasconcelos, N. (2010). Risk minimization, probability elicitation,and cost-sensitive svms. In ICML. 6.2.3[89] McLachlan, G. J. and Peel, D. (2004). Finite mixture models. John Wiley & Sons. 2.1[90] Mei, S. and Montanari, A. (2019). The generalization error of random features regression:
Precise asymptotics and the double descent curve. Communications on Pure and Applied
Mathematics. 4.1.4[91] Menon, A. K., Jayasumana, S., Rawat, A. S., Jain, H., Veit, A., and Kumar, S. (2021). Longtail learning via logit adjustment. In International Conference on Learning Representations.
6.1, 6.2.3[92] Michel, P., Hashimoto, T., and Neubig, G. (2021). Modeling the second player in distributionally robust optimization. In International Conference on Learning Representations.
3.7[93] Mignacco, F., Krzakala, F., Lu, Y. M., and Zdeborová, L. (2020). The role of regularizationin classification of high-dimensional noisy gaussian mixture. arXiv preprint arXiv:2002.11544.
4.4.2, 4.4.2, 4.4.3[94] Mohri, M., Rostamizadeh, A., and Talwalkar, A. (2018). Foundations of machine learning.
MIT press. 25, 27[95] Montasser, O., Hanneke, S., and Srebro, N. (2019). Vc classes are adversarially robustly
learnable, but only improperly. arXiv preprint arXiv:1902.04217. 2.1.2[96] Namkoong, H. and Duchi, J. C. (2016). Stochastic gradient methods for distributionallyrobust optimization with f-divergences. In Advances in neural information processing systems,
pages 2208–2216. 3.1[97] Oren, Y., Sagawa, S., Hashimoto, T., and Liang, P. (2019). Distributionally robust language
modeling. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language
Processing and the 9th International Joint Conference on Natural Language Processing(EMNLP-IJCNLP), pages 4227–4237, Hong Kong, China. Association for Computational
159

August 7, 2022
DRAFT
Linguistics. 3.1, 5.1.1[98] Owen, A. B. (2007). Infinitely imbalanced logistic regression. Journal of Machine Learning
Research, 8(4). 4.1.4[99] Pan, S. J. and Yang, Q. (2009). A survey on transfer learning. IEEE Transactions on
knowledge and data engineering, 22(10):1345–1359. 3.1[100] Papernot, N., McDaniel, P., Jha, S., Fredrikson, M., Celik, Z. B., and Swami, A. (2016).
The limitations of deep learning in adversarial settings. In 2016 IEEE European symposium on
security and privacy (EuroS&P), pages 372–387. IEEE. 1, 2.1[101] Patel, V. M., Gopalan, R., Li, R., and Chellappa, R. (2015). Visual domain adaptation: A
survey of recent advances. IEEE signal processing magazine, 32(3):53–69. 3.1[102] Prasad, A., Balakrishnan, S., and Ravikumar, P. (2020). A robust univariate mean estimatoris all you need. In Chiappa, S. and Calandra, R., editors, Proceedings of the Twenty Third
International Conference on Artificial Intelligence and Statistics, volume 108 of Proceedingsof Machine Learning Research, pages 4034–4044. PMLR. 3.5[103] Prasad, A., Suggala, A. S., Balakrishnan, S., and Ravikumar, P. (2018). Robust estimation
via robust gradient estimation. arXiv preprint arXiv:1802.06485. 1, 3.1[104] Qiao, X. and Liu, Y. (2009). Adaptive weighted learning for unbalanced multicategoryclassification. Biometrics, 65(1):159–168. 4.1.4[105] Qiao, X., Zhang, H. H., Liu, Y., Todd, M. J., and Marron, J. S. (2010). Weighted distanceweighted discrimination and its asymptotic properties. Journal of the American Statistical
Association, 105(489):401–414. 4.1.4[106] Quionero-Candela, J., Sugiyama, M., Schwaighofer, A., and Lawrence, N. D. (2009).
Dataset shift in machine learning. The MIT Press. 3.1[107] Raghunathan, A., Xie, S. M., Yang, F., Duchi, J., and Liang, P. (2020). Understanding andmitigating the tradeoff between robustness and accuracy. arXiv preprint arXiv:2002.10716.
2.1.2[108] Raudys, Š. and Young, D. M. (2004). Results in statistical discriminant analysis: A reviewof the former soviet union literature. Journal of Multivariate Analysis, 89(1):1–35. 4.1.2[109] Rawls, J. (2001). Justice as fairness: A restatement. Harvard University Press. 1, 3.1, 5.1.1[110] Sagawa, S., Koh, P. W., Hashimoto, T. B., and Liang, P. (2020a). Distributionally robustneural networks for group shifts: On the importance of regularization for worst-case generalization. In International Conference on Learning Representations. 3.1, 3.7, 3.9.1, 5.1, 5.1.1,
5.2.1, 5.2.1, 5.2.2, 5.4, 5.4.1, 5.4.1, 6.1, 6.3.3[111] Sagawa, S., Koh, P. W., Lee, T., Gao, I., Xie, S. M., Shen, K., Kumar, A., Hu, W., Yasunaga,M., Marklund, H., Beery, S., David, E., Stavness, I., Guo, W., Leskovec, J., Saenko, K.,
Hashimoto, T., Levine, S., Finn, C., and Liang, P. (2022). Extending the WILDS benchmark
for unsupervised adaptation. In International Conference on Learning Representations. 6.1[112] Sagawa, S., Raghunathan, A., Koh, P. W., and Liang, P. (2020b). An investigation of
why overparameterization exacerbates spurious correlations. In III, H. D. and Singh, A.,160

August 7, 2022
DRAFT
editors, Proceedings of the 37th International Conference on Machine Learning, volume 119
of Proceedings of Machine Learning Research, pages 8346–8356. PMLR. 3.1, 5.1, 5.1.1, 5.3.3[113] Schmidt, L., Santurkar, S., Tsipras, D., Talwar, K., and Madry, A. (2018). Adversariallyrobust generalization requires more data. In Advances in Neural Information Processing
Systems, pages 5019–5031. 2.1, 2.1.1, 2.1.2, 2.3, 1, 3, 4, 2.1, 2.4, 4.1.4[114] Scott, A. and Wild, C. (1991). Fitting logistic regression models in stratified case-control
studies. Biometrics, pages 497–510. 4.1.4[115] Shen, Y. and Sanghavi, S. (2019). Learning with bad training data via iterative trimmed loss
minimization. In International Conference on Machine Learning, pages 5739–5748. PMLR.
3.4, 3.7[116] Shimodaira, H. (2000). Improving predictive inference under covariate shift by weighting
the log-likelihood function. Journal of statistical planning and inference, 90(2):227–244. 3.1,
3.7, 5.1, 5.2.1, 6.1[117] Silver, D., Huang, A., Maddison, C. J., Guez, A., Sifre, L., Van Den Driessche, G.,
Schrittwieser, J., Antonoglou, I., Panneershelvam, V., Lanctot, M., et al. (2016). Mastering thegame of go with deep neural networks and tree search. nature, 529(7587):484. 1, 2.1[118] Stanforth, R., Fawzi, A., Kohli, P., et al. (2019). Are labels required for improving
adversarial robustness? arXiv preprint arXiv:1905.13725. 2.1.2[119] Suggala, A. S., Prasad, A., Nagarajan, V., and Ravikumar, P. (2018). Revisiting adversarialrisk. arXiv preprint arXiv:1806.02924. 2.1.2[120] Sur, P., Chen, Y., and Candès, E. J. (2019). The likelihood ratio test in high-dimensional
logistic regression is asymptotically a rescaled chi-square. Probability theory and related fields,
175(1):487–558. 4.1.4[121] Szegedy, C., Zaremba, W., Sutskever, I., Bruna, J., Erhan, D., Goodfellow, I., and Fergus,
R. (2013). Intriguing properties of neural networks. arXiv preprint arXiv:1312.6199. 1, 2.1[122] Tan, C., Sun, F., Kong, T., Zhang, W., Yang, C., and Liu, C. (2018). A survey on deeptransfer learning. In International conference on artificial neural networks, pages 270–279.
Springer. 3.1[123] Tatman, R. (2017). Gender and dialect bias in youtube’s automatic captions. In Proceedingsof the First ACL Workshop on Ethics in Natural Language Processing, pages 53–59. 5.1[124] Thompson, A. C. and Thompson, A. C. (1996). Minkowski geometry. Cambridge University
Press. 2.2[125] Thrampoulidis, C., Abbasi, E., and Hassibi, B. (2018). Precise error analysis of regularized
m-estimators in high dimensions. IEEE Transactions on Information Theory, 64(8):5592–5628.
4.1.4, 4.6, 4.8.1[126] Thrampoulidis, C., Oymak, S., and Hassibi, B. (2015). Regularized linear regression: A
precise analysis of the estimation error. In Conference on Learning Theory, pages 1683–1709.
PMLR. 4.2.2[127] Thrampoulidis, C., Oymak, S., and Soltanolkotabi, M. (2020). Theoretical insights into mul-

161

August 7, 2022
DRAFT
ticlass classification: A high-dimensional asymptotic view. arXiv preprint arXiv:2011.07729.
4.4.2, 4.4.2, 4.4.3, 4.4.3[128] Tsipras, D., Santurkar, S., Engstrom, L., Turner, A., and Madry, A. (2018). Robustness
may be at odds with accuracy. arXiv preprint arXiv:1805.12152. 2.1.2[129] Tukey, J. W. (1960). A survey of sampling from contaminated distributions. Contributions
to probability and statistics, pages 448–485. 1, 3.1[130] Van der Vaart, A. W. (2000). Asymptotic statistics, volume 3. Cambridge university press.
2.1.3[131] Van Horn, G. and Perona, P. (2017). The devil is in the tails: Fine-grained classification in
the wild. arXiv preprint arXiv:1709.01450. 1, 4.1[132] Vershynin, R. (2010). Introduction to the non-asymptotic analysis of random matrices.
arXiv preprint arXiv:1011.3027. 22[133] Viering, T., Mey, A., and Loog, M. (2019). Open problem: Monotonicity of learning. In
Conference on Learning Theory, pages 3198–3201. 4.1[134] Wainwright, M. J. (2019). High-dimensional statistics: A non-asymptotic viewpoint,
volume 48. Cambridge University Press. 2.6.1, 4.1.2[135] Wang, C., Jiang, B., et al. (2018a). On the dimension effect of regularized linear discriminant analysis. Electronic Journal of Statistics, 12(2):2709–2742. 4.1.4, 4.9[136] Wang, H., Zhu, R., and Ma, P. (2018b). Optimal subsampling for large sample logistic
regression. Journal of the American Statistical Association, 113(522):829–844. 4.1.4[137] Wang, K. A., Chatterji, N. S., Haque, S., and Hashimoto, T. (2022). Is importanceweighting incompatible with interpolating classifiers? In International Conference on Learning
Representations. 6.1[138] Wang, M. and Deng, W. (2018). Deep visual domain adaptation: A survey. Neurocomputing,
312:135–153. 3.1[139] Wang, Y., Jha, S., and Chaudhuri, K. (2017). Analyzing the robustness of nearest neighbors
to adversarial examples. arXiv preprint arXiv:1706.03922. 2.1.2[140] Weng, T.-W., Zhang, H., Chen, H., Song, Z., Hsieh, C.-J., Boning, D., Dhillon, I. S., andDaniel, L. (2018). Towards fast computation of certified robustness for relu networks. arXiv
preprint arXiv:1804.09699. 2.1.2[141] Wiles, O., Gowal, S., Stimberg, F., Rebuffi, S.-A., Ktena, I., Dvijotham, K. D., and Cemgil,
A. T. (2022). A fine-grained analysis on distribution shift. In International Conference on
Learning Representations. 6.1[142] Williamson, R. and Menon, A. (2019). Fairness risk measures. In International Conference
on Machine Learning, pages 6786–6797. PMLR. 4.1.4[143] Wu, Y. and Zhou, H. H. (2019). Randomly√initialized em algorithm for two-component gaussian mixture achieves near optimality in o( n) iterations. arXiv preprint arXiv:1908.10935.
4.1.4[144] Xu, D., Ye, Y., and Ruan, C. (2021). Understanding the role of importance weighting for
162

August 7, 2022
DRAFT
deep learning. In International Conference on Learning Representations. 6.1[145] Xu, H., Caramanis, C., and Mannor, S. (2009a). Robust regression and lasso. In Advancesin neural information processing systems, pages 1801–1808. 2.1.2[146] Xu, H., Caramanis, C., and Mannor, S. (2009b). Robustness and regularization of support
vector machines. Journal of machine learning research, 10(7). 2.1.2[147] Xu, H. and Mannor, S. (2012). Robustness and generalization. Machine learning, 86(3):391–
423. 2.1.2[148] Xu, Z., Dan, C., Khim, J., and Ravikumar, P. (2020). Class-weighted classification:
Trade-offs and robust approaches. In III, H. D. and Singh, A., editors, Proceedings of the37th International Conference on Machine Learning, volume 119 of Proceedings of Machine
Learning Research, pages 10544–10554. PMLR. 3.1, 3.1, 4.1, 5.1.1, 5.6, 6.1[149] Yang, Y. and Xu, Z. (2020). Rethinking the value of labels for improving class-imbalanced
learning. arXiv preprint arXiv:2006.07529. 4.1[150] Ye, H.-J., Chen, H.-Y., Zhan, D.-C., and Chao, W.-L. (2020). Identifying and compensating
for feature deviation in imbalanced deep learning. arXiv preprint arXiv:2001.01385. 6.1, 6.2.3[151] Yin, D., Ramchandran, K., and Bartlett, P. (2018). Rademacher complexity for adversariallyrobust generalization. arXiv preprint arXiv:1810.11914. 2.1.2[152] Zafar, M. B., Valera, I., Gomez Rodriguez, M., and Gummadi, K. P. (2017). Fairness
beyond disparate treatment & disparate impact: Learning classification without disparate
mistreatment. In Proceedings of the 26th international conference on world wide web, pages
1171–1180. 3.1, 4.1.4, 5.1.1[153] Zemel, R., Wu, Y., Swersky, K., Pitassi, T., and Dwork, C. (2013). Learning fair representations. In International Conference on Machine Learning, pages 325–333. 3.1, 5.1.1[154] Zhai, R., Cai, T., He, D., Dan, C., He, K., Hopcroft, J., and Wang, L. (2019). Adversariallyrobust generalization just requires more unlabeled data. arXiv preprint arXiv:1906.00555.
2.1.2[155] Zhai, R., Dan, C., Kolter, Z., and Ravikumar, P. (2021a). Doro: Distributional and outlier
robust optimization. In Meila, M. and Zhang, T., editors, Proceedings of the 38th International
Conference on Machine Learning, volume 139 of Proceedings of Machine Learning Research,
pages 12345–12355. PMLR. 5.1.1, 5.6, 6.1, 6.2.1, 6.4.1, 6.4.1[156] Zhai, R., Dan, C., Kolter, Z., and Ravikumar, P. (2022). Understanding why generalized
reweighting does not improve over erm. arXiv preprint arXiv:2201.12293. 6.1[157] Zhai, R., Dan, C., Suggala, A., Kolter, J. Z., and Ravikumar, P. K. (2021b). Boosted CVarclassification. In Thirty-Fifth Conference on Neural Information Processing Systems. 6.1[158] Zhang, H., Yu, Y., Jiao, J., Xing, E. P., Ghaoui, L. E., and Jordan, M. I. (2019). Theoreticallyprincipled trade-off between robustness and accuracy. arXiv preprint arXiv:1901.08573. 2.1.2[159] Zhu, B., Jiao, J., and Steinhardt, J. (2020). Generalized resilience and robust statistics. 3.1

163