Model Patching: Closing the Subgroup Performance Gap with Data Augmentation
Authors: Karan Goel
Albert Gu
Yixuan Li
Christopher Re
Venue: ICLR 2021 Poster (2021)
Keywords: Robust Machine Learning
Data Augmentation
Consistency Training
Invariant Representations

Abstract: Classifiers in machine learning are often brittle when deployed. Particularly concerning are models with inconsistent performance on specific subgroups of a class, e.g., exhibiting disparities in skin cancer classification in the presence or absence of a spurious bandage. To mitigate these performance differences, we introduce model patching, a two-stage framework for improving robustness that encourages the model to be invariant to subgroup differences, and focus on class information shared by subgroups. Model patching first models subgroup features within a class and learns semantic transformations between them, and then trains a classifier with data augmentations that deliberately manipulate subgroup features. We instantiate model patching with CAMEL, which (1) uses a CycleGAN to learn the intra-class, inter-subgroup augmentations, and (2) balances subgroup performance using a theoretically-motivated subgroup consistency regularizer, accompanied by a new robust objective. We demonstrate CAMEL’s effectiveness on 3 benchmark datasets, with reductions in robust error of up to 33% relative to the best baseline. Lastly, CAMEL successfully patches a model that fails due to spurious features on a real-world skin cancer dataset.

Published as a conference paper at ICLR 2021
MODEL PATCHING: CLOSING THE SUBGROUP PERFORMANCE GAP WITH DATA AUGMENTATION
Karan Albert Sharon Li, Christopher Ré Department of Computer Science, Stanford University (kgoel, a allaa bert.gu, sharonli, chrismre smre êcs sts.ssaffoorddd
ABSTRACT
Classifiers in machine learning are often brittle when deployed. Particularly con- cerning are models with inconsistent performance on specific subgroups of a class, e.g., exhibiting disparities skin cancer classification in the presence absence of a spurious bandage. To mitigate these performance differences, we introduce model patching, a two-stage framework for improving robustness that encourages the model to be invariant to subgroup differences, and focus on class information shared by subgroups. Model patching first models subgroup features within a class and learns semantic transformations between them, and then trains classifier with data augmentations that deliberately manipulate subgroup features. We instantiate model patching with CAMEL, which (1) uses CycleGAN learn the intra-class, inter-subgroup augmentations, and (2) balances subgroup performance using a theoretically-motivtteed subgroup consistency regularizer, accompanied by robust objective. We demonstrate CAMEL's effectiveness on 3 benchmark datasets, with reductions in robust error of up 33% relative the best baseline. Lastly, CAMEL successfully patches model that fails due to spurious features on a real-world skin cancer dataset.
INTRODUCTION
Machine learning models typically optimize for average performance, and when deployed, can yield inaccurate predictions on important subgroups of a class, For example, practitioners have noted that on the ISIC skin cancer detection dataset (Codella et al., 2018), classifiers are more accurate on images of benign skin lesions with visible bandages, when compared to benign images where no bandage is present (Bissoto et al., 2019; Rieger et al., 2019).
This subgroup performance gap is an undesirable consequence classifier's reliance on subgroup- specific features, e.g. spuriously associating colorful bandages with benign cancer class (Figure 1). common strategy to side-step this issue is to use manual data augmentation to erase the differences between subgroups, e.g., using Photoshop (Winkler et al., 2019) or image tools (Rieger al., 2019) to remove markings on skin cancer data before retraining a classifier. However, hand-crafting these augmentations may be impossible if the subgroup differences are difficult to manually express.
Ideally, we would automatically learn the features differentiating the subgroups of a class, and then encourage classifier to be invariant to these features when making its prediction. To this end, introduce model patching, a framework that encapsulates this solution two stages:
Learn inter-subgroup transformations. Isolate features that differentiate subgroups within class, learning inter-subgroup transformations between them. These transformations change an example's subgroup identity but preserve the class label.
Train to patch the model. Leverage the transformations controlled data augmentations that manipulate subgroup features, encouraging the classifier to be robust to their variation.
In the first stage of model patching (Section 2.1), we learn, rather than specify, the differences between the subgroups of class. We assume that these subgroups known to the user, e.g. this is common when users perform error analysis (Oakden-Rayy et al., 2019). Our key insight here is to learn these differences inter-subgroup transformations that modify the subgroup membership of examples,

---

Published as a conference paper at ICLR 2021
while preserving class membership. Applying these semantic transformations data augmentations in the second stage allows us to generate "imagined" versions of an example in the other subgroups of its class. This contrasts with conventional data augmentation, where heuristies such rotations, flips, MixUp or CutOut (DeVries & Taylor, 2017; Zhang et al., 2017) are hand-crafted rather than learned. While these heuristics have been shown improve robustness (Hendryeks et al., 2019), the invariances they target are not well understood. Even when augmentations are learned (Ratner et al., 2017a), they are used address data scareity, rather than manipulate examples to improve robustness in prescribed way. Model patching is the first framework for data augmentation that directly targets subgroup robustness.
The goal of the second stage (Section 2.2) is to ap- propriately use the transformations to remove the classifier's dependence on subgroup-specifie features. We introduce two algorithmic innovations that target subgroup robustness: (i) subgroup robust objective and; (ii) a subgroup consistency regularizer. Our sub- group robust objective extends prior work group robustness (Sagawa et al., 2020) to our subgroup setting, where classes and subgroups form hierar- chy (Figure? left). Our new subgroup consistency regularizer constrains the predictions on original and augmented examples to be similar. While recent work on consistency training (Hendrycks et al., 2019; Xie et al., 2019) has been empirically successful in con- structing models that are robust to perturbations, our consistency loss carries theoretical guarantees on the model's 's robustness. We note that our changes are easy to add top of standard classifier training.
Suigroup
Subgreup colored spot
vanilla model
model patching
Figure 1: A vanilla model trained on skin
cer dataset exhibits a subgroup performance gap between images of malignant cancers with and without colored bandages. GradCAM (Selvaraju et al., 2017) illustrates that the vanilla model spuri- ously associates the colored spot with benign skin lesions. With model patching, the malignancy is predicted correctly for both subgroups.
We contribute a theoretical analysis (Section 3) to motivate end-to-end framework. Our analysis codifies the distributional assumptions underlying the class-subgroup hierarchy and motivates our new consistency regularizer, which has a simple information theoretic interpretation under this framework. First, introduce natural model for the data generating process that decouples an example from its subgroup. Under this model, we prove that our consistency loss in Stage 2, when applied subgroup augmentations from Stage 1, bounds the mutual information between the classifier output and the subgroup labels. Thus, training with our end-to-end framework forces the classifier to be invariant to subgroup-specifte features.
We conduct an extensive empirical study (Section 4) that validates CycleGAN Augmented Model Patching (CAMEL)'s ss ability to improve subgroup invariance and robustness. We first evaluate CAMEL on controlled MNIST setup. where it cuts robust error rate to third of other approaches while learning representations that are far more invariant, as measured by mutual information estimates. On two machine learning benchmarks CelebA and Waterbirds, CAMEL consistently outperforms state-of-the-an approaches that rely on robust optimization, with reductions subgroup performance gap by up to 10%. Next, we perform ablations on each stage of our framework: (i) replacing the CycleGAN with state-of-the-ar heuristic augmentations worsens the subgroup performance gap by 3.35%: (ii) our subgroup consistency regularizer improves robust accuracy by up to 2.5% over prior consistency losses. As an extension, we demonstrate that CAMEL can be used in combination with heuristic augmentations, providing further gains in robust accuracy of 1.5%. Besides CycleGANs, we show that other GAN-based augmentation methods can also be made significantly more robust by combining them with Stage 2 of model patching. Lastly, on the challenging real-world skin cancer dataset ISIC, CAMEL improves robust accuracy by 11.7% compared a group robustness baseline.
Our results suggest that model patching is a promising direction for improving subgroup robustness in real applications. Code for reproducing our results available on GitHub.
2

---

Published as a conference paper at ICLR 2021
Y black hair
Y blonde hair
hair
blonde hair
Apply Subgroup Augmentations
Subgroup Consistency Regularizer
Femala
Illlo [ir
Gassriyy
JS
male
Z female
Maie Made
brown hair
Classify
Mi
KL
Coasity
hil
Classifcation Loss
Intn
male
Z female
Real Image Kyz
Compute maximum loss subgroups
loss over classes
Stage 1: Learn Inter-Subgroup Augmentations
Stage 2: Retrain Patch the Model
Figure 2: The model patching framework. (Left) The class-subgroup hierarchy with each class Y divided into subgroups (e.g. blonde hair into Z E {male, female)) We learn inter-subgroup augmentations to transform examples between subgroups of a class. (Right) To patch the classifier, we augment examples by changing their subgroup membership and then train with our subgroup consistency loss and robust objective.
CAMEL: CYCLEGAN AUGMENTED MODEL PATCHING
In this section, we walk through CAMEL'S two-stage framework (Figure 2) in detail. Section 2.1, we introduce Stage of model patching, learning class-conditional transformations between sub- groups. In Section 2.2, Stage 2 uses these transformations as black- -boc--bxx augmentations to train classifier using new subgroup robust objective (Section 2.2.1) and consistency regularizer (Section 2.2.2). Section outlines our theoretical analysis on the invariance guarantees of our method. glossary for all notation is included in Appendix A.
Setup. We consider a classification problem where X' C IR is the input space, and ) 21.2....,C C} is set of labels over C classes. Each class y y may be divided into disjoint subgroups Zy C Z Jointly, there is distribution P over examples, class labels, and subgroups labels (X, Y. Z). Given a dataset { Vss 2=) our goal is to learn class prediction model fo X A parameterized by 0, where A denotes probability distribution over y
2.1 STAGE 1: LEARNING INTER- SURR-SU TRANSFORMATIONS
The goal of the first stage is to learn transformations F FS X' - X that translate examples in subgroup z to subgroup z', for every pair of subgroups 2, z' E Zy in the same class y.
Recent work has made impressive progress on such cross-domain generative models, where examples from one domain are translated another, ideally preserving shared semantics while only changing domain-specifte features. In this work, we use the popular CycleGAN model (Zhu et al., 2017) to learn mappings between pairs of subgroups, although we show that it is possible to substitute other models (see Section 4.2.4). Given datasets {xz ) (x:., yin from pair of subgroups 2, z' E Zy we train CycleGAN F=-+z/ transform between them. When classes have more than two subgroups, pairwise models can be trained between subgroups, multi-domain models such as the StarGAN (Choi et al., 2018) can used. We include review of CycleGANs Appendix C.I.
Given these transformations {F=->z ) J,, we generate augmented data for every training example (x, y, by passing it through all Fe-sz',2 E Zy. We denote these generated examples ãz, := (Z Je'z, where Z.'- = F. (x). For convenience, & denotes the number of subgroups [Zy]-
Prior work that uses data augmentation to improve robustness has mostly relied on heuristic aug- mentations, and focused on robustness to out-of-distribution examples (Hendrycks et al., 2019) with
Note that this allows each class to have the same subgroups, or for classes to have overlapping subgroups special cases. Many of the datasets consider in our experiments (Section 4) have this property.
3

---

Published as a conference paper at ICLR 2021
Table Comparison of metrics and losses for classifier training. Here Pa and are marginal distributions of (z, y) for the subgroup and (2(2, y) = l[((rg max fa())) denotes correct prediction on an example.
Metric of Interest
Loss L())
ERM
(poe(r,)) y)
Ep((So(r).) v) max Ep, {(f(r)))) Eyer {maxze2, E B. ((fo(z).)))
GDRO
minsez Ep_ og y)
SGDRO
m Ep, and (2( y) minsez, Bp, (x v)l
empirical studies. contrast, we learn to transform examples rather than specifying augmentations directly, and focus on improving worst-case subgroup robustness. We emphasize that while others have used cross-domain generative models for data augmentation, our novelty lies in targeting in- variance subgroup features using this style of augmentation. Past work has focused on domain adaptation (Huang et al., 2018), 8), few-shot learning (Antoniou et al., 2017). and data scarcity (Bowles al., 2018; Ratner et al., 2017b), but has not attempted to explicitly control the invariance of the classifier using the learned augmentations. As we describe in our theoretical analysis (Section 3), our use of cross-domain models a natural consequence of the class-subgroup setting.
2.2 STAGE 2: SUBGROUP ROBUSTNESS WITH DATA AUGMENTATION
The goal of the second stage is to learn classifier fo on both the original and augmented data from Stage I, using our subgroup robust objective (Section 2.2.1) and consistency regularizer (Sec- tion 2.2.2). Our robustness objective targets worst-case subgroup robustness, while our consistency regularizer forces the learned classifier to be invariant to subgroup features. Where relevant, we include discussion here differences prior work, with an extended related work in Appendix B.
2.2.1 A SUBGROUP ROBUSTNESS OBJECTIVE
We review two established objectives for training classifiers with their associated metrics and loss functions, and introduce our new objective target subgroup robustness (cf. Table
Prior work: Empirical Risk Minimization (ERM). The usual training goal is to maximize the aggregate accuracy, optimized using the empirical risk w.z.. a proxy loss function (Table 1, top).
Prior work: Group Robustness (GDRO). In our setting, aggregate performance is too coarse a measure of risk, since classes have finer-grained groups of interest. This can be accounted for by optimizing the worst-case performance over these groups Letting P, denote the conditional distribution of examples associated with subgroup E Z, the robust accuracy can be quantified by measuring the worst-case performance among all groups. This can be optimized by minimizing the corresponding group robust risk (Table 1, middle right). A stochastic algorithm for this group distributionally robust optimization (GDRO) objective was recently proposed (Sagawa ct al., 2020).
Nass-conditional Subgroup Robustness (SGDRO). The GDRO objective treats group structure as flat hicrarchy, While this approach accounts for worst-case subgroup performance, loses the class-subgroup hierarchy of our setting. Tailored to this, we create the SGDRO training objective (Table 1, bottom right) to optimize class-conditional worst-case subgroup robustness, aggregated over all classes (Figure 2 right). To measure subgroup robustness, we define the subgroup performance gap (Table 1, bottom left) for class as the gap between its best and worst performing subgroups.
We note that both GDRO and SGDRO assume knowledge of the subgroups, which is standard assumption in group robustness (Arjovsky et al., 2019; Ganin al., 2016; Sagawa et al., 2020).
2.2.2 SUBGROUP INVARIANCE USING A CONSISTENCY REGULARIZER
Standard models learn to rely spurious subgroup features when making predictions. Subgroup consistency regularization targets this problem by enforcing consistency on bagrroup-augmenter data, encouraging the classifier to become invariant -ubgroups-eaturrs.
4

---

Published as a conference paper at ICLR 2021
Recall that Stage 2 connects Stage by receiving augmented data žz,, representing "imagined" versions of an example x in all other subgroups of its class y. We define the cefficonsisteccy loss C, and -oosssstoo-ccnsssteeeeeyy loss C, as follows, where mn Σ (0z) denotes the average output distribution on the augmented examples.
1 Σ KL. KI. (fo(())))) (1) ZEZ,
(z,,,
0z, = KT. (fo(x))))
(2)
The self-consistency loss is the more important component, encouraging predictions on augmented examples to be consistent with each other. As these augmented examples correspond one "imagined" example per subgroup, self-consistency controls dependence on subgroup features. Translation consistency additionally forces predictions on the original example be similar those of the average CycleGAN -rcceGaN-translated translated examples, ignoring potential artifacts that the CycleGANs generate.
We note that consistency losses have been used before, c.g. UDA (Xie et al., 2019) and Aug- Mix (Hendrycks et al., 2019) use different combinations of KL divergences chosen empirically. Our regularization (1) is tailored to the model patching setting, where it has theoretical interpretation relating subgroup invariance (Section 3). We show empirical improvements over these alternate consistency losses in Section 4.2.2.
Overall Objective.
The total consistency loss averages over all examples,
(0) -
2 (zz))> (I, Zz,; + (x, ēz,
Combining our SGDRO robust objective and the consistency loss with the consistency strength hyper-parameter yields the final objective,
CCAMEL (0) LSCDRO (O)
AN INFORMATION THEORETIC ANALYSIS OF SUBGROUP INVARIANCE
We introduce framework analyze our end-to-end approach, showing that it induces subgroup invariances the model's features. First, we review a common framework for treating robustness over discrete groups that aims create invariances, or independences between the learned model's features o(X (X) and groups Z. We then define a new model for the distributional assumptions underlying the subgroup setting, which allows us to analyze stronger invariance guarantees by minimizing a mutual information (MI) upper bound. Formal definitions and full proofs are deferred to Appendix C.
Prior work: Class-condittoned Subgroup Invariance. Prior work (Ganin et al., 2016; Li al., 2018; Long et al., 2018) uses adversarial training to induce subgroup invariances of the form (o(X) Z) Y. so that within each class, the model's features ((X) appear the same across subgroups Z. We call this general approach class- class-condition conditionii domain adversarial training (CDAT). Although these works are motivated by other theoretical properties, we show that they induce above invariance by minimizing a variational lower bound of the corresponding mutual information.
Lemma 1. CDAT minimizes a lower bound on the mutual information I(x(X); Z Y).
Since the model's features matter only insofar as they affect the output, for the rest of this discussion assume without loss of generality that ((X) = Y is simply the model's prediction.
Natural Distributional Assumption: Subgroup Invariance on Coupled Sets. Although prior work generally has no requirements how the data X among the groups Z relate to each other, we note that a common implicit assumption is that there a "correspondence" between examples among different groups. We codify this distributional assumption explicitly.
Informally, we say that every example belongs to coupled set [z), containing one example per subgroup its (x's class (Figure 3) (Appendix C.3, Definition 1). [X] is the random variable for coupled sets, i.e. it denotes sampling an example and looking at its coupled set. Intuitively, x' E [ær represent hidden examples the world that have identical class features to x and differ only their subgroup features. These hidden examples may not be present in the train distribution and model patching "hallucinates" them, allowing models to directly learn relevant class features.
5

---

Published as a conference paper at ICLR 2021
? 7
plaiin
Zigcug
dotted
This idea of coupled sets underlies both stages of the framework and en- ables stronger invariance guarantees. Given this notion, all examples x in coupled set [z] should have identical predictions in order to be robust across subgroups, modeled by the desired invariance (Y 1 2) [X]. Instead of Lemma 1, we aim to minimize T(Y; Z [X]). Note that T(;: Z [X] 2 I(²: Z Y), which follows from the chain rule for MI (proof in Appendix C), so this is stronger notion of invariance than CDAT permits. Additionally, the losses from the CycleGAN (Stage 1) and consistency regularizer (Stage 2) combine form an upper bound the mutual information rather than lower bound, so that optimizing our loss is more appropriate.
Po
paydmoo
[X]
7
Figure 3: Coupled sets for subgroups of the Y= 7 class.
Theorem 1. For model fe with outputs Y, the MI (Y;; Z [X]) is the Jensen-Shannon Divergence (JSD) of predictions coupled sets [[z]-[x] JSD (fe())))))))) In the case of k = 2 subgroups per class, this can be upper bounded by the CycleGAN and consistency losses
(F,1,,, + Σ [c
=EZy
In particular, the global optimum of the trained CAMEL model induces Y Z [X].
The main idea that the conditional MI I(;; [X]) can be related to model's predictions on all elements in coupled set [a:] using properties of the JSD. However, since we do not have true coupled sets, the consistency loss (3) only minimizes a proxy for this JSD using the augmentations z Using standard GAN results, the divergence between the true and augmented distributions can be bounded by the loss of a discriminator, and the result follows from metric properties of the JSD.
Thus, the CycleGAN augmentations (Stage 1) and our consistency regularizer (Stage 2) combine to provide an upper bound on our MI objective, tying together the model patching framework neatly.
4 EXPERIMENTS
We demonstrate that CAMEL can take advantage of the learned subgroup augmentations and consis- tency regularizer to improve robust and aggregate accuracy, while reducing the subgroup performance gap (defined in Table 1). We validate CAMEL against both standard training with subgroup knowledge (ERM) and other baselines aimed at improving group robustness across 4 datasets. We also conduct extensive ablations to isolate the benefit of the learned inter-subgroup transformations over standard augmentation, and the subgroup consistency regularizer over prior consistency losses.
Datasets. We briefly describe the datasets used, with details available in Appendix D.1.
MNIST-Correlation. We mix data from MNIST (LeCun et al., 1998) and MNIST-Corrppte (Mu Gilmer, 2019) to create a controlled setup for analyzing subgroup performance. Digit parity classes Y E (even, odd} are divided into subgroups Z E {clean, zigzag) from MNIST and MNIST-Corrupted Y and are highly correlated, so that most even (odd) digits are clean (zigzag).
CelebA- Uneeb--Undersmmppee. Following Sagawa et al. (2020), we classify hair color Y (non-blonde, blonde} in the CclebA dataset (Liu et al., 2015). Subgroups are based on gender Z = (female, male). We subsample the set of non-blonde women so that most non-blonde (blonde) examples are men (women).
Waterbirds. In this dataset to analyze spurious correlations (Sagawa al., 2020), birds Y E {landbird, waterbird] are placed against image backgrounds Z E {land, water], with waterbirds (landbirds) more commonly appearing against water (land).
ISIC. In this skin cancer dataset (Codella et al., 2018), we classify Y E (benign, malignant) cancers, with bandages Z appearing on ~ 50% of only benign images.
Methods. CAMEL instantiates model patching described in Section 2. We use the original CycleGAN model with default hyperparameters (Appendix D.2). We compare against ERM and GDRO (Table I), which respectively minmmize the standard risk and robust risk (over all subgroups)

---

Published as a conference paper at ICLR 2021
Table 2: comparison between CAMEL and other methods on benchmark datasets. Evaluation metrics include robust & aggregate accuracy and the subgroup performance gap. calculated on the test set. Results are averaged over 3 trials (one standard deviation indicated in parentheses).
Dataset
Method
Subgroup
Acc. (%)
Y
Z
Aggregate Acc. (%)
Robust Acc. (%)
Subgroup Gap (%)
y
even clean
even rigrag
clean
odd
odd zigrng
even
odd
MNIST-
ERM
86.96
73.51
71.47
75.21
76.75
(1.60)
71.47 (1.50)
13.45
3.73
Correlation
IRM
94.68
69.30
81.77
93.53
84.85 42)
69.30 (3.29)
25.38
11.76
CDAT
94.63
72.85
79.21
92.97
84.93 (5.84)
72.85 (3.47)
21.78
13.76
GDRO
98.10
93.31
96.82
97.15
96.35 (0.49)
93.31 (1.30)
4.79
0.79
CAMEL
98.85
97.89
97.98
97.87
97.55 (0.46)
97.77 (0.42)
0.96
0.17
blonde Temale
blonde
blande
blonde
non-blonde
blonde
male
Jemale
male
CelebA-
ERM
81 1..99
98.08
98.13
60.04
88.26 (1.86)
02.22 (6.83)
16.99
38.09
Undersampled
GDRO
89.26
92,24
94.08
82.20
90.91 (0.78)
82.20 (3.13)
2.98
11.88
CAMEL
92.15
93.73
91.13
83.53
92.90 (0.35)
83,90 (1.31)
1.83
8,07
landhird land
landhird
waterhird land
waterhind
landbird
waterbird
Waterbirds
ERM
98.92
75.12
72.71
94.95
86.31 (0.39)
72.71 (2.36)
23.80
22.24
GDRO
94.46
83.81
88.19
92.36
89.39 (0.
83.81 (0.39)
10.65
4.17
CAMEL
90.84
90.40
89.69
89.58
90.89 (0.87)
89.12 (0.36)
0.43
1.04
on the training set. On MNIST-Correlaion,, we additionally compare against the IRM (Arjovsky et al., 2019) and CDAT (Li et al., 2018) baselines which target invariance assumptions (details in Appendix D.6). All classifiers are fine-tuned using a ResNet-50 architecture, with pretrained ImageNet weights. Detailed information about experimental setups are provided in Appendix D.
Metrics. We evaluate aggregate accuracy, robust accuracy, and subgroup gap for class y, which are the metrics interest for ERM, GDRO, and our subgroup robustness setting (Table 1).
4.1 SUBGROUP ROBUSTNESS AND INVARIANCE ON BENCHMARK DATASETS
We first compare all methods on the benchmark datasets, with results summarized in Table 2.
CAMEL increases aggregate and robust accuracy while closing the subgroup gap. On all datasets, CAMEL improves both aggregate and robust accuracy by up to 5.3%. mitigating the tradcoff' that other methods experience. CAMEL also balances out the performance of subgroups within each class, e.g., Waterbirds, reducing this subgroup gap by
10.2 10.22% on landbirds compared to GDRO.
CAMEL learns mubgroup-invarian representa- tions. To measure the invariance of models, we re- port estimate of the mutual information defined in Lemma 1, calculated using conditinniiio domain prediction heads (Appendix D.5). Table illustrates that CAMEL is the only method that successfully makes the model invariant subgroups.
Table 3: Estimated MI between predictions and subgroups computed on MNIST-Corrclation.
ERM
IRM
CDAT
GDRO
CAMEL
MI Estimate
0.67
0.69
0.69
0.33
0.02
4.2 MODEL PATCHING ABLATIONS
We perform ablations on the major components of our framework: (1) substituting learned augmenta- tions with alternatives like heuristic augmentations in Stage 1, and (2) substituting prior consistency losses for our subgroup consistency regularizer in Stage
4.2.1 EFFECT OF LEARNED AUGMENTATIONS
We investigate the interaction between the type of augmentation used and the strength of consistency regularization, by varying the consistency loss coefficient À on Waterbirds (Table 4). We compare to: (i) subgroup pairing, where consistency is directly enforced on subgroup examples from class without augmentation and (ii) heuristic augmentations, where the CycleGAN is substituted with
7

---

Published as a conference paper at ICLR 2021
Table 4: Ablation analysis (Section 4.2.1) that varies the consistency penalty coefficient X. brevity, we report the maximum subgroup performance gap over all classes.
Robust Acc. (%)
Method
Max Subgroup Gap
À 20
À 50
= 200
Subgroup
74.22
71.88
74.22
Pairing
19.53
Heuristic
87.50
88.54
79.17
Augmentation
6,95
6.48
37.50
82.03
83.33
89.06
CAMEL
12.50
10.84
3.13
CA CAMELLC + Heuristic
89.06
90.62
53.45
0.21
1.30
19.39
state-of-the-art heuristic augmentation pipeline (Hendryeks et al., 2019) (Appendix D.6) containing rotations, flips, cutout etc. Our goal to validate our theoretical analysis, which suggests that strong consistency training should help most when used with the coupled examples generated by the CycleGAN. We expect that the ablations should benefit less from consistency training since, (i) subgroup pairing enforces consistency on examples across subgroups that may not lie in the same coupled set; and (ii) heuristic augmentations may not change subgroup membership at all, and may even change class membership.
Strong consistency regularization enables CAMEL's success. As À increases from 20 to 200, CAMEL's robust accuracy rises by 7% while the subgroup gap is 9.37% lower. For both ablations, performance deteriorates when À large. Subgroup pairing is substantially worse (14.84% lower) since it does
any augmentation, and Heuristic augmentations (e.g. rotations, flips) are not targeted at subgroups and can distort class information (e.g. color shifts in Mig,ix), and observe that strongly enforcing consistency (A = 200) makes these models much worse. Overall, these results agree with our theoretical analysis.
expected does not benefit from increasing A.
CAMEL combines flexibly with other augmentations. Empirically, we find that using heuristic augmentations in addition to the CycleGAN (method CAMEL + Heuristic) can actually be beneficial, with robust accuracy of 90.62% and a subgroup gap that 1 1.83% lower than using CAMEL alone.
4.2.2 ANALYZING THE SUBGROUP CONSISTENCY REGULARIZER
Next, investigate our choice of consistency regularizer, by substituting it for (i) triplet Jensen- Shannon loss (Hendrycks et al., 2019) and (ii) KL-divergence loss (Xie et al., 2019) in CAMEL (Figure 4). Our goal is show that our theoretically justified regularizer reduces overfitting and better enforces subgroup invariance.
Consistency regularization reduces overfitting. Figure 4 illustrates the train and validation cross- entropy loss curves for CAMEL and GDRO on the small (landbird, water) Waterbirds subgroup (184 examples). Consistency regularization shrinks the gap between train and validation losses, strongly reducing overfitting compared to GDRO.
Alternative consistency losses deteriorate perfor- mance. As expected, substituting the subgroup consis- tency loss with either the triplet-JS loss or the KI. loss in CAMEL reduces robust accuracy significantly (-2.5% on Waterbirds). Interestingly, our subgroup consistency regularizer improves over prior consistency losses even when used with heuristic augmentations.
Table 5: MNIST-Correlatin results when training to predict Z labels and testing Y labels. Test robust accuracy bolded.
Method
Subgroup Accuracies
ERM
99.18
13.18
12.24
99.46
GDRO
99.02
84.00
94.20
98.78
4.2.3 TRAINING WITH SUBGROUP INFORMATION
Since subgroup labels are available at train time, a natural baseline to consider is training predict the finer-grained
8

---

Published as a conference paper at ICLR 2021
0.8
GORO Train
CAMEL Train CAMEL
Learned Aug.
Heuristic Aug.
0.6
GDRO Val
Val
0.5-
Triplet JS
KL
Triplet JS
KL.
Performance
Change
02
CAMEL
-2.50
-0.83
-2.08
-1.04
0.1
Consisteney Loss)
2000
4000
6000
8000
10000
# training steps
Figure 4: Consistency loss ablations on Waterbirds. (Left) loss curves on the (landbird, water) subgroup. The addition of the CAMEL consistency loss to GDRO reduces overfitting. (Right) Robust accuracy decrease with alternate consistency losses (Triplet JS (Hendrycks al., 2019) and KL (Xie et al., 2019)) CAMEL- generated data heuristic augmentations.
Z label using ERM or GDRO. Evaluation is still performed by comparing against the coarser coarser-grained label corresponding to the predicted value. We use MNIST-Correlation in the setting of Table 2.
In Table 5, we find that while both ERM and GDRO perform well on the larger subgroups, the performance on minority subgroups drops compared the simpler setting in Table 2 where they are trained only with Y labels. These results are consistent with the intuition that being asked to predict labels forces models learn information about the spurious attributes instead of the real class, and highlights the importance of erasing subgroup information in order to learn invariant representations.
4.2.4 ADDITIONAL GAN ABLATIONS
As the quality of images generated by GANs continues to improve, previous works highlighted in Appendix B have considered them as data augmentation methods, where training an ERM classifier on GAN-augment -agmenngett images can improve model accuracy. However, in Appendix D.8, we show that this does not improve robust accuracy. In contrast, we show that CAMEL can flexibly incorporate three other GAN baselines alternatives to CycleGAN in Stage 1, whereby combining them with Stage 2 in the model patching pipeline improves robust accuracy by over points.
4.3 REAL-WORLD APPLICATION IN SKIN CANCER CLASSIFICATION
We conclude by demonstrating that CAMEL can improve performance substantially on the real-world ISIC (Codella et al., 2018) skin cancer dataset (Table 6). We augment only the benign class, which is split into subgroups due to the presence contains no subgroups. We also report AUROC, as is conventional in medical applications.
colored bandage (Figure 1) while the malignant class
CAMEL substantially improves robust accuracy by 11.7% and importantly, increases accuracy on the critical malignant cancer class from 65.59% (ERM) and 64.97% (GDRO) to 78.86%. While standard ERM models spuriously correlate the presence of the colored bandage with the benign class, CAMEL reduces the model's dependence on spurious features. We verify this by constructing a modified ISIC subgroup (Appendix D.7) for the malignant class that also contains bandages. Figure illustrates using GradCAM (Selvaraju et al., 2017) that CAMEL removes the model's is reliance on the spurious bandage feature, shifting attention to the skin lesion instead.
Table 6: Comparison on ISIC. Aver- age of trials (one standard deviation indicated in parentheses).
Evaluation Metric
Method
Robust Acc.
AUROC
ERM
65.59 (1.17)
92.48 (0.80)
GDRO
64.97 (3.15)
89.50 (2.50)
CAMEL
77.45 (0.35)
92.47 (0.38)
5 CONCLUSION
Domain experts face a common problem: how can classifiers that exhibit unequal performance on different subgroups of data be fixed? To address this, introduced model patching, new frame- work that improves a classifier's 's subgroup robustness by encouraging subgroup-feature invariance. Theoretical analysis and empirical validation suggest that model patching can be useful tool for domain experts in the future.

---

Published as a conference paper at ICLR 2021
ACKNOWLEDGMENTS AND DISCLOSURE OF FUNDING
We thank Pang Wei Koh, Shiori Sagawa, Geoff Angus, Jared Dunnmon, and Nimit Sohoni for assistance with baselines and datasets and useful discussions. We thank members of the Hazy Research group including Mayee Chen, Megan Leszczynski, Sarah Hooper, Laurel Orr, and Sen Wu for useful feedback on previous drafts. KG and AG are grateful for Sofi Tukker's assistance throughout this project. We gratefully acknowledge the support of DARPA under Nos. FA86501 1877665 (SDH) and FA86501827882 (ASED); NIH under No. U54EB020405 (Mobilize), NSF under Nos. CCF1763315 15 (Beyond Sparsity), CCF1563078 (Volume to Velocity), and 1937301 (RTMLL): ONR under No. N000141712266 (Unifying Weak Supervision); the Moore Foundation, NXP, Xilinx, LETI-CEA, Intel, IBM, Microsoft, NEC, Toshiba, TSMC, ARM, Hitachi, BASF, Accenture, Ericsson, Qualcomm, Analog Devices, the Okawa Foundation, American Family Insurance, Google Cloud, Swiss Re, the Salesforce Deep Learning Research grant, the HAI-AWSS Cloud Credits for Research program, and members of the Stanford DAWN project: Teradata, Facebook, Google, Ant Financial, NEC, VMWare, and Infosys. The U.S. Government is authorized to reproduce and distribute reprints for Governmental purposes notwithstanding any copyright notation thereon. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views, policies, or endorsements, either expressed or implied. of DARPA, NIH, ONR, or the U.S. Government.
REFERENCES
Amjad Almahairi, Sai Rajeswar, Alessandro Sordoni, Philip Bachman, and Aaron Courville. Augmented cyclegan: Learning many-to-many mappings from unpaired data, arXiv preprint arXiv: 10151, 2018.
Antreas Antoniou, Amos Storkey, and Harrison Edwards. Data augmentation generative adversarial networks arXiv preprint 0tff::/711.044444 2017.
Martin Arjovsky, Léon Bottou, Ishaan Gulrajani, and David Lopez-Paz. Invariant risk minimization. arXiv preprint 1rXiv:1907.02893, 2019.
Henry S Baird, Document image defect models. In Structured Document Image Analysis, pp. 546-556. Springer, 1992.
Shumeet Baluja and Ian C Fischer. Adversarial transformation networks: Learning to generate adversarial examples. ArXiv, hts//770309387,, 2017.
Sara Beery, Yang Liu, Dan Morris, Jim Piavis, Ashish Kapoor, Markus Meister, and Pictro Perona, Synthetic examples improve generalization for rare classes. ArXiv, abs/1904.05916, 2019.
David Berthelot, Nicholas Carlini, Ian Goodfellow, Nicolas Papernot, Avital Oliver, and Colin A Raffel. Mixmatch: A holistic approach to semi-supervised learning. In Advances in Neural Information Processing Systems, pp- 5050-5060, 2019.
Alceu Bissoto, Michel Fornaciali, Eduardo Valle, and Sandra Avila. (de) constructing bias on skin lesion datasets. 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pp. 2766-2774, 2019.
Christopher Bowles, Liang Chen, Ricardo Guerrero, Paul Bentley, Roger Gunn, Alexander Hammers, David Alexander Dickie, Maria Valdés Hernández, Joanna Wardlaw, and Daniel Rueckert. Gan augmentation: Augmenting training data using generative adversarial networks. arXiv preprint 1rXiv:1810.10866: 2018.
Andrew Brock, Theodore I Lim, James M. Ritchie, and Nick Weston. Neural photo editing with introspective adversarial networks. ArXiv, hbbs/1609.07993 2016.
Wen chen Sun, Fangai Liu, and Weizhi Xu. Unlabeled samples generated by gan improve the person re- identification baseline. In ICCTA 2019, 2019.
Yunjey Choi, Minje Choi, Munyoung Kim, Jung- Woo Ha, Sunghun Kim, and Jacgul Choo. Stargan: Unified generative adversarial networks for multi-domain image-to-image translation. In Proceedings of the IEEE conference computer vision and pattern recognition, pp. 3789-8797, 2018
Yunjey Choi, Youngjung Uh. Jaejun Yoo, and Jung- Woo Ha. Stargan v2: Diverse image synthesis for multiple domains. In Proceedings of the IEFF/CVF Conference Computer Vision and Partern Recognition, pp. 8188-8197, 2020.
10

---

Published as a conference paper at ICLR 2021
Noel CF Codella, David Gutman, M Emre Celebi, Brian Helba, Michael A Marchetti, Stephen W Dusza, Aadi Kalloo, Konstantinos Liopyris. Nabin Mishra, Harald Kittler, Skin lesion analysis toward melanoma detection: challenge at the 2017 international symposium on biomedical imaging (isbi), hosted by the international skin imaging collaboration (isic). In 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018), pp. 168-172. IEEE, 2018.
Ekin D Cubuk, Barret Zoph, Dandelion Mane. Vijay Vasudevan, and Quoc V Le. Autoaugment: Learning augmentation strategies from data. In Proceedings of the IEEE conference on computer vision and pattern recognition,, pp. 113-123, 2019a.
Ekin Cubuk, Barret Zoph, Jonathon Shlens, and Quoc Le. Randaugment: Practical data augmentation with no separate search. arXiv preprint arXiv: 13719, 2019b.
Xiaodong Cui, Vaibhava Goel, and Brian Kingsbury. Data augmentation for deep neural network acoustic modeling. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 23:1469-1477 2015.
Tri Dao, Albert Gu, Alexander J. Ratner, Virginia Smith, Christopher De Sa, and Christopher Ré, A kernel theory of modern data augmentation. Proceedings of machine learning research, 97:1528 1537, 2018.
Koen Deschacht and Maric-Francine Moens. Semi-supervised semantic role labeling using the latent words language model. In EMNLP, 2009.
Terrance DeVries and Graham W Taylor. Improved regularization of convolutional neural networks with cutout. arXiv preprint arXiv:1708.0455555 2017.
Nikita Dvomik, Julien Mairal, and Cordelia Schmid. On the importance of visual context for data augmentation scene understanding. IEEE transactions on pattern analysis and machine intelligence, 2018.
Debidatta Dwibedi, Ishan Misra, and Martial Hebert. Cut, paste and learn: Surprisingly easy synthesis for instance detection. In Proceedings of the IEEE International Conference on Computer Vision, pp. 1301-1310, 2017.
Logan Engstrom, Dimitris Tsipras, Ludwig Schmidt, and Alcksander Madry, A rotation and translation suffice: Fooling cnns with simple transformations. ArXiv, abs/1712.02779, 2017.
Marzich Fadaee, Arianna Bisazza, and Christof Monz. Data augmentation for low-resource neural machine translation. In ACI., 2017.
Yaroslav Ganin, Evgeniya Ustinova, Hana Ajakan, Pascal Germain, Hugo Larochelle, François Laviolette, Mario Marchand, and Victor Lempitsky. Domain adversarial Learning Research, 17(1):2096 2030, 2016.
Domain-adversarial training of neural networks, The Journal Machine
Jacoh R. Gardner, Matt Kusner, Yixuan Li, Paul Upchurch, Kilian Q. Weinberger, and John Hoperoft. Deep manifold traversal: Changing labels with convolutional features. ArXiv, abs/1511.06422 1.06421, 2015.
Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial nets. In Advances in neural information processing systems, pp.
2014a.
Ian J. Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. CoRR, abs/1412.6572, 2014b.
Sven Gowal, Chongli Qin, Po-Sen Huang. Taylan Cemgil, Krishnamurthy Dvijotham, Timothy Mann, and Pushmeet Kohli. Achieving robustness the wild via adversarial mixing with disentangled representations. arXiv preprint arXiv:1912.03192. 2019.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Identity mappings in deep residual networks. In European conference on computer vision, pp. 630-645 Springer, 2016.
Christina Heinze-Deml and Nicolai Meinshausen. Conditional variance penalties and domain shift robustness.
arXiv preprint 171iv 1469, 2017.
Dan Hendrycks, Norman Mu, Ekin D Cubuk, Barret Zoph, Justin Gilmer, and Balaji Lakshminarayanan. Augmix: A simple data processing method to improve robustness and uncertainty. arXiv preprint 1rXiv:1912.0278.. 2019.
Daniel Ho, Eric Liang, Ion Stoica Pieter Abbeel, and Xi Chen. Population based augmentation: Efficient. learning of augmentation policy schedules. arXiv preprint /rX.v:9905.05399 2019.
11

---

Published as a conference paper at ICLR 2021
Zhiting Hu, Bowen Tan, Ruslan Salakhutdinov. Tom Michael Mitchell, and Eric P. Xing. Learning data manipulation for augmentation and weighting. In NewrIPS, 2019.
Sheng-Was Wei Huang, Che- Thung Lin, Shu-Ping Chen, Yen-Yi Yi Wu, Po-Hao Hsu, Shang- Hong Lai. Auggan: Cross domain adaptation with gan-based data augmentation. In ECCV, 2018.
Phillip Isola, Jun- Yan Zhu, Tinghui Zhou, and Alexci A Efros. Imago-to-image translation with conditional adversarial networks. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 1125-1134, 2017.
Navdeep Jaitly and E. S. Hinton. Vocal tract length perturbation (vtip) improves speech recognition. In Proc. ICML Workshop on Deep Learning for Audio, Speech and Longuage, 2013.
Robin Jia and Perey Liang. Data recombination for neural semantic parsing. ArXiv, abs/ 1606.03622...... 2016.
Can Kanbak, Seyed- Moyed-Mohsen Moosavi Deetool- and Pascal Frossard. Geometric robustness of deep networks: Analysis and improvement. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 1441, 2017.
Harini Kannan, Alexey Kurakin, and lan Goodfellow, Adversarial logit pairing. arXiv preprint arXi: 1803.06373, 2018.
Tero Karras, Samuli Laine, and Timo Aila. A style-hased generator architecture for generative adversarial networks. 2019 IEEF/CVF Conference Computer Vision and Pattern Recognition (CVPR), pp. 4396-4405, 2018.
Michael Kearns, Seth Neel, Aaron Roth, and Zhiwei Steven Wu. Preventing fairness gerrymandering: Auditing and learning for subgroup fairness. In International Conference on Machine Learning, pp. 2564-2572, 2018.
Tom Ko. Vijayaditya Peddinti. Daniel Povey, and Sanjeev Khudanpur. Audio augmentation for speech recogni- tion. In INTERSPEECH, 2015.
Sosuke Kobayashi. Contextual augmentation: Data augmentation by words with paradigmatic relations. ArXiv, 0bs//885.0622) 2018.
Oleksandr Kolomiyets, Steven Bethard, and Marie-Francine Moens. Model-portabilitty experiments for textual temporal analysis. In ACI, 2011.
Alex Krizhevsky, Ilya Sutskever, and Geoffrey E. Hinton. Imagenet classification with deep convolutional neural networks. In NIPS, 2012.
Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11):22 (6(11)) 2324, 1998.
Ya Li. Xinmei Tian, Mingming Gong, Yajing Liu, Tongliang Liu, Kun Zhang, and Dacheng Tao. Deep domain generalization via conditional invariant adversarial networks. In Proceedings of the European Conference on Computer Vision (ECCV). pp. 624 0449639,220 2018.
Sungbin Lim, Iddo Kim, Tacsup Kim, Chiheon Kim, and Sungwoong Kim. Fast autoaugment. In Advances in Neural Information Processing Systems, pp. 6662-6672, 2019.
Ziwei Liu, Ping Luo, Xiaogang Wang, and Xiaoou Tang. Deep learning face attributes in the wild. In Proceedings of the IEEE international conference computer vision, pp. 3730-3738, 2015.
Mingsheng Long, Zhangjic Cao, Jianmin Wang, and Michael Jordan. Conditional adversarial domain adaptation. In Advances in Neural Information Processing Systems, pp. 1640-1650, 2018
Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. ArXiv, abs/ 700.06683,, 2017.
Giovanni Mariani, Florian Scheidegger, Roxana Istrate, Costas Bekas, and Cristiano Malossi. Bagan: Data augmentation with balancing gan. arXiv preprint arXiv:1803.0065:: 2018.
Marian Mazzone and Ahmed Elgammal. Art, creativity, and the potential of artificial intelligence. In Arts, volume 8, pp. 26. Multidisciplinary Digital Publishing Institute, 2019.
Ninarch Mehrabi, Fred Morstatter, Nripsuta Saxena, Kristina Lerman, and A. Galstyan. A survey bias and faimness machine learning- ArXiv, hbs/1908.00633. 2019.
12

---

Published as a conference paper at ICLR 2021
Juan Marořas Molano, Roberto Paredes, and Daniel Ramos-Castro. Generative models for deep learning with very scarce data. In CIARP, 2018
Seyed-Mohsen Moosavi-Dezfooll, Alhussein Fawzi, Jonathan Uesato, and Pascal Frossard. Robustness via curvature regularization, and vice versa. 2019 IEEE/CVF Conference Computer Vision and Pattern Recognition (CVPR), pp. 9070-9078, 2018.
Saypraseuth Mounsaveng, David Vázquez, Ismail Ben Ayed, and Marco Pedersoli. Adversarial learning of general transformation for data augmentation. ArXiv, ahs/1909.09801, 2019.
Norman Mu and Justin Gilmer. Mnist-e: A robustness benchmark for computer vision. arXiv preprint arXiv:I9000 02337, 2019.
Luke Dakden-Rayner, Jared Dunnmon, Gustavo Carncir, and Christopher Ré. Hidden stratification causes clinically meaningful failures in machine learning for medical imaging. arXiv preprint arXiv: 1209. 2019.
Augustus Odena. Christopher Olah, and Jonathon Shlens. Conditional image synthesis with auxiliary classifier gans. In ICMI., 2016.
Nicolas Papernot, Patrick D. McDaniel, Xi Wu, Somesh Jha, and Ananthram Swami. Distillation as defense to adversarial perturbations against deep neural networks. 2016 IEEE Symposium on Security and Privacy (SP), pp. 2015.
Mchran Pesteic, Purang Abolmacsumi, and Robert Rohling, Adaptive augmentation of medical data using independently conditional variational auto-encoders. IEEE Transactions on Medical Imaging, 38:2807-2820, 2019.
Haonan Qiu, Chaowei Xiao, Lei Yang, Xinchen Yan. Honglak Lee, and Bo Li. Semanticadv: Generating adversarial examples via attribute-condittonall image editing. ArXiv, abs/1906.07927,, 2019.
Alexander Ratner, Henry Ehrenberg, Zeshan Hussain, Jared Dunnmon, and Christopher Ré. ..earning to compose domain-spectie transformations for data augmentation. In Advances in neural information processing systems, pp. 3236-3246. 2017a.
Alexander J. Ratner, Henry R. Ehrenberg, Zeshan Hussain, Jared Dunnmon, and Christopher Ré. Learning to compose domain-specific transformations for data augmentation. Advances in neural information processing systems, 30:3239-3249, 2017b.
Scott E. Reed, Kihyuk Sohn, Yuting Zhang, and Honglak Lec. Learning to disentangle factors of variation with manifold interaction. In ICML, 2014.
Scott F. Reed, Yi Zhang, Yuting Zhang, and Honglak Lee. Deep visual analogy-making. In NIPS, 2015.
Laura Rieger. Chandan Singh, W. James Murdoch, and Bin Interpretations are useful: penalizing explanations to align neural networks with prior knowledge. ArXiv, 1ss/1999.138844 2019.
Shiori Sagawa, Pang Wei Koh, Tatsunori B Hashimoto, and Percy Liang. Distributionally robust neural networks group shifts: On the importance regularization worst-case generalization. The International Conference on Learning Representations (ICLR), 2020.
Veit Sandfort, Ke Yan, Perry J. Pickhardt, and Ronald M. Summers. Data augmentation using generative adversarial networks (cyclegan) to improve generalizability in et segmentation tasks. Scientific Reports, 2019.
Ramprasaath Selvaraju. Michael Cogswell, Abhishek Das. Ramakrishna Vedantam, Devi Parikh, and Dhruv Batra. Grad-cam: Visual explanations from deep networks via gradient-based localization. In Proceedings of the IEEE international conference on computer vision, pp. 618-626, 2017.
Rico Sennrich, Barry Haddow, and Alexandra Birch. Improving neural machine translation models with monolingual data. ArXiv, abs/1511.066 2015.
Miikka Silfverberg, Adam Wiemerslage, Ling Liu, and Lingshuang Jack Mao. Data augmentation morpho- logical reinflection. In CoNLL Shared Task, 2017.
Patrice Y. Simard, Bernard Victori. Yann LeCun, and John S. Denker. Tangent prop formalism for specifying selected invariances an adaptive network. NIPS, 1991.
Patrice Y. Simard, Yann LeCun, and John S. Denker. Efficient pattern recognition using new transformation distance. In NIPS, 1992.
13

---

Published as a conference paper at ICLR 2021
Patrice Y. Simard, Yann LeCun, John S. Denker, and Bernard Victori. Transformation invariance in pattern recognition tangent distance and tangent propagation. In Neural Networks: Tricks of the Trade, 1998.
Patrice Y. Simard, David Steinkraus, and John C. Platt. Best practices for convolutional neural networks applied to visual document analysis. Seventh International Conference on Document Analysis and Recognition, 2003. Proceedings. pp. 958-963, 2003.
Yang Song, Rui Shu, Nate Kushman, and Stefano Ermon. Constructing unrestricted adversarial examples with generative models. In NeurlPS, 2018.
Yannis Stylianou, Olivier Cappé, and Eric Moulines. Continuous probabilistic transform for voice conversion. IEEE Trans. Speech and Audio Processing, 6:131-142, 1998.
Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian J. Goodfellow, and Rob Fergus. Intriguing properties of neural networks. CoRR, abs/1312.6199, 2013.
Christian Szegedy, Wei Liu, Yangqing Jia, Pierre Sermanet, Scott Reed, Dragomir Anguelov, Dumitru Erhan, Vincent Vanhoucke, and Andrew Rahinovich. Going deeper with convolutions. 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1-9, 2014.
Toan Tran, Trung Pham, Gustavo Carneiro, Lyle Palmer, and Ian Reid. bayesian data augmentation approach for learning deep models. ArXiv, abs/1710. 10564, 2017.
Paul Upchurch, Jacob Gardner, Geoff' Pleiss, Robert Pless, Noah Snavely, Kavita Bala, and Kilian Weinberger. Deep feature interpolation for image content changes, In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 7064-7073, 2017.
William Yang Wang and Diyi Yang. That's annoying!!!: A lexical and frame-semantic embedding based data augmentation approach automatic categorization of annoying behaviors using petpeeve tweets. In EMNLP, 2015.
Yaxing Wang, Chenshen Wu, Luis Herranz, Joost van de Weijer, Abel Gonzalez- Garzzal Garcia, and Bogdan Raducanu. Transferring gans: generating images from limited data. In Proceedings of the European Conference on Computer Vision (ECCV), pp. 218-234, 2018.
Jason Wei and Kai Zou. Eda: Fasy data augmentation techniques for hoosting performance on text classification tasks. In FMNIP//JCNIPP, 2019.
Julia K Winkler, Christine Fink, Ferdinand Toberer, Alexander Enk, Teresa Deinlein, Rainer Hofmann-Welleeee Luc Thomas, Aimilios Lallas, Andreas Blum, Wilhelm Stolz, et al. Association between surgical skin markings in dermoscopic images and diagnostic performance of deep learning convolutional neural network for melanoma recognition. JAMA dermatology, 155(10): 1 1141, 2019.
Chaowei Xiao, Bo Li, Jun- Yan Zhu, Warren Hc, Mingyan Liu, and Dawn Xiaodong Song. Generating adversarial examples with adversarial networks. In IJCAI, 2018.
Qizhe Xie, Zihang Dai, Eduard Hovy, Minh-Thang Luong, and Quoe V Le. Unsupervised data augmentation for consistency training. arXiv preprint arXiv:
2019.
Ziang Xic, Sida I. Wang, Jiwei Li, Daniel Lévy, Aiming Nic, Dan Jurafsky, and Andrew Y. Ng. Data noising as smoothing in neural network language models. ArXiv, abs/1 703.02573, 2017.
Larry S. Yaeger, Richard F. Lyon, and Brandyn J. Webb. Effective training of neural network character classifier for word recognition. In NIPS, 1996.
Adams Wei Yu, David Dohan, Minh-Than Thang Luong, Rui Zhao, Kai Chen, Mohammad Norouzi, and Quoe V. Le. Qanet: Combining local convolution with global self-attention for reading comprehension. ArXiv, abs/1804.0954) 2018.
Sangdoo Yun, Dongyoon Han, Scong Joon Oh, Sanghyuk Chun, Junsuk Choe, and Youngjoon Yoo. Cutmix: Regularization strategy to train strong classifiers with localizahle features. In Proceedings of the IEEE International Conference on Computer Vision, pp.
2019.
Hongyi Zhang, Moustapha Cisse, Yann N Dauphin. and David Lopez-Paz. mixup: Beyond empirical risk minimization. arXiv preprint 171.0427,. 2017.
Xiang 7hang, Junbo Jake Zhao, and Yann LeCun. Character-level convolutional networks for text classification. In NIPS, 2015.
14

---

Published as a conference paper at ICLR 2021
Xiaofeng Zhang, Zhangyang Wang. Dong Liu, and Qing Ling. Dada: Deep adversarial data augmentation for extremely low data regime classification. ICASSP 2019 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 2807-2811 2018.
Stephan Zheng. Yang Song. Thomas Leung.
Goodfellow. Improving robustness deep neural networks via stability training. In Proceedings of the ieee conference on computer vision and pattern recognition, pp. 4480-4488, 2016.
Jun-Yan Yan Zhu, Taesung Park, Phillip Isola, and Alexci A Efros. Unpaired amage-to-image translation using cycle-consistent adversarial networks. In Proceedings of the IEEE international conference on computer vision, pp. 2223-2232, 2017.
Barret Zoph, Ekin Dogus Cubuk, Golnaz Ghiasi, Tsung-Yi Yi Lin, Jonathon Shlens, and Quoe V. Lc. Learning data augmentation strategics for object detection, ArXiv, abs// 1906. 1 1172, 2019.
15

---

Published as a conference paper at ICLR 2021
A GLOSSARY OF NOTATION
We provide a glossary of notation used throughout the paper.
Table Summary of notation used throughout this work.
Notation
Description
Preliminaries
3, X,Y.Z P Pg. P. X.Y.Z
Example, cluss, subgroup
Random variables for examples, classes, and subgroups
joint distribution for X.
The distribution for X conditioncd on class or subgroup
Domains for X, Y.
Zy C
The subgroups belonging to class
E
The class of a subgroup
fo:
A
The purameterized class prediction model, returning categorical distribution over
A random variable with support y indicating a random sample from the output of fo
Coupled sets augmentations
[r]
A coupled set
XX]
Random variable for coupled sets
coupled set [a] The coupled set (Definition 1) of examples in z's class y. Same [at]. An augmented coupled set
Example belonging subgroup
z7.
al [pt]a
Example helonging to suhgroup = the augmented coupled set augmented coupled set of examples á class yi Same as (f].
Zzy
k
Number subgroups in any (generic) class
Model components and losses
Loo
Sum of CycleGAN consistency and identity losses
L.
Sclf-consistency loss (Eq 1)
Translation-cone ceation-consistencyy loss (Eq 2)
Total consistency loss (Eq 3)
L: X2 RR
distance function, used CycleGAN consistency losses
A
Hyperpuramettr controlling the strength of the consisteney The KL divergence
KLO))
JS(-) T(-)
The Jensen-Shannon divergence (Definition 2)
The Mutual Information
B EXTENDED RELATED WORK
We provide a comprehensive overview of related work and highlight connections to our work below.
B.1 OVERVIEW OF DATA AUGMENTATION
Data augmentation is widely used for improving the aggregate performance of machine learning models in computer vision (Krizhevsky et al., 2012; Szegedy al., 2014), natural language processing (Kolomiyets al., 201 Scnnrich et al., 2015; Zhang et al., 2015) and audio (Cui ct al., 2015; Ko ct al., 2015). The theoretical motivation for data augmentation is largely based on the tangent propagation formalism (Dao et al., 2018; Simard et al., 1991; 1992; 1998) which expresses the desired invariances induced by data augmentation as tangent constraints the directional derivatives of the learned model.
Early work considered augmentations image defects (Baird, 1992) stroke warping (Yaeger et al., 1996) for character recognition. Since then, augmentation is considered an essential ingredient computer vision (LeCun et al., 1998; Simard et al., 2003), with commonly used augmentations including random flips, rotations and crops (He et al., 2016; Krizhevsky et al., 2012; Szegedy et al., 2014). Applications of augmentation in computer vision include object detection (Dwibedi al., 2017: Zoph al., 2019) and understanding (Dvornik al., 2018)
In natural language processing, common data augmentation techniques include fack-translation (Sennrich et al., 2015; Yu et al., 2018), synonym or word substitution (Fadaee et al., 2017; Kobayashi, 2018; Kolomiyets et al., 2011; Wang & Yang. 2015; Zhang al., 2015), noising (Xie et al., 2017), grammar induction (Jia & Liang, 2016), text editing (Wei Zou, 2019) and other heuristies (Deschacht & Moens, 2009; Silfverberg al., 2017). speech and audio applications, augmentation is also commonly used, through techniques such as vocal tract length warping (Jaitly & Hinton, 2013; Ko et al., 2015) and stochastic feature mapping (Cui et al., 2015; Stylianou et al., 1998).
In this work, we perform empirical evaluation image classification tasks although our ideas can be extended classification of other modalities such as speech and text.
16

---

Published as a conference paper at ICLR 2021
B.2 AUGMENTATION PRIMITIVES AND PIPELINES
Next, we highlight the particular augmentation primitives that have been used in prior work. Our work differentiated by the use of learned augmentation primitives using CycleGANs (Zhu ct al., 2017), as well as a theoretical justification for this choice.
Hand-Crafted Augmentation Primitives. Commonly used primitives are typically heuristic transfor- mations, such as rotations, flips or crops (Krizhevsky al., 2012; Szegedy et al., 2014). Recent work has hand-crafted more sophisticated primitives, such as Cutout (DeVries & Taylor, 2017), Mixup (Zhang et al., 2017), CutMix (Yun et al., 2019) and MixMatch (Berthelot et al., 2019). While these primitives have culminated in compelling performance gains (Cubuk et al., 2019a;b), they produce unnatural images and distort image semantics.
Assembling Augmentation Pipelines. Recent work has explored learning augmentation policies - the right subset of augmentation primitives, and the order in which they should be applied. The learning algorithm used can be reinforeement learning (Cubuk et al., 2019a; Ratner et al., 2017a) or random sampling (Cubuk al., 2019b). More computationally efficient algorithms for learning augmentation policies have also been proposed (Ho et al., 2019; Lim et al., 2019).
These pipelines are primarily derived from the fixed set of generic image transformations we discussed earlier, and do not directly target specific attributes. By contrast, we consider learning augmentation primitives that target subgroup robustness, and additionally demonstrate in Section 4.2.2 that heuristic augmentations can complement CAMEL yield additional performance gains.
Learned Augmentation Primitives. There is substantial prior work learning image transformations that produce semantic, rather than superficial changes to an image. A common paradigm is to learn semanti- cally meaningful data representation, and manipulate embeddings in this representation to produce desired transformation, Transformations can then be expressed as vector operations over embeddings (Reed et al., 2015; Upchurch et al., 2017) manifold traversals (Gardner et al., 2015; Reed et al., 2014). Alternative approaches rely on training conditional generative models (Almahairi et al., 2018; Brock al., 2016; Choi et al., 2018; Isola et al., 2017: Zhu et al., 2017) that learn a mapping between two or more image distributions. Much of this prior work is motivated by the need for sophisticated tools for image editing (Karras et al., 2018; Upchurch et al., 2017) e.g. for creative applications of machine learning (Mazzone & Elgammal, 2019).
Closer to our setting work that explores the use of these transformations for data augmentation. A prominent use case focuses on imbalanced datasets, where learned augmentations are used to generate examples for underrepresented classes or domains. Examples include BaGAN (Mariani et al., 2018), 8), DAGAN (Antoniou et al., 2017), TransferringGAN (Wang et al., 2018) and others (Beery et al., 2019; Hu et al., 2019; Molano et al., 2018; Mounsaveng et al., 2019; Tran et al., 2017; Zhang ct al., 2018). Applications to medical data (Pesteic al., 2019; Sandfort et al., 2019) and person re- ce-ientifiiation (chen Sun et al., 2019) have also been explored.
Our model patching framework differs substantially from these papers, since we focus on robusiness. We discuss this intersection next.
B.3 DATA AUGMENTATION AND MODEL ROBUSTNESS
Prior work on model robustness has mostly focused on learning models that are robust to bounded ty-norm perturbations (Goodfellow al., 2014b; Goooavvi-ezzfoo et al., 2018; Papemot et al., 2015; Szegedy et al., 2013) using ideas such as adversarial training (Madry et al., 2017). A separate line of work considers consisteney training (Hendrycks et al., 2019; Kannan et al., 2018; Zheng ct al., 2016), where predictions are made invariant to input perturbations, often by minimizing a divergence between the predictions for the original and perturbed examples. Consistency regularization has also been shown to be effective for semi-supervised learning (Xie al., 2019).
Consistency training. contrast equation (3) with consistency losses from prior work. Unsupervised Data Augmentation (UDA) (Xie et al., 2019) simply controls an asymmetric divergence between the original example and each augmented example individually E. (=)) AugMix (Hendrycks et al., 2019) uses a Jensen- Shassnn-haanon divergence
(L(F())] KL (L(EE))) =
1:
where m 1 [f(x) E,, f(t.). This can he seen version of our consistency, but with different weights and different mean distribution that the KL.s heing computed against. Our loss (3) has an important
17

---

Published as a conference paper at ICLR 2021
asymmetry between the original example x and the augmentations ž; One reason to prefer it is simply noting that as the number & of subgroups grows, the AugMix loss tends to the second term, and does not control for the discrepancy between predictions on the original domain (x) and the augmented ones f(ā.). Our consistency regularization instead allows us to bound mutual information objective between variables in the joint subgroup distribution, yielding tractable and interpretable objective (Section 3). In addition, we compare with these consistency losses and provide empirical results in Section 4.2.2.
Robustness to more general augmentations has also been explored (Baluja Fischer, 2017; Engstrom et al., 2017; Kanbak et al., 2017; Odena et al., 2016; Qiu et al., 2019; Song et al., 2018; Xiao et al., 2018), but there is limited work on making models more robust to semantic data augmentations. The only work we are aware of is AdvMix (Gowal et al., 2019), which combines a disentangled generative model with adversarial training to improve robustness.
Our work contributes to this area by introducing the model patching framework to improve robustness in a targeted fashion. Specifically, under the data-generating model that we introduce, augmentation with a CycleGAN (Zhu et al., 2017) model allows us to learn predictors that are invariant to subgroup identity.
B.4 LEARNING ROBUST PREDICTORS
Recent work (Sagawa et al., 2020) introduced GDRO, a distributionally robust optimization method to improve worst-case accuracy among set of pre-defined subgroups. However, optimizing the GDRO objective does not necessarily prevent model from learning subgroup-spectio features. Instead, strong modeling assumptions on the learned features may be required, e.g. Invariant Risk Minimization (Arjovsky et al., 2019) attempts to learn an invariant predictor through a different regularization term. However, these assumptions are only appropriate for specialized setups where extreme out-of-domain generalization is desired. Unfortunately. these approaches still suffer from standard learning and generalization issues stemming from a small number of examples in the underperforming subgroup(s)- even with perfect subgroup information. Additionally, they necessarily trade off average (aggregate) accuracy against different robust metric.
B.5 RELATIONSHIP TO SUBGROUP FAIRNESS
common goal in subgroup fairness is to ensure statistical parity of predictions across groups, and a variety of fairness criteria have been proposed (Mehrabi et al., 2019). Others have considered a theoretical setting where the number of subgroups can be large (possibly infinite) (Kearns et al., 2018). Typically, this line of work assumes that groups common across classes, while in our setting, consider the possibility of different subgroups each class. The bulk of this work focuses on prediction problems on small datasets, rather than with high-dimensional image data.
C DETAILED ANALYSIS
We begin with background material on the CycleGAN (Appendix C.I) Jensen-Shannee Shannon Divergence (Appendix C.2). Appendix C.3 contains a longer discussion of the modeling assumptions in Section 3, fleshing out the distributional assumptions and definition of coupled sets. Appendix C.4 and Appendix C.5 completes the proofs of the results in Section 3.
C.I BACKGROUND: CYCLEGAN
Given two groups A and B, CycleGAN learns mappings F: B - A and G A B given unpaired samples PA.b b Pg. Along with these generators, it has adversarial discriminators DA Dp trained with the standard GAN objective, i.e. DA distinguishes samples a ~ PA from generated samples F(b), where 6 PB. In CAMEL, A and B correspond to data from a pair of subgroups z, 2 of class.
CycleGAN uses cycle consistency loss to ensure that the mappings F and G are nearly inverses of each other, which biases the model toward learning meaningful cross-domain mappings. An additional identity loss is sometimes used which also encourages the maps F, G preserve their original domains i.e. F(a) a for Pa. These cycle consistency and identity losses can be modeled by respectively minimizing Leo (a, F(G())) and Lo (a, F())) for some function Leg which measures some notion of distance on A (with analogous losses for B). Figure 5 visualizes the CycleGAN model.
Definition 1. The sum the CycleGAN cycle consistency LoG(a, F(G(o)) and identity LeGla, F(a)) losses on domain A is denoted La 0) for overall CycleGAN parumeters 0, and similarly for domain B. In the context of Stage I of model patching, let Foo(x;80 f) denote the loss when the domain is one of the subgroups 2.
The original CycleGAN uses the & distance Ta a) ||a-ãllı- However, we note that many other functions be used to enforce similarity. In particular, we point that pair-conditioned discriminator D(a, a} + [0, 1)²
18

---

Published as a conference paper at ICLR 2021
also be used, which accepts coupled pair of original and translated examples and assigns probability each being the original example. If the guesses for the true translated examples are D. and Da respectively, then the distance is L(a, ä) = maxp log Da + log(1 - Da) + log To sanity check that this has properties of a distance, note that L decreases as a, are more similar, as the discriminator has trouble telling them apart.
Intuitively, the discriminator loss is measure of how sim- ilar the original and generated distributions are, which be used in Section C.5 to prove our main result.
A
C.2 BACKGROUND: PROPERTIES OF THE WOWSEN-SHANNON DIVERGENCE
ait
A
We define the Censen-Shanoor divergence (JSD) and its properties that will be used in method and analysis.
Cycle consistency loss:
Identity loss: I.(0,0")
L (a.a")
Definition 2. The
Divergence (JSD)
of distributions PI...., Pe is JS(PI..... P&) Ei (I)PI.))) where M = E P.
Figure 5: CycleGAN learns mappings domains A AU B, where F maps examples and maps to B. To model possible distribution shift intro- duced by the generative model, we denote their images as Im(F) A, Im(G) B respectively, Semantically consistent mappings are encouraged with the cycle consistency and identity losses, c.g. to ensure that F F(a) a for all E A.
We overload the JS(.) function in the following ways. The JSD of random variables X1...., Xx. is the JSD of their lows (distributions).
Additionally, we define the JSD of vector-valued inputs if they represent distributions from context. For example, for model I thut outputs a vector representing categoricul distribution, JS([a(21)....... fo(2x) the JSD those distributions.
We briefly review important properties of the JSD. Un- like the KI. divergence and other notions of distributional distance, the JSD can be related metric.
Proposition 1. The JSD is the square of a metric. In particular, any three distributions p, q, satisfy JS(p, q) rS( r)
Finally, the following fact about the JSD relating it to the mutual information of a mixture distribution and its indicator variable will be useful in our analysis.
Proposition Let 7 be a uniform categorical indicator variable with support [k]] and Ps, E [ki] be distribu- tions. Let X ~ P2, Z ou 7. be the random variable associated with the mixture distribution of the P, controlled by the indicator 7. Then T(X; 7.) JS P1. Pa).
Finally, we review standard results (e.g., g. from the GAN literature) on the relationship between discriminators and the IS divergence, which relates the loss of an optimal discriminator to the JSD of the two distributions. We include proof for completeness.
Proposition 3. Consider two domains A and À (Le., distributions on a common support A), with densities p(a), p(a) respectively. Consider discriminator D A + R optimized maximize the loss
C(D) Ea-pita log D(a) +
2 Ba-B(a) log(1 D(a)).
Then the value of this for the optimal discriminator Dr JS(A, A) -log
Proof. Differentiate the loss with respect to the discriminator's output ((a) any example E A, which yields
1 1
2 p(o) D(a) - Ø(a)
- ((a)
19

---

Published as a conference paper at ICLR 2021
The loss is maximized at (a) p(o)+b(a)' and using Definition 2:
p(a)
The result follows from plugging this discriminator into the loss
p(a)
+ 1
p((a) Ba- p(a) + i(a)
[(D") = E Fampla) log
p(a) + p((a
1 KL
All A +
KL
Al|
A + A
2
2
2
(oo((2)
JJSAA,
log 2.
C.3 SUBGROUP INVARIANCE USING COUPLED DISTRIBUTIONS
common framework treating robustness over discrete groups aims to create invariances, or independencies between the learned model's 's features and these groups. We review this approach, hefore defining new model for the distributional assumptions used in this work. The notion of coupled sets we introduce underlies both stages of the framework and allows for stronger invariance guarantees than previous approaches, which will be analyzed in Appendix C.5.
flass-onddiiooee Subgroup Invariance. In order for a model to have the same performance over all values of Z. intuitively it should learn "Z "Z-invariant features", which can be accomplished in a few ways. Invariant Risk Minimization (IRM) (Arjovsky et et., 2019) calls the Z labels environments and aims to induce (Y I (xX)) 1 Z, where (xX) are the model's features, so that the classifier does not depend on the environment. Another line of work treats Z as domains and uses adversarial training to induce invariances of the form (o(X) 1 2) YY (Ganin et al., 2016; Li et al., 2018; Long et al., 2018), so that within each class, the model's features look the same across domains. We call this general approach caas--oondiional domain adversarial training (CDAT), which attaches domain 2 prediction head per class Y. and adopts an adversarial minmax objective so that the featurizer (xX) erases related information and reduces the model's dependence on Z.
Subgroup Invariance. Although previous works generally make no assumptions on how the data X among the groups 7 relate to each other, we note that common implicit requirement is that there is a "correspondence" between examples among different groups We codify this distributional assumption explicitly with notion coupling, which allows us to define and analyze stronger invariances.
Goupling-conditionn
particular, we assume that the underlying subgroups are paired or coupled, so that every example can be translated into the other subgroups. Definition formalizes our distributional notion of coupled sets.
Definition 1. For a given distribution a coupled set within class y a set {x=)=EZ, consisting of one example from each subgroup of y, where each example has the same probability. A coupling for distribution P on (X, Y, 7) is a partition of examples in into coupled sets. For any example x. E X, let [s] denote its coupled set. Let [x]]...., [[)) denote the elements of a coupled set [x] in a class with & subgroups. Let [X] denote the random variable that samples coupled set; i.e. taking [x) for random sampled from any fixed subgroup 2.
Additionally, we say that a distribution is subgroup-coupled if it satisfies Definition 1, i.e it has a coupling.
In the context of subgroups of a class this assumption entails that every example can be factored into its subgroup and coupled set membership. All examples that are members of particular coupled set can be thought of as sharing set of common features that signal membership the class. Separately, examples that are members of a particular subgroup can be thought to share common features that signal subgroup membership. Together, these two pieces of information identify any example from class c.
We represent this assumption by letting the (unobserved) random variable [X] represent the "class identity" of example X, which can be thought of as the class features that aren't 't specific to any subgroup. Thus, the full generating process of the data distribution (X, Y, Z, [X]) consists and subgroup Z within a class Y. which together control the actual example X. Note that [X] and Z are both more fine-grained and thus carry more information than Y. This pracess is illustrated in Figure 6a. Figure 6b illustrates this concept for the MNIST-Corrupted dataset (Mu & Gilmer, 2019). Given digit class such as
independently choosing a coupled set [X]
2 Note that this will typically not hold for the training distribution, since some subgroups may be underrepre- sented, making it much less probable that examples from those subgroups are sampled in a coupled set. However, we are concerned with robustness to test distribution where the subgroups are of equal importance and equally likely.
20

---

Published as a conference paper at ICLR 2021
Y = 3. subgroups correspond corruptions such as zigzags and dotted lines applied to the digits. A coupled set consists of these corruptions applied a clean digit.
Definition allows us to reason about the following stronger invariances. Given class E Y every example in subgroup 2 E Zy implicitly has corresponding examples in all subgroups Zy within its class, and the learned features for each of these coupled sets should be identical in order equalize performance between subgroups. the stronger coupling-oonditonee invariance
Thus instead of the weaker goal (x(X) 1 2) Y.
(o(X) 1 2) Y, [X] (6(X) 2) [X].
selgnuup
clatppie
prediction
OXX
Y
y-5
coupled
G it
(a) Joint distribution of examples X with their class labels Y, subgroup labels Z, and coupled sets [X].
(b) Illustration with the MNIIT-Corrupted dataset (Mu & Gilmer, 2019), where subgroups % different types of corruptions.
Figure 6: Subgroup-coupled distributions separate the coupled set to which an example belongs (with respect to their class), from its subgroup label.
Note since features matter insofar as their effect on final output y, it suffices to look case ((X) Ý. We first show in Section C.4 that CDAT methods target the invariance (Y - 2) Y by minimizing a lower bound for the conditional mutual information, [(Y; (1Y22 Z Y) (Lemma 1).
In Section C.5, prove main result: our combined objective function (4) targets the stronger invariance
( I Z) [X] by upper bounding the corresponding MI, which can be interpreted as forcing matching outputs for the examples in every coupled set.
C.4 MI BOUNDS FOR CLASS-CONDITIONED INVARIANCE
Recall that the high-level goal of CDAT is to induce independencies between subgroup information and the model's feature representation. In order to induce the desired invariance (ø(X) 1 Z) Y of class features from subgroup identities, natural approach
minimize conditional mutual information (ex(X;; Z Y). which is minimized at 0 when the invariance is satisfied and grows when 6(X) and Z are predictive of each other. This mutual information can be estimated using standard techniques.
Lemma 1. CDAT minimizes lower bound on the mutual information T()(X); Z Y), where ((X) is the feature layer where the domain prediction head is attached.
Proof. We have
1(0(X); H(Z Y) H(Z (()),))
H(Z Y) + E Ex,yyp(E,, Eemp(s [log(p(=(((z))))) v))
2 H(Z Y) + Ex.ymp(a Be)p(a)(()) [log(p> (=lø(x),
H(Z Y) + E..,,0())) [log(po (=lo(z))
which bounds the MI variationally through parametrized conditional model Pe. Up additive term H(Z Y) which is constant of the data distribution, this simply the cross-entropy loss model trained on top of the featurizer & predict from ((X) and Y. which coincides with the domain adversarial training approach.
By specializing ((X) to Ý. we obtain
Corollary 1. If CDAT attaches a domain prediction head to the prediction layer it optimizes lower bound on 2 Y).
Thus, although approaches involving domain adversarial training (Ganin et al., 2016; Li et al., 2018) motivate their approach through altemate concepts such as H- H--iiergeencss and GAN-based adversarial games, we see that they are implicitly minimizing simple variational estimate for mutual information.
In Section 4, Table 3's reported estimate of the mutual information uses Comllary 1.
21

---

Published as a conference paper at ICLR 2021
C.5 MI BOUNDS FOR COUPLING-CONDITTONN INVARIANCE
The stronger distributional assumptions Definition 1 allow us analyze the invariance ((X) [X]. which can be interpreted as forcing matching features for the data in every coupled set.
True Coupled Sets. Given a subgroup-coupled distribution, access to coupled sets allows analysis of stronger invariance assumptions.
First, we confirm that this is indeed stronger notion of invariance, that is
T(Z: [x]) 2 T(2; (x); y)-
(5)
This follows from the chain rule for mutual inequality:
((Z: [X]] (()) YX[X])
[X] [Y) + (();(()) [x]
(6)
=X] ((X) Y)
-T(Z; Y) T(Z; (.)()).
Here, the first two equalities follow from Definition (in particular, [X] and Z are more fine-grained than Y), and the last two follow from the chain rule for mutual information.
In particular, equation (5) quantifies the intuition that conditioning on an example's 's coupled set reveals more information then just conditioning on its class, Conversely, minimizing the LHS of (5) necessarily minimizes the objective [2; (X) Y) in (Li ct al., 2018), and an additional non-negative term 1(2; [X] 6(X), Y)
relating the features and identity of examples.
Moreover, the features (xX) are only relevant insofar as their ability to predict the label. Specializing (XX). this stronger conditional MI is related to the model's predictions; is exactly equal to the self-consistency regularizer (I) if the model had access to true coupled sets [x:].
Thus, in the case where ((X) = Ý is simply the model's prediction, this MI is simply the ShasnnnShannon divergence of the model's predictions.
Lemma
r(z; y [X]) JS (fo(e))))_s((())).............. ((o)()))
Proof. For any features o. the mutual information can be written
I(Z; [X]) x (B[Z [(]]: [X])
|5 (z)) [x]])
where the random variable [[o(X) [X] denotes the formal conditional expectation. The second equality follows since (Z 1 [X]) Y.
Consider specializing this to the case when o(X) y, i.c. it represents the random variable where an output class prediction Ý is sampled from the final class probability predictions fx( of the model. Since this is distributed as P3 fo(X.). we obtain
1
]
I(Z; Y (xx) = E(a)~1X1
2: K 1ETH
fo((=),)
(8)
= -E
fo((x)))
where the second equality follows by Proposition
Augmented Coupled Sets. In practice, may not have coupled sets [x]. Instead, we a gen- erative model such as a CycleGAN as proxy that provides noisy versions of the coupled set. denoted [i] = ([)].... [ā]x) where [G], are individual augmented examples per subgroup. However, the genera- tive augmentation model may not perfectly model the subgroup distribution; for example, it may introduce artifacts.
We model this distributional assumption explicitly:
Definition 3. Each subgroup 2, which has a distribution Ps over X. has corresponding augmented subgroup ? with distribution P3 representing augmented examples through the generative model(s). In particular, we suppose for any coupled set [zr), it has realizations [x) in subgroup and [i], subgroup ž.
22

---

Published as a conference paper at ICLR 2021
We also the notation [i]] for generated coupled set and [ā]: its realization in subgroup (a specific augmented example). Note that [ā] and the notation ž2, from Section 2.2 refer the same thing, the of augmented examples.
Figure also illustrates the concept of Definition 3: original domains A, B have corresponding domains A, B that are the images of the generators F, G.
We can control the difference between augmented and true subgroup distribution in two ways. First, the translation-loss L. (2) regularizes the average predictions from the augmentations to match those of the original example, constraining the prediction model to ignore general distribution shifts introduced by the generative models.
Morcover, the discrepancy between the loss are minimizing via CycleGAN CyeleGAN-auggenteer -uggmented examples C, = Ez , fo ([x) x )) can bounded by the loss of the pair-conditioned CycleGAN discriminators (Section 2.1), via metric properties of the JSD
JS (fo(())),, ..., ()))) (1) and the true objective JS ( ...,
Models such CycleGAN directly control the deviation of augmentions from the original examples, via the GAN discriminators and consistency losses. The following Lemma says that CycleGAN discriminator loss is the divergence between the original distribution in subgroup 2, and the generated distribution of subgroup 2. paralleling standard GAN results (Goodfellow et al., 2014a).
Lemma 3. The optimal discriminator between original subgroup distribution P2 and augmented subgroup Pss has loss Lio { [+]=) log 2.
Proof of Lemma 3. By Proposition 3,
[p)))x [=]=) log [ log ([))(= + [ log(1 - (())))
where Di is discriminator for this coupled set (within subgroup z). Instead of training separate discriminator per example coupled set, enough train single discriminator D conditioned this specific coupled set ([x] [x] ). In other words this is discriminator whose input is both the original example [ar]: and a generated
version (x] s, and for cach input guesses its chance of being a real example. This is exactly the pair-conditioned discriminator described in Section C.I.
Proof of Theorem 1. We finally put the pieces together to prove the main result, restated here for conve- nience.
Theorem 1. For model fo with outputs Y. the MI I(Y; Z (X) is the Jensen- Divergence (JSD) of predictions coupled sets [ (fo(z))ze(a)) In the case of 2 subgroups per class, this can be upper bounded by the CycleGAN and consistency losses
Bux (fr (z,, (F,,))) 0)
Σ ())
zEZy
particular, the global optimum of the trained CAMEL model induces Y 1 Z [X].
First, the equivalence of the quantity we care about ((2; Y; [X]) and the consistency loss on true coupled sets given by Lemma 2. It remains to bound [ fo ((]]a)), which can be bounded by the consistency loss on augmented examples S fo (ā) [=]2)) and the optimal CycleGAN losses (.S((f((=)))) fo(().))
by metric properties of the JSD.
Proof of Theorem I. Consider any fixed subgroup and let X, denote the R. V. from the mixture distribution of P. and Pa. i.c. cither a true example or an augmented example from subgroup x. Let W denote the (binary) indicator of this mixture. Then
JS(fo((z)=), fo((())) .)) = I(W; fx(.)) < I(; = [S( (3),,
(9)
where the equalities are Proposition 2 and the incquality is an application of the data processing inequality on the Markov chain W + X, fx.).)
Combining equation (9) with Lemma applying the definition of Lec. and summing over two groups 1, 2 yields
(8(6(()))) (St
(10)
(x 0) + (x;
By definition of the self-consistency loss (1) and Definition 2,
JS ([)) fo({))) Lo (3):0).
(11)
23

---

Published as a conference paper at ICLR 2021
Table 8: Number of training, validation and test examples in each dataset.
Dataset
Split
Subgroup Size (Y, 2)
even, clean
even, zigzag
odd, clean
odd. zigzag
MNIST-Corrclation
train
9900
100
100
9900
validation
9900
100
100
9900
test
4926
4926
5074
5074
landhird, land
landbird, water
waterbird, land
waterbird, water
Waterbirds
train 3498
184
56
1057
validation
467
466
133
133
2255
2255
642
642
non-hlonde, female
non-blonde, male
blonde, female
blonde,
CelebA-Undersan undersa-ddr
train
4054
66874
22880
1387
validation
8535
8276
2874
182
test
9767
7535
2480
180
benign. no bandage
benign, bandage
malignant, no bandage
malignant, bandage
ISIC
train 8062
7420
1843
validation
1034
204
test
1026
895
239
for any sample and where [+] denotes the generated coupled (F)( F2(x)) usual. Denoting the right hand side L. (r:; 0) for shorthand, summing equations (10) and (11), and using the metrie property of the ISD (Proposition 1) gives
JS (L)) s (x; (z) (x; 0)+
Finally, squaring and averaging over the dataset and applying Lemma gives the result of Theorem
I(;; Z [X]) & Ex~x (Ca(z;0)² + (2; (x;
These pieces he combined show that the GAN-hased modeling of subgroups (Stage and the consistency regularizer (Stage 2) together minimize the desired identity-condie condity--ondiiooneed mutual information, which completes the proof of Theorem 1.
D EXPERIMENTAL DETAILS
We provide detailed information about our experimental protocol and setup for reproducibility. including dataset information in D.I.
D.1 DATASET INFORMATION
We provide details for preprocessing and preparing all datasets in paper. Table summarizes the sizes of the subgroups present in each dataset. All datasets will be made available for download.
We mix data from MNIST (LeCun et al., 1998) and MNIST-Cornppe (Mu & Gilmer, 2019) create controlled setup. We classify digit parity Y feven, odd), where each class is divided into subgroups Z {clean, zigzag), drawing digits from MNIST and MNIST-Corrpped (with the zigzag corruption) respectively.
MNIST-CorreLation.
To generate the dataset, we use the following procedure:
Fix total dataset size N, and desired correlation p.
Sample
(o+D)
even digits from MNIST
N
(o+1)N
even digits from MNIST-Corrppeed
7
4
N
(
odd digits from MNIST
7
24

---

Published as a conference paper at ICLR 2021
[o+1)N odd digits from MNIST-Coruupted
This generates a datasct with balanced and Z with size N 0.98. This makes Y and Z highly correlated, so that most even (odd) digits are clean (zigzag). For validation, we use 50% of the training data.
2 each. For our experiments, we use 40000, p=
Beleb--Unddrrsmppled We modify the CelebA dataset (Liu et al., 2015) by undersampling the (Y non-blonde, Z = female) subgroup in the training set. The original dataset contains 71629 exam- ples in this training subgroup, and we keep random subset 4054 examples. This number chosen make the ratio subgroup sizes equal both classes datasets.
4054
1387
We do not modify the validation or test.
66874
22880
This modification introduces a spurious correlation between hair-color and gender, which makes the dataset appropriate for our setting. We preprocess images by resizing 128 128 3 before use.
Waterbirds. We use the Waterbirds dataset (Sagawa et al., 2020) and resize images to 224 x 224 x 3 before use. Note that this differs from the preprocessing used by (Sagawa al., 2020), who first resize to 256 256 x and then center-crop the image part of the (spurious) background is cropped out, while we retain the full image.
224 x The preprocessing they use makes the task easier, since some
ISIC.
We use the ISIC dataset (Codella et al., 2018) and resize images to 224 x 224 x before use.
D.2 CYCLEGAN TRAINING DETAILS
We use the default hyperparameters suggested by (Zhu et al., 2017) for CycleGAN training, with batchnorm for layer normalization. We use Adam for optimization (B: = 0.5) with constant learning rate of 0.0.000 0.0002 for both generators and both discriminators.
MNIST- Corrs-Correlation. Train 200 images each from both MNIST and MNIST- (100 images per class) for 2500 epochs with a batch size of 25, cycle loss coefficient of 10.0 and identity loss coefficient of 1.0. We randomly rotate, pad and crop every image for training. Figure shows some CycleGAN generated examples.
CelebA-Unin -ndorsspedds Train separate CycleGANs for both classes. Train on 1000 images each from both subgroups within the class for 4000 epochs with a batch size of 16, cycle loss coefficient of 10.0 and identity loss coefficient of 1.0. We flip inputs randomly (with prohability 0.5) and randomly crop up to 10% of every image. Due to instahility during training, we visually inspected samples generated on the training set several checkpoints to pick the best model. Figure shows some CycleGAN generated examples.
and 184 images each from both subgroups for the landbird and waterbird classes respectively. Train for 4000 epochs with batch size 4, cycle loss coefficient of 10.0 and identity loss coefficient of 1.0. We flip inputs randomly (with probability 0.5) and randomly crop upto 10% of every image. Figure 10 shows some CycleGAN generated examples.
Waterbirds. Train separate CycleGANs for both classes. Train
4
0820
6
6
4
6
7
Figure 7: An example of data in MNIST-CCrrclation. Most even digits are clean while most odd digits contain zigzag corruption.
25

---

Published as a conference paper at ICLR 2021
3
1
9
Z
(
677
9
S
3
0
1
OATE
000
401
4000
Z
3
33
(top) real images from MNIST
(top) real images from MNIIS-Correlation (bottom) CycleGAN generated images
(bottom) CycleGAN generated images
Figure 8: Results of inter-subgroup transformations on MNIST-Corelation.
ISIC.
Train on 100 images each from both benign subgroups (with and without bandaids) for 4000 epochs with a batch size of 4, cycle loss coefficient of 10.0 and identity loss coefficient of 10.0. We flip inputs randomly (with probability 0.5) and randomly crop upto 10% of every image.
D.3 ARCHITECTURES AND TRAINING INFORMATION
All training code is written in Python with tensorflow-2.00 2.0. All models are trained with Stochastic Gradient Descent (SGD), with momentum of 0.9. In order to isolate the effect of our method, we do not use any data augmentation (such as pad and crop operations or random flips) when training the classifier.
We train convolutional neural network from scratch, initialized with random weights. The architecture is provided below,
26

---

Published as a conference paper at ICLR 2021
(top) male image from CelebA
(top) female image from CelebA
(bottom) CycleGAN generated female image
(bottom) CycleGAN generated male image
Figure 9: Results of inter-subgroup transformations on CelebA-Undersampled. Generation examples use the CycleGAN trained on the non-blonde class.
Conv2D(filters = 32, kernel=3) ReLU Conv ReLU - MaxPPollngg2 2) + 00020 + Conv2D(64, ReLU + Conv2D(64, ReLU
MaxPooling2D(2) - (002ont(0.25) +
Flatten + Dense(units + ReLU + (005 + Dense(10) + Softmax.
Other datasets. All models are fine-tuned using ResNet-50 architecture, with pretrained ImageNet weights The only preprocessing common to all methods is standard ImageNet normalization using ja [0.485, 0.456, 0.406), [0.229, 0.224, 0.225).
D.4 HYPERPARAMETERS
model selection, we use robust accuracy on validation set". The selected model's hyperparameters are then run 3 times, and the results averaged over these trials are reported Table Below, we provide details of all hyperparameter sweeps, and in Table 12, we include the best hyperparameters found for each method and dataset.
D.4.1 CELEBA-UNDERSANIN
We run sweeps for all methods over 50 epochs.
ERM. Sweep over learning rates (0.0001, 0.00005, 0.00002, 0.0000000000000000000 with weight decay fixed to 0.05,
GDRO. Sweep over adjustment coefficients in (1.0, 3.00 and learning rates (0.0001, with weight decay fixed to 0.05.
3 'hhe particular model used was taken from https: /ttp://ggttt//// http:/tttt..com/g///ee/eeeeeeeee css fi on models. s.
*orr the ISIC dataset, additionally performed model selection using AUROC, illustrated in Table
27

---

Published as a conference paper at ICLR 2021
(top) land background image from Waterbirds (bottom) CycleGAN generated water background image
(top) water background image from Waterhirds
(bottom) CycleGAN generated land background image
Figure 10: Results of inter-subgroup transformations on Waterbirds. Generation examples use the CycleGAN trained on the landbirds class.
28

---

Published as a conference paper at ICLR 2021
CAMEL. Sweep consistency penalties in (5.0, 10.0, 20.0, 50.0). Learning fixed 0.00005. weight decay fixed to 0.05 and the adjustment coefficient fixed 3.0.
D.4.2 WATERBIRDS
We rum sweeps for all methods over 500 epochs.
ERM. Sweep over learning rates {0.001, 0. 0.0001, 0.00001} and weight decays {0.5, 0.001}.
GDRO.
Sweep over learning rates (0.00001, 0.00005) and weight decays {0.5, 0.05) with adjustment cocfficient fixed to 1.0 and batch size 24, Wc also separately swept weight decays {1.0, 0.001 and adjustment coefficients over (1.0, 2.0}.
CAMEL. Sweep consistency penalties (100.0, 200.0) and learning rates {0.00005, 0.0001). Weight decay fixed 0.001 and adjustment coefficient is fixed 2.0. Separately, sweep learning rates {0.00001, 0.00002, 0.00005, 0.0001}, fixing the consistency penalty to 200.0, weight decay to 0.05 and ad- justment coefficient to 1.0.
D.4.3
We run sweeps for all methods over 100 epochs.
ERM. Sweep over learning rates (0.0001,0.0002, 0.0002, 0.0005, 0.001} } and weight decays {0.0005, 0.05).
GDRO. Sweep over learning rates {0.0001, 0.0002, 0.0005, 0.001 and weight decays (0.0005, 0.05). Ad- justment coefficient is fixed to 1.0.
CDAT. Sweep over domain loss coefficients (-0.1, 0.1, 1.0). We fix learning 0.001 and weight decay 0.0005. We run CDAT 400 epochs, since takes much longer converge.
Sweep over IRM penalty f0.01, 1.0, 10, 100, 1000, 10000} and learning rates {0.0005, 0.001 Weight decay is fixed to 0.0005.
IRM.
CAMEL. Sweep over consistency penalty weights {0.0, 2.0, 5.0, 10.0, 50.0}. Learning rate is fixed to 0.001 and weight decay is fixed to 0.0005.
D.4.4 ISIC
We sweeps for all methods 75 epochs.
ERM.
Sweep over weight decays {0.5, 0.05, 0.00005). Learning rate fixed 0.0001.
GDRO. Sweep over learning rates (0.0001, (0.0001,0.00001 0.00001 } and weight decays (0.5, 0.05, 0.00003). Adjustment coefficient fixed to
CAMEL. Sweep over learning rates (0.0001, 0.00005). weight decays (0.01, 0.05). consistency penalties (10.0, 50.0} 11..0,50.0) and anncaling rates {0.005, 0.002).
D.5 MUTUAL INFORMATION MEASUREMENT
For the mutual information measurement experiment on MNIST-Correlation in Section 4.1, we additionally attach domain prediction head to the final feature layer. This domain prediction head is then used to predict the suhgroup of any example v. Note that this domain prediction head does not pass hack gradients the main model, merely observes the subgroups using this. Intuitively, this captures how much information about the subgroups is available to be "squeezed-out" by the domain prediction head. This constitutes use of Lemma to estimate the mutual information, and report the average cross-entropy loss (added log 2).
learned representation
attempts to improve prediction accuracy
D.6 BASELINE COMPARISONS
We describe the baselines that compare to, with implementations for each of these available in our code release.
29

---

Published as a conference paper at ICLR 2021
Table 9; Ablation analysis (Section 4.2.1) that varies the consistency penalty coefficient on the MNIST-
Correlation dataset. For brevity, we report the maximum subgroup performance gap over all classes.
Robust Acc. (%) Max Subgroup Gap
Method
== 1
A = 2
X = 5
X = 10
Heuristic
91.90
92.29
92.08
93.22
Augmentation
4.37
3.95
4.94
3.88
97.88
97.02
98.31
98.40
CAMEL
1.16
1.16
1.28
1.21
D.6.1 METHODS
ERM. We use standard training with cross-entropy loss. ERM cannot take advantage of knowledge of the subgroups, this constitutes standard baseline that practitioner might solve task.
GDRO. This is our main baseline as described in Section 2, and uses a stochastic optimization method (Sagawa al,, 2020). GDRO uses subgroup information optimize the worst-case loss over all subgroups. We note that GDRO requires the specification of adjustment coefficient, and we describe the best found coefficients in Table
We use generic domain adversarial training approach using domain prediction head attached to the last feature layer the model o(X). The domain head predicts the subgroup identity of the given example, we use gradient reversal order to crase domain information from the representation (()). vary magnitude of the gradient reversal on the domain loss (which we call the domain loss coefficient in Table 12) in order to find the best-performing model.
CDAT.
IRM. We implement IRM penalty (Arjovsky at,,. 2019), and treat subgroups as separate environments across which the model should perform well.
D.6.2 ABLATIONS
Subgroup Pairing. We simply take pairs of examples that lie in different subgroups and enforce consistency on them.
Heuristic Augmentations. We build pipeline inspired by AugMix (Hendrycks al., 2019) using the following operations: shearing, translation, rotation, flipping, contrast normalization, pixel inversion, histogram equalization, solarization, posterization, contrast adjustment, color enhancement, brightness adjustment, sharp- ness adjustment, cutout and mixup. We sample between and 3 of these augmentations in random order and apply them to the image.
We include an additional ablation the coefficient À in Table Compared heuristic augmentations, CAMEL provides substantial improvements that are stable across different values of
dataset where vary the consistency penalty
D.7 ISIC SPURIOUS CORRELATIONS
For completeness, we include a detailed evaluation for the ISIC dataset in Table 10. Here, we highlight that regardless of what criterion is used for model selection between robust accuracy and AUROC, CAMEL exceeds the performance of the other methods.
For ISIC, we also create an alternate evaluation dataset with artificial images in order test whether model spuriously correlates the presence of bandage with the benign cancer class. construct this dataset, we use image segmentation to automatically extract images of the bandages from the benign cancer class, and superimpose them on images with malignant cancers. This allows us to generate the artificial subgroup of the malignant cancer class that would contain images with bandages. We use this dataset to highlight how CAMEI. improves the model's dependence this spurious feature in Figure 1.
30

---

Published as a conference paper at ICLR 2021
Table 10: Performance on the ISIC validation set.
Evaluation
Model Selection Criterion
Method
Metric
Robust Acc.
AUROC
Robust
HRM
65.59 (1.17)
52.93 (10.27)
Acc.
GDRO
64.97 (3.15)
51.23 (1.93)
CAMEL
77.45 (0.35)
66.67 (3.03)
AUROC
ERM
92.48 (0.80)
93.38 (0.14)
GDRO
89.50 (2.50)
91.83 (0.11)
CAMEL
92.47 (0.38)
93.41 (0.52)
Table 11: Comparisons GAN Baselines Waterbirds and CelebA-Undersamplee..
Aabbes/Aggregate Acc.
Dataset
GAN Model
GAN + ERM
GAN Model Patching
Waterhirds
CycleGAN
76.88/91.75
$9.12/90.89
Augmented CycleGAN
63.12/91.08
84.87/86.44
DAGAN
73.12/90.28
CelebA- Undersaldde
StarGAN v2
65.91/90.58
80.68/89.33
D.8 ALTERNATIVE GAN AUGMENTATION BASELINES
As noted in Section 2. Stage 1 of the model patching pipeline be integrated with alternative domain translation models. an additional bascline, we compare alternative GAN augmentation methods. Typically, these methods are as data augmentation method, but not evaluated on robustness.
We consider the Augmented CycleGAN (Almahairi ct al., 2018), Data Augmentation GAN (DAGAN) (Antoniou et al., 2017) and StarGAN-v2 (Choi et al., 2020) models, either when used in combination with ERM, or when part of the model patching baseline. When used as part of model patching, replace the CycleGAN in Stage with alternative GAN model.
We used released code Augmented CycleGAN and DAGAN to generate data for the Waterbirds dataset. StarGANv2, we used pre-trained models for Celeb-A. We note that DAGAN is meant to be self-contained data augmentation pipeline, so we did not consider it in conjunction with Model Patching.
The results of this comparison is are shown in 11. In particular, these alternate models have poor robust performance when used purely for data augmentation. Their performance improves when integrated in the model patching pipeline.
2The consistency penalty is increased linearly every step, from O to À with rates 0.002 and 0.005 for = 50.0 and À 10.0 respectively.
31

---

Published as a conference paper at ICLR 2021
Table 12: The values of the best hyperparameters found for each dataset and method.
Method
Dataset
Hyperparametters
Learning
Weight Decay
Size
MNIIS-CCorrilttion
0.0001
0.05
100
CelebA-Undersann Undersampled
0.00005
0.05
16
ERM
Waterbirds
0.001
0.001
ISIC
0.0001
0.005
24
0.0001
0.00005
24
Learning Rate
Weight Decay
Batch Size
GDRO Adjustment
MNNIST-CorrcLation
0.0005
0,0005
GDRO
CelebA-Undennan Undersampled
0.0001
0.05
16
3.0
Waterbirds
0.00001
0.05
24
1.0
ISIC
0.0001
0.00005
24
0.0
Learning Rate
Weight Decay
Batch Size
GDRO Adjustment
A
MNIST-CCorrelatio
0.001
0.000S
100
1.0
5.0
Beleb--Undersampee
0.00005
0.05
CAMEL
Waterbirds
0.0001
0.001
16
2.0
100.0
ISIC
0.0001
0.01
24
3.0
50.0*
0.0001
10.0
Learning
Weight Decay
Domain Coefficient
CDAT
MNIST-Corrriation
0.001
0.0005
100
-0.10
Learning Rate
Weight Decay
Batch Size
IRM Amneal Steps
IRM Penalty
IRM
MNIST-Correlation
0.0005
0.0005
100
2000
0.1
32


Reviews:
[{"review_id": "nLhiuMV0iOh", "reviewer": "AnonReviewer4", "summary": "", "questions": "", "limitations": "", "rating": 7, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Machine learning models are trained to optimize performance on the entire training set, and can often exhibit inaccurate performance on a subgroup. Such inaccuracy often results from the model’s dependence on spurious features. This paper proposes model patching — a two-step method to avoid this problem. The first step learns inter-subgroup transformations where an example from one subgroup is transformed into examples of all the other subgroups within a class. The second stage uses these transformations as controlled data augmentations to learn a classifier that is robust to subgroup-specific variation.\n\nIn particular, the paper uses CycleGAN to learn the transformations between pairs of subgroups. The second stage uses the original data and the augmented data from stage 1 and minimizes a subgroup robust objective plus a self-consistency loss. The subgroup robust objective captures the discrepancy between the best and the worst performing subgroup within a class. On the other hand, the self-consistency loss enforces consistency on the augmented data. \n\nExperiments: The authors perform extensive experiments on three benchmark datasets — MNIST, CelebA, and Waterbirds. On all the datasets CAMEL improves both aggregate and robust accuracy by at least 5.3% and also reduces the subgroup gap significantly. Then the authors perform ablations on the two major components of the framework. It seems that learned augmentations perform better than other heuristic augmentations, and substituting the consistency loss with other losses reduces the robust accuracy by 2.5%. Finally, the authors apply the proposed framework on the real-world ISIC skin cancer dataset and found that it improves robust accuracy by 11.7% compared to the other methods.\n\n\nStrengths:\n- Extensive experimentation: I thought that the experiments were sufficient to demonstrate the effectiveness of the proposed approach. In particular, I liked the Model patching ablations experiments, which showed that both the learned augmentations and the subgroup consistency regularizer are important.\n- I thought that the use of CycleGAN and SGDRO is well-motivated and appropriate for this setting. \n\nWeaknesses:\n- No overlapping subgroups across classes. This paper considers a setting where each class is partitioned into multiple subgroups. I am not sure whether the proposed framework can be easily generalized for the setting when the subgroups are overlapping across multiple classes.\n\n\n\nQuestions for the authors:\n1. What happens if the group information is not known or noisy/incorrect? Can the objective function in subsection 2.2.1 be modified to handle such situations?\n2. If I expand the square term given in theorem 1, there will also be a product term in addition to the consistency loss and CycleGAN loss. However, this additional product term is absent from the objective considered by CAMEL. So I did not follow why CAMEL induces the desired conditional independence (\\hat{Y} \\perp Z | [X]).\n3. In table 4, why does CAMEL + Heuristic have very low robust acc (~53%) for \\lambda=200? Moreover, for this value of \\lambda, the maximum subgroup gap is quite large.\n4. Subsection 4.2.2 does not report performance for the datasets — CelebA and MNIST. What is the effect of consistency loss ablations on these datasets?\n\n\nIn summary, this paper proposes a model patching based two-step method to design a classifier that is robust and reduces performance disparity across different subgroups. Experiments on standard datasets clearly show that the proposed method is more effective than some of the existing approaches.", "extra_scores": {}, "extra_text": {}}, {"review_id": "iQsC1W67P_", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The authors consider the problem  of differential  performance across subgroups commonly present in classifiers, and propose to mitigate it. They augment subgroup data for each class using CycleGAN, and balance the performance across subgroups in each class using a consistency regularizer and a robust objective that minimizes the difference between the minimum and maximum subgroup performance.  This approach is demonstrated in several datasets including the ISIC skin cancer data.\n\nThe paper is written well, easy to follow, and I was able to understand and appreciate the contributions quickly. The appendices are also very  thorough and the code is organized well. There is sufficient detail to help readers reproduce the results.\n\nComments:\n1. The authors can specify very early on that the subgroups are pre-specified by the user, and not automatically discovered.\n2. Relations to the  area of group fairness may be helpful. The goal there is also  to ensure parity of predictions among groups, and various methods are used. The groups though are assumed to be common across classes. It may be possible to extend this approach to that area, and the idea of data augmentation may be very appealing there. A small discussion on this can be quite informative to the community. There is also the notion of  subgroup fairness discussed here which considers exponentially or infinitely many subgroups\nhttp://proceedings.mlr.press/v80/kearns18a.html \n3. Have the authors considered how their methods can be modified when the subgroups are common  across classes (this points directly to the group fairness comment  before)?\n4. Just to clarify, the robustness metric used in the experiments is the same as the metric of interest for GDRO in Table 1, correct? And is the gap same as the metric of interest for SGDRO? Does the performance for groups in each class vary between robust accuracy, and robust accuracy + gap? How are the metrics reported in the experiments consolidated across classes? It may be helpful to define the experimental metrics explicitly.\n5. Saying that your minimization of I(\\hat{Y}; X | [X]) is parallel to Lemma 1 is confusing, since you minimize the upper bound (in Thm. 1) and Lemma 1 minimizes the lower bound. Agree that your minimization is stronger.\n6. Have you tried  any preliminary experiments with greater than 2 subgroups per class? Just curious what it takes to use something like StarGAN like you mention.\n7. What are the risks of letting the users specify the subgroups? In some problems it may be hard to diagnose which subgroups are meaningful. Is minimizing the worst case performance over many possible automatically discovered subgroups an interesting future  direction in your opinion?  What does it take to do that?\n8. How well  will  this method work when there is considerable imbalance in the subgroups considered?\n9. In table 5,  what do the quantities in the parenthesis mean?\n\nTypo:\nSec. D.4.2 (GDRO): weighte ->  weight", "extra_scores": {}, "extra_text": {}}, {"review_id": "SDF4s3X66g", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 7, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The paper focuses on data augmentation in cases when the classifier performance is worse on specific parts of the data. The problem is also closely related to that of the spurious correlation problem, the setting where the classifier might pick up on \"random\" patterns in the data to make its decision. The solution proposed by the paper is quite intuitive: first, given the subgroups in a class, a CycleGAN model is used to learn different versions of the same training example, each corresponding to a different subgroup. Once the augmented versions of the examples are available, the the classifier is then trained with additional penalty terms, ensuring that the predictions are consistent across different versions of the same training example. Empirical results show that the proposed method does reasonably well as compared to the competitors.\n\nWhile the reviewer is not an expert in the area, the contribution of the paper indeed sounds appealing. The proposed method is quite intuitive and simple to implement; and the empirical results are quite encouraging. On top of that, the proposed method is quite general (does not seem to be limited to just images) and can be applied to a variety of domains. One drawback of course is the need for manual identification of the subgroups, but given that state-of-the-art methods also need manual annotations (at least to my knowledge), that is probably fine. Finally, the paper is quite well written and easy to follow.\n\nA few comments and suggestion:\n\n- The current assumption is that the subgroups are specific to a single class. However, that may not always be the case in the real world. Consider for instance the problem of fair classification where subgroups (e.g., socially salient groups) might span multiple classes. Does the proposed method extend to such cases? From the first sight, that does not seem to be the case.\n\n- It is not clear what is meant by the statement \"handcrafting these augmentations may be impossible if the subgroup differences are difficult to express\". However, if the differences are difficult to express, wouldn't it also mean that separating these subgroups is difficult in the first place? In such cases, even the proposed method would also have a problem.\n\n- One potential problem with the proposed approach is its application in domains with small training datasets. It looks like training the CycleGAN would require a relatively large amount of data. How does the proposed approach expected to perform in such cases?\n\n- In Eqs. 2 and 3, the KL divergence is computed between the predicted distributions (presumably, softmax output). Given that DNNs tend to be quite badly calibrated (https://arxiv.org/pdf/1706.04599.pdf), is it worth computing the full KL divergence? Shouldn't minimizing the difference between the argmax class be sufficient?", "extra_scores": {}, "extra_text": {}}, {"review_id": "NFcmEqN9PES", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 7, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "# Summary\n\nThis paper introduces a method (CAMEL) to make CNN models robust to the effect of subgroups in classes. CAMEL uses CycleGAN to transfer the subgroup of each input image in each class and applies consistency regularization among transferred images. This paper additionally introduces a novel objective (SGDRO). CAMEL shows preferable performance on various data, including a real-world dataset of skin cancer classification.\n\n# Strengths\n\n1. The proposed method is straightforward and sound.\n1. The authors conducted extensive experiments with various datasets, including a real-world dataset of skin cancer classification. They also performed comprehensive ablation studies to show how and why CAMEL works.\n1. CAMEL can be combined with heuristic augmentation (e.g., rotations) to improve robustness.\n\n# Weaknesses\n\n1. The datasets used in the experiments are too simple to show the effectiveness of CAMEL and SGDRO. Specifically, the number of subgroups in each class of all datasets is at most 2. On the other hand, especially SGDRO assumes that subgroups have structure. A more complicated subgroup setting should be considered.\n1. When the number of subgroups in each class and the number of classes is limited as the experiments, one can treat $\\mathcal{Y}\\times\\mathcal{Z}$ as target and use ERM. This should be a strong baseline.\n1. I concern the scalability of this method. If CycleGANs are used as the paper, $\\sum_{y\\in\\mathcal{Y}}\\frac{\\|\\mathcal{Z}_y\\|!}{2}$ CycleGAN models need to be trained on a pair of subgroups beforehand and used during CAMEL training. Even StarGANs are used, $\\|\\mathcal{Y}\\|$ StrarGANs are required.\n\n# Feedbacks\n\nThe paper will be improved if generated images of CycleGANs are presented to visually show how CycleGANs change subgroups.", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper presents an approach for mitigating subgroup performance gap in images in cases when a classifier relies on subgroup specific features. The authors propose a data augmentation approach, where synthetically produced examples (by GANs) act as instantiations of the real samples in all possible subgroups. By matching the predictions of original and augmented examples, the prediction model is forced to ignore subgroup differences encouraging invariance. The proposed method of ‘controlled data augmentations’ (as precisely called by R4) is relevant and well-motivated, the theoretical justifications support the main claims, and the experimental results are diverse and demonstrate merits of the proposed approach. As rightly pointed out by R3, ‘The appendices are also very thorough, and the code is organized well’.

In the initial evaluation, the reviewers have raised (in unison) concerns regarding overlapping subgroups per class, and an imbalance problem in the subgroups when training GANs. There were also questions reg. theoretical justifications, and empirical evaluations of the baseline methods. The authors have addressed all major concerns in the rebuttal. Pleased to report that based on the author respond with extra experiments and explanations, R2 has raised the score from 6 to 7. In conclusion, all four reviewers were convinced by the author’s rebuttal, and AC recommends acceptance of this paper – congratulations to the authors!

There is a colossal effort in the community addressing a goal similar to this work – learning invariant representations w.r.t. sensitive features by means of algorithmic fairness methods. (R1 and R3 relate to it). When preparing the final version, the authors are encouraged to elaborate more on the discussion/comparison to fairness-based methods, ideally including empirical evidence where possible (where subgroups overlap, e.g. CelebA). The AC believes this will strengthen the final revision and will have an even broader impact in the community.

Decision: Accept (Poster)