Diversify and Disambiguate: Out-of-Distribution Robustness via Disagreement
Authors: Yoonho Lee
Huaxiu Yao
Chelsea Finn
Venue: ICLR 2023 poster (2023)
Keywords: Out-of-distribution robustness
spurious correlations
underspecification
ambiguity
ensembles

Abstract: Real-world machine learning problems often exhibit shifts between the source and target distributions, in which source data does not fully convey the desired behavior on target inputs. Different functions that achieve near-perfect source accuracy can make differing predictions on test inputs, and such ambiguity makes robustness to distribution shifts challenging. We propose DivDis, a simple two-stage framework for identifying and resolving ambiguity in data. DivDis first learns a diverse set of hypotheses that achieve low source loss but make differing predictions on target inputs. We then disambiguate by selecting one of the discovered functions using additional information, for example, a small number of target labels. Our experimental evaluation shows improved performance in subpopulation shift and domain generalization settings, demonstrating that DivDis can scalably adapt to distribution shifts in image and text classification benchmarks.

Published as a conference paper at ICLR 2023
DIVERSIFY AND DISAMBIGUATE:
OUT-OF-DISTRIBUTTON ROBUSTNESS VIA DISAGREEMENT
Yoonho Lee
Huaxiu Yao Stanford University
Chelsea Finn Stanford University
Stanford University
ABSTRACT
Real- world machine learning problems often exhibit shifts between the source and target distributions, in which source data does not fully convey the desired behav- ior on target inputs. Different functions that achieve near-perfect source accuracy can make differing predictions on test inputs, and such ambiguity makes robust- ness to distribution shifts challenging. We propose DivDis, simple two-stage framework for identifying and resolving ambiguity in data. DivDis first learns a diverse set of hypotheses that achieve low source loss but make differing predic- tions on target inputs. We then disambiguate by selecting one of the discovered functions using additional information, for example, small number of target la- bels. Our experimental evaluation shows improved performance in subpopulation shift and domain generalization settings, demonstrating that DivDis can scalably adapt to distribution shifts in image and text classification benchmarks.
INTRODUCTION
Datasets are often underspecified: multiple plausible hypotheses each describe the data equally well (D' Amour et al., 2020), and the data offers further evidence to prefer such ambiguity, machine learning models typically choose only one of the possible explanations of given data. Such choices can be suboptimal, causing these models to fail when the data distribution is shifted, as common in real-world applications. For example, examination of a chest X-ray dataset (Oakden-Raynes et al., 2020) has shown that many images of patients with pneumothorax include thin drain used for treating the disease. A standard classifier trained on this dataset can erroncously identify such drains as predictive feature of the disease, exhibiting degraded accuracy on the in- tended distribution of patients not yet being treated. To not suffer from such failures, it is desirable to have a model that can discover a diverse collection of alternate plausible hypotheses.
another. Despite
The standard empirical risk minimization (Vapnik, 1992, ERM) paradigm performs poorly on under- specified data, because ERM tends to select the solution based on the most salient features without considering alternatives (Geirhos et al., 2020; Shah et al., 2020; Scimeca et al., 2021). This sim- plicity bias occurs even when training an ensemble (Hansen Salamon, 1990; Lakshminarayanan et al., 2017) because each model is still biased towards simple functions. While many recent meth- ods (Ganin et al., 2016; Sagawa et al., 2020; Liu et al., 2021) improve robustness in distribution shift settings, we find that they fail on data with more severe underspecification. This is because, similarly to ERM, these methods only consider a single solution even in situations where multiple explanations exist.
We propose Diversify and Disambiguate (DivDis), a two-stage framework for learning from under- specified data. Our key idea is to learn collection of diverse functions that are consistent with the training data but make differing predictions on unlabeled test datapoints. DivDis operates as follows. We train neural network consisting of a shared backbone feature extractor with multiple heads, each representing a different function. As in regular training, each head is trained to predict labels for training data, but the heads are additionally encouraged to represent different functions from each other. More specifically, the heads are trained to make disagreeing predictions on a sepa- rate unlabeled dataset from the test distribution, a setting close to transductive learning. At test time, we select one member of the diversified functions by querying labels for the datapoints most infor- mative for disambiguation. We visually summarize this framework in Fig. 1. DivDis is designed for
"Email: yoonho stanford.dddd Code is available http:///ghhaa..comyyyooooe/e/did

---

Published as a conference paper at ICLR 2023
Stage 1: Diversify
Stage 2: Disambiguate
Diversity
Disa mibig alde
leboled Target
Nost Deoigredd Datapoirt
Near- near-pptimall al Functions for Source Disribution
Figure 1: Our two-stage framework for learning from underspecified data. In the DIVERSIFY stage. train each head a multi-headed neural network accurately predict labels source data while also outputting differing predictions for unlabeled target data. the DISAMBIGUATE stage, we choose one of the heads by observing labels for an informative subset of the target data.
scenarios with underspecified data and distribution shift, and its heads will not yield a set of diverse functions in settings where only one function can achieve low training loss.
We evaluate DivDis several settings in which underspecification limits the performance of prior methods, such as standard subpopulation shift benchmarks (Sagawa et al., 2020) or the large-scale CXR and Camelyon17 datasets (Wang et al., 2017; Sagawa et al., 2022). DivDis achieves an over 15% improvement in worst-group accuracy on the Waterbirds task when tuning hyperparameters without any spurious attribute annotations, and outperforms cxisting semi-supervised methods on the Camelyon17 task. We also consider challenging problem settings in which labels are completely correlated with spurious attributes, so a classifier based on the spurious feature can achieve zero loss. In these completely correlated settings, experiments find that DivDis is substantially more comple-efficient DivDis with target domain labels outperforms two fine-tuning methods that use 128 labels.
LEARNING FROM UNDERSPECIFIED DATA
We consider a supervised learning setting in which we train a model J that takes input x E X' and predicts its corresponding label y V. We train f with a labeled dataset Ds ((11,31) (I2, y2),.... drawn from data distribution ps(z, y). The model f is selected from hy- pothesis class f FF by approximately minimizing the predictive risk [ data distribution. The model f is evaluated via its predictive risk held-out samples from ps (2r, v)- Standard procedures such as regularization and cross-validation encourage such generalization.
v)] on the
However, even if function f generalizes unseen data sampled from the same distribution ps(r,3) y). performance often deteriorates in distribution shift conditions, when we evaluate on target data sampled from different distribution pr (z, y). In many distribution shift scenarios (Koh al., 2021), the overall data distribution can be modeled as mixture of domains, where each domain E ID' corresponds subpopulation shift setting, where the source and target distributions are different mixtures of the same underlying domains: Dss EeED w.Pe Pa and pr = EdED w² Pa. where {w JEED d * [wi] Jaen-
fixed data distribution Pa y). In this paper, we specifically consider a
Conditions like subpopulation shift can be inherently underspecified because the generative process underlying the data distribution, i.e. the domains and coefficients, has so many possibilities. We formalize this intuition through a notion of near-optimal sets of hypotheses. We define the e-optimal set for a data distribution as follows:
Definition (e-optimal set). Let p(z, y) be a distribution over X x V, and set of predictors f: X + Y. Let E, F + R be the risk with respect to p(z, u). The e-optimal set with respect to F at level e 2 is defined as
F F=(FEF(C(())
(I)
Put differently, the e-optimal set consists of all functions that generalize within the distribution p(z, y). The constant controls the degree of generalization, and we consider small e here onwards.
Note that a model's predictions on samples from ps(x, w)-wheh Ds or a held-out validation set-cannot be used to distinguish between different near-optimal functions with respect to (z, y)- This is because by defimition, the predictions of any two models f1, f2 E J are nearly identical on (s(x. y) for small Based on souree data alone, have insufficient reason to prefer any member of Jer over another. Our state of belief should therefore cover J as comprchensively as possible, putting nonzero weight many functions that embody different inductive biases. This reasoning
2

---

Published as a conference paper at ICLR 2023
is consistent with existing principles for reasoning under uncertainty such the maximum entropy principle (Keynes, 1921; Jaynes, 1957): in the absence of complete knowledge, one's beliefs should be uppropriately spread across all possibilities that are consistent with the available information.
Unfortunately, these principles are notably difficult to implement in practice because of the size and dimensionality of _Fe. We thus incorporate mild additional assumption into our problem statement that substantially reduces the set solutions to consider. We use an unlabeled target dataser DT sampled from pr (x:). The functions inside J can be compared based how their predictions differ on Dr. This simplifies the initial infiniteddimensional problem of comparing functions to one of comparing finite sets of predictions, significantly reducing the scope of search. The target set Dr can also be seen as specifying the directions of functional variation that are most important to us.
We formalize the underspecification with respect to predictions on unlabeled target data through the following variant of c-optimal sets:
Definition 2 (De-optimal set). Let D = {21....,, InF be an unlabeled dataset, p(æ, y) data distribution, and F hypothesis class. Let F be the e-optimal set with respect to p(z. y) and The DE Do-optimal set FDz is the set of equivalence classes of Fe defined by the following relation m
D
between two functions:
1 5 0 if ((x) 9(() Ve ED
Compared to Fe, the dataset-dependent set D is typically much smaller and easier to manipulate, since defined through predictions (E )) rather than raw functions.
Problem statement. To summarize, we use labeled source dataset Ds = { Vi) Vi)....) . along with an unlabeled targer dataset D = .......). We assume there is subpopulation shift between the two datasets, and our goal is to find a function that performs well in the target distribution. Such a function will lie inside the near-optimal set Je of ps (Z. y), and the learner leverages DT to find diverse functions inside this set.
DIVERSIFY AND DISAMBIGUATE
We describe Diversify and Disambiguate (DivDis)) two-stage framework for learning from underspecified data. We first describe the general framework (Sec. 3.1), and then our specific imple- mentation of the two DIVERSIFY (Sec. 3.2) and DISAMBIGUATE (Sec. 3.3) stages.
3.1 GENERAL FRAMEWORK
As a running example to motivate our algo- rithm, consider an underspecified cow-camel image classifiation task in which the source data includes images of cows with grass back- grounds and camels with sand backgrounds. We imagine two completely different clas- sifiers each achieving perfect accuracy in the source distribution: one that classifies by ani- mal and the other by background. After iden- tifying both possible functions, we can resolve the ambiguity by observing the label of sin- gle image of cow in the desert.
Algorithm 1 DivDis training
Input: Source data Ds. Target data Dr, Initial parameters 0g. Heads N, Weights X1, A2
Stage one: DIVERSIFY
Sec. 3.2
while Not converged do
(Xs, Ys) ~ Ds. X1 ~ Dr
Erem t- Lxem (f: Ys)
M + Lmi(0; (0:00 XT)
P (3)
Ereg <- L (0; XT,
D (4)
O + - aV (Exent + A1 bi + Ag
P (5)
Stage two: DISAMBIGUATE
Sec, 3.3
With this motivation, DivDis aims to first find a set of diverse functions and then choose the best member of this set with minimal supervi- sion. The DivDis framework consists of two stages. In the first stage, we DIVERSIFY by training a finite set of functions that together approximate the DE-optimal set for target data Dr. This stage uses both the source and target datasets for training. The source data ensures that all functions achieve low predictive loss on ps(r, y), while the target data reveals whether not the functions rely different predictive fea-
DDi +- Top datapoints w.r.t. disagreement
for Head = 1..N do
Acc; + Accuracy of head on D Return arg max; Acc;
3

---

Published as a conference paper at ICLR 2023
tures. In the second stage, DISAMBIGUATE by choosing the best member among this set of functions, for example, by observing the label of target datapoint for which the heads disagree on.
3.2 DIVERSIFY: TRAIN DISAGREEING HEADS
As previously described, the DIVERSIFY stage learns diverse collection of functions by compar- ing predictions for the target set while minimizing training error. In our hypothetical cow-camel task, diversifying predictions in this way will produce functions that disagree on ambiguous data- points like cows in the desert. Such disagreement can cause one head to become an animal classifier and another to be a background classifier.
We represent and train multiple functions using a multi-headed neural network with N heads, For an input datapoint x, we denote the prediction of head i as fi(x) = Ø. We ensure that each head achieves low predictive risk on the source domain by minimizing the cross-entropy loss for each head (f.) = En [((((z),))) v)].
Ideally, each function would rely on different predictive features in the input. While the most straightforward solution is to enforce differing predictions from each function, this does not nec- essarily indicate that the two functions rely different features. As extreme example, consider function f for a binary classification problem and its "adversary") f which outputs the exact opposite of f the target dataset. Even though f and f disagree completely on the target distribution, they rely on the same input features.
We train each pair of heads produce predictions that are close to being statistically independent from each other. Independence of prediction values directly implies disagreement in predictions: for example, mutual information is maximized when two prediction heads completely agree with each other. Concretely, we minimize the mutual information between each pair of predictions:
[ f,) DxL (p(ū), ()) || p(T)) 8 p(ú)))
where DKL (- II-) is the KL divergence and Si is the prediction f((x) for x Dr. In practice, optimize this quantity using empirical estimates of the distributions p(T. 05)) and p()) @ p(o)).
To prevent functions from collapsing degenerate solutions such predicting single label for the entire target set while maintaining good source accuracy, we also minimize an optional regulariza- tion loss which regularizes the marginal prediction of each head across the target dataset:
(fi) DKL (p(1)) P(U)),
(4)
where p(y) is a hyperparameter. Our experiments in Fig. 14 show that DivDis is not very sensitive to the choice of this hyperparameter. If we expect the label distribution of the source and target datasets to be similar, we can simply set p(y) to be the label distribution in the source dataset Ds-
The overall objective for the DIVERSIFY stage is a weighted sum with hyperparameters \ι de E R:
Σ, Exem + A1 Σ iyg Cmilfe: fz) + X2
(5)
We note that the quantities needed for the mutual information term (3) is easily computed in parallel across examples within batch using modern deep learning libraries; we show code snippet in Appendix B. In practice, the cost of computing the objective (5) is dominated by the cost of feeding two batches-oee one source and one target-to the network. The time- and space- complexity of one step in the DIVERSIFY stage is approximately x2 compared to standard SGD step in optimizing ERM with the source data, and both can be reduced by using smaller batch size.
3.3 DISAMBIGUATE: SELECT THE BEST HEAD
After learning diverse set of functions that all achieve good training performance in the DIVER- SIFY stage, we DIS DISAMIIGUTTE by selecting one of the functions. For example, once our model has learned both the animal and background classifiers for the cow-camel task, we can quickly see which is right by observing the ground-truth label of an image of a cow in the desert. As such an example is not present in the given labeled source unlabeled target data, this stage requires infor- mation beyond what was used in the first stage. We now present three different strategies for head selection during the DISAMBIGUATE stage.
4

---

Published as a conference paper at ICLR 2023
Active querying. To select the best head with minimal amount of supervision, we propose an active querying procedure, in which the model acquires labels for the most informative subset of the unlabeled target dataset Dr. Since larger difference in predictions indicates more information for disambiguation, we sort each target datapoint & E DDr according to the total distance between head predictions Lizs (x) - [z()]] We select small subset of the target dataset, which has the m datapoints (i.e. m < (Drl) with the highest value of this metric. We measure the accuracy of each of the N heads with respect to this labeled set and select the head with the highest accuracy.
Random querying. A related alternative to the active querying strategy is random querying, in which we label a random subset of the target dataset Dr. Beyond its simplicity, an advantage of this procedure is that one can perform labeling in advance, because the datapoints to be labeled do not depend on the results of the DIVERSIFY stage. However, random querying is substantially less label-efficient than active querying because the set will likely include unambiguous datapoints for which labels are less informative for head selection.
Disambiguation on source data. Even if two functions are near-optimal in terms of labels, other properties of the source data D3 can distinguish between the two. Visualization techniques such as Grad-CAM (Selvaraju et al., 201 7) reveal which region of the input is relevant to each head. Such regions can be compared against the true predictive features by human observer. This comparison could also be performed in an automated way using existing bounding box annotations or pixel-wise segmentation labels, such as those available for the ImageNet and COCO datasets (Deng al., 2009; Lin et al., 2014). A key advantage of this method is that we do not require anything from the target domain beyond unlabeled datapoints.
We note that the active and random query strategies do not use any information unavailable to previous approaches: existing methods (Nam al., 2020; Liu et al., 2021) use labeled target data to tune hyperparameters. Unless stated otherwise, we use active querying because of its superior label efficiency. In Appendix E, we further analyze such label efficiency through generalization bound that depends only the number of heads and the difference between the best and second best head. In our experiments, the active strategy required as little as a single label. We emphasize that as long as the best head is selected, the choice of disambiguation method only affects label efficiency, and the final performance of the selected head is the same for all three methods. We summarize the overall structure of DivDis in Algorithm 1.
4 EXPERIMENTS
Through our experimental evaluation, we aim to answer the following questions. (1) What functions does the DIVERSIFY stage discover simple, oow-dimensional problems? (2) Can DivDis tackle image and language classification problems with severe underspecification, in which simplicity bias hinders the performance of existing approaches? (3) How sensitive is DivDis to hyperparameters, and what data assumptions are needed to tune them? (4) How does DivDis compare to unsupervised domain adaptation algorithms, which also leverage unlabeled data from the target domain?
Unless stated otherwise, DivDis uses a network with 2 heads and the DISAMBIGUATE stage uses the active querying strategy with 16 labels, which found suffices to recover the best between two heads. We closely follow the experimental settings of previous works, and all experimental details including datasets, architectures, and hyperparameters are in the appendix.
4.1 ILLUSTRATIVE TOY TASK
2D classification task. We start with synthetic 2D binary classification problem to see what functions the DIVERSIFY stage discovers. The task is shown in Fig. (left): inputs are points on plane, and the source distribution includes points in the second and fourth quadrants while the target distribution covers all four quadrants. By design, the labels for the points in the first and third quadrants are ambiguous. We train network with two heads and measure target accuracy of cach head throughout the DIVERSIFY stage. We visualize learning curves and decision boundaries in Fig. 3, which show that while the two heads initially learn to fit the source data, they later diverge to functions based on different predictive features. We also show an extended visualization with additional metrics in the appendix (Fig. 8).
Coverage of near-optimal set. To further understand how well DivDis can cover the set of near- optimal functions, trained 20-head model on the same 2D classification task, where each head
5

---

Published as a conference paper at ICLR 2023
1.0
Head
ERM
0.-
Uniabeled
0.--
Left: synthetic 2D classi- fication task with underspecification. Right: decision boundaries 20 linear functions discovered by the DIVER- SIFY stage. Together, these funccions span the set of linear decision bound- aries consistent with the data.
Figure 2:
200
800
2000
Figure 3: Learning curves for DivDis with heads trained the synthetic 2D classification task. We show the target domain racy each head, along with that ERM for comparison. The two heads initially represent similar functions, but later on diverge to represent classifiers that rely on entirely different features while remaining consistent with the source data. Head 2 achicves substan- tially higher accuracy than ERM.
a linear classifier. Results in Fig. 2 (right) show that the heads together span the set of linear classifiers consistent with the source data. Note that this set includes the function with the diagonal decision boundary (y x). This result suggests that given enough heads, the set of functions learned by DivDis can sufficiently cover the of near-optimal functions, including the simplest function learned by existing methods.
Comparison with ensembles. We compare the diversity of the functions produced by the DI- VERSIFY stage to that of independently trained models on a 3-dimensional version of the binary classification task. We measure how much each function relies on each of the three input dimensions through the Pearson correlation coefficient between each input dimension and the prediction. Visu- alizations in Fig. 9 of Appendix show that the functions learned by DivDis depend on different input features, whereas independently trained models use roughly equal mix of all features. This experiment demonstrates that the diversity vanilla ensemble cannot effectively cover the set of near-optimal functions, and is therefore insufficient for underspecified problems.
4.2 TASKS WITH COMPLETE SPURIOUS CORRELATION
We evaluate DivDis on datasets with a complete correlation, where the source distribution has spuriously correlated attribute that can predict the label perfect accuracy. To make this problem tractable, we leverage unlabeled target data Df for which the spurious attribute is not completely correlated with labels, as in the toy classification task (Fig. 2). Introducing complete correlations makes the problem considerably harder than existing subpopulation shift problems, because even the e-optimal set with s = 0 includes classifiers based on the spurious attribute.
Real data with complete correlation. We evaluate DivDis on several benchmarks for spurious correlation (Waterbirds, CelebA. MultiNLD) that we modified to exhibit complete correlation be- tween the label and the spurious attribute. Speficially, we alter the source dataset to include only the majority groups (e.g. waterbird with water background) and (e.g. landbird with land background). We use the original target validation set as the unlabeled data for the DIVERSIFY stage. We denote these tasks Waterbirds-CC etc., to distinguish from the original benchmarks. These tasks are con- siderably more difficult than the original, and they introduce a specific challenge not addressed in the existing literature: leveraging the difference in source and target data distribution encode and subsequently disambiguate tasks with high degrees of underspecification. To our best knowledge, no prior methods are designed address such complete correlations.
To examine the sample efficiency of the additional label queries used by DivDis during the DIS- AMBIGUATE stage, we experiment with N (4, 8, 16, 32, 64, 128) labeled target examples on the Waterbirds-CC. and CelebA-CC tasks, evaluating the performance using both active and random querying. We consider two baseline methods which similarly use datapoints from the target distri- bution in addition to the completely correlated source data: (1) ERM on the Waterbirds-CC training dataset with N additional minority datapoints and (2) Deep Feature Reweighting (Kirichenko et al., 2022, DFR), which first trains an ERM model on the training dataset and then fine-tune the last layer on a group-balancee balanced set size N. Fig. 4 shows that while DFR is substantially more sample- efficient than ERM, possibly due to its two-stage nature, even DivDis with random querying shows

---

Published as a conference paper at ICLR 2023
Waterbirnts
CelebA
70
70-
DaDa
40-
20
Ditts leet D-dia Frador
13
Randim Guesding
30-
20
HMM Piwety
10
12 Begez Labub
26
:28
16
32
61
124
Mic Satto
Trrper Labe
(a) Accuracy vs mix ratio.
(b) Grad-CAM visualization.
Figure 4: Worst-group accuracy Waterhirds-CC and CelebA-CC data, given different numbers of la- beled target examples. DivDis significantly more label-efficient than other methods in learning from non-majority labels.
Figure S: (a) Accuracy of DivDis, ERM. Group DRO, and JTT on MNIST-CIFAR data with different correla- tion ratios. (b) Grad CAM visualization of two Icarned heads on random datapoint from the MNIST-CIFAR source dataset. See Sec. 4.2 for details.
Avg Acc, Source Dist
Avg Acc, Target Dist
Warst Acc, Target Dist
Worst (%)
Ie:
84.1
839
77.0
50.2
85.8
26.5
50.1
72.8
72.5
60.0
Ensemble
34.8 + 1.0
Mi weight
88.3
88.4
86.4
88.0
90.1
74.0
84.4
23.4
Teney et al. (2021)
33.4 + 1.6
1+1
83.4
83.5
892
83.4
53.7
69.3
09.5
89.9
88.9
59.0
81.0
05.6
64.
83.9
18.9
Ensemble Dis
36.0 1.2
1e2-
21.5
82.4
702
12.1
54.
07.5
90.2
50.0
47.3
70.6
75.2
18.0
DivDis (ours)
82.4 1.9
Table I: Worst-group accuracy of various ensemble methods on the Waterbirds-CC dataset. All meth- ods use two heads ensemble members.
160
10-2
16-2
1=2
Regularizer weight
Regulartzer weight
Rogularierr
Figure 6: Hyperparameter grids for DivDis on the Waterbirds dataset. We show three metrics: average accuracy the source and target dis- tributions and worst-group accuracy on the target distribution. The high correlation between the three metrics indicates that we can tune the hy- perparameters of DivDis using only held-out labeled source data.
substantially higher sample efficiency. This indicates that the small set of diverse functions learned during the DIVERSIFY stage is critical for quickly learning from additional minority data.
As an additional naive point of comparison in the complete correlation setting, evaluate the per- formance of existing methods for addressing spurious correlations: ERM, JTT (Liu et al., 2021), and Group DRO (Sagawa et al., 2020)). As these methods are designed for settings with milder spurious correlation and do not leverage additional information through unlabeled target data, they are expected to fail in the complete correlation setting. Results for the Waterbirds- CCC, CelebA-CC, and MultiNLI-C tasks in Tab. 6 of Appendix D show that these methods show subpar performance: their worst-grous accuracy is worse than that of random guessing in some settings, and have a per- formance gap with DivDis of up to 35% in worst-group accuracy. In Tab. 1, we also compare DivDis to a vanilla ensemble and our implementation of Teney et al. (2021), where we observe no further benefits from these ensemble methods in our complete correlation setting. This experiment also shows that DivDis achieves substantially higher worst-group accuracy compared to applying the Dis stage independent ensemble, demonstrating that that the Div stage critical for performance.
Overcoming simplicity bias on MNIST-CIFAR data. In the MNIST-CIFAR task (Shah et al., 2020), each datapoint is concatenation of one MNIST image and one CIFAR image, and labels are binary. The source dataset is completely correlated: the first class consists of (MNIST zero, CIFAR car) images, and the second class is (MNIST one, CIFAR truck). The unlabeled target dataset is constructed from the validation sets of MNIST and CIFAR, and has no such correlation. By design, the unseen combinations are ambiguous. We evaluate on variants of MNIST-CIFAR with different levels of underspecification. We denote the mix ratio of DD5 indicates the distribution of the target set.
E [0. 1]. where = 0 indicates
completely correlated data as described above, and Fig. 5a shows the target domain accuracy of DivDis, ERM, Group DRO (Sagawa et al., 2020), and JTT (Liu et al., 2021) after training with various mix ratios. Existing methods fail to do better than random guessing (50%) in the completely correlated setting (ratio= 0.0), whereas DivDis achieves over 75% accuracy. Higher ratios make the problem closer to an i.i.d. setting where the source and target distributions identical, and the two methods achieve similar accuracy.
Comparison of DISAMBIGUATE strategies on MNIST-CIFAR data. Using MNIST-CIFAR data with complete correlation, plot the average final accuracy after the DISAMBIGUATE
7

---

Published as a conference paper at ICLR 2023
Waterbirds Worst Ace
CelebA Worst Ace
Test Acc
Tuned with:
Worst
Average
Worst
Average
Pseudo-Label
67.7 = 8.2
CVaR DRO
75.9%
62.0%
64.4%
36.1%
DANN
68.4 = 9.2
LIF
78.0%
44.1%
77.2%
24.4%
FixMatch
71.0 - 4.9
JTT
86.7%
62.5%
81.1%
40.6%
CORAL
77.9 = 6.6
NoisyStudent
86.7 = 1.7
DivDis (ours)
85.6%
81.0%
55.0%
55.0%
DivDis (ours)
90.4 + 1.8
Table Worst-group test accuracies the Waterbirds and CelebA tasks, when tuning hyperparameters with respect to average and worst-group accuracies. DivDis is substantially more robust to hyperparameter choice in both tasks, allowing tune hyperparameters without group labels.
Table 3: OOD test accuracy on Camelyon17- W--WILDS. All meth- ods leverage unlabeled target data.
stage for both the active query and random query strategies, for different number of labels used. Fig. 11 of Appendix shows that active querying in particular is very efficient, and one label suffices for finding the head with highest target data accuracy. We further verify the possibilityy of disambiguation on source data: Fig. 5b shows Grad-CAM (Selvaraju et al., 2017) visualizations of two heads randomly sampled image from the source dataset. Even though the two heads predict the same label, they respectively focus on distinct features of the data: the MNIST region and the CIFAR region. Since we know that the true predictive feature is the CIFAR image, we can select the second head based on this single datapoint, and nothing beyond what was used during training.
4.3 UNDERSPECIFCATTIN FROM DISTRIBUTION SHIFT
Comparison with unsupervised domain adaptation methods. Finally, we evaluate DivDis on the Camelyon17 17- WILDS dataset (Sagawa et al., 2022), large-scale tumor classification dataset con- sisting of 455, 954 labeled source datapoints and 600, 030 unlabeled target datapoints. The target datapoints collected from hospital not seen in the source data. As the unlabeled set for DivDis, we use the official val unlabeled set provided by Sagawa et al. (2022). We compare against several approaches that can also leverage this unlabeled data: Pseudo-Label (Lee al., 2013), FixMatch (Sohn et al., 2020), CORAL (Sun et al., 2016), and NoisyStudent (Xie et al., 2019). All results other than DivDis are from Sagawa et al. (2022). Quantitative results in Tab. 3 show that DivDis outper- forms these methods, achieving over 90% OOD test set accuracy. This experiment demonstrates that the approach of DivDis scales to large datasets, effectively capturing the implicit ambiguity in- side large unlabeled datasets from related but different distribution. This experiment demonstrates that DivDis can effectively leverage unlabeled data for underspecification arising from variation in real-world data collection conditions.
Do we need group labels for hyperparameter tuning? Existing methods for learning from data with subpopulation shift typically tune hyperparameters using group label annotations (Levy et al., 2020; Nam et al., 2020; Liu et al., 2021), 1).. making them deployable only in scenarios where group labels are available. To examine the dataset assumptions required to successfully tune DivDis's 's hyperparameters, we ran a hyperparameter sweep over (A1, A2). For each setting, we measure three metrics using held-out data: (1) average accuracy on Ds. (2) average accuracy Dr, and (3) worst-group accuracy on Dr. These metrics correspond to different assumptions about available information: (1) labeled source data, (2) labeled target data, and (3) labeled target data with group annotations, respectively.
Results in Fig. 6 show that the three metrics have a clear correlation. Notably, this implies that that tuning the hyperparameters of DivDis with respect to average accuracy on Ds yields close an optimal model for worst-group accuracy on the target distribution. Additional experiments on the CelebA dataset exhibit similar trend, as shown in Appendix D. In Tab. 2, we compare DivDis to existing methods as reported by Liu al. (2021). Note that the "Average" columns here correspond to our second weakest data assumption of labeled target data, whereas Fig. 6 implies that we can tune DivDis's hyperparameters using only labeled source data. This experiment demonstrates that compared to previous methods for distribution shift, DivDis's 'ss hyperparameters require substantially less information to tune.
5 RELATED WORK
Underspecificationn Prior works have discussed the underspecified nature of many datasets (D' Amour al., 2020; Oakden-Rayner al., 2020). Underspecification is especially problem-
8

---

Published as a conference paper at ICLR 2023
atic when the bias of deep neural networks towards simple functions (Arpit et al., 2017; Gunasekar et al., 2018; Shah et al., 2020; Geirhos et al., 2020; Pezeshki et al., 2021:?) is not aligned with the true function. Yet, these works do not present a general solution. As we find in Section 4, DivDis can address underspecified datasets, even when one viable solution is much simpler than another, since only one of the heads represent the simplest solution. Our notion of near-optimal sets can be seen as an extension of Rashomon sets (Fisher et al., 2019; Semenova ct al., 2019) to the unsuper- vised domain adaptation setting. Active learning methods (Cohn et al., 1996; Hanneke et al., 2014) are also related in that they handle underspecification by reducing ambiguity. Our MI-based diver- sity term resembles a common active learning criterion (Houlsby et al., 2011), 1), but key difference is that we directly optimize a set of models with respect to our criterion.
Ensemble methods. Our approach is related to ensemble methods (Hansen & Salamon, 1990; Diet- terich, 2000; Lakshminarayanan et al., 2017), which aggregate the predictions of multiple learners. Ensembles have been shown to perform best when each member produces errors independently of one another (Krogh et al., 1995), a property we exploit by maximizing disagreement on unlabeled test data. Previous works have extended ensembles by learning a diversified set of functions (Pang et al., 2019; Parker-Holde Holder et al., 2020; Wortsman al., 2021; Rame & Cord, 2021; Sinha et al., 2021). While the DIVERSIFY stage similarly learns a collection of diverse functions, our approach differs in that we directly optimize for diversity separate target dataset. Prior works have learned a diverse set of reinforcement learning policies both with an underlying task Mouret & Clune 5); Conti al. (2018); Kumar et al. (2020) and with no task at all Lehman & Stanley (201 I); Eysenbach et al. (2019); Sharma et al. (2020). While similarly motivated, these works operate in a very different setting, since we consider supervised learning with underspecification. So-called quality diversity methods aim to balance performance and novelty in a population Lehman & Stanley (2011); Mouret & Clune (2015); Cully & Demiris (2017).
Two recent works leverage unlabeled target data to learn set of diverse functions. Teney et al. (2021) introduces gradient orthogonality constraint with respect to features from a pre-trained backbone. We provide further experimental comparison to this method in Appendix A. Concurrently to our work, Pagliardini et al. (2022) propose to sequentially train a set of functions with a diversity loss on target data. In contrast, DivDis requires a single network and training loop regardless of the number of heads. Furthermore, Sec. demonstrates that DivDis scales to larger datasets.
Robustness and causality. Many recent methods aim to produce robust models that succeed even in conditions of distribution shift (Tzeng et al., 2014; Ganin et al., 2016; Arjovsky et al., 2019; Sagawa al., 2020; Nam al., 2020; Creager et al., 2021; Liu et al., 2021). While our work is similarly motivated, we address a class of problems that these previous methods fundamentally cannot handle. By nature of learning only one function, these robustness methods cannot disambiguate problems where the true function is truly ambiguous, in the sense that functions based on two different fea- tures can both be near-optimal. DivDis handles such scenarios by learning multiple functions in the DIVERSIPY stage and then choosing the correct one in the DISAMBIGUATE stage with minimal added supervision. This research direction is also related to inferring the causal structure (Pearl, 2000; Schölkopf, 2019) of observed attributes. Although many causality works focus on situations in which interventions are impossible, we explore inherently ambiguous problems where some form of intervention is necessary to succeed. Additionally, recent methods for extracting causality from observational data have been most successful in low-dimensional settings (Louizos et al., 2017; Goudet et al., 2018; Ke et al., 2019), whereas our method easily scales to large convolutional net- works for image classification problems.
CONCLUSION
We proposed Diversify and Disambiguate (DivDis), a two-stage framework for learning from un- derspecified data. Our experiments show that DivDis has substantially higher performance when learning from datasets with high degrees of underspecification, at the modest cost of unlabeled tar- get data and a few corresponding labels. To our knowledge, our method is the first to address this problem setting in the context of underspecification. An appealing property of DivDis is its auto- matic discovery of meaningful complementary features, as demonstrated in Fig. 6 and Tab. 2. This capability is related to that of disentangled feature learning (Chen et al., 2016; Higgins et al., 2017; Kim & Mnih, 2018; Chen et al., 2018), which is often posed as unsupervised problem. Our experiments show the alternative possibility of learning different possible meanings of labels in an underspecified supervised learning setting.
9

---

Published as a conference paper at ICLR 2023
ACKNOWLEDGEMENTS
We thank Damien Teney for help in reproducing the published Collages task experiments. We also thank Pang Wei Koh, Henrik Marklund, Annic S. Chen, other members of the IRIS and RAIL labs, and anonymous reviewers for helpful discussions and feedback. This work was supported in part by KFAS, Google, Apple, Juniper Networks, and Open Philanthropy. Chelsea Finn is a fellow in the CIFAR Learning in Machines and Brains program.
REFERENCES
Martin Arjovsky, Léon Bottou, Ishaan Gulrajani, and David Lopez-Paz. Invariant risk minimization. arXiv preprint (xXxv:997.00299) 2019.
Devansh Arpit, Stanislaw Jastrzębski, Nicolas Ballas, David Krueger, Emmanuel Bengio, Maxin- der S Kanwal, Tegan Maharaj, Asja Fischer, Aaron Courville, Yoshua Bengio, et al. A closer look at memorization deep networks. In International Conference on Machine Learning, 2017.
Ricky TQ Chen, Xuechen Li, Roger Grosse, and David Duvenaud. Isolating sources disentangle- ment in variational autoencoders. Conference Neural Information Processing Systems, 2018.
Xi Chen, Yan Duan, Rein Houthooft, John Schulman, Ilya Sutskever, and Pieter Abbeel. Infogan: Interpretable representation learning by information maximizing generative adversarial nets. In Conference on Neural Information Processing Systems, 2016.
David Cohn, Zoubin Ghahramani, and Michael I Jordan. Active learning with statistical models. Journal of artificial intelligence research, 4:129-145, 1996.
Edoardo Conti, Vashisht Madhavan, Felipe Petroski Such, Joel Lehman, Kenneth O Stanley, and Jeff Clune. Improving exploration in evolution strategies for deep reinforcement learning via a population of novelty-seeking agents. Conference Neural Information Processing Systems, 2018.
Elliot Creager, Jörn-Henrie Jacobsen, and Richard Zemel. Environment inference for invariant learning. International Conference on Machine Learning. 2021.
Antoine Cully and Yiannis Demiris. Quality and diversity optimization: A unifying modular frame- work. IEEE Transactions on Evolutionary Computation, 22(2):245-22 2599 2017.
Alexander D' Amour, Katherine Heller, Dan Moldovan, Ben Adlam, Babak Alipanahi, Alex Beu- tel, Christina Chen, Jonathan Deaton, Jacob Eisenstein, Matthew D Hoffman, et al. Under- specificatioo presents challenges for credibility in modern machine learning. arXiv preprint arXiv:2011.03395, 2020.
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hi- erarchical image database. In 2009 IEEE conference on computer vision and pattern recognition, pp.
2009.
Li Deng. The mnist database of handwritten digit images for machine learning research. IEEE Signal Processing Magazine, 29(6):141-142, 2012.
Thomas G Dietterich. Ensemble methods in machine learning. In International workshop on multi- ple classifier systems, pp. 1-15. Springer, 2000.
Benjamin Eysenbach, Abhishek Gupta, Julian Ibarz, and Sergey Levine. Diversity all you need: Learning skills without reward function. Conference Neural Information Processing Systems, 2019.
Aaron Fisher, Cynthia Rudin, and Francesca Dominici. All models are wrong. but many are useful: Learning a variable's s importance by studying an entire class of prediction models simultaneously. J. Mach. Learn. Res., 20(177): 2019.
Yaroslav Ganin, Evgeniya Ustinova, Hana Ajakan, Pascal Germain, Hugo Larochelle, François training of neural net-
Laviolette, Mario Marchand, and Victor Lempitsky. works. The journal of machine learning research, 17(1):2096-200 2016.
10

---

Published as a conference paper at ICLR 2023
Robert Geirhos, Jörn-Henrik Jacobsen, Claudio Michaelis, Richard Zemel, Wieland Brendel, Matthias Bethge, and Felix A Wichmann. Shortcut learning in deep neural networks. Nature Machine Intelligence, 2(11):665-672) 2020.
Olivier Goudet, Diviyan Kalainathan, Philippe Caillou, Isabelle Guyon, David Lopez-Paz, and Michele Sebag. Learning functional causal models with generative neural networks. In Explain- able and interpretable models in computer vision and machine learning, pp. 39-80 80. Springer, 2018.
Suriya Gunasekar, Jason Lee, Daniel Soudry, and Nati Srebro. Implicit bias of gradient descent on linear convolutional networks. In S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa- Bianchi, and R. Garnett (eds.), Advances in Neural Information Processing Systems, 2018.
Suchin Gururangan, Swabha Swayamdipta, Omer Levy, Roy Schwartz, Samuel R Bowman, and Noah A Smith. Annotation artifacts in natural language inference data. arXiv preprini arXiv:1803.02324, 2018.
Steve Hanneke et al. Theory of disagreement- active learning. Foundations and Trends in Machine Learning, 7(2-3):131-309) 2014.
Lars Kai Hansen and Peter Salamon. Neural network ensembles. IEEE transactions on pattern
analysis and machine intelligence, 12(10)9993-001, 1990.
Irina Higgins, Loic Matthey, Arka Pal, Christopher Burgess, Xavier Glorot, Matthew Botvinick, Shakir Mohamed, and Alexander Lerchner. beta-vae: Learning basic visual concepts with constrained variational framework. International Conference on Learning Representations, 2017.
Neil Houlsby, Ferenc Huszár, Zoubin Ghahramani, and Máté Lengyel. Bayesian active learning for classification and preference learning. arXiv preprint arXiv:1112.5745 2011.
Edwin T Jaynes. Information theory and statistical mechanics. Physical review, 106(4):620, 1957.
Nan Rosemary Ke, Olexa Bilaniuk, Anirudh Goyal, Stefan Bauer, Hugo Larochelle, Bernhard Schölkopf, Michael C Mozer, Chris Pal, and Yoshua Bengio. Learning neural causal models from unknown interventions. arXiv preprint arXiv: 1910011910 2019.
John Maynard Keynes. A treatise on probability Maemillan and Company, limited, 1921.
Hyunjik Kim and Andriy Mnih. Disentangling by factorising. In International Conference Machine Learning. 2018
Polina Kirichenko, Pavel Izmailov, and Andrew Gordon Wilson. Last layer re-training is sufficient for robustness to spurious correlations. arXiv preprint ar arXiv:2204.02937, 2022.
Pang Wei Koh, Shiori Sagawa, Sang Michael Xie, Marvin Zhang, Akshay Balsubramani. Weihua Hu, Michihiro Yasunaga, Richard Lanas Phillips, Irena Gao, Tony Lee, et al. Wilds: A benchmark of in-the-will distribution shifts. In International Conference Machine Learning, pp. 5637- 5664. PMLR, 2021.
Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. 2009.
Anders Krogh, Jesper Vedelsby, al. Neural network ensembles, cross validation, and active leamn ing. Advances in neural information processing systems, 1995.
Saurabh Kumar, Aviral Kumar, Sergey Levine, and Chelsea Finn. One solution is not all you need: Few-shot extrapolation via structured maxent rl. Advances in Neural Information Processing Systems, 2020.
Balaji Lakshminarayanan, Alexander Pritzel, and Charles Blundell. Simple and scalable predic- tive uncertainty estimation using deep ensembles. Conference on Neural Information Processing Systems, 2017.
Dong-Hyun Lee et al. Pseudo-label: The simple and efficient supissupee learning method for deep neural networks. In Workshop challenges in representation learning, ICML., 2013.
11

---

Published as a conference paper at ICLR 2023
Joel Lehman and Kenneth O Stanley. Novelty search and the problem with objectives. In Genetic programming theory and practice IX. pp. 37-56. Springer, 2011.
Daniel Levy, Yair Carmon, John C Duchi, and Aaron Sidford. Large-scale methods for distribution- ally robust optimization. In Advances in Neural Information Processing Systems, 2020.
Tsung-Yi Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick. Microsoft coco: Common objects in context. In European conference on computer vision, pp. 740-755. Springer, 2014.
Evan Z Liu, Behzad Haghgoo, Annie S Chen, Aditi Raghunathan, Pang Wei Koh, Shiori Sagawa, Percy Liang, and Chelsea Finn. Just train twice: Improving group robustness without training group information. In International Conference on Machine Learning, pp. 6781-6792. PMLR, 2021.
Ziwei Liu, Ping Luo. Xiaogang Wang, and Xiaoou Tang. Deep learning face attributes in the wild. In Proceedings of International Conference on Computer Vision, 2015.
Christos Louizos, Uri Shalit, Joris Mooij, David Sontag, Richard Zemel, and Max Welling. Causal effect inference with deep latent-variabl models. Conference on Neural Information Processing Systems, 2017.
lean-Baptiste Mouret and Jeff Clune. Illuminating search spaces by mapping elites. arXiv preprint 1rXiv:150.004000 2015.
Junhyun Nam, Hyuntak Cha, Sungsoo Ahn, Jaeho Lee, and Jinwoo Shin. Learning from failure: Training debiased classifier from biased classifier. Conference on Neural Information Processing Systems, 2020.
Yuval Netzer, Tao Wang, Adam Coates, Alessandro Bissacco, Bo Wu, and Andrew Ng. Reading digits in natural images with unsupervised feature learning, In NIPS Workshop Deep Learning and Unsupervised Feature Learning 2011, 2011.
Luke Oakden- Bakden-Rayner, Jared Dunnmon, Gustavo Carneiro, and Christopher Ré. Hidden stratification causes clinically meaningful failures machine learning for medical imaging. In Proceedings of the ACM conference health, inference, and learning, pp. 151-159, 2020.
Matteo Pagliardini, Martin Jaggi, François Fleuret, and Sai Praneeth Karimireddy. Agree to dis- agree: Diversity through disagreement for better transferability. arXiv preprint arXiv:2202.04411, 2022.
Tianyu Pang, Kun Xu, Chao Du, Ning Chen, and Jun Zhu. Improving adversarial robustness via promoting ensemble diversity. In International Conference on Machine Learning, 2019.
Jack Polker- Luke Metz, Cinjon Resnick, Hengyuan Hu, Adam Lerer, Alistair Letcher, Alex Peysakhovich, Aldo Pacchiano, and Jakob Foerster. Ridge rider: Finding diverse solutions by following eigenvectors of the hessian. Conference on Neural Information Processing Systems, 2020.
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Kopf, Edward Yang, Zachary Vito,, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. Pytorch: An imperative style, high- gigh-performncc deep learning library. Conference Neural Information Processing Systems, 2019.
Judea Pearl. Causality: models, reasoning and inference, volume 19. Cambridge university press, 2000.
Mohammad Pezeshki, Sékou-Ouma Kaba, Yoshua Bengio, Aaron Courville, Doina Precup, and Guillaume Lajoie. Gradient starvation: A learning proclivity in neural networks. In A. Beygelz- imer, Y. Dauphin, P. Liang, and Wortman Vaughan (eds.), Advances Neural Information Processing Systems, 2021.
12

---

Published as a conference paper at ICLR 2023
Alexandre Rame and Matthieu Cord. Dice: Diversity in deep ensembles via conditional redundancy adversarial estimation. International Conference on Learning Representations, 2021.
Alex Rogozhnikov. Einops: Clear and reliable tensor manipulations with cinstein-like notation. In International Conference on Learning Representations, 2022.
Shiori Sagawa, Pang Wei Koh, Tatsunori B Hashimoto, and Percy Liang. Distributionally robust neural networks for group shifts: On the importance of regularization for worst-case generaliza- tion. International Conference Learning Representations, 2020.
Shiori Sagawa, Pang Wei Koh, Tony Lee, Irena Gao, Sang Michael Xie, Kendrick Shen, Ananya Kumar, Weihua Hu, Michihiro Yasunaga, Henrik Marklund, Sara Beery, Etienne David, Ian Stavness, Wei Guo, Jure Leskovec, Kate Saenko, Tatsunori Hashimoto, Sergey Levine, Chelsea Finn, and Percy Liang. Extending the WILDS benchmark for unsupervised adaptation. In Inter- national Conference on Learning Representations, 2022.
Bernhard Schölkopf. Causality for machine learning. arXiv preprint arXiv:1911.10500 2019.
Luca Scimeca, Seong Joon Oh, Sanghyuk Chun, Michael Poli, and Sangdoo Yun. Which short- cut cues will dnns choose? study from the parameter-space perspective. arXiv preprint arXiv:2110.03095, 2021.
Ramprasaath R Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedantam, Devi Parikh, and Dhruv Batra. Grad-cam: Visual explanations from deep networks via gradient-based local- ization. In Proceedings of the IEEE international conference on computer vision, pp. 618-626, 2017.
Lesia Semenova, Cynthia Rudin, and Ronald Parr. A study rashomon curves and volumes: A new perspective on generalization and model simplicity in machine learning. arXiv preprint arXiv: 01755, 2019.
Harshay Shah, Kaustav Tamuly, Aditi Raghunathan, Prateek Jain, and Praneeth Netrapalli. The pitfalls of simplicity bias in neural networks. Conference on Neural Information Processing Systems, 2020.
Archit Sharma, Shixiang Gu, Sergey Levine, Vikash Kumar, and Karol Hausman. Dynamics-ave aware unsupervised discovery of skills. International Conference Learning Representations, 2020.
Samarth Sinha, Homanga Bharadhwaj. Anirudh Goyal, Hugo Larochelle, Animesh Garg, and Flo- rian Shkurti. Dibs: Diversity inducing information bottleneck in model ensembles. P Proceedings of the AAAI Conference on Artificial Intelligence, 2021.
Kihyuk Sohn, David Berthelot, Chun-Liang Li, Zizhao Zhang. Nicholas Carlini, Ekin D. Cubuk, Alex Kurakin, Han Zhang, and Colin Raffel. Fixmatch: Simplifying semi-supervised learning with consistency and confidence. Advances in Neural Information Processing Systems, 2020.
Baochen Sun, Jiashi Feng, and Kate Saenko. Correlation alignment for unsupervised domain adap- tation. arXiv preprint arXiv: Arxiv-1612.01939, 2016.
Damien Teney, Ehsan Abbasnejad, Simon Lucey, and Anton van den Hengel. Evading the simplicity bias: Training diverse of models discovers solutions with superior ood generalization. arXiv preprint arXiv:2105.05617 2021.
Damien Teney, Maxime Peyrard, and Ehsan Abbasnejad. Predicting is not understanding: Recogniz- ing and addressing underspecification in machine learning. In European Conference on Computer Vision, pp. 458 476. Springer, 2022.
Eric Tzeng, Judy Hoffman, Ning Zhang, Kate Saenko, and Trevor Darrell. Deep domain confusion: Maximizing for domain invariance. arXiv preprint arXiv: 141.3444, 2014.
Vladimir Vapnik. Principles risk minimization for learning theory. In Advances in neural infor- mation processing systems, pp. 831-838, 1992.
13

---

Published as a conference paper at ICLR 2023
Xiaosong Wang, Yifan Peng, Le Lu, Zhiyong Lu, Mohammadhadi Bagheri, and Ronald M Sum- mers. Chestx-ray8: Hospital-see chest x-ray database and benchmarks on weakly-supervie classification and localization of common thorax diseases. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 2097-2106, 2017.
Mitchell Wortsman, Maxwell Horton, Carlos Guestrin, Ali Farhadi, and Mohammad Rastegari. Learning neural network subspaces. International Conference on Machine Learning. 2021.
Han Xiao, Kashif Rasul, and Roland Vollgraf. Fashion-mnist: a novel image dataset for benchmark- ing machine learning algorithms. arXiv preprini arXiv: 10::: 170..07747 2017.
Qizhe Xie, Minh-Thame Thang Luong, Eduard Hovy, and Quoe V. Le. Self-training with noisy student improves imagenet classification. Conference on Computer Vision and Pattern Recognition, 2019.
Bolei Zhou, Agata Lapedriza, Aditya Khosla, Aude Oliva, and Antonio Torralba. Places: A 10 mil- lion image database for scene recognition, IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017.
14

---

Published as a conference paper at ICLR 2023
Method
Parameters
MNIST
SVHN
FMNIST
CIFAR
Upper bound oracle
99.7
89.7
77.4
68.7
Evading (N=8) Evading (N=16)
27152
97.3
82.1
59.6
55.8
54304
96.6
72.1
64.6
58.4
Evading (N=16, weight sharing)
3904
99.7
50.8
50.3
50.2
Evading (N=32, weight sharing)
4448
99.6
50.7
50.1
50.2
DivDis (N=2)
3428
98.1
66.5
62.5
51.5
DivDis (N=4)
3496
99.4
75.1
66.1
53.4
Table 4: Accuracies for the 4-way collage task. We denote the method in Teney et al. (2021) as "Evading". DivDis with 4 heads shows competitive performance to the Evading method with up to 16 independent models. We note that by default, Evading separate models, meaning parameter count and computation scales linearly with N. Evading method fails when networks share backbone parameters.
14-
Figure 7: Samples from the original 16 x 16 collage dataset from Teney et al. (2021) (leff) and the higher- resolution 32 x 32 dataset used in Tab. (right). A substantial amount of visual detail is lost in the original datasct, especially in harder datasets such FashionMNIST and CIFAR.
A THE COLLAGES TASK: CAN DIVDIS LEARN MORE THAN Two FUNCTIONS?
To further investigate whether DivDis can learn more than two functions, we evaluate DivDis on the block collages dataset (Teney et al., 2021). This dataset is a binary classification task in which four attributes have a complete spurious correlation with labels in the source dataset. Each datapoint is constructed by concatenating image each from the MNIST (Deng, 2012), SVHN (Netzer
Method
MNIST
SVHN
FMNIST
CIFAR
Evading (N=8)
97.3
82.1
59.6
55.8
Evading (N=16)
96.6
72.1
64.6
58.4
Evading (N=8, ours)
99.5
70.0
56.3
50,9
Evading (N=16, ours)
99.7
86.3
65.9
60.8
Evading (N=8, ours, x2)
99.5
49.7
51.6
49.7
Evading (N=16, ours, x2)
99.8
49.6
52.0
49.7
DivDis (N=2, x2)
97.6
63.9
61.8
51.7
DivDis (N=4, x2)
96.5
73.0
69.2
52.7
Table Collage dataset accuracies with higher resolution images (16 16 - x 32). re-implemented the "Evading" method of Teney et al. (2021), and denote results from our codebase as "ours". The Evading method fails on 32 x 32 images, likely because input-space diversification sensitive to input dimensionality. In contrast, DivDis performs output-space diversification and is robust change in input dimensionality.
15

---

Published as a conference paper at ICLR 2023
et al., 2011). FashionMNIST (Xiao et al., 2017), and CIFAR (Krizhevsky et al., 2009) datasets in 2 x 2 grid. This task aims to learn the predictive patterns corresponding to each of the four datasets based on labeled data with complete correlation between all four. After training on the source data, we separately test for each of the four features using de-correlated datasets labeled according to one of the four features. We report the highest accuracy achieved among functions inside the learned ensemble for each of the four features.
We evaluated DivDis with 2 or 4 heads on this task. For more than two heads, we minimize the average mutual information loss (3) between all pairs of heads. As the unlabeled set for DivDis, we use decorrelated 4-way collage data constructed from the validation split of each dataset. Results in Tab. 4 compare feature-wise test accuracies with that of "Evading" (Teney et al., 2021). DivDis with heads shows competitive performance to Evading with up to 16 independent models, indicating that DivDis requires substantially fewer functions to achieve the same performance as Evading. We also note that crucial to use the correct number of heads for DivDis. As expected, training only N 2 heads results in subpar performance because this dataset has 4 different features. The two methods differ in scalability: Evading uses N independent models, meaning that the number of parameters and computation cost scales linearly with N. In contrast, DivDis shares all weights except for the last layer. As shown in Tab. 4, Evading fails when the N networks perform weight sharing.
As shown in Fig. 7, the images in the original collages dataset have resolution 16 x 16, which loses substantial amount of visual detail because each image from (MNIST, SVHN FMNIST, CIFAR) is reduced to an 8 x 8 image. We evaluate both methods on larger 32 x 32 collage data in Tab. 5, where Evading fails discover features besides MNIST. Evading is an input-space diversification method, making it sensitive to input dimensionality. In contrast, DivDis performs output-space diversification, making its performance more robust to changes input dimensionality.
Diversification is an effective approach to handling underspecification, and the two broad approaches of input-space diversification (Teney et al., 2021; 2022) and output-space diversification (e.g.. Di- vDis) are complementary approaches with different strengths and weaknesses. As shown in Tables 4 and 5, DivDis is more scalable in terms of model count and input size. We also note that this scalabil- ity is also reflected in results the large-scale Camelyon dataset: DivDis achieves 90.4% accuracy on test hospitals (Tab. 3), whereas the method in Teney et al. (2022) achieves 82.5%. Conversely, while DivDis was shown to require fewer functions, Evading (Teney al., 2021) achieves higher performance on the original collage dataset when using up to 96 models. An exciting direction for future work is understanding and consolidating these tradeoff' between input-space and output- space diversification methods for underspecified tasks.
B PARALLEL IMPLEMENTATION OF MUTUAL INFORMATION OBJECTIVE
import torch from ei einops import rearrange
def nutual_info_ll
(probs):
Input: predicted probabilites on target batch. B, H, probs. shape B-batch_ssit size, H=heads, D=pred_ dim marginal p probs...
arrobs.meaniieo mean (dim=0) $ H, marginal_p torch. corch.einsum ("hd, ge->hdee, marginal_F, marginal_p) $ H1, H, D, D.
marginal_p rearrange (marginal_s, "h d e -> (h g) (d e) 8 H*2, D-2
joint_p torch. bge probs, probs) mean (dim=0) joint_p rearrange (fointt_. "h g e -> (h g) (d ") $ H-2, D^2
H, H, D,
kl_divs = jeint_p (jognt_p.log() - marginal_p.og()) krng rearrange (kl (dim=-1), " (h g) -> h g", b=E) $ H, H pairwise_mie torch. (k1_ grid, diagonal=1) # Get only off-diagonal HL divergences return pairwise mis.
This implementation is based on the PyTorch Paszke et al. (2019) and einops Rogozhnikov (2022) libraries. It demonstrates that the mutual information objective (3) is easily parallelized across the input batch using standard tensor operations.
16

---

Published as a conference paper at ICLR 2023
C EXPERIMENTAL SETUP
C.I DETAILED DATASET DESCRIPTIONS
Toy classification task. Our toy binary classification data is constructed as follows. The source
dataset Dy has binary labels with equal aggregate probability p'y = 0) = p = 1) = 1 Each
2.
datapoint is 2-dimensional vector, and the data distribution for each class in the source dataset is:
p(u y 0) Unif([ ( 1, 0] [0, 1J)
p(z y = 1) = Unif( (0, 1] x [)),0)-
In contrast, the data distribution for each class in the target dataset is:
p( 0) Unii([-1111111 1, x [-1,1])
p( y 1) Unif([0, 1] x [-1,1]).
Labels are balanced for the target dataset. Put differently, the target dataset has a larger span than the source dataset, and the labels of the target dataset reveal that the true decision boundary is the Y- Yaaxis.
CXR-14 pneumothorax classification. The CXR-14 dataset Wang al. (2017) is large-scale dataset for pathology detection in chest radiographs. We evaluate on the binary pneumothorax clas- sification task, which has been reported to suffer from hidden stratification: subset of the images with the disease include a chest drain, a common treatment for the condition Oakden-Rayner et al. (2020).
Waterbirds dataset. Each image in the Waterbirds dataset constructed by pasting a waterbird or landbird image background drawn from the Places dataset Zhou al. (2017). There two backgrounds in this dataset water and land, where each category of birds is spuriously correlated with one background. Specifically, there are 4,795 training samples, where 3,498 samples are from "waterbirds in water" and 1,057 samples are from "landbirds in land". "Waterbirds in land" and "landbirds in water" are considered as minority groups, where 184 and 56 samples are included, respectively.
CelebA dataset. The CelebA dataset Liu et al. (2015) large-scale image dataset with over 200, 000 images of celebrities, each with 40 attribute annotations. We construct four different com- pletely correlated problem settings, each based on a pair of attributes. The pair of attributes consists of a label attribute and spurious attribute, and we remove all examples from the two minority groups in the source dataset. The four problem settings are summarized below. Our task construc- tion is similar to that of Sagawa al. (2020), which uses hair color as the label and gender as the spurious attribute.
Label attribute
Spurious attribute
CelebA-CC.A CelebA-CC-2
Mouth Slightly_Opp
Wearing Lipstick
Attractive
Smiling
CelehA-CC-3
Wavy_ Hair
High_Chees Cheekbones
CelebA-CC-4
Heavy Makeup
Lips
MultiNLI dataset. Given a hypothesis and promise, the task of MultiNLI dataset is to predict if the hypothesis is entailed by, neutral with, or contradicts with the promise. The spurious correla- tion exists between contradictions and the presence of the negation words nobody, no, never, and nothing Gururangan et al. (2018). The whole MultiNL. dataset is divided into six groups, where each spurious attribute belongs to {" ('no negation", "negation") } and each label belongs to (entailed, neutral, contradictory,, ). There are 206, 175 samples in total, where the smallest group only has 1,521 samples (entailment with negations).
Camelyon17-WILDS dataset. (2022). Input images of patches from lymph node sections given, and the task is to classify as either a tumor or normal tissue. Evaluation is performed on OOD hospitals for which labels are unseen during training. The model is given unlabeled validation images from the OOD hospitals.
This dataset is part of the U-WILDS benchmark Sagawa et al.
17

---

Published as a conference paper at ICLR 2023
C.2
DIVDIS HYPERPARAMETER SETTINGS
We show below the hyperparameters used in our experiments:
Toy Clasification
MNIST-CIFAR
Waterbirds
(4R-14
Waterhinds-GG CC
CelehA-CC
MultiNLI-CC
N
2,20
10
10
1,10, 100. 1000
10
10
10
1000
22
10
10
0,1, 10, 100
10
0, 10
0,
0,10
0.0.1
m
1
16
16
16
16
16
D ADDITIONAL EXPERIMENTS
Extended learning curves toy task. In Fig. 8, show extended version of the learning curve shown in Fig. 3. The extended plot includes cross-entropy loss and mutual information loss during training. The learning curves show that cross-entropy loss decreases first, at which point both of the heads represent functions similar to the ERM solution. Afterwards, the mutual information loss decreases, causing the functions represented by the two heads to diverge.
Visualization of functions on 3D toy task. We examine the extent to which the DIVERSIFY stage can produce different functions by visualizing which input dimension each function relies on. We modify the synthetic binary classification task to have 3-dimensional inputs and train DivDis with {2, 3, 5] heads. For each head, we visualize the Pearson correlation coefficient between each input dimension and output. We normalize this 3-dimensional vector to sum to one and plot each model as a point on a 2-simplex in Fig. 9, with independently trained functions as a baseline. The results show that the DIVERSIFY stage acts as repulsive force between the functions in function space, allowing the collection of heads to explore much closer to the vertices. This experiment also demonstrates why vanilla ensembling is insufficient for underspecified problems: the diversity due to random seed is not large enough to effectively cover the set of near-optimal functions.
Noisy dimension. To if DivDis effectively combat simplicity bias, further evaluate on a harder variant of the 2D) classification problem which we add noise along the x-axis. This noise makes the "correct" decision boundary have positive non-zero risk, making it harder to learn than the other function. Results in Fig. 10 demonstrate that even in such scenario, DivDis recovers both the x-axis and y-axis decision boundaries, suggesting that DivDis can be effective even in scenarios where ERM relies on spurious features due to simplicity bias.
Evaluation on completely correlated data. We evaluate DivDis on existing benchmarks modified to exhibit complete correlation. Using the Waterbirds Sagawa et al. (2020), CelebA Liu et al. (2015), and MultiNL Gururangan et al. (2018) datasets, we alter the source dataset to include only majority groups while keeping target data intact. We denote these tasks as Waterbirds-CC, MultiNLI-CC, etc to distinguish from the original benchmarks. These -CC tasks are considerably more difficult than the original benchmarks, and introduce specific challenge not addressed in the existing literature: leveraging the difference in source and target data distribution to encode and subsequently disam- biguate tasks with high degrees of underspecification. To our best knowledge, no prior methods designed to address such complete correlations.
As the closest existing problem setting is subpopulation shift, we show the performance of ERM, JTT Liu et al. (2021), and Group DRO Sagawa et al. (2020) as naive point of comparison. We also include random guessing baseline as a lower bound on performance. As expected, Tab. shows that existing subpopulation shift methods show subpar performance on these tasks, notably failing do better than random guessing the Waterbirds- CCC task. This is hardly surprising, as methods based on loss upweighting such as JTT and Group DRO require minority examples in the training data to upweight. In contrast, DivDis is well-suited to this challenging setting, and can deal with complete correlation by leveraging unlabeled target data to find different predictive features of the labels.
CelebA hyperparameter grid. CelebA dataset. This grid shows a strong corrleation between metrics with respect to hyperparametrr choice, indicating that DivDis can be tuned using only labeled source data.
In Fig. 13, show an additional hyperparameter grid for the
18

---

Published as a conference paper at ICLR 2023
Waterbirds-CC
CelebA-CC-I
CelebA-CC-2
MulÜNLI-CC
Avg (%)
Worst (%)
Avg (%)
Worst (%)
Avg (%)
Worst (%)
Avg (%)
Worst (x)
Random
50.0
50.0
50.0
50.0
50.0
50.0
33.3
33.3
ERM
60.5
2.0
57.0
73.1 0.9
53.2
2.5
JTT
44.6 + 1.9
26.5 + 1.4
71.4 1.9
51.2 = 5.4
78.7 + 0.8
59.8 = 1.1
80.0 1: 4.0
40.5 = 2.3
Group DRO
55.6 4.8
47.1 4 8.9
71.6 0.3
59.3
2.6
71.6
61.3 = 2.3
79.1 1: 3.4
39.8
1.4
DivDis reg
87.2
77.5
01.0 0.4
85.9 + 1.0
79.7
69.3
80.3
67.6
4.0
DivDis
87.6
1.4
82.4 1.9
90.8 4: 0.4
85.6 + 1.1
79.5
68.5
79.9 : 1.2
71.5
2.5
Table 6: Modified Waterbirds, CelebA, and MultiNLI datasets with complete correlation between labels and spurious attribute. DivDis outperforms previous methods in terms of both average and worst- group accuracy.
Acc
AUC
AUC (drain)
AUC (no-drain)
ERM 0.883
0.828
0.904
0.717
Pseudo-label
0.898
0.835
0.904
0.721
DivDis
0.934
0.836
0.902
0.737
Table 7: Pneumothorax classification metrics the test of CXR-14. In addition to overall accuracy and AUC, we measure AUC separately on two subsets the positive class, drain no-drain. DivDis shows higher AUC on the no-drain subset, which is more indicative the intended population of patients not yet being treated. Performance gains on the no-drain subset also contribute positively to the overall metrics (Acc and AUC).
Additional Disambiguate experiments MNIST-CIFAR data. In Fig. 11, we plot the accuracy of the head selected by the active and random querying strategies, when using different numbers of label queries. Notably, the active querying strategy successfully chooses the best head with even one label query. In Fig. 12, we show additional Grad-CAM plots for MNIST-CIFAR data, on 6 more random datapoints from the source dataset. Compared to the example given in the main text, these examples are just as informative in terms of which head is better.
Effeet of ratio. We test various values between 0 and for the regularizer loss (4), on the Wa- terbirds benchmark. Fig. 14 shows that even using a ratio of 0.1 yields close 80% worst-group accuracy, demonstrating that the performance of DivDis is not very sensitive to this hyperparameter.
CXR pneumothorax classification. investigate whether DivDis can disambiguate naturally oc- curring spurious correlations, we consider the CXR-14 dataset Wang al. (2017), large-scale dataset for pathology detection in chest radiographs. We evaluate on the binary pneumothorax clas- sification task, which has been reported to suffer from hidden stratification: subset of the images with the disease include a chest drain, a common treatment for the condition Oakden-Rayna et al. (2020). We train DivDis with two heads to whether tures of chest drains and lungs as a predictor for pneumothorax. The unlabeled data for DivDis is a subset of validation data sampled we compare against the semi-supervised learning method Pseudo-label Lee et al. (2013), to see how much of the performace gain of DivDis can be attributed to the unlabeled target set alone. In Tab. 7, we show test split accuracy and AUC, along with AUC for the subset of positive samples with and without a chest drain. Our experiments show that DivDis achieves higher AUC in the no-drain split while doing marginally worse on the drain split, indicating that the chosen head is relying more on the visual features of the lung. The overall metrics (Ace and AUC) indicate that this performance gain in the no-drain subset also leads to better performance in overall metrics.
disambiguate between the visual fea-
that the ratio of drain
drain is 1:1. In addition ERM,
E FINITE-HYPOTHESS GENERALIZATION BOUND FOR HEAD SELECTION
Proposition Let the heads have risk b S l2... S Iy E on the target dataset, and let A l2 41- The required number of i.i.d. labels from the target set to select the best head with probability 8 is m 2(log 2N -log
19

---

Published as a conference paper at ICLR 2023
Accuracy
0.8
0.6
Cross Entropy
1.0
0.0
Am on pol
200
400
600
800
1000
Figure 8: Extended visualization for the 2D classification task (Fig. 2), with additional curves for the cross- entropy and mutual information losses. Note that only accuracy is measured with target data, and the cross- entropy and mutual information losses are the training metrics for DIVERSIFY measured on source data. Until around iteration 100, the model initially decreases cross-entropy the cost of increasing mutual information. The decision boundaries at this stage similar for the two heads. Afterwards, both the mutual information and cross-yntropy decrease, leading the heads having very different decision boundaries.
Table CelebA dataset with complete correlation between 4 different pairs of attributes. DivDis outperforms previous methods in all but one setting.
CelebA-CCC CC-1
CelebA-CC-2
CelebA-CC-3
CelebA-CC-4
Avg (%)
Worst (%)
Avg (%)
Worst (%)
Avg (%)
Werst (%)
Avg (%)
Worst (%)
ERM
70.9 1: 2.0
57.0 5.8
73.1 0.9
41.1
87.0 1 0.7
71.9 :: 2.6
63.9 3.5
1.4
71.4+119
64.8
34.0
67.4
49.3
GDRO
71.6 0:6:::033
59.3
71.6
61.3
88.2 0.6
83.7 0.71:88
65.0
21.
Div wlo reg
91.0 + 0.4
85.9 1.0
79.7
69.3 1.9
7..6
62.0 2.6
84.7
6..4
DivDis
90.8
85.6
79.5
68.5
80.6
67.1
818
7..5
Proof. Given m i.i.d. samples, Hoeffding's inequality gives for all 0,
P p
{]
< 2 exp (-2me²)
(6)
The event of failing to select the best head is superset of the following event, for which we can bound the probability as:
[[11 -
) 1/2
12
A .... ... v (IN
IN (IN-TN>> ) 2N exp
mA?
2
20

---

Published as a conference paper at ICLR 2023
Figure 9: Visualization of 32, 5) functions trained independently (top row) and with DivDis (hottom row).
Vertices of the 2-simplex represent the three dimensions of the input data. The functions learned by DivDis much more diverse compared independent training.
Head 41
Head #2
Figure 10: Functions learned by DivDis on variant of the synthetic classification task, where the labeled source dataset has noise along the a-axis. The second head recovers the V-axis decision boundary even though it is harder learn due the noise. This indicates that DivDis successfully overcome simplicity bias and learn functions FRM would not consider.
mA?
Solving for 2N exp
we get the sample size bound
2(log 2NV
log 8)
m
(8)
A
Table 9: CXR dataset test set metrics
Accuracy
AUC
AUC (drain)
AUC (no-drain)
ERM
0.883 0.006
0.828 +:2 0.001
0.904 0.008
0.717 + 0.005
Pseudolabel
0.898 + 0.015
0.835 + 0.004
0.904 +: 0.007
0.721 + 0.007
DivDis
0.934 0.014
0.836 + 0.007
0.902 + 0.006
0.737 + 0.001
21

---

Published as a conference paper at ICLR 2023
75
70
Accuracy (%)
65
55
Active Random
50
2
2
2
2
2
2
Number labels
Figure 11: CIFAR label accuracy of chosen head in DivDis vs label used for the Disambiguate stage. The active querying strategy chooses the best head with one label query, and the random query strategy similarly works with modest budget of queries.
0
Figure 12: Additional GradCAM visualizations of two learned heads on examples from the source datasct of the MNIST-CIFAR task. These examples sufficiently differentiate the best of the two heads, demonstrating the viability of the source data inspection strategy for the DISAMBIGUATE stage.
22

---

Published as a conference paper at ICLR 2023
Avg Acc, Source Dist
Avg Acc, Target Dist
Worst Acc. Target Dist
1e-2
82.3
85.0
84.3
81.8
84.5
83.4
44.6
53.7
49.6
1e-1
1e0
82.3
82.6
76.8
80.8
81.5
75.3
44.6
45.0
28.3
161
1e1
1e1
Figure 13: Grids for DivDis's two hyperparameters (A1, the CelebA dataset. Rows indicate and columns indicate λz. show three metrics measured with held-out datapoints: average accuracy on source and target distributions and worst-group accuracy on the target distribution. We average each metric across three random seeds. The high correlation between the three metrics indicates that we can tune the hyperparameters of DivDis using only held-out labeled source data.
Worst-gmen Iccursy
0.8
0.7
0.6
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
Figure 14: Worst-group accuracy the Waterbirds benchmark when using different ratio values for p(y) in the regularizer loss (4). The plot shows that the performance of DivDis not very sensitive to this hyperparameter.
23


Reviews:
[{"review_id": "gezSAr3TxJ", "reviewer": "Reviewer_jJSw", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper considers the problem of ***underspecification*** of the dataset. To address this issue, this paper proposes a *two-stage* framework, Diversify and Disambuguate (DivDis), for learning from underspecified data. For the first stage (***I. diversity***), DivDis learns a diverse set of hypotheses that gain low source loss but make different predictions on target data. Then, in the second stage (***II. disambiguate***), DivDis aims to select one of the learned functions. Note that, the work uses additional information for the function selection. In the experiment, this work validates the effectiveness of DivDis on datasets with *a complete correlation* (the source distribution has a spuriously correlated attribute that can predict the label perfect accuracy).\n\n", "summary_of_the_review": "--- **Post Rebuttal** ---\n\nThe proposed method (DivDis) is reasonable to me. The idea of learning multiple functions is already explored and studied for the same problem of underspecification. However, ***the new contributions based on this idea are convincing and sufficient***. The authors also discuss the effect of the number of heads (N), which corresponds to the number of functions. Therefore, I tend to accept this paper.\n", "strength_and_weaknesses": "**[Strength]**\n\n+ This work is well written. The problem, method, and experiment are clearly presented. \n\n+ The related work is nicely organized.\n\n+ The proposed DivDis is straightforward and has a clear motivation. Learning multiple and diverse functions on the underspecified dataset is reasonable. Then, selecting the most suitable function is also straightforward.\n\n+ The three strategies in stage two of DivDis are good. This work clearly presents them. Active querying required as little as a single label is interesting to me. In addition, this work highlights in Section 3.3 that \"We emphasize that existing OOD methods tune hyperparameters using target set labels\" Please clarify how the existing methods use target labels and give some references or examples.  \n\n**[Weakness]**\n\n- While the framework is reasonable, its novelty is somewhat weak. First, existing methods already study the idea of learning diverse functions. For example, works [1,2] also study the underspecification issue and propose to learn multiple different functions. From this point, the novelty of this work is limited.\n\n- Following up on the above, the first stage of this work (i.e., diversity) uses a mutual information loss to learn diverse functions. In comparison, [1] uses a penalty on the alignment of their input gradients; [2] uses a local independence loss and an on-manifold loss. Then, a question is: which manner is more effective to learn multiple diverse functions? A discussion or experimental analysis would be helpful.\n\n- The comparison with [1,2] should be clearly reported in the experiment. I noticed that methods of [1,2] do not have a simple function selection while this work has (stage 2). So, this work can use the same selection manner of [1,2] for comparison.  \n\n    [1] Teney, Damien, et al. \"Evading the simplicity bias: Training a diverse set of models discovers solutions with superior ood generalization.\" In CVPR, 2022.\n\n    [2] Teney, Damien, Maxime Peyrard, and Ehsan Abbasnejad. \"Predicting is not understanding: Recognizing and addressing underspecification in machine learning.\" In ECCV, 2022\n\n - Crucial details are missing: 1) according to Section B.2, this work only uses N=2 heads. Namely, this work only learns two functions. Why use N=2? What is the effect of N on learning? 2) when the dataset is more complex (e.g., collages dataset of [2]), using N=2 might not suitable. Thus, this work needs to study N. \n\n-  In domain adaptation, using the disagreement of two classifiers is beneficial for achieving high accuracy on the target data [3]. Table 3 compares several domain adaptation methods. It would be better if this work could include [3] in the table. \n\n--- **Post Rebuttal** ---\nThe authors have provided responses for the above questions. The rebuttal is helpful and addresses most of my concerns.\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "NNjx7oXNkW", "reviewer": "Reviewer_5h8Y", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper adresses distribution shift between train and test samples. The authors leverage the fact that different models can perform equally on the train distribution while exhibiting diversity in their prediction on the test distribution. The diversity among trained model is enforced by explicitly requesting different predictions on unlabeled test data. More precisely, this is done by adding a loss term that minimizes mutual information between pairs of predictions from two models during their respective trainings. Another loss term avoids learning naive classifiers such as constant functions. \nAt inference time, one needs to select the appropriate model. This is done by querying the label of few data points for which disagreement among trained models is high and examining average accuracy on these points. \n\n", "summary_of_the_review": "I really enjoyed reading this paper and I have only a few remarks. \n\nHow does the model scale to large number of heads ? I suppose the MI term will be required for each pair of heads. Could this be a bottleneck ?\n\nI was expecting a bit more experiments of the number of necessary test point label reveals necessary. If possible, it would be nice to extend this. \n\nSince the methods is useful for underspecified data how can we assess to match this use case when start with a new dataset ?\n", "strength_and_weaknesses": "Pros : \n- The paper is very clear and easy to understand.\n- The contributions are well framed and positioned\n- The contributions are supported by clear experimental evidence \n\nCons : \n- The methods required unveiling the class labels of a selection of test points. \n- The approach is limited to underspecified data otherwise diverse predictors cannot be trained on the source distribution.\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "7cgq42fquQM", "reviewer": "Reviewer_kpaT", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper proposes a solution to underspecification. The authors show how to learn multiple models/hypotheses compatible with the training data (low source risk). They then propose multiple options to select the best one (using additional data such as OOD annotations).", "summary_of_the_review": "The paper studies an important problem, proposes an innovative solution, and demonstrate very interesting results on a range of datasets. I recommend it for acceptance.", "strength_and_weaknesses": "### Strengths\n- The paper studies a problem of great importance (OOD generalization) that is receiving increasing attention.\n- The paper proposes a solution that is complementary to other contributions proposed very recently (appropriated discussed by the authors). This paper encourages diversity across models in prediction space, whereas other solutions work in the space of features or input gradients.\n- The disambiguation stage is cleverly addressed with a variety of strategies, which allow comparing a complete method (i.e. including model selection, which some competing works do not address).\n- Experiments are performed on a variety of tasks and datasets.\n\n### Weaknesses/questions\n- W1: In the conclusion: \"An appealing property of DivDis is its automatic discovery of disentangled features, as demonstrated in Sec. 4.\" Which results does this refer to specifically?\n\n- W2: All experiments (except one on toy data) seem to use 2 heads. Is that right? What happened if the method is applied with >2 heads on the other datasets? The results in (Teney et al. 2021) seem to indicate that more heads are often better. The method in the two papers are different, but it would be interesting to investigate if/why (not) more heads could also help here.\n\n- W3: This paper includes a comparison with the method of (Teney et al. 2021) only on Waterbirds-CC, which seems unfair. I would have liked to see a comparison with the same CIFAR-MNIST (with 4 tiles) used by these other authors, since this dataset can only work with >= 4 heads.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "0VDtybl10AO", "reviewer": "Reviewer_X9wJ", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "DivDis tackles the problem of underspecification by generating diverse hypotheses and then disambiguating to choose the best one. Diversification is performed by training multiple heads to yield different predictions on an unlabeled target distribution. Disambiguation is performed by first actively querying to obtain labels for a small target set and then selecting the head with the best accuracy on this set.", "summary_of_the_review": "Overall, I vote to accept the paper since it addresses an important shortcoming of most DNNs: the inability to consider multiple, plausible ways of performing the task, especially in the absence of a clear signal from the training set. The proposed approach is sound and logical. \n\nClarifying some of the points I mentioned in the weaknesses could strengthen the paper.", "strength_and_weaknesses": "**Strengths (S):**\n\n[S1] The paper addresses a very important issue where the training data alone may not uniquely specify the best solution for the target distribution. This is a real issue for the current DNNs, since they are limited to producing a single hypothesis, which may or may not be the best way to perform the task. The work provides a principled way to uncover multiple plausible hypotheses and select the best one given a target distribution.\n\n[S2] The process of training diverse heads on unlabeled target distribution is logical. The experiments with the toy dataset and completely/partially correlated spurious factors clearly demonstrate the ability to learn different functions and the ability to choose the best one. However, I think the paper needs to clarify the points mentioned in the weaknesses section.\n\n[S3] The results showing that the hyperparameters that work on held-out source data work for target distribution too is important. This eases hyperparameter tuning. \n\n**Weaknesses (W):**\n\n [W1] Is it realistic to expect to have access to unlabeled data that has the SAME distribution as the test data? Wouldn't it be more realistic that the unlabeled data is distributed different from both train and test? How would the approach handle that situation?\n\n[W2] For each dataset/experiment, I think the paper should clarify how the unlabeled, target distribution was constructed. Where do the samples come from (a separate validation set?)? Does the size of unlabeled target set matter to ensure diversity?\n\n[W2.1] For Camelyon17, are both unlabeled data and test data from the same hospital? \n\n[W3] Most experiments are limited to 2 heads, so it is unclear how well the method generalizes beyond that (for non-toy datasets). \n\n[W4] Moreover, the separate head approach assumes prior knowledge of the number of plausible hypotheses. I wonder what happens when there is a mismatch between the actual number of plausible hypotheses for the task and the number of heads in the model (and not just toy dataset).\n\n[W5] For the sake of completeness, I think the paper should include methods (JTT/gDRO) for MNIST-CIFAR data with various mixing ratios.\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}]

Author rebuttal:

Decision: Accept: poster