On Calibration and Out-of-Domain Generalization
Authors: Yoav Wald
Amir Feder
Daniel Greenfeld
Uri Shalit
Venue: NeurIPS 2021 Poster (2021)
Keywords: Out-of-Domain Generalization
Calibration
Invariance
Causality

Abstract: Out-of-domain (OOD) generalization is a significant challenge for machine learning models. Many techniques have been proposed to overcome this challenge, often focused on learning models with certain invariance properties. In this work, we draw a link between OOD performance and model calibration, arguing that calibration across multiple domains can be viewed as a special case of an invariant representation leading to better OOD generalization. Specifically, we show that under certain conditions, models which achieve \emph{multi-domain calibration} are provably free of spurious correlations. This leads us to propose multi-domain calibration as a measurable and trainable surrogate for the OOD performance of a classifier. We therefore introduce methods that are easy to apply and allow practitioners to improve multi-domain calibration by training or modifying an existing model, leading to better performance on unseen domains. Using four datasets from the recently proposed WILDS OOD benchmark, as well as the Colored MNIST, we demonstrate that training or tuning models so they are calibrated across multiple domains leads to significantly improved performance on unseen test domains. We believe this intriguing connection between calibration and OOD generalization is promising from both a practical and theoretical point of view.



On Calibration and Out-of-domain Generalization
Yoav Wald
Amir Feder*
Daniel Greenfeld Jether Energy Research
Johns Hopkins University yoav. wald@gmail. com
Technion amir amirf amirfeder@@mail.com com
Gandelggreenfeld@@@mall.... com
Uri Shalit
Technion urishalit@toccnion..a. ic
Abstract
Out-of-domaal (OOD) generalization is significant challenge for machine learn- ing models. Many techniques have been proposed to overcome this challenge, often focused on learning models with certain invariance properties. In this work, we draw a link between OOD performance and model calibration, arguing that calibration across multiple domains can be viewed as a special case of invariant representation leading to better OOD generalization. Specifically, we show that un- der certain conditions, models which achieve free of spurious correlations. This leads us to propose multi-domas calibration measurable and trainable surrogate for the OOD performance of classifier. We therefore introduce methods that are easy to apply and allow practitioners improve multi-domain calibration by training modifying existing model, leading to better performance on unseen domains. Using four datasets from the recently proposed WILDS OOD benchmark 22) well the Colored MNIST dataset (21). we demonstrate that training or tuning models so they are calibrated unseen
calibration provably
across multiple domains leads significantly improved performance test domains. We believe this intriguing connection between calibration and OOD generalization is promising from both a practical and theoretical point of view.
Introduction
Machine learning models have recently displayed impressive success in plethora of fields However, as models are typically only trained and tested on in in-domain (ID) data, they often fail to generalize to out-of-domain (OOD) data [23]. The problem is especially pressing when deploying machine learning models in the wild, where they are required to perform well under conditions that
were not observed during training. For instance, a medical diagnosis system trained on patient data from few hospitals could fail when deployed in a new hospital.
Many methods have been proposed improve the OOD generalization of machine learning models. Specifically, there is rapidly growing interest in learning models that display certain invariance properties under distribution shifts and do not rely on spurious correlations in the training data 34 17]
M. While highlighting the need for learning robust models, so far these attempts have limited success scaling realistic high-dimensional data, and in learning truly invariant representations
20)
In this paper, we argue that an alternative and relatively simple approach for learning invariant representations could be achieved through model calibration across multiple domains. Calibration asserts that the probabilities of outcomes predicted by a model match their true probabilities. Our claim that simultancous calibration over several domains can be used as observable indicator
"qqual contribution
35th Conference on Neural Information Processing Systems (NcurIPS 2021).

---

for favorable performance on unseen domains. For example, if we take all patients for whom classifier outputs a probability of 0.9 for being ill, and in one hospital the true probability of illness in these patients is 0.85 while in the other it is 0.95, then we may suspect the classifier relies on spurious correlations. Intuitively, the features which lead the classifier to predict a probability of 0.9 imply different results under different experimental conditions, suggesting that their correlation with the label is potentially unstable. Conversely, if the true probabilities in both hospitals match the classifier's 's output, it may be sign of its robustness.
Our contributions are as follows: We prove that in Gaussian- models, under a general-positie condition, being concurrently calibrated across sufficient number of domains guarantees a model has no spurious correlations. We then introduce three methods for encouraging multi-domain calibration in practice. These are, in ascending order of complexity: (i) model selection by a multi-domain calibration score, (ii) robust isotonic regression as a post-processing tool, and (iii) directly optimizing deep nets with multi-domain calibration objective, based on the method introduced by Kumar et al. 20 We show that multi-domain calibration achieves the correct invariant classifier in a learning scenario presented by Kamath et al. [00 unlike the objective proposed in Invariant Risk Minimization M. Finally, we demonstrate that the proposed approaches lead to significant performance gains on the WILDS benchmark datasets |
and also succeed on the colored MNIST dataset 22
2 Calibration and Invariant Classifiers
2.1 Problem Setting
Consider observable features X, label Y and an environment (or domain) E with sample spaces X.Y, accordingly. We mostly focus on regression and binary classification, therefore y = IR or )'= (0,1), 1). To lighten notation, our definitions will be given for the binary classification setting and we will point out adjustments to regression where necessary. There is no explicit limitation on (g). but we assume that training data that has been collected from finite subset of the possible environments Ewain C E. The number of training environments is denoted by k, and Emain = [ei C. E, so that our training data is sampled from a distribution P[X, Y E' ej] Vi [k.. Our goal to learn models that will generalize to new, unseen environments in E.
Ideally, we would like to learn classifier that is optimal for all environments E. Unfortunately, only observe data from the limited set Erain and even if this set is extremely large, the Bayes optimal classifiers each environment do necessarily coincide. Following other recent work [34] 17 we therefore aim for a different goal - learning classifiers whose per-instance output will be stable across environments E, as we explain below.
We assume the data generating process for E,X, Y follows the causal graph in Figure I We differentiate between causal and anti-causal components of X, and further differentiate between the anti-causal variables which are affected or unaffected by E, denoted as X Suc-spuriuus and Xae-non spu-riosu spurious. respectively. As illustrative example, consider again predicting illness across different hospitals. When predicting lung cancer, Y, from patient health records, Xeasaal could be features like smoking. Xac-non n-c-o--spurious are symptoms of Y such infections that appear in chest X-rays, while X Xuc-quurious can be marks that technicians put on X-rays as in [51] Smoking habits may vary across hospital populations, might X-ray markings; but the influence of smoking on cancer and the manifestation of cancer in an X-ray do not vary by hospital.
We do not assume to know how to partition X into Xemusal Xac-sparious X Xac-non-puriious The main assumptions made in the causal graph in Fig. [ are that there are hidden variables, and that there is no edge directly from environment E to the label Y. Such an arrow would imply the conditional distribution of Y given X can be arbitrar- ily different in unseen environment E, compared to those present in the training set. Note that for simplicity do not include arrows from Xeausal to X spariouss and X nec-nonsppurios but they may be included as well.
E
X Xemusal
Y
X ac-sparious
ac-non-spurious Figure 1: Learning in the pres- ence of causal and anti-causal fea- tures. Anti-causal features can be either spurious (X (Xne-suurioas) or non-spurious (X sc--oo--purious).
See Appendix A.3 for brief introduction to causal graphs.

---

We will say a representation (X) contains a spurious
correlation with respect to the environments E and label Y, if + E (xX); this motivates our naming of Xx spuriou and Xmc- nom-siiru in Fig. 1 as Y M. E X ae sp-riioo but Y Il E. Xac-moo-sumn spurious Similar observations have been made by [ Having spurious correlation implies that the relation between (()) and Y depends on the environment it is not transferable nor stable across environments. In this work we will simply consider the output f f(X) of a classifier f: X - [0, as representation. The crux of this paper is the observation that having [E[Y f(X), E = f(X) for every value of E, i.e. f being a calibrated classifier across all environments, equivalent up-to simple transformation to having Y IL E f(X), and thus to f having spurious correlations with respect to E. We prove this assertion in section 2.2 and as a demonstration of this principle we prove (section B) that linear models which are calibrated across diverse set of environments E are guaranteed to discard Xac sparious as viable features for prediction.
2.2 Invariance and Calibration on Multiple Domains
We define calibration, along with straightforward generalization to the multiple environment setting. Definition 1. Let P(X, Y] be joint distribution over the features and label, and f X - [0. 1] a classifier Then f fxx is calibrated w.nt to if for all a E [0.1] the range of f. | f(X)= a] a. In the multiple environments setting, f(x) is calibrated on Evain if for all 6, E Emain and a in the range of f restricted to e,, E|Y fX) = a, E e; =
For regression problems, we consider regressors that output estimates for the mean and variance of Y, and say they are calibrated if they match the true values similarly to the definition above. The precise definition can be found in the supplementary material.
We now tie the notion of calibration on multiple environments with OOD generalization, starting with its correspondence with our definition of spurious correlations. Recall that a representation (XX does not contain spurious correlations if Y E (X)). Treating the output f(XX ) of a classifier as a representation of the data, and considering classifiers satisfying the above conditional independence with respect to training environments, we arrive at definition of an invariant classifier.
Definition 2. Let f : X [0, 1]. f is an invariant classifier w.r. Evain for all a [0, and environments ei, ej E Emain where a is in the range of f restricted to each of them:
E|Y | = a, E = e.] = E f(X) = aa ey].
(1)
Lemma D gives the correspondence between invariant classifiers and classifiers calibrated multiple environments. The proof is in Section A. of the supplementary material.
Lemma 1. Ifa binary classifier f is invariant w.r. Evain then there exists some g IR + 10, such
that (i) g o f is calibrated on all training environments, and (ii) the mean squared error of I on each environment does not exceed that of f. On the other hand, if a classifier calibrated on all training environments it is also invariant w.nt Evrain-
Now, we can note how the above notion of invariance relates to that of Invariant Risk Minimization shared classifier w H - [0, 1], w o & heing optimal on all environments w.r.t a loss 7 [0, 1] x y - Rgo- Under the representation +()) S(X), and the cross-entropy squared losses it turns out that the original IRM definition coincides with Equation F Hence we aim for a similar notion of conditional independence, yet we approach it from the point-of-view of calibration. In Section we will see that taking this approach leads to different methods that are highly effective in achieving and assessing invariance. We further note that the original IRM objective was deemed too difficult to optimize by the original IRM authors, leading them to propose an alternative called IRMv1. This alternative however does not capture the full set of required invariances, as shown by 20 whereas we show in section 6.1 that multi-domain calibration does indeed capture the required invariances.
M. where invariance of representation & X' - H. is linked
Having established the connection between calibration multiple environments and invariance, there are several interesting questions and points to consider:
Calibration and sharpness. Calibration alone is not enough to guarantee that a classifier performs well; on a single environment, always predicting E| will give a perfectly calibrated classifier.
See Observation 2 in 20 for proof.
3

---

Hence, multi-domain calibration should be combined with some sort of guarantee on accuracy. In the calibration literature, this is often referred as sharpness. To this end, in Section 5 we will propose regularizing models during training fine-tuning with Calibration Loss Over Environments (CLOOVE. Combining this regularizer with standard empirical loss functions helps balance between sharpness and multi-domain calibration. Even without training new model, we will propose methods for model selection and post-processing that are very casy to apply and help improve multi-domain calibration without significant effect on the sharpness of the models.
Generalization and dependence on xac-sprriuss- Suppose that f(X) is calibrated on Erain Under what conditions does this imply it is calibrated on E? It is easy to show that calibration several environments entails calibration on any distribution which can be expressed as a linear combination of the distributions underlying said environments. However, set & we would like to know what conditions and how many training environments are required for calibration to generalize. We also wish to understand when does calibration over finite set of training environments indeed guarantee that a classifier is free of spurious correlations. We now turn to answer these questions in the setting of Gauss-Ga models.
go beyond that? Given general
3 Motivation:
Model
Let us consider data where is a multivariate Gaussian. Since we will be considering Gaussian data, the set of all environments & will be parameterized using pairs of real vectors expressing expectations and positive definite matrices of an appropriate dimension expressing covariances: (1, Σ) μ Rd
$ $4+).
For two scenarios ((a) and (b) in Figure 2 we prove that when provided with data from training enviromments, where i is linear in the number of features, and the environments satisfy some mild non-degeneracy conditions, any predictor that is calibrated on all training environments will not rely on any of the spurious features sp. and will also be calibrated on all E E.
In scenario (a), we take Y be a binary vari- able drawn from Bernoulli distribution with parameter 1 E [0, I]. and observed features generated conditionally on Y. The features Xac-ns E dh are invariant, meaning their con- ditional distribution given Y is the same for all environments, whereas Kac-sp E R& are spu- rious features, their distribution may shift between environments, altering their correlation with Y. The data generating process for training environment i E [k] in Fig. 2 a) is given by:
E
E
Kac-ns
X Xac-sp
Xe
Y
Xxe
(a)
(b)
Figure 2: Graphs describing our theoretical anal- ysis. Features are: (a) anti-causal, some spurious while others invariant. (b) causal and may undergo covariate shift, or anti-causal and spurious.
w.p O.W
X as
Y y N 112-1//)m Ens) Ex).
X Xac-sp 1
(2)
For x [Xac-ns Xac-spl we consider a linear classifier (x: w, b) ( where a R [0 some invertible function (e.g. sigmoid). Since the mean of spurious features, pi, is determined by y. these features can help predict the label in some environments. Yet, these correlations do not carry to all environments, and fxx) might rely on spurious correlations whenever the coefficients in w corresponding to Xac-sp are non-zero. Any such classifier can suffer an arbitrarily high loss in an unseen environment, because new environment can reverse and magnify the correlations observed in Eraii- Using these definitions, we may now state our result for this case:
Theorem 1. Given > 2dl training environments where data generated according to Equation with parameters {μ4, E, Men we say they lie in general position for all non-zero X E Rdv;
span { Σxx+
{ p4
dim
dup + 1.
(E(N)
If a linear classifier is calibrated on k training environments which lie in general position, then its coefficients for the features Xac-op are zero. Moreover, the set of training environments that do not lie in general position has measure zero in the set of all possible training envirouments Ek.

---

As corollary, we see that calibration on training environments generalizes to calibration on E. The proof of this theorem given in the supplementary material, Section A.4 The data generating process closely resembles the one considered by 37 who use diagonal covariance matrices.
In the second scenario consider the addition of causal features subject covariate shift x, RR., as shown in Figure Pp. The covariate shift is induced when the environments E' alter the distribution of the causal features x. [40] In this case, we analyze a regression problem since it is amenable to exact analysis. The data generating process for training environment i E [K] is:
X. ~ (N),, Y = w. X. + E~ (O,
apc-sp = 914 + 7, (O
(3)
For x = [Xee Xac-spl it turns out that in this case, calibration on multiple domains forces fx) to discard sps but also forces to use w since characterizes P(Y xx. which is the invariant mechanism in this scenario. The exact statement and proof are in Section A.5 of the supplement.
Theorem 2 (informal). Let f(x: w) w x be linear regressor and assume we have k > max {de + 2, dsp) } training environments where data is generated according to Equation D Under mild non degeeereyyyy conditions, the regressor is calibrated across all training environments then the coefficients corresponding to X, equal w and those that correspond to X xacssp are zero.
Together, these results show calibration can generalize across environments, given that the number of environments is approximately that of the spurious features. They also show that for the settings above, the relatively stable and well-known notion of calibration implies avoiding spurious correlations.
Related Work
As discussed in Section - multi-domain calibration an instance of an invariant representation M. Many extensions the above work have been proposed, e.g. 24 Yet, recent work claims that many of these approaches still fail to find invariant relations in cases of interest 120 37 [3] where a significant challenge seems to be the gap between what is achieved by the regularization term used in practice and the goal of conditional independence Y IL E ((X). Guirajani et al. ||] give sobering view on methods for OOD generalization, emphasizing the power of ERM and data augmentation, and the challenge of model selection. We claim that compared to the above approaches, multi-domain calibration studied here is a simpler form of invariance. Furthermore, calibration attractive because there are standard tools to quantify it such as calibration scores [3 and vast literature on its properties and how it can be obtained 50
2
Learning models which generalize OOD is a fruitful area of research with many recent developments. Most work focuses on the case of Domain Adaptation where unlabeled samples are available from the target domain, including recent work on OOD calibration 48. However, important work has also been done on the area of our focus the so-called "proactive" case 43 where no OOD samples 39].
are available whatsoever 228
Calibration also plays an important role in uncertainty estimation for deep networks [12,, and recently in fairness, where calibration on subgroups of populations is sought (35). This has interesting resemblance to the multiple environments calibration we consider here. A more general notion of
multi-calibration has also been studied in this context [16), with recent results on sample complexity
42 which may provide tools to finite sample analysis of domain generalization. Finally, multiple
methods for training calibrated models (26 29 36 have also been proposed. In Section 5 we propose
generalization of (2 the multi-domain case achieve multi-domain calibration.
5
Proactively Achieving Multi-Domain Calibration
So far we have seen a general argument why calibration can limit spurious correlations, and that in linear-Gaussin models multi- calibration guarantees OOD generalization. Now we turn to a more applied perspective and show how can we optimize models so they achieve this type of calibration in practice. We propose three approaches: (1) using calibration measures for model selection, (2) post-proesssing calibration, and (3) a calibration objective building on a method proposed by [26]. Section A. in the supplementary provides a slightly broader introduction to notions we use here. We will assess model calibration by the Expected Calibration Error (ECE) of the calibration curve 77, which is the average deviation between model accuracy and model confidence.
5

---

5.1 Model selection with average ECE
Model selection challenging when aimed OOD generalization. As recently observed by [- since OOD accuracy is often at odds with In- Do-Dommin (ID) accuracy, selection based on ID validation error eliminates the advantage of domain generalization methods over vanilla ERM with data augmen- tation. We suggest that model selection towards OOD generalization should balance ID validation error with another observable surrogate for the stability of a model to distribution shifts between domains. Motivated by multi-domain calibration, we propose using the average ECE across training environments as this surrogate. Concretely, we propose choosing model with lowest average ECE from those obtaining ID validation accuracy that is above a certain user-defined threshold.
5.2 Post-Processing Calibration
Practitioners interested in (single-domain) calibrated models often apply post-processing calibration methods to binary classifiers, where the most widely used approach is Isotonic Regression Scaling Unlike standard calibration problems, in case there are multiple domains to calibrate over. We give two ways of extending Isotonic Regression to the multi-domain setting, which we term "naive calibration" and "robust calibration". Naive Calibration takes predictions of trained model f on validation data pooled from all domains and fits an isotonic regression x². We then report the performance of 2* o f the OOD test set.
50 50
Robust Calibration: In a multiple domain setting, Naive calibration may produce model that well calibrated on the pooled data. but uncalibrated on individual environments. Since our goal is simultaneous calibration, the following alternative attempts to bound the worst-case mis- calibration across training environments. For each environment e E Fimins we denote the num- ber of validation examples we have from it by Ne. and by Se,; the prediction of the model on the i-th example. Then in a similar vein to robust optimization, we fit an isotonic regressor that
solves: 2" arg min max
N Σ (=(fe,) - v.) Since Isotonic Regression can be formulated as E
quadratic program, and Equation (5.2) minimizes a pointwise maximum over such objectives, can cast Eq. 5.2 as a convex program and solve with standard optimizers. We then evaluate the OOD performance of 2 o f.
5.3 Learning with Multi-Domain Calibration Error
The above model selection and post-processing methods are easy to apply and (as we will soon see) surprisingly effective. However, both are limited in their power to learn a model that is truly well-calibrated across multiple domains. We now propose more powerful approach: an objective function that directly penalizes calibration errors on multiple domains during training. Specif- ically, we propose learning a parameterized classifier fx(x) using a learning rule of the form: ming Eee (fo) + A -rfo), where 1: R x R + R is an empirical loss function (e.g. cross- entropy) and (Jo) denotes the expected loss over data from training environment and r(fo) regularization term over multiple environments. Using this notation the method proposed by w & with regularizer given by r(f) = EeeE TiRMI (f), where
11] learns a classifier
riRMvI() (ʃ) V / 12.
Our proposed regularizer r(fe) is based on the work of Kumar et al. 26 who introduce a method they call Maximum Mean Calibration Error (MMCE). MMCE harnesses the power of universal kernels to express the ECE as an Integral Probability Measure, and works as follows: For a dataset D D= (x; 3/s - denote the confidence of classifier on the i-th example by fo;: max fo(x)) 1, fo(:;)) and its correctness by
1 I For given universal kernel k R x RR - R. MMCE over the dataset D is given by: , -XMMC((f) = m Ei.jED (a - fea)(c, - fe (
fea)- Calibration Loss Over Environments (CLOvE), Given multiple training domains with a dataset De for each Evain: we arrive at our proposed regularizer by aggregating MMCE over them: .De
TCLOvE (fe) Eee Eugin (MMCE(fe) (fe). A key property of CLOvE is that its minima correspond to perfectly calibrated classifiers over all training domains, a consequence of the correspondence between MMCE and perfect calibration.
Corollary (of Thm. 1 in (6)). CLOvE is a proper scoring rule. That is, equals if and only if fo(x) is perfectly calibrated for every Eqrim-

---

Additional properties of CLOvE, such as large deviation bounds and relation to ECE, can also be derived; see results in 266 for further details. In the following section, we will see how these properties translate into favorable OOD generalization in practice when training with CLOVV..
Experiments and Results
6.1 Colored MNIST and Two-Bit Environments
In order to explore the challenges of OOD generalization and how they relate to learning from multiple environments, [ used the colored MNIST dataset (21. In this dataset certain digits tend to be colored either red or green in the train set, but the correlation between colors and digits is flipped in the OOD test set, making color a spurious feature. This dataset was then further simplified into "Two-Bit" environments by [0)] who proved that the IRMv1 penalty proposed in M| does nof in fact achieve the correct invariant solution on the simplified setting. The Two-Bit environments problem setting has two binary features, X1, X2 E (-1,1), 1). corresponding respectively to digit identity (0- - 4 or 5 - 9) and digit color in the original colored MNIST. The environments are parameterized by (a, B) E [0, 1 controlling the correlation of the features with the label:
Y - (.5().5), X1 + Rad(a) X2 + Y Rad(3), where Rad(o) is a random variable given data from two environments 61 = (0,33),.s 62 = (a, 32), 31 * 32. The learned model is tested on a new environment C33 (a, B3) with B3 significantly different from 31, 82. Only model discarding the spurious feature X2 will maintain its accuracy moving from train to OOD test.
equal to -1 with probability 5 and with probability 1 - 5. At training
Calibration discards spurious correlation in Two-Bit environments. Figure Ba), which we adapt from Figure 6 in Appendix B of (20) illustrates the merits of CL.OVE in this setting. The figure shows the space of odd classifiers, i.e. those for which (11--11
f(- - The true invariant classifiers are those for which in addition f), f(1, -1), corresponding to models lying on the diagonal of Figure [a), denoted by the dashed gray line. In the figure, we plot in solid lines the classifiers for which ri Mv (f) equals O, and in solid circles the classifiers for which TMMCF( (f) equals 0 (due Corollary I these coincide with calibrated classifiers on environment e). Note that in this parameterization, the zeros of r 'iRMy1 (f) (f) are lines whereas the zeros of "MCE (f) are isolated points. Intersections of the zeros of riRMv1(.7 (.f) denote solutions for which the corresponding regularization terms are O on all respective environments, while intersection of zeros of r "MME (f) are the zeros of (c). As observed by 20, when Frrain {er, e2) the solution denoted by RP has the lowest empirical loss. yet this solution has a spurious correlation with X₂ and thus will incur a higher loss on the test environment E3. This means the corresponding IRMv1 I learning rule cannot retrieve the optimal invariant classifier. On the other hand, learning with CLOvE does retrieve the optimal invariant classifier in this case, in addition to the trivial constant classifier. This means CLOvE discards spurious correlations in cases where IRMv1 I does not. In Section C we present experiments reproducing the above scenario on the Colored MNIST dataset.
-f(-1, 1), and f(1,
Model selection based on average ECE We train models with varying hyperparameters on Colored MNIST using ERM, CLOvE and IRM, (100 models with each algorithm, see Section C of the supplement for details). We then calculate the ECE and IRMv1 penalties of each model over held-out validation set from each training environment, and evaluate the average of these against OOD accuracy. Figure ()) presents the results across all trained models. The ID) ECE penalty displays a very strong correlation across the entire range and every training regime (Pearson corr. -0.92,, while ID IRMv1 I behaves more erratically (Pearson corr. = -0.59). Since quantities used for model selection should be agnostic to choices made at training time, we suggest that ID ECE is better choice for use in model selection. Further results model selection can be found in the supplement, Section C
6.2 WILDS Benchmarks
WILDS is a recently proposed benchmark of in-the-wild distribution shifts from several data modal- ities and applications] Table I presents the four WILDS datasets experiment with, chosen to represent diverse OOD generalization scenarios. We follow the models and training algorithms
explained 20 the optimal solutions are odd so we may focus on them for visualization purposes.
https://wildd htt.:/iwsssssssf.dd..soa www.soor...deeeoooooo
7

---

06
0.5
Orey
0.4
0C0
0.3
02
Collroted [Rb
0.0
0.00
0.04
10"
10
sggt 11)
Average ECE Training urments
Average FiMM1 Training Emvorments
(a)
(b)
Figure 3: (a) Zeros of MMCE and IRMv1 are indicated by circles and by solid lines respectively, in a color corresponding to each environment. The dashed diagonal is the space of invariant solutions. Some zeros intersect across environments, and these are therefore the domain-invariant solutions. Among the domain-invariant invariant solutions, OPT IRMv1 has the lowest empirical loss when training on é1, E2. Hence learning with IRMv1 will prefer this model over OPT which discards the spurious correlation with X2. (b) Correspondence between observable criteria and OOD accuracy in CMNIST. Each point corresponds to a model trained with some training algorithm (marked by color) and hyperparameter setting. Size of marker proportional the ratio between OOD and ID accuracies.
In order to perform multi domain calibration we modify the splits include multi-domain validation set whenever possible. See supplemental Section B for details and for additional results on Amazon Reviews. As in [23], we use three different training algorithms to train our models: ERM, IRM, DeepCORAL, and further use GroupDRO for one of the datasets, compatible with WILDS version 1.0.0. We apply three calibration approaches described in 552 and [5.3] above to each trained model: naive calibration and robust calibration, which are post-processing methods and therefore applied on the models' outputs; and CLOvE, which we apply as a fine-tuning approach to the top layers of each trained model. We train each (algorithm calibration) combination four times with different random seeds, and report average results and their standard deviations.
proposed by
Dutaset
Type
Label (g)
Input (a)
Domain (e)
Model (f(z))
Map
Regression
Wealth Index
Satellite Image
Country
ResNet
Camelyon17
Binary
Tumur Tissue
Iistopatholooical Image
lospital
DenseNet
CivilComments
Binary
Comment Toxicity
Online Comment
Demographics
BERT
FMoW
Multi-class
Land Use Type
Satellite Image
Region
DenseNet
Table 1: Description of each of the datasets used in our WILDS experiments.
Table presents our main results on the FMoW (left) and Camelyon17 (right) datasets. On both datasets, robust calibration already improves performance, and CLOvE then significantly outperforms robust calibration, improving performance by 7% and 2.8% (absolute) over the strongest alternative on FMoW and Camelyon/7, respectively. When compared to the original model, the performance of CLOvE is even more striking, with CLOVE outperforming it by more than 10% (absolute) on FMoW and 6% on Camelyon17. Another appealing property of CLOvE is the low variance exhibited across different runs. Indeed, CLOvE has lower variance than both naive and robust calibration approaches, and has lower variance than the original (uncalibrated) model on 4 of the 6 experiments.
FMoW
Camelyon17 Naive Cal.
Algorithm
Orig.
Naive Cal.
Rob. Cal.
CLOVE
Orig.
Rob. Cal.
CLOvE
ERM
32.63
33.09
37.19
44.16
66.66
71.23
71.22
75.75
(0.016)
(0,021)
(0.035)
(0.018)
(0.144)
(0.089)
(0.086)
(0.049)
DecpCORAL
31.73
31.75
33.86
40.05
72.44
75.97
76.8
79.96
(0.01)
(0.01)
(0.016)
(0.009)
(0.044)
(0.054)
(0.065)
(0.039)
IRM
31.33
31.81
34.41
42.24
70.87
73.25
73.4
73.95
(0.012)
(0.016)
(0.015)
(0.014)
(0.068)
(0.066)
(0.069)
(0.061)
Table 2: Left: worst unseen region accuracy OOD test set in FMoW. Right: Accuracy on unseen hospital test set in Camelyon17. Orig.: original algorithm, no changes applied. Best OOD result for each domain in bold. Standard deviation across runs in brackets, lowest OOD std. is underlined.
Analysis. As can be seen in Figure improvements in ID calibration are associated with better OOD performance. Interestingly, when our post-processing does not improve OOD performance,
8

---

is often linked to our inability to substantially improve ID calibration. This is most visible in IRM experiments, where robust calibration unable outperform naive calibration both in terms ID calibration and in OOD performance. Finally, we find it interesting that merely post-processing the data (as in robust calibration) can already have such a marked effect on OOD accuracy, though still inferior to actually optimizing for multi-domain calibration as done by CLOvE.
Results on alternative settings. While our theoretical analysis is focused on OOD generalization of classification models, we also experiment with alternative settings from WILDS to test the power of ID calibration in improving OOD performance. Specifically, we experiment with the PovertyMap dataset, which introduces regression task, and the CivilComments dataset, which introduces sub- population shift scenario for binary classifier. As can be seen in Table results on the CivilComments dataset (right), show that calibration consistently improves worst- case performance, with an average improvement of 21.5% across training algorithms. While CLOvE does outperform naive and robust calibration on average, the gain is lower in comparison to FMoW and Camelyon17.
ERM IRM DecPCOSAL
000 Accurey
Racust
Naleu
Driginal
0.023
0.000 0.0ss u.000 0.125 0.100 0.175 0.200
Average ECE ove To ring Envirnments
Figure 4: OOD accuracy as a function of average ECE over training domains, for all models on the Camelyon17 dataset.
In PovertyMap (left), the model solves regression task, so we cannot use CLOvE to improve OOD performance. Still, robust calibration improves performance across all experiments, though by a smaller margin. In the case of models pre-trained by IRM, robust calibration improves OOD performance substantially, outperforming the original model by 0.08% (absolute). Interestingly, calibration also leads to more stable results both in PovertyMap and in CivilComments, as can be seen in the standard deviation across different model runs.
PovertyMap
CivilComments
Algorithm
Orig.
Naiee Cal.
Rob. Cal.
Algorithm
Orig.
Naive Cal.
Rob. Cal.
CLOvE
ERM
0.832
0,827
0.834
ERM
63.65
76.98
78.99
80.39
(0.011)
(0.014)
(0.006)
(0.026)
(0.005)
(0.008)
(0.007)
IRM
0.735
0.812
0.815
IRM
40.61
68.97
68.92
68.45
(0.117)
(0.016)
(0.015)
(0.16)
(0.013)
(0.013)
(0.02)
DeepCORAL.
0.832
0.835
0.837
GroupDRO
71.67
76.2
78.54
80.07
(0.011)
(0.009)
(0.012)
(0.007)
(0.013)
(0.008)
(0.003)
Table 3: Left: Pearson correlation r on in-domain (ID) and OOD (unseen countries) test sets in
PovertyMap. Right: worst-case group accuracy on the test set in the CivilComments dataset.
7 Conclusion
In this paper we highlight novel connection between multi-domain calibration and OOD generaliza- tion, arguing that such calibration can be viewed as an invariant representation. We proved in a linear setting that models calibrated multiple domains are free of spurious correlations and therefore generalize out of domain. We then proposed multi-domain calibration as a practical and measurable surrogate for the OOD performance of a classifier. We demonstrated that actively tuning models to achieve multi-domain calibration significantly improves model performance on unseen test domains, validation set is useful criterion for model selection. A major
and that in-domain calibration limitation of our work is that our theoretical findings are limited to linear models in a population (as opposed to finite-sample) setting; we thus consider them more as motivation rather than full justification of using multi-domain calibration in practice as we do. We look forward to expanding the scope of theoretical understanding of the conditions under which multi-domain calibration can provably guarantee out-of-domain generalization, including the finite-sample setting and the analysis of specific algorithms. We also expect new practical methods, building on our findings, will help push forward the real-world ability to generalize to unseen test domains.
Acknowledgments
We wish to thank Ira Shavitt for his helpful comments and to Alexandre Ramé for pointing to an error in the original manuscript. This research was partially supported by the Israel Science Foundation (grant No. 1950/19).

---

References
[1] M. Arjovsky, L. Bottou, I. Gulrajani, and D. Lopez-Paz. Invariant risk minimization. arXiv preprint arXiv: 02899. 2019.
[2] P. Bandi, O. Geessink, Q. Manson, M. Van Dijk, M. Balkenhol, M. Hermsen, B. E. Bejnordi. B. Lee, K. Pueng, A. Zhong, et al. From detection of individual metastases to classification of lymph node status at the patient level: the camelyon17 challenge. IEEE transactions on medical 2018.
imaging, 38(2):500-533)
[3] A. Bellot and M. van der Schaar. Generalization and invariances the presence of unobserved confounding, arXiv preprint arXhv:2007.10653, 2020.
[4] D. Borkan, L. Dixon, J. Sorensen, N. Thain, and L. Vasserman. Nuanced metrics for measuring unintended bias with real data for text classification. In Companion proceedings of the 2019 world wide web conference, pages 491-500, 2019.
[5] G. W. Brier al. Verification of forecasts expressed in terms of probability. Monthly weather review, 78(1):1-3, 1950.
[6] G. Christie, N. Fendley, J. Wilson, and R. Mukherjee. Functional map of the world. In Proceedings of the IEEE Conference Computer Vision and Pattern Recognition, pages 6172-6180, 2018.
[7] M. H. DeGroot and S. E. Fienberg. The comparison and evaluation of forecasters. Journal of the Royal Statistical Society: Series D (The Statistician), 32(1-2):12-22, 1983.
[8] S. Desai and G. Durrett. Calibration of pre-trained transformers. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 295-302, 2020.
[9] Devlin, M.-W. Chang, K. Lee, and K. Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume I (Long and Short Papers), pages 4171-4186, 2019.
[10] J. Duchi, E. Hazan, and Y. Singer. Adaptive subgradient methods for online learning and stochastic optimization. Journal of machine learning research, 12(7), 2011.
[11] Gulrajani and D. Lopez-Paz. arXiv:2007.01434 2020.
search of lost domain generalization. arXiv preprint
[12] C. Guo, G. Pleiss, Y. Sun, and K. Q. Weinberger. On calibration of modern neural networks. In International Conference on Machine Learning, pages 1321-1330. PMLR, 2017.
[13] R. Guo, Zhang, H. Liu, and E. Kiciman. Out-of-distribttoon prediction with invariant risk minimization: The limitation and an effective fix. arXiv preprint arXiv:2101.0771 2021.
[14] C. Gupta, A. Podkopaev, and A. Ramdas, Distribution-free binary classification: prediction sets, confidence intervals and calibration. Advances in Neural Information Processing Systems, 33, 2020.
[15] K. He, X. Zhang, S. Ren, and J. Sun. Identity mappings in deep residual networks. In European conference on computer vision, pages 630-645. Springer, 2016.
[16] U. Hébert-Johnson, M. Kim, Reingold, and G. Rothblum. Multicalibration: Calibration for the pepputaaioonlll-ddennfifaa pages 1939-1948. PMLR, 2018.
masses. In International Conference on Machine Learning.
[17] C. Heinze-Deml, J. Peters, and N. Meinshausen. Invariant causal prediction for nonlinear models. Journal of Causal Inference, 6(2), 2018.
[18] W. Hu, G. Niu, I. Sato, and M. Sugiyama. Does distributionally robust supervised learning give robust classifiers? International Conference on Machine Learning, pages 2029-2037. PMLR, 2018
10

---

[19] G. Huang, Z. Liu, L. Van Der Maaten, and K. Q. Weinberger. Densely connected convolutional networks. Proceedings of the IEEE conference on computer vision and pattern recognition. pages 4700-4708, 2017.
[20] P. Kamath, A. Tangella, D. J. Sutherland, and N. Srebro. Does invariant risk minimization capture invariance? In AISTATS, 2021.
[21] B. Kim, H. Kim, K. Kim, S. Kim, and Kim. Learning not to learn: Training deep neural networks with biased data. In Proceedings of the IEEE/CVF Conference on Computer Vision 2019.
and Pattern Recognition, pages
[22] D. P. Kingma and Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv: 1412v94412.6980, 2014.
[23] P. W. Koh, S. Sagawa, H. Marklund, S. M. Xie, M. Zhang, A. Balsubramani, W. Hu, M. Ya- sunaga, R. L. Phillips, Beery, et al. Wilds: benchmark of in-the-wild distribution shifts. arXiv preprini arXiv:20/2.07421, 2020.
[24] D. Krueger, E. Caballero, J.-H. Jacobsen, A. Zhang, J. Binas, R. L. Priol, and A. Courville. Out-of-distribution generalization via risk extrapolation (rex). arXiv preprint arXiv:2003.006 00688, 2020.
[25] V. Kuleshov, N. Fenner, and S. Ermon. Accurate uncertainties for deep learning using calibrated regression. In International Conference on Machine Learning, pages 2796-28044 PMLR, 2018.
[26] A. Kumar, S. Sarawagi, and U. Jain. Trainable calibration measures for neural networks from kernel mean embeddings. In International Conference Machine Learning, pages 2805-2814, 2018.
[27] M. Lee. Smooth manifolds. Introduction to Smooth Manifolds, pages 1-31. Springer, 2013.
[28] Magliacane, van Ommen, T. Claassen, Bongers, P. Versteeg, and J. M. Mooij. Domain adaptation by using causal inference to predict invariant conditional distributions. In Proceedings of the 32nd International Conference on Neural Information Processing Systems, pages 10869- 10879, 2018
[29] J. Mukhoti, V. Kulharia, A. Sanyal, S. Golodetz, P. H. Torr, and P. K. Dokania. Calibrating deep neural networks using focal loss. arXiv preprint arXiv:2002. 2020.
[30] A. Niculescu-Minz and R. Caruana. Predicting good probabilities with supervised learning. In Proceedings of the 22nd international conference on Machine learning, pages 625-632, 2005.
[31] J. Nixon, M. W. Dusenberry, L. Zhang, G. Jerfel, and D. Tran. Measuring calibration in deep learning. In IEEE Conference on Computer Vision and Pattern Recognition Workshops, CVPR Workshops 2019, Long Beach, CA, USA, June 16-20, 2019, pages 38-41. Computer Vision Foundation IEEE, 2019.
[32] J. Pearl. A probabilistic calculus actions. In Uncertainty Proceedings 1994, pages 454-462. Elsevier, 1994.
[33] J. Pearl. Causality. Cambridge university press, 2009.
[34] J. Peters, P. Bühlmann, and N. Meinshausen. Causal inference by using invariant prediction: identification and confidence intervals. Journal of the Royal Statistical Society. Series (Statistical Methodology), pages 947-1012, 2016.
[35] G. Pleiss, M. Raghavan, F, Wu, J. Kleinberg, and K. Q. Weinberger. On faimesss and calibration. arXiv preprint arXiv: 700.020099022... 2017.
[36] A. Rahimi, A. Shaban, C.-A. Cheng, R. Hartley and B. Boots. Intra order-preserving functions for calibration of multi-class neural networks. Advances in Neural Information Processing Systems, 33, 2020.
[37] E. Rosenfeld, P. Ravikumar, and A. Risteski. The risks of invariant risk minimization. arXiv preprint a:Xiv:2010.05761, 2020.

---

[38] D. Rothenhäusler, N. Meinshausen, P. Bühlmann, and J. Peters. Anchor regression: heteroge- neous data meets causality. arXiv preprint arXiv: wwt.:0011.0000 2018.
[39] S. Sagawa, P. W. Koh, T. B. Hashimoto, and P. Liang. Distributionally robust neural networks for group shifts: On the importance of regularization for worst-case generalization. arXiv preprint arXiv: 1et./99111...00 1.08731, 2019.
[40] B. Schölkopf. Janzing, J. Peters, E. Sgouritsa, K. Zhang, and J. Mooij. On causal and anticausal learning. arXiv preprint arXiv: (206::/206.677, 2012.
[41] A. W. Senior, R. Evans, J. Jumper, J. Kirkpatrick, L. Sifre, Green, C. Qin, Zidek, A. W. R. Nelson, A. Bridgland, H. Penedones, S. Petersen, K. Simonyan, S. Crossan, P. Kohli, D. T. Jones, Silver, K. Kavukcuoglu, and D. Hassabis. Improved protein structure prediction using potentials from deep learning, Nat., 577(7792)7006-7711 2020.
[42] E. Shabat, Cohen, and Y. Mansour. Sample complexity of uniform convergence for multicali- bration. arXiv preprint arXiv:2005.01757 2020.
[43] A. Subbaswamy, Schulam, and Saria. Preventing failures due to dataset shift: Learning predictive models that transport. In The 22nd International Conference on Artificial Intelligence and Statistics, pages 3118-3127. PMLR, 2019.
[44] B. Sun and K. Saenko. Deep coral:
Correlation alignment for deep domain adaptation. In European conference on computer vision, pages 443-450, Springer, 2016.
[45] J. Vaicenavicius, D. Widmann, C. Andersson, F. Lindsten, J. Roll, and T. Schön. Evaluat- ing model calibration in classification. In The 22nd International Conference on Artificial Intelligence and Statistics, pages 3459-3467. PMLR, 2019.
[46] A. Vaswani, Shazeer, N. Parmar, Uszkoreit, L., Jones, N. Gomez, Kaiser, and I. Polosukhin. Attention is all you need. In Advances in neural information processing systems, pages
2017.
[47] V. Vovk, G. Shafer, and I. Nouretdinov. Self-calibrating probability forecasting. In Proceedings of the 16th International Conference on Neural Information Processing Systems, pages 1133- 1140, 2003.
[48] X. Wang, M. Long. J. Wang, and M. Jordan. Transferable calibration with lower bias and variance in domain adaptation. arXiv preprint arXiv:2007.08255: 2020.
[49] C. Yeh, A. Perez, A. Driscoll, G. Azzari, Z. Tang, D. Lobell, S. Ermon, and M. Burke, Using publicly available satellite imagery and deep learning to understand economic well-being in africa. Nature communications, 11(1):1-11, 2020.
[50] B. Zadrozny and C. Elkan. Obtaining calibrated probability estimates from decision trees and naive bayesian classifiers. In C. Brodley and A. Danyluk, editors, Proceedings of the Eighteenth International Conference Machine Learning (ICML 2001), Williams College, Willianstown, MA, USA, June 28 July 1, 2001, pages 609-616. Morgan Kaufmann, 2001.
[51] J. R. Zech, M. A. Badgeley, M. Liu, A. B. Costa, J. J. Titano, and E. K. Oermann. Variable generalization performance a deep learning model to detect pneumonia chest radiographs: a cross-sectional study. PLoS medicine, 15(11):e 1002683, 2018
Checklist
1. For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope? [Yes]
(b) Did you describe the limitations of your work? [Yes] Limitations that we found most significant are discussed in Section 7
12

---

(c) Did you discuss any potential negative societal impacts of your work? [No] The work does not treat specific task where we see an immediate risk for negative societal impact. Though the methods described in this paper might be used by practitioners in safery-critical domains such as healthcare, and we suggest doing so with care since the suggested methods need to be studied further in controlled settings before being used in such domains.
(d) Have you read the ethics review guidelines and ensured that your paper conforms to them? [Yes] The paper conforms with the provided guidelines.
2. If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results? [Yes] Most assump- tions, such as population setting (infinite data) assumption, Gaussian data and linearity of models have been given in Section E] Other assumptions regarding specific general- position conditions under which the theorems hold are given in the supplementary material. along with detailed statements of the theorems.
(b) Did you include complete proofs of all theoretical results? [Yes] The proofs of main theoretical claims can be found in the supplementary material.
3. If you ran experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main experi- mental results (either in the supplemental material or as a URL)? [No] We are preparing the code for publication and will do our best to have it ready by the end of the review period. Major parts of our code are based on existing publicly available code from papers that we cite, while noting that used their published code. Hence some parts of our results can be reproduced quite easily by using it.
(b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)? [Yes] We specify hyperparameters and training details in the supplementary material (for both the WILDS benchmark and ColoredMNIST), When using training setup from other works (c.g. in Colored MNIST), we give a reference to the work and specify changes
made upon their setup.
(c) Did you report error bars (e.g., with respect the random seed after running experi- ments multiple times)? [Yes]
(d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)? [Yes]
4. If you are using existing assets (c.g., code, data, models) or curating/releasing new assets...
(a) If your work uses existing assets, did you cite the creators? [Yes] Some of the code
use adapted from the works of [1] 200 which we cite several times and specifically in
Section 6 with regards to the used code. Our experiments the WILDS benchmark
23 use assets from the project that was also cited.
(b) Did you mention the license of the assets? [Yes] Licenses are included in the supple- mentary material.
(c) Did you include any new assets either in the supplemental material or as a URL? [No]
(d) Did you discuss whether and how consent was obtained from people whose data you're roe're using/curating? [No] All assets we
allowed for public use for the purposes of
sceintific research.
(e) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content? [No] The data we use is taken from publicly available datasets, where all personally identifiable information have been removed.
5. If you used crowdsourcing or conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, applicable? [N/A] We did not conduct research with human subjects
(b) Did you describe any potential participant risks, with links Institutional Review Board (IRB) approvals, if applicable? [N/A]
(c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation? [N/A]
13


Reviews:
[{"review_id": "iD_pzTLw0bK", "reviewer": "Reviewer_C1X7", "summary": "The authors suggest a different regularizer than IRM based on the gradient  - which has been shown to have failure modes\nThey instead propose calibration and, focusing on classification (but also with some regression experiments),\n(i) show for linear model that under similar conditions as the general position condition for IRM, for k > 2 d_sp a multi-domain calibrated w has zero coefficient on the spurious features\n(ii) use MMCE for nonlinear setting on WILDS dataset that illustrate that their method works. \n\n*Update:* I've carefully read the author's response that has addressed my comments and I still think it's a good paper and should be accepted.\n", "questions": "", "limitations": "", "rating": 7, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strengths\n- Clearly and well written\n- principled derivation for using multi-class calibration (in practice via the MMCE proxy), quick glance at the proof makes sense \n- good experiments demonstrating impressive effectiveness of calibration on WILDs dataset, comparing also with some alternative calibration methods\n\nCritical comments: \n\nIn terms of theory\n- Theorem (that calibration works for sufficiently different training environments) is not surprising given IRM Theorem, uses similar technique -> don’t really see it as a really novel or groundbreaking contribution\n- In contrast, more useful/convincing for their story would have been a theorem that shows where calibration works but IRM does not\n\nIn terms of methodological contribution \n- the numbers in orig column for Camelyon 17 in Table 2 are not matching the WILDS - in the appendix you can find that the problem is in the choice of the training dataset, i.e. you are using 4 rather than 3 training environments. in the original WILDS paper with 3 training environments ERM had best OOD generalization. how does Clove+ERM compare with IRM for example, for different (i.e. smaller) number of training environments? how about other architectures?\n- would be good to add a discussion about computational costs compared to other methods \n\nExperimental details:\n- need some more experimental details on how you use the clove regularizer in practice even though one can guess that you use the same settings as in Kumar et al, Laplace kernel …? it would greatly improve reproducibility (no code was given to us)\n- how have the authors chosen the parameters for measuring ECE and for post-hoc calibration (e.g. number of bins)?\n- what if you don’t use Clove as fine-tuning but e.g. the IRMv1 regularizer on the entire network when trained from scratch? \ngenerally: are there scenarios in which post-hoc calibration fails to perform better than methods that exploit multi-domain information during training (e.g. GroupDRO, DeepCORAL, IRM)? what is characteristic for these failure cases? moreover, when does a decrease in ECE not imply an improvement in OOD accuracy?\n\nMinor comments:\n- important typo odd classifiers: f(1,-1) = - f(-1,1) - the minus was missing\n- in Theorem 1, it would be much clearer to write x \\in R^{d_{sp}} (apart from the fact that the subscript sp was a typo). \n- in appendix d_e = d_sp\n- the label set for binary classification is not consistent across the paper (sometimes it’s {0, 1}, other times {-1, 1})\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "They could have a more thorough discussion about limitations (as proposed in detailed comments). Potential negative societal impact same as out-of-distribution detection in general. ", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "7"}, "extra_text": {}}, {"review_id": "U4WZRAnzNO1", "reviewer": "Reviewer_jGwe", "summary": "This work suggests that for well calibrated models will achieve better performance on OOD data. It is shown empirically that expected calibration error (ECE) is negatively correlated with OOD performance, and various methods of selecting calibrated models are suggested. The best performing method involves regulation to encourage calibration during training.", "questions": "", "limitations": "", "rating": 7, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "This paper is clearly written and explores interesting ideas. The naive and rob. calibration methods make excellent baselines, and the empirical performance gain, in particular for FMoW/Camelyon17, is impressive. Limitations are addressed, in particular that only fine-tuning is explored and that theoretical findings are limited to the linear setting. A few questions:\n\n- Are these models still calibrated out of distribution? This would strengthen the argument.\n- Why does this work only consider OOD performance. Could this be a method to improve the calibration of models during training instead of needing to apply post-processing? Does it result in lower ECE than standard post processing, or could this method negatively impact accuracy?\n- As I am less familiar with these ideas, I would have appreciated additional intuition. I did not find the motivating hospital example particularly obvious/insightful. The following motivation makes more sense to me: If I have two classifiers which have similar accuracy but one is more calibrated, I would rather choose the calibrated classifier as I would assume it is using the right information. However, I am not sure why multi-domain is important for this motivation or method?\n\n-- Added --\n\nI acknowledge the response and will keep my score as is", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Adequately addressed.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "2.5h"}, "extra_text": {}}, {"review_id": "GbiH6TePxbr", "reviewer": "Reviewer_QH4q", "summary": "This paper draws an important connection between out-of-domain generalization and calibrated models. Specifically, it is argued that calibrated models are related to invariant representations w.r.t spuriously correlated features, which is necessary for generalization across multiple domains. The paper proves this statement in the case of a multivariate linear-Gaussian model. The authors empirically find a correlation between the Expected Calibration Error (ECE) and OOD performance on Colored MNNIST. Based on these insights, the authors propose three methods for achieving multi-domain calibration of trained models in practice: Approach #1 suggests doing model selection based on the Expected Calibration Error; Approach #2 is intended to augment the trained model by post-processing calibration methods and Approach #3 introduces a calibration objective at train time. In multiple experiments across 5 datasets, reported results suggest that their method outperforms the baselines algorithms without calibration on OOD test environments. \n", "questions": "", "limitations": "", "rating": 7, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "**Originality:** To the best of my knowledge this work is an original and novel contribution in relating the problem of finding invariant representations for out-of-domain generalization with the field of calibrated models. It offers important insights, both theoretically and practically and proposes approaches that are agnostic to well-known baseline algorithms in OOD generalization. The related work seems to be adequately covered but I am not familiar enough with the entire literature.\n\n**Quality:** This paper seems technically correct and the claims are well supported and motivated by theorems in the case of a multivariate linear gaussian model. The proposed methods appear to be the effective approaches once the link between calibration and OOD generalization has been established. I did not find any technical flaws in the experiments both on Coloured MNIST and the four chosen datasets from the WILDS Benchmark. Analysis and discussion of the experimental results are adequate. The theoretical claims are somewhat limited in that they are constrained to linear models but the authors are upfront about this and the experiments support that calibration still offers benefits beyond linear settings. \n\n**Clarity:** I found most parts of this work clearly written and well organized.\n\nI had some problems parsing section 5.3 and how the CLOvE method works in detail. From reading this part, I assumed this method trains the models from scratch under the proposed CLOvE regularizer but in the experiments you only applied it as a fine-tuning method, which confused me because training from scratch seems to be the natural first choice to test this method. What was the reason for this particular choice of only fine tuning the top layers? \n\nAdditionally, I think the paper could benefit if the assumed graph and problem setting in 2.1 is supported or accompanied with a concrete simple example, as with the hospital example in the introduction. This could help to get some intuition about what could be a spurious or non-spurious feature and why we would like to avoid them.\n\n**Significance:** I can imagine results in this paper are clearly important from a theoretical and practical point of view as this work establishes an interesting link with a seemingly unrelated field. While the proposed method already shows improvements upon existing algorithms this opens up a path for further research w.r.t to theory and improved methods.\n\n----\nUpdate: I want to thank the authors for their response and running the additional experiments. I also read the other reviews and remain very positive about this work and will therefore maintain my score. ", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "There is a very brief discussion on limitations in section 7, which could be discussed more prominently in the respective sections. Societal Impact is addressed properly.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "10"}, "extra_text": {}}, {"review_id": "-sEr6Ssco2d", "reviewer": "Reviewer_5YBQ", "summary": "The paper proposes out-of-domain (OOD) generalization of the machine learning models by calibrating them across multiple domains. The paper argues that multi-domain calibration makes the model learn invariant features and therefore free of spurious correlations. The relationship between calibration and invariance has been clearly stated via lemma. The authors prove how multi-domain calibration makes the model free of spurious correlations in the case of a linear Gaussian model and in two different scenarios. Different ways of practically achieving multi-domain calibration, i.e., during model selection, as post-processing techniques on the trained model, and during training have been proposed. Experimental results on the WILDS OOD benchmark and Colored MNIST dataset illustrate the efficacy of the proposed ways for calibrating a model on multi-domains for OOD generalization.\n", "questions": "", "limitations": "", "rating": 7, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strengths:\n1) All the definitions and lemmas are very well written.\n2) The novel theoretical results on the relation between multi-domain calibration and spurious correlation are a promising step towards OOD generalization.\n3) The paper is not restricted to the theory of why multi-domain calibration works for OOD generalization but also proposes practical ways of achieving it during model selection, as post-processing techniques on the trained model, or during training.\n4) The experimental results are quite rich and good.\n\nWeakness:\n1) The paper is not so easy to follow due to the lack of background or citation(s) for the definitions of causal, anti-causal, spurious features, and isotonic regression, etc. Also, a small paragraph on the mathematical notations used throughout the paper would be helpful. Adding an exact reference to the appendix sections will be helpful to link the paper with the appendix.\n2) The motivation for using the causal graph in Figure 1 of Section 2.1 (problem definition) is missing. Is it the standard causal graph used to describe the relationship between the types of features, environment, and the label? Also, the causal graphs used in Section 3 for proving that multi-domain calibrated models are free of spurious features are different from the one in Figure 1. Why did the authors split the graph used in defining the problem statement into two and then proved the properties for these two sub-graphs? The justification for doing so is missing in the paper. Is it possible to prove the desired properties on the entire graph from Figure 1?\n3) The authors use invariance and free of spurious correlations interchangeably in the paper. A lemma for invariance <=> free of spurious correlations would be good.\n4) Theorem 1 and 2 are proved for binary classifier and regression, respectively. Do both of these theorems hold for multi-class classifier and regression?\n5) The authors mention that CLOvE is a calibration technique applicable during training in Line 247. But in experiments, they use CLOvE as a fine-tuning approach on the models already trained with the existing OOD generalization techniques. This has been acknowledged in conclusion (Line369) but it would be good to evaluate the effectiveness of CLOvE by using it in training models from scratch (not based on other OOD generalization techniques).\n6) There is a disconnect between the theorems (1 and 2) on calibrated models and the proposed ways of calibrating models on multi-domains. Do the proposed way of calibrating models satisfy the conditions required in the theorems?\n\nMinor comments:\n1) All the definitions given in the paper are for a binary classifier. Are they generalizable for a multi-class classifier?\n2) What is w* in line 111?\n3) There is a notation mismatch for R^d_{sp} in lines 152 and 164.\n4) 6 - 9 in line 278 should be 5 - 9.\n5) What is table S1 in the appendix? It is not referred to anywhere.\n6) It should be \"Section 6\" instead of \"Section 7\" in line 847 of the appendix.\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "The motivation for OOD generalization in Line 23 of the introduction states that \"a medical diagnosis system trained on patient data from a few hospitals could fail when deployed in a new hospital.\".  OOD generalization might increase trust in the use of these models in the medical domain. The medical domain is one of the most safety-critical domains. The question to be asked here by these multi-domain calibrated models is: Are they good enough to be trusted in safety-critical domains?\n", "review_text": "", "extra_scores": {"ethical_concerns": "The authors adhere to the ethical standards mentioned in the NeurIPS Ethics Guidelines.\n", "needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "~11.5 hours"}, "extra_text": {}}]

Author rebuttal:

Meta-review: Authors make substantive contributions to the connections between out-of-distribution generalization and calibration. I agree with the reviewers that the contributions are significant and interesting. To make the paper more accessible, I encourage the authors to make the exposition (at least the first few sections) more friendly to readers who don’t have a causal inference background.
Decision: Accept (Poster)