An Information-theoretic Approach to Distribution Shifts
Authors: Marco Federici
Ryota Tomioka
Patrick Forré
Venue: NeurIPS 2021 Poster (2021)
Keywords: Information Theory
Selection Bias
Representation Learning
Distribution Shift
Fair Machine Learning

Abstract: Safely deploying machine learning models to the real world is often a challenging process. For example, models trained with data obtained from a specific geographic location tend to fail when queried with data obtained elsewhere, agents trained in a simulation can struggle to adapt when deployed in the real world or novel environments, and neural networks that are fit to a subset of the population might carry some selection bias into their decision process.
In this work, we describe the problem of data shift from an information-theoretic perspective by (i) identifying and describing the different sources of error, (ii) comparing some of the most promising objectives explored in the recent domain generalization and fair classification literature. From our theoretical analysis and empirical evaluation, we conclude that the model selection procedure needs to be guided by careful considerations regarding the observed data, the factors used for correction, and the structure of the data-generating process.

An Information-theorriic Approach to Distribution Shifts
Marco Federici AMLab University of Amsterdam m. federieecce ceteeriiciiaa....m nl
Ryota Tomioka Microsoft Research Cambridge. UK
ryotommicroooft.ccm
Patrick Forré
AI4Science Lab, AMLab University of Amsterdam
pod.forreeuva.i
Abstract
Safely deploying machine learning models to the real world often challenging process. Models trained with data obtained from specific geographic location tend to fail when queried with data obtained elsewhere, agents trained in simulation struggle to adapt when deployed in the real world novel environments, and neural networks that are fit to a subset of the population might carry some selection bias into their decision process. In this work, describe the problem of data shift from a novel information-theoreti perspective by (i) identifying and describing the different sources of error, (ii) comparing some of the most promising objectives explored in the recent domain generalization and fair classification literature. From our theoretical analysis and empirical evaluation, we conclude that the model selection procedure needs to be guided by careful considerations regarding the observed data, the factors used for correction, and the structure of the data-gen getta-gnerraing process.
Introduction
One of the most common assumptions for machine learning models is that the training and test data are independently and identically sampled (IID) from the same distribution. In practice, this assumption does hold in many practical scenarios (Bengio al. 2020). A machine learning
model trained to recognize land usage from satellite images using pictures from the early 2000s may struggle recognize the style of modern architectures (Christie et al. 2018). data collected limited set of hospitals may not be representative of the variation introduced by the use of different machines or procedures (Zech et 2018 Beede et al. 2020). Other kinds of distribution shifts more subtle and difficult to recognize despite having a noticeable impact on the model's 's predictive performance. Examples include represeepree or over-represented population groups (Popejoy Fullerton) 2016 /Bollmmini & Gebru 2018) or biased annotations collected from crowd-sourcing services (Zhang et al.]|[1100 2017] Xia al. 2020).
Different approaches in literature address these issues by using some external source of knowledge such domain environment annotations (Wang & Deng 2018), protected attributes (Mehrabil et al. 2019) or sub-population groups (Santurkar et al. 2020) to reduce bias and minimize the model error outside of the training distribution. Despite the progress in the field of domain generalization literature, Gulrajani & Lopez-Pazi (2020) has shown that the effectiveness of some of the most common algorithms heavily relies on the syper-parameter tuning strategy, revealing limitations of
35th Conference on Neural Information Processing Systems (NcurIPS 2021).

---

examined models when compared to a more traditional empirical risk minimization strategy. One of the major issues behind this observed behavior is a lack of clarity and applicability for the underlying assumptions regarding the problem statement and the data-generating process (Zhao et al. 2019 Rosenfeld al,, 2020; Mahajan al. 2020).
With this work, we aim to present a new perspective on the problem to analyze and clarify the fundamental differences between some of the most common approaches by:
1. Introducing a novel information-theoretinn framework to describe the problem of distribu- tion shift and connecting it the test error and its components (section (2).
2. Analyzing four main families of objectives and describing some of their guarantees and assumptions (section 3
3. Demonstrating that the effectiveness of different criteria is determined by the structure of the underlying data-gen setta-enerating process (section 4
4. Showing that the results obtained by popular models designed according to the aforemen- tioned criteria can drastically differ from the theoretically expected performance (section 4)
The analysis the boutlenec,, independence, sufficiency, and separation criteria reveals that some of the most popular models can systematically fail to reduce the test error even for simple datasets. No unique objective is simultaneously optimal for every problem, but additional knowledge about the selection procedure and better approximations can help to mitigate the bias.
1.1 Problem Statement
Consider x and y as the features and targets respectively with joint density p(x. y) for the predictive problem of interest. Let t be binary random variable representing which data is selected for training (t (t= 1) and which is not (t (t= 0). We will refer to p(x, y/t = 1) as the joint Training distribution and p(x 0) as the Test distribution that are induced by the selection t. Throughout the paper, we will consider selections that can be expressed as function of the features x, targets y, independent noise and other variables c, which represent other factors that are affecting the data collection procedure (e.g. geographic location, time intervals, population groups).
Let g(y| (x) represent a learnable model for the predictive distribution p(y[x). We use the Kullback- Leibler divergence express the Train and Test error respectively[i
DKL (e (yyix) = (KI /o///xxtt t= (oaaat=1]]llaa 1)llx())x)),
Train envor
DEL PRL(PyRR (1y)2lyy)x) PRu(plylxt== 0)]a())))))
Test error
Although only the train distribution can be accessed at training time, we are interested learning models g(y (x) that result in small test error.
1.2 Characterizing Distribution Shift
As first step. we characterize the difference between training and test distribution function of the selection. The effect that the selection has in the joint distribution p(x, y) can be quantified by considering the mutual information between the features-target pair xy and the selection variable t, divergence: I(xy; DKL (p(x,(x,) y //t)//(x..)) y)).
which can be also expressed as a Kullback- This measure of distribution shift intuitively represents how many bits the selection variable carries about the joint distribution of targets and features or, equivalently, how much the joint density y)x has changed as a result of the selection.
Using the chain rule of mutual information, one can express distribution shift as the sum of two separate components:
t(xy;t)
t(x:t)
tl;:t
(3)
Distribution shift
Covariate shift
Concept shift
Further details regarding the convention used for conditional KL-divergence be found in appendix A

---

high
Cholesteral [v]
X1
pain no pain p(y|<)
mid
@lylx)
001k)
X2
(b)
low
mid
high
Blood Pressure [X]
Figure 1: (a) Example of distribution shift due to selection bias (chest pain complaints) for the prediction of cholesterol levels (y-axis) given blood pressure (x-axis). A maximum likelihood approach (green dashed line) results in a model that over-estimates for any given x when compared to the best model in the class (red dashed line). (b) Example of a data generating process in which the features are composed of two parts X1 and X2. When both feature components and X2 are observed, concept shift can be strictly positive. Discarding information about X2 will reduce the effect of the selection bias, while removing X1 might increase it ((y;t(x)) S ((y;t(xxx2) S I [y;
which can be interpreted as the amount of covariate shift (Shimodaira 2000) and concept shift (Widmer & Kubat, 1996 Moreno- Torres al,) (012), respectively. These two quantities refer the changes in the predictive distribution p(y x) and the marginal features distribution p(x) respectively, which add up to represent the changes the joint distribution.
Whenever the data selection is perfectly IID, the selection variable can be expressed as a function of some independent noise f()) and the corresponding distribution shift / [xxy; t) is zero. On the other hand, if the data collection procedure has been influenced by other factors, we do not have such a guarantee, even when the selection does not depend on features and targets directly f(e, c)).
2 Information-theoreti Framework
Using the quantities described in the previous sections, we can show that the sum train and test error lower-bounded by the amount concept shift:
Proposition For any model q(y(x) and a min(f(t 0), =)}:
1
DKL (1y + DKL ( [ypp))
2 [y; t/x),
As a consequence, whenever the selection induces concept shift [(y; > 0, any sufficiently flexible model f(y|x) trained using a maximum likelihood approach must incur in strictly positive test enror. Intuitively, whenever the selection induces a change in the predictive distribution (p(y|x) + p(y x, t = 1)) fitting the model the training distribution will incorporate the selection bias into the model prediction. necessarily resulting in errors when evaluated on the test distribution, shown in the example reported figure Ia
The different approaches analyzed in this work are based in the introduction of a latent representation z, which allows for the definition of different regularization strategies. Using the latent representation as an intermediate variable, the model g(y(x) can be re- parametrized with an encoder g x) and latent classifie gg)l2))
q(y|x) = Ezwg(zlx) [g(y|z lz = =)]
(5)
The effect of the introduction of a latent representation can be observed by expressing the training and test error function of the encoder and the classifie
2 The name classifier will used for convenience to express both classification and regression problems.
The expressions in equations 0 and hold with equality for deterministic encoders shown in appendix
3

---

Proposition 2. For any encoder q(z(x) and classifier q(ylz):
DKL(Pyje 1l S
1 y/z)
+ DEL (pyjal Poyjlllyy lyyl))
(6)
Train information loss
Latent train ennor
DKL (exPPiyylly S
ox: (Z)
+ DKL (yyle))))))
information lows
Latent test ervor
The two terms 4=11 (x: y/z) and Ie=o(x;) y/z) represent the amount of predictive information that is lost by encoding the features x into z on train and test distribution respectively, while DKL (P and (r
(P [|qylz) 2) refer to the train and test error when using instead of the original observations as the predictive features, which will be referred respectively. Test information loss and latent test error capture two intrinsically different kinds of error. The former indicates the increase in the prediction uncertainty as result of the encoding procedure, while the latter represents the discrepancy between the model q (z) and the latent test predictive distribution p(y iz, t 0).
latent training error and latent test error
2.1 Latent test error
Minimizing the latent test error makes the latent predictor ((ylz) approach the test latent predictive
distribution p(y 2,, 0). We show that the Gensen-Shannon divergence between the two distributions is upper-bounded by monotonic function of the latent training error and the amount of concept shift in the latent space (latent concept shift ((z))
Proposition 3. For any q(2)x), g(ylz) and any representation that satisfies p(z =2|t= = 0) 0 and p(z = 2 1) 0:
( 2a
1(y:t/z= = 2) 2)+ V DKL (لل===|l0))== > D P
2
(8)
Whenever the Jensen-Shannon divergence between the test predictive distribution p(y|z, = 0) and the classifier g(y 22) is small, the latent test error (measured in terms of Kl--divergence) must also be small least the regions that have positive probability according to both train p(x 1) and test p(x t 0) data distributions. Since the train predictive distribution p(y|x, 1) is not defined for æ that have zero probability on the train distribution, we have no guarantees regarding the model predictions in those regions unless other inductive biases are considered.
The left hand side of the expression in proposition can be minimized by addressing I(y; t/z) and DKL (p)77) [gyxx) with respect to the encoder and classifier respectively. In other words, we can minimize the latent
error by simultancously fitting g(ylz) to the train predictive distribution ply z, t 1) and minimizing the latent concept shift induced by the encoder (z) When the latent concept shift and latent error approach zero, the latent train predictive distribution p(y|z, 1) approaches the true (unselected) predictive distribution p(y|z). and so does the modeled classifier q(y(z). In this ideal scenario, the only source of test error the additional uncertainty that due to the information lost in the encoding procedure.
2.2 Minimizing the information loss
Since losing information generally results in increased test error (equation 7 will consider objectives that discard the minimal amount of information required to reduce the latent concept shift. This can be done by minimizing the KL-divergence between the training predictive distribution p(y x; = 1) and the latent classifier model g(ylz):
min T (x; = min (xxxx)
DKL (v) [ KT - (1411))
g(xl))))(((z)
min ((xl))
DKL (p (1)1)))))
In addition to reducing the amount of information lost in the encoding procedure, minimizing the right hand side of the expression in equation 9
makes the model q(ylz) approach the latent predictive

---

distribution p(y|z, t 1). Note that only the selected training distribution p(y, = 1) is available at training time. practice, minimizing the train information loss also generally results in decreased test information loss since the same features are usually informative for both train and test distributions.
2.3 A General Loss function
To summarize, the overall objective consists in finding the maximally informative representation that minimizes latent concept shift so that the same learned predictor ((y(z) can perform similarly on both train and test settings. This is achieved by (i) minimizing the amount of latent concept shift induced by the representation. (ii) maximizing the amount of predictive information in the representation, and (iii) matching ((ylz) and the training latent predictive distribution p(y t 1). The three requirements can be enforced by considering a loss function with the following form:
C( Gylzi X) = DKL (P (pyT)alaa))))) + XR(ga)x).
(10)
in which, the first term addresses (ii) and (iii), while the second term represent regularization term that acts on the encoder ((((x) to minimize latent concept shift t/y; following (i). For sufficiently flexible family of latent predictors, requirement (iii) depends only g(y/z), while the paper--rrameter λ defines the trade-off between latent concept shift (i) and predictive information loss (ii).
Regularization Criteria
Since only selected data (t = 1) is accessible at training time, the latent concept shift can not be computed minimized directly. Most of the approaches considered this analysis make use of regularization R((xx)) that is based on the observation of an additional variable e which relates to the selection criteria. This variable is usually referred to as domain or environment in the domain adaptation and generalization literature, while the name protected attribute is used in the context of fair classification. We will refer to this variable
environmental factors factors in the following sections.
We analyze four families of criteria proposed in the representation learning (Tishby & Zaslavsky 2013), domain generalization (Koyama & Yamaguchi 2020) and fair classification (Barocas et al. 2018) literature focusing on their underlying assumptions and theoretical guarantees. The different regularization strategies and models can be seen specific instance of the loss function in equation I0 An empirical comparison between instances of the different criteria can found in section 4 proots are reported in appendix B while the relation between the reported criteria is further discussed in appendix B.9
3.1 Information Bottleneck Criterion
Combining the results from propositions and 2 we can infer that reducing the latent test error necessarily requires the representation to discard some train predictive information (4=1 (x; y/))>0). This is because a lossless encoder (zz(x) together with an optimal latent classifier q(y|z (z) would result in overall model x(y|x) that matches the train distribution p(y|x, 1), and, therefore, results in positive latent test error (proposition) D.
The Information Bottleneck criterion (Tishby & Zaslavsky 2015) introduces a regulariza- tion term R(gxx) It= (x; z) to control the amount of information in the representation, defining lossy compression scheme (Alemi et al. 2017) that can be regulated using the hyper-parameter X. Note that this criterion does use any environmental information and blindly discards data-features depending on the regularization strength X. As shown in the example reported in figure lb] discarding information without any additional constraint can increase the amount of latent concept shift depending on the structure of the underlying data generating process. This is because discarding information a necessary but not sufficient condition to reduce the latent concept shift.
In contrast with the domain adaptation and generalization literature, we will consider the more general case which e represented by vector.
5

---

3.2 Independence Criterion
Whenever the data selection depends only on some observed variable (t= c(e;))), the most intuitive approach to reduce the latent test error is to make the representation independent of the environmental factors e. This can done by minimizing mutual information between e and z: R((xxx)= == (e= z). This criterion, known independence or statistical parity in the fair classifi- cation literature (Dwork et al. [011] Corbett- Davies et al. 2017). J aims to remove any environmental information from z, resulting in representation that satisfies ( 1) p(z e, t 1).
Despite the usefulness of this criterion in the fairness and differential privacy literature, enforeing independence does not necessarily reduce the test error (Zhao et al., 2019, Johansson et al. 2019). This is because a consistent marginal across different environments does not imply consistent predictive distribution (p(y)e zz. e, = 1) * ply zz, = 1)), and enforcing independence may even increase the latent concept shift and the test error (as shown in figure
3.3 Sufficiency Criterion
Instead of enforcing property the marginal feature distribution, one consider stable properties of the joint distribution of features z and labels y. The requirement of creating a representation that yields a stable classifier for different values of the environmental factor can be captured by the following regularization: ((xxx) == I-=1 (y; e/z). Intuitively minimizing 4 (y: e elz) corresponds to minimizing the distance between the predictive distribution p(y e, z, t = 1) for each one of the observed environmental conditions. We can show the following:
Proposition 4. Whenever the selection can be expressed as function of e and some independent noise e, the latent concept shift induced by a representation z of x is upper-bounded by I(y; e|z):
Af: t= [(e,e) = T(y; S ely:))z).
(11)
In other words, for given selection it is possible to remove the effect of concept shift by enforcing the sufficiency constraint using the variable (or variables) that are responsible for that selection.
The result from proposition iis applicable only if the two following conditions are met: (i) sufficient representation must exist (Koyama & Yamaguchil 2020); (ii) enforcing sufficiency on the selected train distribution results in sufficiency for the overall joint distribution (4=1 (y; e|z) 0
ely:) = 0) (Rosenfeld et al., 2020). Even if assumptions (i) and (ii) are often acceptable in practice, lack of environment variety training time direct dependencies between environmental factors and targets (such as in the y-CMNIST dataset in section 4.2) can compromise the effectiveness of the sufficiency criterion. An in depth discussion with simple example is reported in appendix G
3.4 Separation Criterion
The last family of objectives and design principles includes approaches that aim to capture the stability in the latent feature distribution when the target is observed across different environmental conditions p(zly) p(zly, y, e) (Chouldechova 2017 Li et al. 2018b). This requirement can be enforced by minimizing the dependency between environmental factors and the representation when the label is observed: R((x)x) := I (e; z|y). The resulting separation criterion (since y separates and e) can be used to identify stable properties of the joint distributions even when the selection depends on both targets y and environmental factors e:
Proposition 5. If the selection t can be expressed as a function ofe, and some independent noise e, the latent concept shift of a representation z ofx x and T [e: y):
by the sum of prior-shift I [ t)
3f t= f(e.y,c) = I(y; S [(y I(e;zly).
(12)
When selection and targets y are marginally independent, proposition guarantees concept shift of representation that enforees separation is zero. Furthermore, whenever the marginal distribution p(ylt= = 0) is known, it is possible to adjust the prediction of the latent classifier on the test d
gggaaatton that the latent
Further details the re-weighting procedure can be found in appendix B.8

---

Latent information
CMNIST
d-CMNIST
y CMNIST
1.75
CMNIST
d-CMNIST
V-CMNIST
y
1.50
Test error
1:25
1.00
d
d
0.25
0.00
Piduro
Cobor
Dot
Indkemndents sumfttnoy Separstion
Picture
Coler
rigt
mospanddntt Aitreerry tepwratio
Priit
Colr
Digt
Independonns Sultiaenty
Figure Graphical models (a) and error components (b) for the CMNIST, d-CMNIST and y-CMNIST data distributions. (a) Dashed lines are used to underline marginal independence between color and environment e, while red arrows denote dependencies added to the original CMNIST distribution. (b) Models trained with strong regularization (A ~ 107) ) for the different criteria are compared against the classifiers trained using only color, digit, picture, or prior information. The colors show the proportion of the test error (in nats) due to the predictive information loss (1==0(x; y/z), in orange) and the latent test error (DKL (p)72 (ylu), in blue) according to the decomposition in equation 7
Similarly to the other criteria, one needs to assume that enforcing separation on train (7== z = 0) suffices to guarantee /( [(e; z Although a representation that enforces separation always exists, the effectiveness of the separation criteria depends on the data-generating process, since the requirement could be exclusively satisfied by a constant representation.
4 Experiments
We evaluate the effectiveness of the criteria presented in section 3 and some of their most popular implementations multiple versions of the CMNIST dataset (Arjovsky et al. 2019) produced by altering the data-generating process (figure 2ah to underline the shortcomings of the different methods.
The main advantage of using a synthetic dataset such as CMNIST lies in the possibility to directly optimize for the different criteria using differentiable discrete mutual information measurements. This is possible since the joint occurrence of color and digit x := [c, d] is a low-dimensional discrete sufficient statistic of the pictures x, and, consequently, all the mutual information quantities of interest involving x E [0, 1] ] 28xxxx x2 can be computed using x E {0. 1} x (o.....9) 99 instead. Further details regarding the direct optimization of the criteria are reported in appendix
In figure 3 the theoretical performance for each criterion is compared against the results obtained by training different models on the pictures x using neural network architectures parametrize the encoder x and classifier g(y (z) for different regularization strength A. For the comparison, we consider diverse popular models designed according to the criteria defined in section
Variational Information Bottleneck (VIB) (Alemi et al. 2017 : a variational tractable approximation of the Information Bottleneck criterion;
Domain Adversarial Neural Network (DANN) (Ganin et al. 2016): an adversarial model
based on min-max game with discriminator d(elz) that is optimized to predict environment information from the representation z to enforce the Independence criterion:
Invariant Risk Minimization (IRM) (Arjovsky al. 2019): A model designed following the Sufficiency criterion that aims to create a representation from which the same latent classifier is simultaneously optimal on all environments.
Conditional Domain Adversarial Neural Network (CDANN) al.ttal 2018b): an adversarial approximation of the Separation criterion in which, analogously to DANN, a discriminator tries to predict the environment when the representation and the true label y are given.
7

---

CMNIST
d-CMNIST
y- Y-CMNIST
picture
13-
1.
Cross-entroes entropy
14
1.6
14.
Bottlonack
1.4
1.4
Sufficlency
12-
1.-
1.0
1.0
1.8
1.8
1.8
DANN
entropy
144
1.5
IRM
CDAMN
1.4
1.4
1.4
12-
1.0
1.0
Train Cross-entropy
Train Cross-entropy
Train Cross-entropy
Figure 3: Comparison of training cross-entropy (x-axis) and test cross-entropy (y-axis) on the CMNIST, d-CMNIST and y-CMNIST datasets for representations trained using different criteria directly (top row) and corresponding approximations/modd. the trade-off between the two errors for wide range of regularization strength values X. The white symbols in each plot represent the error of models that consider only color (p(ylc, = 1)), digit (p(yld, 1)), picture (p(y)x, x, 1)) prior (p(y't = 1)) information. The shaded area represents the standard deviation observed across three runs with different seeds.
(bottom row). Each trajectory describes
Variance-based Risk Extrapolation (VREx) KKueger et al. 2020): model designed follow- ing the Sufficiency and Independence criteria, based on the minimization of the training eтror variance across the different environments;
For better comparison, all the models the same encoder and classifier neural network architec- tures. Each model has been trained by slowly increasing the regularization strength after an initial pre-training with small A. Further details regarding the neural network architectures, objectives, optimization and specific hyper-parameters can be found in appendix E
4.1 Evaluation metric
The measure of test error, information loss and latent test error reported in figure 2h can be computed only for discrete variables. For neural network models the training and test error defined in section can be estimated up to constant entropy by considering the expected negative lelliiel (empirical cross-entropy):
DKL (PyTx Mylx) ~
log g(y = vilx = a.;) (y)()
(13)
N
1=1
constant
The samples x,, yt are obtained from p(x, y/t respectively. By computing the training and test cross-entropy values for different regularization strength X, each model defines a trajectory from a maximum likelihood solution (A = 0) to the results obtained when the corresponding independence constraint is enforced (large X), as shown in figure Contrarily to accuracy measurements, the use cross-entropy allows us to detect under-confident or over-confident predictive distributions.
y/t = 1) and p(x, y/t = 0) for train and test cross-entropy
'The implementation of the models in this work is available athhtps: //t:::/ggttttttt// http:tubb..coo//teeeerccc/aaaaai
8

---

4.2 Datasets
The two variants of the CMNIST dataset considered in this work have been designed by minimally changing the original distribution to underline the strengths and weaknesses of the different criteria. Across different experiments, each MNIST picture x of a digit d is associated with a color e that The d-CMNIST and y-CMNIST datasets
depends on binary target and an environment are created by adding a dependency from the environment to digit (d-CMNIST) and label y (y- CMNIST), which results in correlation between targets and environment yT( > 0). Further details regarding the conditional distributions used to produce the different datasets can be found in appendix C
The strong correlation between color and label across the different CMNIST versions can be seen as an artifact introduced by the selection t = f(e), and models that consider only digit information (white triangles in figure 3 outperform the ones that capture color information (white crosses) or both (white circle) in terms of test cross-entropy (y-axis).
CMNIST The CMNIST dataset was originally designed to underline the weaknesses of maximum- likelihood and Empirical Risk Minimization strategies (Arjovsky et al. (2019). The results displayed in the first column of figure Jouree]cotffe confirm that both the sufficiency and separation criteria manage to effectively reduce the test cross entropy error for sufficiently strong regularization A, minimizing the latent test error while retaining more predictive information than a constant representation (first column in figure 2b.. On the CMNIST dataset, most of the models in analysis closely follow the trajectory estimated by optimizing the corresponding criterion directly. The trajectory defined by the VIB model differs from the one described by the Information Bottleneck criterion since, in absence of additional constraints, the nature of the information that discarded (cither color, digit or style) depends the inductive bias introduced by the specific architecture.
d-CMNIST The d-CMNIST dataset adds a dependency between environment e and digit d. increas- ing the frequency of specific digits for some environments. Digit information makes environment and label conditionally independent (y and e are d-separated in figure 2a). As claimed in AArjvssy et al. 2019), models based on the sufficiency criterion manage reduce the latent test error (second column of figure while the separation criterion fails because of the direct dependency between environment and digits. The independence criterion does not improve the test performance since both color and digit correlate with the environment. Both DANN and CDANN architectures results in trajectories that are similar to the ones obtained optimizing for the corresponding criteria. Note that enforcing separation or independence does not improve upon the result that can be obtained by blindly discarding information using VIB. Despite the effectiveness of the sufficiency criterion, the model trained with the IRM objective lies far from the optimal solution. We believe that this is due to the relaxations and approximation introduced by the optimized objective as discussed in appendix F
y-CMNIST Adding a dependency between environment and label y simulates the scenario in which some labels are more frequent in some environments. The arrow between and y flips when compared to the d-CMNIST dataset to represent stable p(dly) across different environments. The corresponding Y-CMNIST dataset includes a path from to y that can not be blocked since no representation can achieve sufficiency (I(e; y|zz criterion (red line, third column of figure 3). the model trained with the CDANN objective struggles to minimize the test error due to instabilities of the ad adversaiial training procedure, discussed in appendix Once again, the trajectory described by the VREx objective more favorable when compared to VIB, while the model trained using the IRM objective is far from optimality. Figure 2b confirms that, on the y-CMNIST dataset, the separation criterion is the only one that minimizes the latent test error without discarding the entirety of the picture information. This underlines that the effectiveness of sufficiency and separation criteria strongly depends on the structure of the underlying graphical model.
0). Despite the optimality of the separation
5 Related work
The problem of distribution (or dataset) shift Canonelo-CCanela et al. 2009 Koh et al. 2020 has been explored in different areas in the machine learning literature ranging domain adaptation and generalization (Wang& & Deng 2018) to sub-population shift and fair classification (Mchrabi et al.

---

2019; Santurkar et al. 2020). Although goals. availability of test features and environments at training time, and data selection criteria (Koh et al., (020) may differ, most recent approaches focus on extracting features with some desired properties through three different families of objectives.
The independence criterion has been used to find the transformation which minimizes the distance between the distribution of the encoded features across different environments by using adversarial
training (Ganin et al. 2016; Xie et al. 2017 Li et 2018a), features adjustment (Lum & Johndrow
2016). kernel methods (Muandet et al. 2013), or variational approaches Louizos et al. 200 Moyer
al. 2018 lsse et al.
2020). Despite the success of the independence criterion, Zhao et (0019);
Johansson et al. (2019) have shown that enforcing stability of the marginal feature distribution is not
sufficient to guarantee generalization.
(20188) extends the independence criterion using adversarial training to create representations that are independent of the environment when the label is observed. The corresponding separation criterion has been explored in the context of fair classification as equalized odds (Hardt et al. 2016)
and some tractable relaxations (Darlington 1971 Woodworth et al. 2017 Chouldechoval 2017))
The idea of considering features that lead to a consistent predictive distribution has been explored in
the causality literature (Peters et al. 2016; Magliacane al. 2018)) for linear models as feature
selection criteria. Other work approached the problem in the non- non-linear case by considering gradients
of the classifier (Arjovsky al. [2019] Koyama & Yamaguchil 2020
Parascandolo al. 2020) or
penalizing the variance of error (Krueger et al. 2020] Xie et al. 2020) across different environments.
Other relaxations of the sufficiency criterion such as predictive parity (Chouldechova 2017) have
been considered in the fairness literature.
Despite the promising directions of research, Gulrajani & Lapez-Paz 2020) have shown that the
performance the most popular approaches in the literature strongly depends on the hyper-parameter tuning strategy, underlying the problem of the lack of (i) standardized benchmark procedures and (ii) clarity regarding guarantees and hidden assumptions for the different algorithms. Although the problem of distribution shift has been widely studied in the classic domain generalization literature (Mansour et al., 2009] Schölkopf et al., [012]] Gong et al.) (2016), this work aims to characterize the problem from representation learning and information theoretical perspective, providing different insights and identifying potential issues and benefits of several popular models.
6 Discussion and Conclusion
In this work, we characterize an information-theoreds framework to analyze the distribution shift problem by relating it to the train and out of distribution test error. We identify two sources of errors for models based on latent representations, and we show that different criteria explored in literature can be seen as different strategies to minimize concept shift in the latent space using extra information about the data selection procedure. We demonstrate both theoretically and empirically that their effectiveness depends on the structure of the underlying graphical model with respect the observed variables, training data, and data selection criteria other than the chosen approximation and relaxations for optimization.
Although test information loss and latent test error are challenging to estimate for real-world datasets, argue that the presented analysis selection bias and distribution shift in machine learning models. An accurate estimation of the error decomposition reported in this work could be further used to guide processes of causal discovery.
be useful to better understand and mitigate the effect of
Acknowledgments and Disclosure of Funding
We thank Sindy Löwe and David Ruhe for their insightful comments and feedback. This work was supported by the Microsoft Research PhD Scholarship Programme.
References
Alemi, A. A., Fischer, I., Dillon, J. V., and Murphy, K. Deep variational information bottleneck. In 5th International Conference on Learning Representations, ICLR 2017, Toulon, France, April 24-26, 2017, Conference Track Proceedings. OpenReview.net, 2017.
10

---

Arjovsky, M., Bottou, L., Gulrajani, L,, and Lopez-Paz, D. Invariant risk minimization. CoRR, htss/997.00289. 2019.
Barocas, S., Hardt, M., and Narayanan, A. Fairness and Machine Learning. fairmlbook.org, 2018.
http:/ /twt:://www... fairmlbook. org
Beede, E., Baylor. E., Hersch, F., Iurchenko, A., Wilcox, L., Ruamviboonsuk, P., and Vardoulakis,
L. M. A human-centered evaluation of a deep learning system deployed in clinics for the detection of diabetic retinopathy. In Bernhaupt, R., Mueller, F., Verweij. D., Andres, J., McGrenere, J., Cockburn, A., Avellino, I., Goguey, A., Bjøn, P., Zhao, S., Samson, B. P., and Kocielnik, R. (eds.), CHI '20: CHI Conference on Human Factors Computing Systems, Honolulu, HI, USA, April 25-30, 2020, pp. 1-12. ACM, 2020.
Bengio, Y., Deleu, T., Rahaman, N., Ke, N. R., Lachapelle, S., Bilaniuk, O., Goyal, A., and Pal, J. A meta-transte objective for learning to disentangle causal mechanisms. In 8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020. OpenReview.net, 2020.
Binette, O. A note on reverse pinsker inequalities. IEEE Trans. Inf. Theory, 2019.
Buolamwini, J. and Gebru, T. Gender shades: Intersectional accuracy disparities in commercial gender classification. In Friedler. S. A. and Wilson, C. (eds.), Conference Fairness, Accountability and Transparency, FAT 2018, 23-24 February 2018, New York, NY, USA, volume 81 of Proceedings of Machine Learning Research, pp. 77-91. PMLR, 2018.
Chouldechova, A. Fair prediction with disparate impact: A study of bias in recidivism prediction instruments. Big Data, 5(2):153-163, 2017.
Christie, G., Fendley, N., Wilson, J., and Mukherjee, R. Functional map of the world. In 2018 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2018, Salt Lake City, UT, USA, June 18-22, 2018, pp. 6172 6180. IEEE Computer Society, 2018.
Corbett-Davies, S., Pierson, E., Feller, A., Goel, S., and Huq, A. Algorithmic decision making and the cost of fairness. In Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Halifax, NS. Canada, August 13 17, 2017, pp. 797-806. ACM, 2017.
Darlington, R. B. Another look at "cultural fairness". Journal of Educational Measurement, 8:71-82, 1971. ISSN 00220655, 17453984.
Dwork, C., Hardt, M., Pitassi, T., Reingold, O., and Zemel, S. Fairness through awareness. CoRR. abs/1 104.3913, 2011.
Endres, D. M. and Schindelin, J. E. A new metric for probability distributions. IEEE Trans. Inf. 2003.
Theory,
Ganin, Y., Ustinova, E.,, Ajakan, H., Germain, P., Larochelle, H., Laviolette, F., Marchand, M., and Lempitsky, V. S. Domain-adversarial training of neural networks. J. Mach. Learn, Res., 17: 59:1-59:35, 2016.
Gilardoni, G. L. On pinsker's and vajda's type inequalities for csiszár's f-divergences. IEEE Trans. Inf. Theory, 2010.
Gong, M., Zhang, K., Liu, T., Tao, D., Glymour, C., and Schölkopf, B. Domain adaptation with conditional transferable components. In Balcan, M. and Weinberger, K. Q. (eds.), Proceedings of the 33nd International Conference on Machine Learning, ICML 2016, New York City, NY, USA, June 19-24, 2016, volume 48 of JMLR Workshop and Conference Proceedings, pp. 2839-2848. JMLR. org, 2016.
Gulrajani, and Lopez-Paz, In search of lost domain generalization. CoRR, abs/2001 0144444 2020.

---

Hardt, M., Price, E.,, and Srebro, N. Equality of opportunity in supervised learning, In Lee, D. D., Sugiyama, M., von Luxburg, U., Guyon, I., and Garnett, R. (eds.), Advances in Neural Information Processing Systems 29: Annual Conference on Neural Information Processing Systems 2016,
December 5-10, 2016, Barcelona, Spain, pp. 3315-3323, 2016.
Huszár, F. Invariant risk minimization: An information theoretic view, 2019. URL https: / www inf Inference... vc/ wawariant--oo
Ilse, M.. Tomezak, J. M., Louizos, C., and Welling, M. DIVA: domain invariant variational autoen- coders. In Arbel, T., Ayed, I. B., de Bruijne, M., Descoteaux, M., Lombaert, H., and Pal. (eds.), International Conference on Medical Imaging with Deep Learning, MIDL 2020, 6-8 July 2020, Montréal, QC, Canada, volume 121 of Proceedings of Machine Learning Research, pp. -228448. PMLR, 2020.
Johansson, F. D., Sontag, D. A., and Ranganath, Support and invertibility in domain- invariant representations. In Chaudhuri, K. and Sugiyama, M. (eds.), The 22nd International Conference on Artificial Intelligence and Statistics, AISTATS 2019, 16- 18 April 2019, Naha Okinawa, Japan, volume 89 of Proceedings of Machine Learning Research, pp. 527-536. PMLR, 2019.
Kingma, D. P. and Ba, J. Adam: A method for stochastic optimization. In Bengio, Y. and LeCun, Y. (eds.), 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings, 2015.
Koh, P. W., Sagawa, S.. Marklund, H., Xie, S. M., Zhang, M., Balsubramani, A., Hu, W., Yasunaga,
M., Phillips, R. L., Beery, S., Leskovec, J., Kundaje, A., Pierson, E., Levine, S., Finn, C., and Liang, P. WILDS: A benchmark of wn-the-wild distribution shifts. CoRR, 2020.
Koyama, M. and Yamaguchi, S. Out-of-distribution generalization with maximal invariant predictor. CoRR, 2020.
Krueger, D.,, Caballero, E., Jacobsen, J., Zhang, A., Binas, J., Priol, R. L., and Courville, A. C. -fitof--ditributti generalization via risk extrapolation (rex). CoRR, 2020.
Li, H., Pan, S. J., Wang, S., and Kot, A. C. Domain generalization with adversarial feature learning- In 2018 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2018, Salt Lake City, UT, USA, June 18-22, 2018, pp. 5400-5409. IEEE Computer Society, 2018a.
Li, Y., Tian, X., Gong, M., Liu, Y., Liu, T., Zhang, K., and Tao, D. Deep domain generalization via conditional invariant adversarial networks. In Ferrari, V., Hebert, M., Sminchisescu. C., and Weiss, Y. (eds.), Computer Vision ECCV 2018 15th European Conference, Munich, Germany, September 8-14, 2018, Proceedings, Part XV, volume 11219 of Lecture Notes in Computer Science, pp. 647-663. Springer, 2018b.
Louizos, C., Swersky, K., Li, Y., Welling, M., and Zemel, R. S. The variational fair autoencoder. In Bengio, Y. and LeCun, Y. (eds.), 4th International Conference on Learning Representations, ICLR 2016, San Juan, Puerto Rico, May 2-4, 2016. Conference Track Proceedings, 2016.
Lum, K. and Johndrow, E. A statistical framework for fair predictive algorithms. CoRR, abs/ htt.1660.0807,20111110 2016.
Magliacane, S., van Ommen, T., Claassen, T., Bongers, S., Versteeg, P., and Mooij, J. M. Domain adaptation by using causal inference to predict invariant conditional distributions. In Bengio, S., Wallach, H. M., Larochelle, H., Grauman, K., Sesa-Bianchi, N., and Garnett, R. (eds.), Advances in Neural Information Processing Systems 31: Annual Conference Neural Information Processing Systems 2018, NeurIPS 2018, December 3-8, 2018, Montréal, Canada, pp. 10869-10879, 2018.
Mahajan, D., Tople, S., and Sharma, A. Domain generalization using causal matching. CoRR, abs/2006.07500, 2020.
Mansour, Y., Mohri, M., and Rostamizadeh, A. Domain adaptation: Learning bounds and algorithms. In COLT 2009 The 22nd Conference on Learning Theory, Montreal, Quebec, Canada, June 18-21, 2009, 2009.
12

---

Mehrabi, N., Morstatter, F., Saxena, N., Lerman, K., and Galstyan, A. A survey on bias and fairness in machine learning. CoRR, 2019.
Miyato, T., Kataoka, T., Koyama, M., and Yoshida, Y. Spectral normalization for generative adversarial networks. In 6th International Conference on Learning Representations, ICLR 2018, Vancouver, BC, Canada, April 30 - May 3, 2018, Conference Track Proceedings. OpenReview.net, 2018.
Moreno- Torres, J. G., Raeder, T., Alaíz Rodriguz, R., Chawla, N. V., and Herrera, A unifying view dataset shift in classification. Pattern Recognit,, 45(1):521-553) 2012.
Moyer, D., Gao, S., Brekelmans, R., Galstyan, A., and Steeg, G. V. Invariant representations without adversarial training. In Bengio. S., Wallach, H. M., Larochelle, H., Grauman, K., Cesa-Bianchi, N., and Garnett, R. (eds.), Advances in Neural Information Processing Systems 31: Annual Conference on Neural Information Processing Systems 2018, NeurIPS 2018, December 3-8, 2018, Montréal, Canada, pp. 9102-9111, 11, 2018.
Muandet, K., Balduzzi, D., and Schölkopf, B. Domain generalization via invariant feature represen- tation. Proceedings of the 30th International Conference on Machine Learning, ICMI. 2013, Atlanta, GA, USA, 16-21 June 2013, volume 28 of JMLR Workshop and Conference Proceedings, pp. 10-18. JorR.org, 2013.
Parascandolo, G., Neitz, A., Orvieto, A.. Gresele, L., and Schölkopf, B. Learning explanations that are hard to vary. CoRR, hass2209.003329 2020.
Peters, J., Bühlmann, P. and Meinshausen, N. Causal inference by using invariant prediction: identification and confidence intervals. Journal of the Royal Statistical Society. Series B (Statistical Methodology), pp. 947-1012, 2016.
Popejoy, A. and Fullerton, S. Genomics is failing diversity. Nature, 538:161-164, 2016.
Shionero-Candelle J., Sugiyama, M., Schwaighofer, A., and Lawrence, N. D. Dataset Shift in Machine Learning. The MIT Press, 2009. ISBN 0262170051.
Rosenfeld, E., Ravikumar, P., and Risteski, A. The risks of invariant risk minimization. CoRR, abs/2010.05761, 2020.
Santurkar, S., Tsipras, D., and Madry, A. BREEDS: benchmarks for subpopulation shift. CoRR, abs/2008.04859. 2020.
Sason, I. and Verdú, S. f-divergence inequalities. IEEE Trans. Inf. Theory, 62(11):5973-6006 2016.
Schölkopf, B., Janzing, D., Peters, J., Sgouritsa, E.,, Zhang, K., and Mooij, J. M. On causal and anticausal learning. In Proceedings of the 29th International Conference on Machine Learning, ICML 2012, Edinburgh, Scotland, UK, June 26 July 1, 2012. ieml.ee / Omnipress, 2012.
Shimodaira, H. Improving predictive inference under covariate shift by weighting the log- liceliidl function. Journal of Statistical Planning and Inference, 2000.
Tishby, N. and Zaslavsky, N. Deep learning and the information bottleneck principle. In 2015 IEEE Information Theory Workshop, ITW 2015, Jerusalem, Israel, April 26 May 2015, pp. 1-5. IEEE, 2015.
Wang, M. and Deng, W. Deep visual domain adaptation: A survey. Neurocomputing, 312:135-153, 2018.
Widmer, G. and Kubat, M. Learning in the presence of concept drift and hidden contexts. Mach. Learn., 100,,, 1996.
Woodworth, E., Gunasekar, S., Ohannessian, M. I., and Srebro, N. Learning non-discrininatory predictors. In Kale, S. and Shamir, O. (eds.). Proceedings of the 30th Conference on Learning Theory, COLT 2017, Amsterdam, The Netherlands, 7-10 July 2017, volume 65 of Proceedings of Machine Learning Research, pp. 1920-1953, PMLR, 2017.
13

---

Xia, M., Field, A., and Tsvetkov, Y. Demoting racial bias in hate speech detection. In Ku, L. and Li. (eds.), Proceedings of the Eighth International Workshop on Natural Language Processing for Social Media, SocialNI. ACI. 2020, Online, July 10, 2020, pp. 7-14. Association for Computational Linguisties, 2020.
Xie, C., Chen, F., Liu, Y., and Li, Z. Risk variance penalization: From distributional robustness to causality. CoRR, abs/2006.00 07544,, 2020.
Xie, Q., Dai, Z., Du, Y., Hovy, E. H., and Neubig, G. Controllable invariance through adversarial feature learning. In Guyon, I., von Luxburg, U., Bengio. S., Wallach, H. M., Fergus, R., Vish- wanathan, S. N. and Garnett, (eds.), Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, December 4-9, 2017, Long Beach, CA, USA, pp. 585-596, 2017.
Zech, J. R., Badgeley, M. A., Liu, M., Costa, A. B., Titano, J., and Oermann, E. K. Variable generalization performance of deep learning model to detect pneumonia in chest radiographs: A cross-sectional study. PLOS Medicine, 2018.
Zhang, J., Sheng, V. S., Li, Q. Wu, J., and Wu, X. Consensus algorithms for biased labeling in crowdsourcing. Inf. Sci., 382-383:254-2775 2017.
Zhao, H., des Combes, R. T., Zhang. K., and Gordon, G. J. On learning invariant representations for domain adaptation. In Chaudhuri, K. and Salakhutdinov, R. (eds.), Proceedings of the 36th Inter- national Conference Machine Learning, ICML 2019, 9-15 June 2019, Long Beach, California, USA, volume 97 of Proceedings of Machine Learning Research, pp. 7523-7532. PMLR, 2019.
14


Reviews:
[{"review_id": "OsVjW7YwPH0", "reviewer": "Reviewer_rgnK", "summary": "The authors describe the problem of data shift from a information\ntheoretic perspective. They identify different sources of error and compare\nthe effect of different regularization objectives on creatively derived datasets\nin a controlled setting which cannot be achieved in most normal datasets.", "questions": "", "limitations": "", "rating": 7, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "## Pros\n\n- the description of the problem of distributional shift as being a result of\n  not truly IID sampling the real distribution is informative and intuitive\n- The construction of the datasets is sound and captures complex interactions between different dataset selection biases.\n- The authors are very thorough in their theoretical framework which makes a compelling argument for analyzing future OOD-robust models under this controlled setting.\n\n# Cons\n\n- The colors in figure three are too hard to distinguish from one another. The\nlegends are also too small for being able to make out what they are showing at\na glance.\n\n- I find the naming of the criteria in sections 3.1, 3.2, and 3.3 to be quite\nconfusing. Why is 3.1 called \"Independence Criterion\" when actually all of the\nsections 3.1, 3.2, 3.3 are encouraging some form of independence? I realize\nthat these are cited as coming from another source, but I still think they\nshould be clarified so that readers do not become confused.\n\n- It would be good to include some qualitative samples from the CMNIST dataset\nin order to gain more intuition about what is being said in the paper.\n\n- Why are none of the models evaluated on cross-criteria? The models seem to be\nchosen for the specific inductive biases which predispose them to do better on\ncertain criteria. If this is the case, then it would be very interesting to see\nhow they perform on the other criteria, in order to validate that the inductive\nbiases are correct.\n\n- I think it would be interesting to analyze some popular convolutional models\nsuch as resnets in the controlled setting to see how their performance breaks\ndown (like equation 3) and then verify if the trend continue on larger datasets\nsuch as CIFAR-10/100 with an OOD dataset for testing. If the best model in the\ncontrolled setting also performed the best in this experiment (on some other standard OOD metrics which can be measured), that would be\nextremely strong validation of the main theoretical findings.\n\n## Questions\n\n- In Figure 1 it is not explicitly clear how there is sample bias in the data\n$p(y | x_1)$. It seems as if there was actually no selection bias, but just\nanother feature was added. Is there something I am missing?\n\n- The anonymous code repo in the appendix appears to be incomplete, as I cannot\nsee the actual training script anywhere in these files. Is there something I am\nmissing?\n\n- Equation 48 in the appendix seems to show that the encoder is only a linear\nfunction. Is there really no non-linearity in that function?\n\n- Equation 48 says it was used to compute the latent representations for $z$ in\nfigure 3, but then in section E of the appendix, the authors state that the\nencoder has 3 hidden layers. Which one is correct?\n\n- What is the \"bottleneck\" criterion in figure 3 and how does this relate to\nthe criteria highlighted in sections 3.1 - 3.3?\n\n# Minor\n\n- L93 I find this confusing because the equations show \"information loss\" and\n  then \"latent error,\" but in the text L93 the authors refer to them in the\n  opposite order and then on L94-95 they refer to the \"former and latter.\" I\n  think the former and latter should match the order of the terms in the\n  equations in order to avoid confusion.\n\nOverall, I am extremely positive on the paper and the main theoretical\nmotivations and findings. However, I feel there are further experiments which could be\ndone to further validate the findings (the last two bullet points of the **Cons**\nreview). I feel the organization and clarity of each section could also use improvement, as I had to read each section multiple times in order to grasp what was being said.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Yes, they have adequately addressed the limitations in the appendix. I am not aware of any negative societal impact.  ", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "5-6 hours"}, "extra_text": {}}, {"review_id": "KnUcR_n4rcB", "reviewer": "Reviewer_ukrp", "summary": "This paper investigates distribution shift from an information-theoretic perspective. It outlines a theoretical framework for describing different sources of modeling error, including that due to distribution shift, in the presence of a latent representation. It uses the framework to categorise existing approaches to reducing distribution shift according to the type of regularization criteria they use. It explains the assumptions underlying the success of these approaches, and demonstrates empirically, on synthetic data, that when these assumptions fail to hold, performance drops. ", "questions": "", "limitations": "", "rating": 7, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "**Originality**\n\nI’m not an expert in this area, but to the best of my knowledge the theoretical presentation is original, as is the demonstrated experiments. It appropriately cites the selected approaches, which are investigated experimentally. I would have liked to see references given in section 3.3 on the separation criteria, as in 3.1, 3.2 (this is given in related work, but good to have it here too in my opinion). \n\n**Quality**\n\nThe quality of the theory is very good. Ideally I would have liked to see the experiments section pushed further, in particular on the neural network approaches. Only one type of architecture for all methods was investigated, and some surprising results were found (VREx performing worse than expected on CMNIST and d-CMNIST; CDANN performing worse than expected on y-CMNIST), which don’t line up well with the theory presented. It’s not really possible with the current experiments to tell whether this is something fundamental about the method, or if it’s something about the architecture, optimizer or other hyperparameters. \n\n**Clarity**\n\nThe paper is well-written overall. \n- One minor suggestion is to move the related work section to either after the introduction, or just before the conclusion (perhaps the latter works better considering the context of the paper is helpful to understand the related work section). In its current position it breaks the flow between theory and experiments.\n- I would also prefer to call the ‘OOD’ terms ‘test’ instead, as this is more standard, and perhaps more correct (just because a sample is not in the training set, doesn’t mean it is out of distribution in some fundamental way - although sometimes this is the case). \n- I found equation 9 and its preceding paragraph hard to follow, consider rewriting, and giving an explanation for the derivation of 9. \n- Line 141: doesn’t lambda also trade off aspects of (i), as well as between (ii) and (iii)?\n-Line 171: suggest rewrite I(e;y|z) as I(y;e|z) for consistency with eq 11 - even though Mutual Info is symmetric, it will be less confusing to reader. \n\n**Significance**\nThe main limit on significance is the use of synthetic datasets - not just because this gives us limited experimental evidence about the effectiveness of methods for distribution shift, but because the theoretical terms would be challenging to estimate in real-world data (the paper acknowledges this shortcoming). Whether the presented analysis is actually useful would have to be shown in follow-up work.\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Technical limitations are briefly commented on. No potential negative societal impact is discussed. ", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "6"}, "extra_text": {}}, {"review_id": "E7VZMqA1Gsc", "reviewer": "Reviewer_S3bF", "summary": "The paper studies the distributional shift setting, in which test and training distribution are different.  The distribution shift is described in terms of mutual information of the data with a binary selection variable $t$ (train or test).\nSeveral propositions describe relations between the distribution of test and train covariates and labels and a latent space.\nBased on those observations, several regularization criteria are suggested.\nThe experiments compare various settings of distributional shifts and popular methods.", "questions": "", "limitations": "", "rating": 5, "confidence": 1, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strengths:\n- Distributional Shift is an interesting and important problem.\n- The information theory approach gives a novel and intuitive description of the problem. \n- The experiments test different settings of distributional shift and compare the performance of popular approaches.\n\n*******************\n\nWeakness:\n- I think the paper lacks significant technical results. I may have missed some insights, but I think that the propositions don't lead to significant and quantifiable conclusions about the distributional shift problem.  \nThe methods tested in the experiments section are claimed to be a relaxation of the criteria derived from the framework. This can be a very interesting conclusion, but it is not studied thoroughly in the paper (Sect 5.1 doesn’t elaborate the description of the methods.)\n\n- There is also no mention of the finite-sample nature of learning problems. The conclusion from Prop 1. about the maximum-likelihood solution seems unfounded if there is no mention of the data used for the estimation.\n\n- The objective function in Sect 2.5 resembles Variational Autoencoders, but it is not discussed in the paper. \n\n- Section 2.3 -  Lines 105-106 - “Whenever the JSD is small then the latent OOD error is small” - this doesn’t seem to follow from (8) since the JSD is a lower bound.\n \n\n*******************\nClarity:\nThe paper is generally well written.\nHowever, the introduction is not informative enough about the contributions of the paper.\nThe different sections feel disjointed with no clear organizing structure.\nFor example, section 2 is titled “Framework and Problem Statement” but contains propositions. It is not clear where the main results of the paper start.\n\n*******************\nMinor comments: \n- Line 98 - the sentence is not clear.\n- Line 191 - typo - “t” \n- It seems to be a problem with the PDF, Fig 3 is missing when viewing the file with Adobe Reader (but it is fine with other software).\n\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "The limitations are discussed in the main review. There is no potential negative societal impact.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "6"}, "extra_text": {}}, {"review_id": "87HEIfozX5", "reviewer": "Reviewer_y2kJ", "summary": "This paper motivates a series of information based analysis and methods. The authors first propose an information theoretic analysis framework for out-of-distribution generalization problem. Some corresponding generalization bounds are brought forward. Three different kinds of regulators are further studied. The effectiveness of proposed methods are demonstrated through numerical experiments.", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The writing of the main paper and appendix are clear.  The reviewer lists some concerns here.\n\n1. Why the authors claim that VREx is a relaxation of sufficiency criterion? VREx requires the stability of joint distribution of $(y, z)$ to achieve a stable loss value across domains. \n\n2. The ablation about combination of different criterions is lacked, which would be an interesting direction.\n\n3. How to numerically estimate the mutual information?", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "The reviewer doesn't foresee a limitation on societal impact.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "5 hours"}, "extra_text": {}}]

Author rebuttal:

Meta-review: The authors study out-of-distribution (OOD) generalisation using an information-theoretic framework, decomposing the contributions of concept and covariate shift to the overall error. They go on to show how the framework yields an upper bound on OOD error in the context of a (latent) representation of the input variables. Consistent with related results, this leads to a conclusion that to minimise OOD error necessitates losing some information which is predictive of the training data. The optimal tradeoff relies on data unavailable at training time and so the authors study 3 proxies for minimising concept shift in the representation: independence, sufficiency and separation. In an empirical evaluation, the authors study minimising these criteria in comparison with existing approaches with related objectives. 

Several reviewers appreciated the information-theoretic formulation of OOD error and the theoretical treatment of it. Reviewer S3bF raised concerns about the significance of the technical contribution but this is to some extent compensated by the insights gained by the change in framework compared to related works. Additionally, several comments were made about the clarity of the organisation of the paper. In particular, transitions between sections are often hard to follow (see e.g., Section 2.2. which starts without any lead-in). The authors would do well to guide the reader better through the sections. Nevertheless, given the reviewers appreciation for the work, and the arguably minor concerns that remain after the discussion phase, I believe that a re-organisation of the paper is sufficient.
Decision: Accept (Poster)