Out-of-Distribution Generalization Analysis via Influence Function
Authors: Haotian Ye
Chuanlong Xie
Yue Liu
Zhenguo Li
Venue:  (2021)

Abstract: The mismatch between training dataset and target environment is one major challenge for current machine learning systems. When training data is collected from multiple environments and the the evaluation is on any new environment, we are facing an Out-of-Distribution (OOD) generalization problem that aims to find a model with the best OOD accuracy, i.e.  the  best  worst-environment  accuracy. However, with limited access to environments, the worst environment may be unseen, and test accuracy is a biased estimate of OOD accuracy.  In this paper, we show that test accuracy may dramatically fail to identify OOD accuracy and mislead the tuning procedure. To this end, we introduce Influence Function, a classical tool from robust statistics, into the OOD generalization problem and suggest the variance of influence function to measure the stability of a model on training environments.  We show that the proposed index and test accuracy together can help us discern whether OOD algorithms are needed and whether a model achieves good OOD generalization.

Under review as a conference paper at ICLR 2021
OUT-OF-DISTRIBUTION GENERALIZATION ANALYSIS VIA INFLUENCE FUNCTION
Anonymous authors Paper under double-blind review
ABSTRACT
The mismatch between training and target data is one major challenge for current machine learning systems. When training data is collected from multiple domains and the target domains include all training domains and other new domains, we are facing an model with the best OOD accuracy. One of the definitions of OOD accuracy is worst-domain accuracy. In general, the set of target domains unknown, and the worst target domains may be unseen when the number of observed do- mains limited. In this paper, we show that the worst accuracy over the observed domains may dramatically fail identify the OOD accuracy. To this end, in- troduce Influence Function, a classical tool from robust statistics, into the OOD generalization problem and suggest the variance of influence function moni- tor the stability of model on training domains. We show that the accuracy on test domains and the proposed index together can help us discern whether OOD algorithms are needed and whether model achieves good OOD generalization.
(OOD) generalization problem that aims to find
INTRODUCTION
Most machine learning systems assume both training and test data are independently and identically distributed, which does not always hold in practice (Bengio et al. (2019)). Consequently, its per- formance is often greatly degraded when the test data from different domain (distribution). A classical example is the problem to identify cows and camels (Beery et al. (2018)) where the em- pirical risk minimization (ERM, Vapnik (1992)) may classify images by background color instead of object shape. As a result, when the test domain is "out-of-distribution" (OOD), e.g. when the background color is changed, its performance will drop significantly. The OOD generalization is to obtain a robust predictor against this distribution shift.
Suppose that we have training data collected from m domains:
== (S E Etr, |Eer m},
S ...., >/00} with 2 Pe.
where P is the distribution corresponding to domain e, Eer is the set of all available domains, including validation domains, and 2, is data point. The OOD problem we considered is to find a model fooD such that
The
fOOD arg min sup P'EEa
((f.P)).
where Eall is the set of all target domains and E(S, P*) is the expected loss of ʃ on the domain pe Recent algorithms address this OOD problem by recovering invariant (causal) features and build the optimal model on top of these features, such as Invariant Risk Minimization (IRM, AAroossyy et al. (2019), ) Risk Extrapolation (REx, Krueger et al. (2020). Group Distributionally Robust Optimiza- tion (gDRO. Sagawa et al. (2019) and Inter-domain Mixup (Mixup. Xu et al. (2020); Yan et al.) 2020); Wang et al. (2020h). Most works evaluate on Colored MNIST (see for details) where we can directly obtain the worst domain accuracy over E.u- Gulrajani & Lopez-Pazz (2020) has assem- bled many algorithms and multi-domain datasets, and finds that OOD al algorithms can outperform ERM in some domain generalization tasks (Gulrajani & Lopez-Pat Paz (2000), e.g. VLCS (Torralbal & Eiros (20IT) and PACS (Li et al. (2017)). This is not surprising, since these tasks only require high performance certain domains, while OOD algorithm is expected to learn truly invariant

---

Under review as a conference paper at ICLR 2021
features and be excellent on a large set of target domains Eall. This phenomenon is described as cesturacy-v--invaiimme trade-off" in Akuzawa et al. 2019).
Two questions arise in the min-max problem (2) First, previous works assume that there is sufficient diversity among the domains in East- Thus the supremacy of e(f, PT) may be much larger than the average, which implies that ERM may fail to discover fOoD- But in reality, we do not know whether it is true. If not, the distribution of E(f. Po) is concentrated on the expectation of E(f, Po), and ERM is sufficient to find an invariant model for Eatt. Therefore, we call for a method judge whether an OOD algorithm is needed. Second, how to judge a model's OOD performance? Traditionally, we consider test domains Etest C E,r and use the worst-domain accuracy over Etest (which we call test accuracy) to approximate the OOD accuracy. However, test accuracy is a biased estimate of the OOD accuracy unless E,r is closed to Eaul More seriously, may be irrelevant or even negatively correlated to the OOD accuracy. This phenomenon is not uncommon, especially when there are features virtually spurious in Eall but show a strong correlation to the target Egy-
We give a toy example in Colored MNIST when the test accuracy fails to approximate the OOD accuracy. For more details, please refer to Section a.n]ad Appendix A.4 We choose three domains from Colored MNIST and use cross-validation (Gulrajani & Lopez-Par Paz (2020)) to select models, ie. we take turns to select a domain S E Eir as the test domain and train on the rest, and select the model with max average test accuracy. Tigure[[ssowwe the comparison between ERM and IRM. One can find that no matter which domain is the test domain, ERM model uniformly outperforms IRM model on the test domain. However, IRM model achieves consistently better OOD) accuracy. Shortcomings of the test accuracy here are obvious, regardless of whether cross-validation is used. In short, the naive use of the test accuracy may result in non-OOD model.
To address this obstacle, we hope to find metric that correlates bet- ter with model's OOD property, even when E,r is much smaller than Eall and the "worst" domain remains wn- known. Without any assumption to Eau, our goal is unrealistic. There- fore, we assume that features that are invariant across Eer should also be across Eatl. This assumption is nec- essary. Otherwise, the only thing we can do is to collect more domains. Therefore, we need to focus what features the model has learnt. Specif- ically, we want check whether the model learns invariant features and avoid varying features.
Test Aecurary Comsatiion
Setup
HM
Toll Anusay
Sctup 2
Setup
CFN
CTN
4
The influence function (Cook & Weisberg (1980)) can serve our pur- pose. Influence function was pro- posed to measures the parameter change when a data point is removed or upweighted by a small perturba- tion (details in B.2). When modified
Figure 1: Experiments in Colored MNIST to show test accu- racy is not enough to reflect model's OOD accuracy. The top left penal shows the test accuracy of ERM and IRM. The other three panels present the relationship between test ac- curacy (x-axis) and OOD accuracy (y-axis) in three setups.
it to devain-level. it measures the influence of a domain instead of a data point on the model. Note that we are not emulating the changes of the parameter when domain is removed. Instead, we are exactly caring about upweighting the domain by + 0+ (will be specified later). Base on this, the variance of influence function allows us to measure OOD property and solve the obstacle.
Contributions we summarize our contributions here: (i) We introduce influence function to domain-level and propose index Vaio (formula 6) based on influence function of the model fo- Our index can measure the OOD extent of available domains, i.c. how different these domains (dis- tributions) are. This measurement provides a basis for whether to adopt an OOD algorithm and to collect more diverse domains. See Section 4.1 and 5.1.1 details. (ii) We point out that the
2

---

Under review as a conference paper at ICLR 2021
proposed index Vyja can solve the weakness of test accuracy. Specifically, under most OOD gener- alization problems, using test accuracy and our index together, we can discern the OOD property of model. See Section 4.2 for details. (iii) We propose use only small but important part of the model to calculate the influence function. This overcomes the huge computation cost of solving the inverse of Hessian. It is not merely for calculation efficiency and accuracy, but it coincides with our understanding that only these parameters capture what features a model has learnt (Section 4.3).
We organize our paper as follows: Section 2 reviews related works and Section 3 introduces the + presents our proposal and detailed
preliminaries of OOD methods and influence function. Section analysis. 3shtoss our experiments. The conclusion given in Section
2 RELATED WORK
The mismatch between the development dataset and the target domain is one major challenge in ma- chine learning (Castro et al. (2020); Kuang et al. (2020). Many works assume that the ground truth can be represented by a causal Direct Acyclic Graph (DAG), and they use the DAG structure to dis- cuss the worst-domaii performance
et al. (2018); Peters et al. (016);; Subbaswamy et al. (2019); Bühlmann al. (2020); Magliacane et al. (2018). All these works employ multiple do- main data and causal assumptions to discover the parents of the target variable. RRoas-Carulle et al.. 2018) and Magliacane al. (2018) also apply this idea Domain Generalization and Multi-Tass Learning setting. Starting from multiple domain data rather than model assumptions, (Arjovsky et al.. (219)) proposes Invariant Risk Minimization (IRM) to extract causal (invariant) features and learn invariant optimal predictor on the top of the causal features, It analyzes the generalization properties of IRM from the view of sufficient dimension reduction (Cook (2009); Cook al. (2002)). Ahuja iet al. (2020) considers IRM as finding the Nash equilibrium of an ensemble game among several domains and develops a simple training algorithm. Krueger al. (2020) derives the Risk Extrapola- tion (REx) to extract invariant features and further derives a practical objective function via variance penalization. Xie et al. (2020) employs a framework from distributional robustness to interpret the
benefit of REx comparing robust optimization (Ben-Tal et al. (2009); Bagnell (205)). Besides, Adversarial Domain Adaption (Li et al. (2018;; Koyama & Yamaguchi (2020) uses discriminator to look for features that are independent of domains and uses these features for further prediction.
Influence function a classic method from the robust statistics literature (Robins et al. (2008) (017);
Van der Laan et al. (2003): ( Tsiatis (2007)) (2007)) the prediction. Koh & I Liang] (2017) proposes a second-order optimization technique to approximate the influence function. They verify their method with different assumptions the empirical risk ranging from being strictly convex and wiice-differentiable to non-convex and non-differentiable losses. Koh et al. (2019) also estimates the effect of removing a subgroup of training points via influ- ence function. They find out that the approximation computed by the influence function is correlated with the actual effect. Influence function has been used in many machine learning tasks. [Ceng et al.) (2019) proposes an explanation method, Fast Influence Analysis, that employs influence function on Latent Factor Model to solve the lack of interpretability of the collaborative filtering approaches for recommender systems. Cohen et al. (2020) uses influence function to detect adversarial attacks.
It can be used to track the impact of a training sample on
Ting & Brochu| (2018) proposes an asymptotically optimal sampling method via an asymptotically linear estimator and the associated influence function. Alaa & Van Der Schaar (2019) develops
model validation procedure that estimates the estimation error of causal inference methods. Besides, Fang et (2020) leverages influence function to select subset of normal users who influential to the recommendations.
PRELIMINARIES
3.1 ERM, IRM AND REX
In this section, we give some notations and introduce some recent OOD methods. Recall the multiple domain setup ① and OOD problem (2. For a domain Pe and hypothetical model f. the population loss is fff, Pr) Ex-pe [L z)] where L(f. is the loss function on z. The empirical loss. which is the objective of ERM, is {f{8) = (1/m) Eeee.. {(SSe) with {ff = (1/n) Σ T(S,2)).
3

---

Under review as a conference paper at ICLR 2021
Recent OOD methods propose some novel regularized objective functions in the form:
((SS)) = E X
(3)
to discover FOoD in 2). Here R(f,S) is a regularization term and À is the tuning parameter which controls the degree of penalty. Note that ERM is special case by setting 0. For simplicity, we will use [(f, 8) to represent the total loss in case of no ambiguity, Arjovsky et al. (2019) focuses on the stability of fOoD and considers the IRM regularization:
R(J,S) = Σ V (ew((w)) (wf). ml
(4)
eEE
where w a scalar and fixed "dummy" classifier, |Arjovsyy et al. (2019) shows that the scalar fixed classifier is sufficient to monitor invariance and responds to the idealistic IRM problem which decomposes the entire predictor into data representation and one shared optimal top classifier for all training domains. On the other hand, Krueger et al. (2020) encourages the uniform performance of fooD and proposes the V-REx penalty:
R(f.))= -((
eEEr
Krueger et al. (202)) derives the invariant prediction by the robustness to spurious features and figure out that REx more robust than group distributional robustness (Sagawa et al. (2019)). In this work, we also decompose the entire predictor into a feature extractor and a classifier on the top of the learnt features. As we will see, different from from Arjovsky et al. (2019) and Krueger we directly monitor the invariance of the top model.
2020
3.2 INFLUENCE FUNCTION AND GROUP EFFECT
Consider a parametric hypothesis f = fe and the corresponding solution: Ô arg ming L(fe, S). By quadratic approximation of C(Je, S) around Ó, the influence function takes the form
=) VeL(f. z) with H Ve
When the sample size of S is sufficiently large, the parameter change due to removing a data point 2 can be approximated by -I=))/ E.ee,. S without retraining the model. Here |S = n stands for the cardinal of the set S". Furthermore, Koh et al. (2019) shows that the influence function can also predict the effects of large groups of training points (i.c. Z {21,..., 2)), although there are significant changes in the model. The parameter change due to removing the group can be approximated by
= -
Σ L)..
=EZ
Motivated by the work Koh al. (2019), we introduce influence function to OOD problem to address our obstacles.
4 METHODOLOOGY
4.1 INFLUENCE OF DOMAINS
We decompose a parametric hypothesis fo(x)) into a top model g and feature extractor 4 i.e. fo(x) g( (2x B), 7) and = 3). Such decomposition coincides the understanding of most DNN, i.e. a DNN extracts the features and build a top model based on the extracted features. When upweighting a domain by a small perturbation 5, we do not upweight the regularized term, i.e.
( 6) = L(O,S) + 8 {(f.S),
since the stability across different domains, which is encouraged by the regularization, should not depend on the sample size domain. For a learnt model fg. fixing the feature extractor 4 i.c. fixing G = 3, the change of top model y caused by upweighting the domain is
lim AO 6-201
IF(f,
--
- VVC(ffg e E
(5)
4

---

Under review as a conference paper at ICLR 2021
Here H4 = S), and we assume L is wiie-diiferentiable in 7. Please see Appendix A.3 for detailed derivation and why & should be fixed. For regularized method, e.g. IRM and REx, the influence of their regularized term is reflected in H and in learnt model fg. As mentioned above, IF (0)) measures change of model caused by upweighting domain Therefore, if g ((0,)) 7) is invariant across domains, the entire model fa treats all domains equally. As a result, small perturbation different domains should cause the same model change. This leads to our proposal.
4.2 PROPOSED INDEX AND ITS UTILITY
On basis of the domain- influence function IF((F, S we propose our index to measure the fluctuation of the parameter change when different domains are upweighted:
Vaia == In (ICovvee,, (17
Here ||- 112 is the 2-norm for matrix, i.c. the largest eigenvalue of the matrix, Covese. (-) refers to the covariance matrix of the domain-level influence function over Etr and In(-) is nonlinear transformation that works well in practice.
OOD Model Under the OOD problem in (2) good OOD model should (i) learn invariant and useful features; (ii) avoid spurious and varying features. Learning useful and invariant features means the model should have high accuracy over set of test domains Etest, no matter which test domain it is. In turn, high accuracy over Etest also means the model truly learns some useful features for the test domains. However, this is not enough, since we do not know whether the useful features are invariant features across En or just spurious features on Erest. On the other hand, avoiding varying features means that different domains are actually the same to the learnt model, so according to the arguments in Section 4.1 V-10 should be small. Combined this, we derive our proposal: if a learnt model fe manage to simultaneously achieve small V Vejō and high accuracy over Etest. should have good OOD accuracy. We prove our proposal in simple but illuminating case, and we conduct various experiments (Section 5 support our proposal. Several issues should be clarified. First, not all OOD problems demand models to learn invariant features. For example, the set of all target domains is small such that the varying features are always strongly correlated to the labels, or the objective is the mean of the accuracy over Eall rather than the worst-domain accuracy. But to our concern, we regard the OOD problem in (2) as bridge causal discover. Thus the set of the target domains is large, and the "weak" OOD problems are out of our consideration. To large extent, invariant features are still the major target and our proposal is still a good criterion to model's OOD property. Second, we admit that the gap between being stable in E,, (small Vyje) and avoiding all spurious features on Eoll does exist. However, to our knowledge, for features that are varying in Eau but are invariant in E,,, demanding model avoid them is somehow unrealistic. Therefore, we make a step forward that we measure whether the learnt model successfully avoids features that vary across E... We leave index about varying features over Eall in our future work.
The Shuffle Vy1 As mentioned above, smaller metric VyIo means strong stablility across Eir, and hence should have better OOD accuracy. However, the proposed metric depends on the dataset S and the learnt model fg- Therefore, there is no uniform baseline to check whether the metric is "small" enough. To this end, we propose a bascline value of the proposed metric by shuffling the multi-domain data. Consider pooling all data points in S and randomly redistributed to new synthetic domains {81 $2,..., 5mh ;= S. We compute the shuffle version of Vy10 for learnt model fø over the shuffled data S:
1 00 : In (ICovveee, :
(IF(S,
and denote the standard version and shuffle version of the metric as V V/y0 and 12y10 respectively. For any algorithm that obtains relatively good test accuracy, if V is much larger than 13 1 So has learnt features that vary across e E Etr, and cannot treat domains in Exr equally. This implics that fø may not be invariant predictor over Eau. Otherwise, if the two values are similar, the model has avoided varying features in E,, and maybe invariant across E,,. Therefore, either the model capture the invariance over the diverse domains, or the domains are not diverse at all. Note that
5

---

Under review as a conference paper at ICLR 2021
this process is suitable for any algorithm, hence providing baseline to whether V و is small. Here also obtain method to judge whether OOD algorithm is needed. Consider fa learnt by ERM. If Viiá 9/ is relatively larger than Vyio. then ERM fails to avoid varying features. In this case, one should consider an OOD algorithm to achieve better OOD generalization. Otherwise, ERM is enough, and any attempt to achieve better OOD accuracy should start with finding more domains instead of using OOD algorithms. This coincides experiments in Guurajani & Lopez-Paz (2020) (Section5.22) Our understanding is that domains in $ are similar. Therefore, the difference between shuffle and standard version of the metric reflects how much varying features a learnt model uses. We show how to use the two version of Vy10 in Section 5.1.1 and Section 5.2
4.3 INFLUENCE CALCULATION
There is a question surrounding the influence function: how to efficiently calculate and inverse Hes- sian? Koh & Liang (017 suggests Conjugate Gradient and Stochastic estimation solve the problem. However, when
obtained by running SGD, it could hardly arrive at the global minimum. Al- though adding a damping term (i.e. let Ha = H + AΠ) can moderately alleviate the problem by transforming it into convex situation, under large neural-network with non-linear activation func- tion like ReLU, this method may still work poorly since the damping term in order to satisfy the transform is large that it will influence the performance significantly. Most importantly, the vari- ation of the eigenvalue of Hessian huge, making the convergence of influence function calculation quite slow and inaccurate (Basu al. (2020)).
In our metric, we circumvent the problem by excluding most parameters 3 and directly calculate Hessian of y to get accurate influence funetion. This modification not only speed up the calculation, but it also coincides our expectation, that an OOD algorithm should learn invariant features does not mean that the influence function of all parameters should be identical across domains. For example, if g( wants to extract the same features in different domains, the influence function should be different on (-)). Therefore, if we use all parameters to calculate the influence, given that y is relatively insignificant in size compared with B, the information of learnt features provided by y is hard to be captured. On the contrary, only considering the influence of the top model will manifest the influence of different domains in the aspect of features, thus enabling us to achieve our goal.
As our experiments show, after this modification, the influence function calculation speed can be 2000 times faster, and the utility (correlation with OOD property) could be even higher. One may not feel surprised given the huge number of parameters in the embedding model D(-). They slow down the calculation and overshadow the top model's 's influence value.
EXPERIMENT
In this section, we experimentally show that: (1) A model fá reaches small Vyjē if it has good OOD property, while a non-OOD model won't. (2) The metric Vy provides additional information on the stability of a learnt model, which overcomes the weakness of the test accuracy. (3) The comparison of Vy and Vyy,s can check whether a better OOD algorithm is needed.
We consider experiments in Bayesian Network, Colored MNIST and VLCS. The synthetic data generated by Bayesian Network includes domain-dependent noise and fake associations between features and response. For Colored MNIST, we already know that the digit the causal feature and the color is non-causal. The causal relationships help us determine the worst domain and obtain the OOD accuracy. VLCS a real dataset, which we show utility of Vy10 step by step. Due to the space limitation, we put the experiments in Bayesian Network to the appendix.
Generally, cross-validation (Gulrajami & Lopez-Pazi (2000) is used to judge a model's OOD prop- erty. In the introduction, we have already shown that the fail to discern OOD properties. We also consider another two potential competitors: conditional mu- tual information and IRM penalty. The comparison between metric and the two competitors postponed into Appendix.
cross-validation may
5.1 COLORED MNIST

---

Under review as a conference paper at ICLR 2021
Colored MNIST (Arjovsky al. (2019) in-
troduces a synthetic binary classification task. The images are colored according to their la- bel, making color a spurious feature in predict- ing the label. Specifically, for domain e, assign a preliminary binary label y = Ldigitss'4 and randomly flip y with p 0.25. Then,
Environment Difference (x)
we color the image according to ÿ but with flip rate of p². Clearly, when p < 0.25
or p" > 0.75, color is more correlated with y than real digit. Therefore, the oracle OOD model FOOD will attain accuracy 0.75 in all do- mains while an ERM model may attain high training accuracy and low OOD property if pa in training domains is too small or too large. Throughout the Colored MNIST experiments, we use three-layer MLP with ReLU activation and hidden dimension 256. Although our MLP model has relatively many parameters and is non-convex due the activation layer. due to the technique mentioned in Section 4.3 the in- fluence calculation is still fast and accurate, with directly calculating influence once spends less than 2 seconds.
stability measurement Vys
The index V~Jo is highly correlated to The plot contains 501 learnt ERM models with x = x 10- i = O, 1, 500. The dashed line the baseline value when the difference between domains is eliminated by pooling and redistribut- ing the training data. The blue solid line is the linear regression of x versus Vy10-
Figure 2:
5.1.1 IDENTIFY OOD PROBLEM
In this section, we show that V-10 can discern whether the training domains are sufficiently diverse Assume Eer has five training domains with
as mentioned Section 4.2
pd {0.2 2z, 0.2 z, 0.2, 0.2 + x, 0.2 + 2x),
where r E [0.0, 0.1] is positively related to the diversity among the training domains. If x is zero, all data points are generated from the same domain (p 0.2) and so the learning task on E,, not an OOD problem. On the contrary. larger means that the training domains are more diverse. We repeat 501 times to learn the model with ERM. Given the learnt model ʃg and the training data, we compute 1 and check the correlation between y and x. [ the results. Our index Vyy is highly related to The Pearson coefficient 0.9869, and the Spearman coefficient is 0.9873.
Also, the benchmark of Vy that learns on the same training domains ($ in 4.2 can be derived from the data by pooling and redistributing all data points, and mark it by the black dashed line. If V is much higher than the benchmark, indicating that æ is not small, an OOD algorithm should be considered if better OOD generalization is demanded. Otherwise. the present algorithm (like ERM) sufficient. The results coincide our expectation that V-10 can discern whether Pe is different.
5.1.2 RELATIONSIIPP BETWEEN V AND OOD ACCURACY
In this section, we use an experiment to support our proposal in Section 4.2 As previously proposed, if a model shows high test accuracy and small Vyj0 simultaneously, it captures invariant features and avoids varying features, so it deserves be an OOD model. In this experiment, we consider a model with high test accuracy and show that smaller 1 generally corresponds to better OOD accuracy, which supports our proposal.
Consider two setups: p" E (0.0.0.1) and p" E {0.1, 0.2, 0.25, 0.3}. We implement IRM and REx with different penalty (note that ERM is X 0) to check relationship between V- and OOD accuracy. For IRM and REx, we run 190 epochs pre-training with X and use early stopping to prevent over-fitting. With this technique, all models successfully achieve good test accuracy (within I of the oracle accuracy) and meet our requirement. Figure presents the results. We can see that Vy,0 are highly correlated to OOD accuracy in IRM and REx, with the absolute of Pearson Coefficient never less than 0.8417. Those models learned with larger À present better OOD property, learning less varying features, and showing smaller V.y 0- The results are consistent with our proposal, except that when À is large in IRM, Vy10 is little bit unstable. We have carefully
7

---

Under review as a conference paper at ICLR 2021
examined the phenomenon and found that it is caused by computational instability when inversing Hessian with eigenvalue quite close to O. The problem of unstable inversing happens with a low probability and can be addressed by repeating the experiment once twice.
5.2 DOMAIN GENERALIZATION: VLCS
In this section, we implement the proposed met- ric for 4 algorithms: ERM, gDRO, Mixup and IRM on the VLCS image dataset, which is widely used for doamin generalization. We em- ulate real scenario with Eaul (V, C.S) and E.r Eau' (S}. As mentioned in Gul- rajani & Lopez-Paz (2020), we use "training- domain validation set" method, i.e. we split a validation set for each S E &rr and the test accuracy is defined as the average accu- racy amount the three validation sets. Note that, our goal is to use the test accuracy and V=10 to measure the OOD generalization, rather than to tune for the SOTA performance on unseen domain {S}. Therefore, we do not apply any model selection method and just use the de- in Gulrajani & Lopez-
Ensiromments
Envronments
CRM
B3224
Van Enxtromments
with Druhanments
F0A5)
P8:009
EMCCCE
-11
Figure 3: The relationship between Vyje and OOD accuracy in REx (left) and IRM (right) with E {0, 50, 100, 500, 1000). We train 400 mod- els for each X. The OOD accuracy and V. en- joy high Pearson coefficient: -0.9745 (up-left), -0.9761 (down-left). -0.8417 (up-right), -0.9476 (dgwn-right). The coefficients are negative be- cause lower Vy forebodes better OOD property.
fault
Paz (2020).
5.2.1 STEP
1: TEST ACCURACY COMPARISON
For each algorithm, run the naive training process 12 times and show the average of test accuracy of each algorithm in Table I Before
calculating Vy13. the learnt model should at least arrive a good test accuracy. Otherwise, there is no need to discuss its OOD performance since OOD accuracy is smaller than test accuracy. In the table, the test accuracy of ERM, Mixup and gDRO is good, but that of IRM is not. In this case, IRM will be eliminated. If an algorithm fails to reach high test accuracy first, we should first change the hyper-parameters until we observe relatively high test accuracy.
5.2.2 STEP 2: SHUFFLE AND STANDARD METRIC COMPARISON
Now we are ready to check whether the learnt models are invariant across Ex,. As mentioned in 4.2) the difference of Vy1 and Dy1j repre- sents whether how much model is invariant across Ey. We calculate the value and the re- sults are in Figure 4 For ERM and Mixup, the two value is nearly the same. In this case, we expect that ERM and Mixup models are invari- ant and should have a relatively high OOD ac- curacy, so no more algorithm is needed. For gDRO, we can clearly see that D is uniformly smaller than Vy1s- Therefore, gDRO models don't. t treat different domains equally, and hence predict that the OOD accuracy will be relatively low. In this case, one who starts with gDRO should turn to other algorithms if a better OOD performance is demanded.
Table 1: Stepl: Test Accuracy (%)
Domain
C
L
V
Mean
ERM
99.29
73.62
77.07
83.34
Mixup
99.32
74.36
78.84
84.17
gDRO
95.79
70.95
75.25
80.66
IRM
49.44
44.76
41.17
45.12
Note that, in the whole process, we know nothing about {S}. so the OOD accuracy is unseen. However, from the above analysis, we know that (1) in this settings, ERM and Mixup is better than gDRO; (2) one who uses gDRO can turn to other algorithms (like Mixup) for better OOD performance; (3) one who uses ERM should consider collecting more environments if he (she) still
8

---

Under review as a conference paper at ICLR 2021
wants to improve OOD performance. So far, finish the judgement using test accuracy and the proposed metric.
ERM
Mixup
Group DRO
0.7
2.2
0.7
Accuracy
Accuracy
OOD Accuracy
000
2.5
FRM
Mixup Shuffie
Group DRO Shuffle
Shuffle
Figure 4: The standard and shuffle version of the metric, ie. Vy1 and Vy1s for ERM, Mixup and gDRO, For each algorithm, each version of the metric, we run the experiments more than 12 times in case of statistical emoor. Similar Vyi and Vy1, represents invariance across &tr, which is the of ERM and Mixup. For gDRO, Vyis is clearly smaller.
5.2.3 STEP 3: OOD ACCURACY RESULTS (ORACLE)
In this step, fortunately obatin Eall and can check whether our judgement is reasonable. Normally, this step will not happen. We now show the OOD accuracy of four algorithms in table 2 Similar to our judgement, ERM and Mixup models achieve higher OOD accuracy than gDRO. The performance also compare the metric of the model from the same algorithm but
Table 2: Step3: OOD
Accuracy (%)
ERM
Mixup
gDRO
IRM
Mean
62.76
63.91
60.17
31.33
Std
1.16
1.57
2.56
13.44
IRM (under this seper-parameters) is lower than test accuracy.
During the above process, with different hyper- parameters (as the same section se.12o Besides, one may notice that even the highest OOD accuracy is just 63.91 %. That is say, obtain OOD accuracy larger than 70%, should consider collecting more environments. In the appendix A.6). we continue our real scenario see that, if initially E,p more diverse, what will our metric lead to.
The whole results in VLCS can also be found in the same appendix, and the comparison of the proposed metric with the IRM penalty in formula 4 can be found there too. Besides, we show the comparison with Conditional Mutual Information in the appendix AA.5] In summary, we use realistic task to see how to judge the OOD property of learnt model using the proposed metric and test accuracy. The judgement coincides well with the real OOD performance.
6 CONCLUSION
In this paper, we focus on two presently unsolved problems, that how can we discern the OOD property of multiple domains and of learnt models. To this end, introduce influence function into OOD problem and propose our metric to help solve these issues. Our metric can not only discern whether multi-domains problem is OOD but can also judge model's OOD property when combined with test accuracy. To make our calculation more meaningful, accurate and efficient, we modify influence function domain-level and propose to use only the top model calculate the influence. Our method is proved in simple cases and it works well in experiments. We sincerely hope that, with the help this index, our understanding of OOD generalization will become more and more precise and thorough.
REFERENCES
Kartik Ahuja, Karthikeyan Shanmugam, Kush Varshney, and Amit Dhurandhar, Invariant risk min- imization games. arXiv preprint XXix:2002.04692 2020.

---

Under review as a conference paper at ICLR 2021
Kei Akuzawa, Yusuke Iwasawa, and Yutaka Matsuo. Domain generalization via invariant represen- tation under domain-clast dependency, 2019. URL https::/ wwwprreewweeweeeeoooo.eeeeee formm?
[d-H3x381C5KX
Ahmed Alaa and Mihaela Van Der Schaar. Validating causal inference models via influence func- tions. volume 97 of Proceedings of Machine Learning Research, pp. 191-201, Long Beach, California, USA, 09-15 Jun 2019. PMLR.
Martin Arjovsky, Léon Bottou, Ishaan Guirajani, and David Lopez-Paz. Invariant risk minimization. arXiv preprint wetp:020000.00000 2019.
J Andrew Bagnell. Robust supervised learning. In Proceedings of the national conference on arti- ficial intelligence, volume 20, pp. 714. Menlo Park, CA; Cambridge, MA; London; AAAI Press; MIT Press; 1999, 2005.
Samyadeep Basu, Philip Pope, and Soheil Feizi. Influence functions in deep learning are fragile. arXiv preprint arXiv:2006. 14651, 2020.
Sara Beery, Grant Van Hor,, and Pietro Perona. Recognition in terra incognita. In Proceedings of the European Conference on Computer Vision (ECCV), pp. 456-473, 2018.
Aharon Ben-Tal, Laurent El Ghaoui, and Arkadi Nemirovski. Robust optimization, volume 28. Princeton University Press, 2009.
Yoshua Bengio, Tristan Deleu, Nasim Rahaman, Nan Rosemary Ke, Sebastien Lachapelle, Olexa Bilaniuk, Anirudh Goyal, and Christopher Pal. A meta-transfer objective for learning to disentan- gle causal mechanisms. International Conference on Learning Representations, 2019.
Peter Bühlmann et al. Invariance, causality and robustness. Statistical Science, 35(3):404 426, 2020.
Daniel Castro, Ian Walker, and Ben Glocker. Causality matters in medical imaging. Nature Communications, 11(1):1-10, 2020.
Weiyu Cheng, Yanyan Shen, Linpeng Huang, and Yanmin Zhu. Incorporating interpretability into latent factor models via fast influence analysis. In Proceedings of the 25th ACM SIGKDD Inter- national Conference Knowledge Discovery & Data Mining, pp. 885-893, 2019.
Gilad Cohen, Guillermo Sapiro, and Raja Giryes. Detecting adversarial samples using influence functions and nearest neighbors. In Proceedings the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2020.
Dennis Cook. Regression graphics: Ideas for studying regressions through graphics, volume 482.
John Wiley & Sons, 2009.
R Dennis Cook and Sanford Weisberg. Characterizations of empirical influence function for detecting influential cases in regression. Technometrics, 22(4):495-508, 1980.
R Dennis Cook, Bing Li, et al. Dimension reduction for conditional mean in regression. The Annals of Statistics, 30(2):45544 474, 2002.
Minghong Fang, Neil Zhenqiang Gong, and Jia Liu. Influence function based data poisoning attacks to top-n recommender systems. In Proceedings of The Web Conference 2020, pp. 3019-3025, 2020.
Ishaan Guirajani and David Lopez-Paz. In search of lost domain generalization. arXiv preprint 23X4 2020.
Pang Wei Koh and Percy Liang. Understanding black-box predictions via influence functions. arXiv preprint /003:/703.044 2017.
Pang Wei W Koh, Kai-Siang Ang, Hubert Teo, and Percy S Liang. On the accuracy of influence functions for measuring group effects. In Advances in Neural Information Processing Systems, pp. 5254-5264, 2019.
10

---

Under review as a conference paper at ICLR 2021
Masanori Koyama and Shoichiro Yamaguchi. Out-of-distribution generalization with maximal in- variant predictor. arXiv preprint arXiv:2008.0188 01883, 2020.
David Krueger, Ethan Caballero, fbern-Heniy Jacobsen, Amy Zhang, Jonathan Binas, Remi Le Priol, and Aaron Courville. preprint urXiv:2003.00666 2020.
generalization via risk extrapolation (rex). arXiv
Kun Kuang, Ruoxuan Xiong, Peng Cui, Susan Athey, and Bo Li. Stable prediction with model misspecification and agnostic distribution shift. In AAAI, pp. 4485-4492, 2020.
Da Li, Yongxin Yang, Yi-Zhe Song, and Timothy M Hospedales. Deeper, broader and artier domain generalization. Proceedings of the IEEE international conference on computer vision, pp. 5542-5550, 2017.
Haoliang Li, Sinno Jialin Pan, Shiqi Wang, and Alex C Kot. Domain generalization with adver- sarial feature learning. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 5400-5409, 2018
Sara Magliacane, Thijs van Ommen, Tom Claassen, Stephan Bongers, Philip Versteeg, and Joris M Mooij. Domain adaptation by using causal inference to predict invariant conditional distributions. In Advances in Neural Information Processing Systems, pp. 10846-10855 2018.
Sudipto Mukherjee, Himanshu Asmani, and Sreeram Kannan. Cemi: Classifier based conditional mutual information estimation. Uncertainty in Artificial Intelligence, pp. 1083-1093, PMLR, 2020.
Jonas Peters, Peter Bühlmann, and Nicolai Meinshausen. Causal inference by using invariant pre- diction: identification and confidence intervals. Journal of the Royal Statistical Society: Series B (Statistical Methodology). 5(78):947-1012 2016.
James Robins, Lingling Li, Eric Tchetgen, Aad van der Vaart, et al. Higher order influence functions and minimax estimation of nonlinear functionals. In Probability and statistics: essays in honor of David A. Freedman, pp. 335-421, Institute of Mathematical Statistics, 2008
James M Robins, Lingling Li, Rajarshi Mukherjee, Eric Tchetgen Tchetgen, Aad van der Vaart, et al. Minimax estimation of a functional on a structured high-dimensional model. The Annals of Statistics, 45(5):1951-1985 2017.
Mateo Rojas- Carulla, Bernhard Schölkopf, Richard Turner, and Jonas Peters. Invariant models for causal transfer learning. The Journal of Machine Learning Research, 19(1):109--1342, 2018.
Shiori Sagawa, Pang Wei Koh, Tatsunori B Hashimoto, and Percy Liang. Distributionally robust neural networks. In International Conference on Learning Representations, 2019.
Rajat Sen, Ananda Theertha Suresh, Karthikeyan Shanmugam, Alexandros G Dimakis. and Sanjay Shakkottai. Mowel-powered conditional independence test. In Advances neural information processing systems, pp. 2951-2961, 2017.
Adarsh Subbaswany, Peter Schulam, and Suchi Saria. Preventing failures due to dataset shift: Learning predictive models that transport. The 22nd International Conference on Artificial Intelligence and Statistics, pp. 1118-1127. PMLR, 2019.
Daniel Ting and Eric Brochu. Optimal subsampling with influence functions. In S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa-Bianchi, and R. Gar- nett (cds.), Advances in Neural Information Processing Systems 31. pp. 3650- 3659. Curran Associates, Inc.. 2018. URL http:/ /ttt:://paper.. nips cc/paper/ iopttima. ima 1- lggglmllllsaaammmgggg caath-inffuennne funct pdf
Antonio Torralba and Alexei A Efros. Unbiased look at dataset bias. In CVPR 2011, pp. 1521-1528. IEEE, 2011.
Anastasios Tsiatis. Semiparametric theory and missing data. Springer Science & Business Media, 2007.
11

---

Under review as a conference paper at ICLR 2021
Mark J Van der Laan, MJ Laan, and James M Robins. Unified methods for censored longitudinal data and causality. Springer Science & Business Media, 2003.
Aad W Van der Vaart. Asymptotic statistics, volume 3. Cambridge university press, 2000.
Vladimir Vapnik. Principles of risk minimization for learning theory. In Advances in neural infor- mation processing systems, pp. 831-838, 1992.
Yufei Wang, Haoliang Li, and Alex C Kot. Heterogeneous domain generalization via domain mixup- In ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Process- ing (ICASSP), pp. 3622-3626 IEEE, 2020.
Sewall Wright. Correlation and causation. J. agric. Res., 20:557-580, 1921.
Chuanlong Xie, Fei Chen, Yue Liu, and Zhenguo Li. Risk variance penalization: From distributional robustness to causality. arXiv preprint arXiv:2006.07544, 2020.
Minghao Xu, Jian Zhang, Bingbing Ni, Teng Li, Chengjie Wang, Qi Tian, and Wenjun Zhang. Adversarial domain adaptation with domain mixup. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 34, pp. 5502-6509. 2020.
Shen Yan, Huan Song, Nanxiang Li, Lincan Zou, and Liu Ren. Improve unsupervised domain adaptation with mixup training. arXiv preprint 0rXiv:2001.0077 2020.
A
APPENDIX
A.1 SIMPLE BAYESIAN NETWORK
In this section, we show that the model with better OOD accuracy achieves smaller Vyj0. We assume the data is generated from the following Bayesian network:
X, < NNO, O?), y W yy + 1),, X2 +
(8)
where X1, X2 E R are the features, y R is the target vector, W1 E R5x5 and W Wy-2 R5x5 are the underlying parameters that are invariant across domains. The variance of gaussian noise is domain. For simplicity,
of that depends here is linearly regress the response y on the input vector (2), i.c. ÿ = 21 W1 + x2 W. According to the Bayesian network (8). is the invariant feature, while the correlation between and y is spurious and unstable since de varies across domains. Clearly, the model based only on X is an invariant model. Any invariant estimator should achieve W, W and W₂ Rs
denote
d. represent domain. The goal
Table Average parameter error W ERM, IRM and REx. Here, "Causal Error" represents [W; Wi-yll and "Non-causal Error"
2 and the stable measurement Vzj0 500 models from
represents [23
Method Vy10
Causal Error
Non-causal Error
ERM
15.844
0.582
0.581
IRM
5.254
0.122
0.109
REx
1.341
0.042
0.033
Now consider five training domains E &,, {0.2, 1.2, each containing 1000 data points. We estimate three linear models using FRM. IRM and REx respectively and record the parameter error as well as V.,0 (note that 7 is here). Table 3 presents the results among 500 repetitions. As expected, IRM and REx learn more invariant relationships than ERM (smaller causal error) and better avoid non-causal variables (W₂ x 0). Furthermore, the proposed measurement Vy is highly related to invariance, i.e. model with better OOD property achieves smaller V This results coincides understanding.
12

---

Under review as a conference paper at ICLR 2021
A.2 PROOF OF EXAMPLE
In this section, we use simple model to illuminate the validity of Vaio proposed in Section Consider structural equation model (Wright (1921)):
X1 21 +N0,1, x2+++N(0,00)
where P is distribution with a finite second-order moment, i.e. Exf +oc, and o is the variance of the noise term in X2. Both P and o² vary across domains. For simplicity, we assume there are infinite training data points collected from two training domains & {(P2 of), (P o2)3 Our goal is to predict from == (x1; x2) using a least-squares predictor Here consider two algorithms: ERM and IRM with À toc. According to Arjovsky et al. 2019), using IRM we obtain 9IRM - (1,
T B == 21 + 12.22
Intuitively, ERM will exploit both and X2, thus achieving better regression model. However, since relationship between y and X2 varies across domains, our index will be huge in such condition. Conversely, FIRM only uses invariant features X1, thus Vy10 -o0. Note that we do not have an embedding model here, so Vz10 V/.
ERM we denote
((3)
Σ (x(
with (c( = E.(y x,3)
Etrl]
eEErr
Note that in Eg, X1 sample from P We then have
al(B)
2
Ex(xx xT
Ex[x2(y x []] )
Σ
Ex(x(y x 3]]
B
1Err]
cEE..
dEEr.
To proceed further, we denote
d
Ex
8
= + =
Exr
cEEtr
By solving the following equations:
Σ
Ee(x1 - x B)] = d(1 31 B2) =0
EEerl
eEEs.
and
1
Ex(x2(y x TB]] (d 1)(1 B B2)
8
32
Exrl nEter
113
we have @ = -(11 with
B
82
Now calculate our index. It is easy to see that
(g) = -2E (y - x 3)] = -2E,x
B1 32)
31
(e)
-2Ecc(x2y x []] -2 +1(1 31- -8)) 81 oz,80l
32
Therefore,
(o} 03) ) and (4(
V( (22
1lo 1(ai-0)) ++2
On the other hand, calculate the hessian and we have
2d
2d
2d 2d + s +2
-2d 2d
)
HERM- =
and H
2d
2d - s + 2
2d(s + +2
2d
13

---

Under review as a conference paper at ICLR 2021
Then have (note that -
Va = In(1Coveee (IF
= Im unl=1la (IFi IF2)
= Im( IFi - IIF2l13
2nn H -1-1-6 (VV(((3) VE2(3))1))
2d +
-2d
) 2d
2
4d(s + 2)
!1
-2d
4
I)
8+2
= 2m 2v - 03.
(s + 2)2
where the third equation holds because the rank of matrix 1. Clearly, when lo o.. (means two domains become identical), our index V/3 + -00. Otherwise, given + 02, we have Via -O, showing that ERM captures varied features.
IRM We now turn to IRM model and show that V/3 + -00 when À + toc, thus proving IRM learnt model ÕIRM does achieve smaller V/3 compared with , in ERM.
Under IRM model, assuming the tuning parameter is X, we have
((B) E
Σ
Eel(y x x31 + 4 [x Bly x 3]]1
eEEe
Then we have the gradient with respect B:
VC(3) =
Σ
(-
-E, - x []] SAE. /x Bly -x xx)) 2x 2x B)))
Errl
REEi
and the Hessian matrix
8A
== HERM +
(E.(x(y 2x Ex - 2x B)) 2x, 3 - x B. [xx T)).
337
(10)
Denote Bx the solution of IRM algorithm on &,r when penalty is X. From Arjovsky et al. (2019) we
know Bx + BIRM == (1,0) T To show limx-+te V3,, DC, we only need show that
lim H (VL((B)) ((2(3))) +00
We prove this by showing that
lim
H HA) (x)= 0
and
vl((3x) - V(2(3x)= = lim x
(1I)
simultaneously.
We add (A) after H to show that H is continuous function of A. Rewrite H
formula [0 as
H(A,BA) = HERM + AF(
where
F(3) =
Σ
(E.lx(y- - 2x B)]E,(x(y 2x TB]T - -2E,|x B(y - x [xx] T))
EErl
cEEir
Ex
) -Eex
lim B8 -PaM
F(Bx)
1- Exx? ) F(BIRM) exists.
ExP
1 - E.x?
cEEir
14

---

Under review as a conference paper at ICLR 2021
Obviously, ((BRR)) is positive definite. Therefore, we have
lim
(A.
lim lim [ERRM (F((3))]
A +0000
A Bg PIRM
lim [ERM AF( A
The first equation holds because lim F(Bx)= F(BR))) has the limit and is not and the last equation holds because the eigenvalue of H goes to +oo when X + to0.
Now consider (L1 V(2(3x).
According to formula 9 we have
lim V(((Bx) (12
lim By 7/BMM
((1 Vl2(BA)
A +
(BI(M))M) V(2(BIRM)
0 2,32 o32)
0
Hence we finish proof of formula 11 and show that V/3 -xi in IRM.
A.3 FORMULA 5
This section shows the derivation the expression 5 Recall that the training dataset $ (S) ...., } } and the objective function
((f.8)= ((, AR(SS), where the second term the right hand side is the regularization. As to ERM, the regularization term zero. With the feature extractor (3) fixed, we upweight a domain S The new objective function is
(- ((..8) 8 ((0,8")
Notice that when upweight an domain, we only upweight the empirical loss the corresponding domain. Further, we denote 7, 5 as the optimal solutions before and after upweighting domain. It is easy to see that 1 71 0 when 5 0. Following the derivation in Koh & Liang 2017 according to the first-order Taylor expansion of (0,, with respect to 7 on 7,
0 T)
= ()( + (. 50(0 (e( - 5) + +(ll - 71)
VV,((0,5))+3(((000 (e s ( -) a(1 - 411
Assume that [=(C(0,8) + 56 3 is invertible, have
a(I プ+ - II)
((0, (V ((,
8
lim 2+ -
Note that this derivation is not fully rigorous. Please refer to Van der Vaart (2000) for more rigorous discussions about influence function.
The reason that / should be fixed is as follows. First, if 3 can be varied, then the change of will become:
Hyr H4S H3g
V V+1(0,5")
Hon
Vol(O,,')
Therefore, the computational cost similar to calculate and inverse the whole hessian matrix. Most importantly, without fixing B, the change of 7 is somehow useless. Say when upweighting $ the use of a feature decreases. It's possible, however, that the parameter in r corresponding to the feature increases while 33 decreases a larger scale. In this case, the use of the feature decreases but increases. Without fixing B, the change of y calculated by influence function may provide no information about the feature. Therefore, argue that, fixing 3 is "double-win" choice.
15

---

Under review as a conference paper at ICLR 2021
A.4 ACCURACY IS NOT ENOUGH
In Introduction, we have given an example where test accuracy misleads us. In this section, we will first supplement some examples where test accuracy not only misjudge different algorithms, but also misjudges the OOD property of models learnt with different penalty within the same algorithm. After that, we will show the universality of these problems and why test accuracy fails.
IRM with pa 0.2
REx with p 0.2
08
OOD Accuracy
OOD Accuracy
0.3
ERM
ERM
IMM50
REX50
IRM100
RCx100
RMM000
REX300
IR10000
REx1000
0.00
0.75
0%
0.58
0.30
0.25
cate
0.45
3150
6.76
0.33
Test Acturacy
Test Accuracy
IRM with past
0.3
REx with past
= 0.3
0OD Accuracy
6.
ODD Accuracy
0.4
6.
ERM
03
ERM
IR550
REx50
82
IRM100
02
REx100
RM5000
REx500
RM10000
REX1000
6.4
3.23
0.22
Accuracy
Test Accuracy
Figure 5: Experiments in Colored MNIST to show test accuracy (X-axis) cannot be used to judge model learnt with different penalty. Consider two test domains with pisst = 0.2 (up penals) and plest 0.3 (down penals). For each A. we run IRM and REx 500 times. We can that when À increases from = 0 to À = 1000, the OOD accuracy also increases, but test accuracy does not. When prest 0.3, their relationship becomes more perplexed.
Consider two training domains
Df E (0.0,0.1}.
and a test domain with flip rate denoted by plest We implement IRM and REx with penalty A {0, 50, 100, 500, 1000} to check the relationship between test accuracy and OOD accuracy. The training process is identical to the experiment in section 5.1.2 As results showed in Figure when OOD property of model gradually improves (caused by gradually increasing A), its relationship with test accuracy is either completely (when plest is 0.2) or partly (when ylu is 0.3) negatively correlated. This phenomenon reveals the weakness of test accuracy. If one wants to select a À when prss is 0.3, judged by test accuracy, À 50 may be the best choice, no matter in IRM or REx. However, the model learnt with X = 50 has OOD accuracy even less than a random guess model.
Whether test accuracy is positively, negatively correlated or irrelevant model's OOD property mainly depends on the "distance" between test domain and the "worst" domain for the model. If test accuracy happens to be the lowest among all the domains, we directly have OOD accuracy equals to test accuracy. In practice, however, their distance may be huge, and this is precisely the difficulty of OOD generalization. For example, we are accessible to images of cows in grasslands, woods and forests, but cows in desert are rare. At this point, the "worst" domain is certainly far from what we can get. If we expect a model to capture the real feature of cows, the model should avoid any usage of background color. However, model based color will perform consistently well (better than any OOD model) no matter in grasslands, woods and forests since all of the available domains are green background in general. In Colored MNIST, test accuracy fails in the same way.
16

---

Under review as a conference paper at ICLR 2021
Such situations quite common. Generally, within domains have, there may be some features that are strongly correlated to the prediction but are slightly varied across domains. These features spurious, given that their relationship with prediction significantly disparate in other domains to which we want to generalize. However, using these features in prediction will easily achieve high test accuracy. Consequently, it will be extremely risky to judge models merely by test accuracy.
A.5 CONDITIONAL MUTUAL INFORMATION
A possible alternative of Va10 may be Conditional Mutual Information (CMJ), For three continuous random variables X, Z, the CMI is defined
[xX;)|Z)= /
p(z, 2) log p(z, y, 2) p(a, =)p(ulz) dxdydz
(12)
where p(.) is the probability density function. Consider [e; (x)) or [(e; ylū), i.e. the mutual information of and true label y. given the features or the prediction ý of The insight is that, the model is invariant across different domains, then little information about e should be contained y given (xx). Otherwise, if the prediction ÿ is highly correlated e. then the mutual information will be high.
REx with 5 Environments
IRM with 5 Environments
0.7
0OD Accuracy
0.5-
OOD Accuracy
6.5-
0.4
0.3
0-3
ERM
ERM
REx10
IRM10
0.1-
REx100
6.1
IRM100
REx1000
IRM10C0
True CMI
True CMI
REX with 2 Environments
IRM with Environments
0.8
0.8
OOD Accuracy
OOD Accuracy
0.5-
6.5
3.
0.4
0.4
0.3
6.3
ERM
ERM
REX10
IRM10
REx100
IRM100
RFx1000
IRM1000
0.0
CCIT
CCIT
Figure 6: Experiments of the relationship between OOD accuracy and CMI (true or estimated using the method in (Sen al. (017)). Models trained by REx (left) and IRM (right) with À (0, 10, 100, 1000]. We train 50 models for each À and calculate the true CMI yl0) or CCIT value. As analyzed in the appendix A.5] true CMI enjoys a highly correlated relationship to OOD accuracy, with Pearson Coefficient -0. -0.9923 (left) and -0.9858 (right). However, the estimated value shows a completely different picture, with Pearson Coefficient -0.0768 (left) and -0.1193 (right).
This metric seems to be promising. However, the numerical estimation of CMI remains challenge. To this end, previous works have done a lot to solve this problem, including CCMI proposed in Mukherjee al. (2000) and CCIT proposed in Sen al. (2017). In this part, will first calculate
17

---

Under review as a conference paper at ICLR 2021
true [(e; ylū) in simple Colored MNIST experiment to show that if there is estimation problem, CMI could be a potential metric to judge the OOD property of the learnt model, at least in simple, discrete task. We then run the code provided by [Sen et al. (2017) (https: //gi thub. com/ rajatssen 91 /CCIT) to show that even in this simple task, the estimation of CMI may severely influence its performance.
Specifically, the experimental setting is similar to that in subsection 5.1.2 with two OOD algorithm and number of training domains in {2,5). 5). For each algorithm, we consider the penalty weight À E {0, 10, 100, 1000), run the algorithm 50 times, and record their OOD accuracy as well as true CMI value or CCIT value. The results are shown in Figure 6 We can see that in the case when true CMI can be easily calculated, especially in the case when the number of domains is small and the task is discrete (not continuous), CMI highly correlated to OOD accuracy. However, in regression task or in a task when directly calculating the value of CMI becomes impractical, the estimation process may severely destroy the correlation, and may also result an inverse correlation. Therefore, we summarized the estimation of CMI has limited its utility. We leave the line-graine analysis of the relationship between CMI, estimated CMI and OOD property to future works.
A.6 RESULTS ON VLCS
A.6.1 CONTINUED SCENARIO
This is continuation of section 5.2 Say in this task, Eall remains the four domains but EE = [L. V3 (empirically we find it more diverse). Similarly, we start with test accuracy shown in table
Table 4: Domain C' out:
Test Accuracy (%)
Domain
L
S
V
Mean
this step, the situation is the same, i.e. IRM should be eliminated un- til proper hyper-parameters are found. step 2, we show the comparison between V/1 and
ERM
73.43
73.87
79.15
75.48
Mixup
73.74
74.54
78.65
75.64
gDRO
71.40
71.95
77.19
73.51
IRM
49.61
38.64
45.35
44.53
Vyi of three algorithms in Figure 7 As we can see, this time the two value are Similar for all three algorithms, including gDRO. This is different from the case when S unseen. this case, predict that all of the three algorithms should achieve high OOD accuracy. In fact, if act as the oracle and calculate their OOD performance, we will find that our judgement is close to the reality: ERM, Mixup and gDRO achieve OOD accuracy from 70.55% to 72.87%. According the confidence interval, they difference are not satistically significant. As for IRM, the OOD accuracy is 38.6 18.64% One who use ERM, Mixup gDRO should be satisfied for the performance, since higher demand is somehow impractical!
ERM
Mixup
Group DRO
0.6
2.8
6.3
00D Accuracy
Accuracy
000 Accuracy
2.
2.2
ERM Shuffle
Mixup
Group DRO
Shuffie
Shuffle
100
125
Figure 7: The standard and shuffle version of the metric, i.e. Vyi and Dyy for ERM, Mixup and gDRO. This time, all three algorithms show similar Vyya and Vy1
A.6.2 FULL RESULTS AND COMPARISON WITH IRM PENALTY
As mentioned in Section 5.2 we consider ERM, gDRO, Mixup and IRM on VLCS image dataset.
We report the full results here, and compare the performance of out metric Vyyo with IRM penalty in
18

---

Under review as a conference paper at ICLR 2021
formula Thorough the whole experiments, Eau- LV, We construct four experimental settings. In each setting, one domain is removed and the rest consists of Etr. For each domain in Exr, we split validation set, and test accuracy is the average accuracy amount validation sets. The results are shown in Table 5 First, our results coincide with Gulrajani & Lopez-Paz, (2020) that ERM nearly outperforms any algorithms. We see that the OO1) accuracy of ERM is either the highest or only slightly lower than Mixup. Meanwhile, it has relatively small V-jo. Second, higher OOD accuracy corresponds to lower V-10. In addition, we notice that IRM has relatively low test accuracy and OOD accuracy. We explain the phenomenon by an improper Slype--paameee IRM, although we didn't 't change the default hyper- paramerrr the code of Gulrajani & Lopez-Paz com/
in
nBed). No matter what, this phenomenon provides good example which we can compare our metric with IRM penalty and discuss their advantages and disadvantages.
(2020) hhttps://itte
Table 5: Experiments in VLCS with 4 algorithms. OOD accuracy means the min accuracy in Eott. We use training-domain validation method mentioned in Gulrajani & Lopez-Paz (2020), so test accuracy is the average accuracy of three split validation set. "Domain" means which domain is excluded, i.e. which domain in Eall' er, In each setting, we run each algorithm 12 times and report the mean and (std). Note that in real implementation, IRM penalty can be negative.
OOD accuracy (%)
Test accuracy (%)
Domain
C
S
Domain
C
S
ERM
72.54
61.48
62.76
65.59
ERM
75.48
84.55
83.33
81.49
(2.62)
(2.31)
(1.16)
(2.27)
(3.37)
(10.61)
(11.64)
(12.83)
Mixup
72.87
62.10
63.91
63.81
Mixup
75.65
84.92
84.17
81.02
(2.04)
(3.10)
(1.57)
(3.64)
(2.80)
(10.54)
(11.07)
(13.52)
gDRO
70.55
61.64
60.17
62.35
gDRO
73,51
82.82
80.66
80.03
(1.91)
(3,92)
(2.56)
(2.11)
(3.27)
(9.41)
(11.17)
(11.50)
IRM
38.64
38.84
31.33
39.50
IRM
44.53
48.83
45.13
51.94
(0.54)
(0.31)
(13.44)
(2.35)
(4.94)
(10.53)
(16.53)
(11.66)
Vyjo (our metric)
IRM penalty (e-4)
Domain
S
Domain
C
S
V
ERM
2.0468
1.9084
1.8476
1.9811
ERM
1.78
1.48
1.43
1.63
(0.3474)
(0.3231)
(0.2887)
(0.3955)
(1.88)
(1.03)
(0.75)
(2.04)
Mixup
2.6996
2.4417
2.5810
2.8780
Mixup
75.4
57.7
65.5
48.3
(0.1926)
(0.2003)
(0.1492)
(0,2304)
(32.6)
(26.4)
(37.2)
(30.6)
gDRO
3,3371
4,8520
5.0915
5.1675
gDRO
9.42
2.13
2.6
1.94
(0.1385)
(0,2515)
(0.278)
(0,3507)
(10.1)
(3.41)
(2.46)
(4.37)
IRM
8.1820
6,8329
7.6234
8.1288
IRM
2.59
0.96
0
2.71
(0.9523)
(0.6646)
(0.6792)
(0.974)
(3.31)
(3.31)
(4.77)
(9.2)
Despite that IRM could be good OOD algorithm, using IRM penalty as the metric to judge the OOD property of a learnt model still has much weakness, and some are severe. First, in different tasks, the value of À obtain an OOD model may be different, as other hyper-parameters like "anneal_steps" in IRM code. Without exhaustive search on the proper value of the hyper- parameters, it's easy that IRM overfits on the penalty term (which the situation in VLCS). When IRM overfits, the IRM penalty will become quite small (higher À often leads to smaller penalty), but absolutely overfitting on penalty term will not result good OOD accuracy. Therefore, the balance between loss and penalty is important. However, how to find balanced point? This is a model selection problem, and Gulrajani & Lopez-Paz (2020) propose that an OOD algorithm without model selec- tion is not complete. No matter what to be used as the metric, it cannot be IRM penalty since we cannot use what is included in the training process as the metric to select training hyper papperr-armmterrs...
Second, IRM penalty shows bias different algorithms. In the Table 5 the IRM penalty of IRM smaller than most algorithms. Besides, although the OOD accuracy of Mixup is similar to ERM, its IRM penalty is significantly higher. This is not strange but will limit the usage of IRM penalty. As for our metric, we mention that small Vy is better. However, the understanding of "smallness" is based on the relative value of the shuffle version and standard version of V-10- As mentioned in section 5.2 when Eau Ver smaller than standard version in gDRO, but in ERM and Mixup, these value are relatively close or indistinguishable. In this case, know that gDRO captures less invariant features and not OOD
{S). we
that shuffle section 5.2 is obviously
19

---

Under review as a conference paper at ICLR 2021
than the other two algorithms. During the whole process, cireumvent the direct comparison of Vyio in different algorithms, which is quite important. In summary, IRM penalty makes IRM good algorithm, but using it as the general metric of OOD performance is completely another picture.
20


Reviews:
[{"review_id": "h-Toz1LLNx", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 7, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Summary:\n     This paper tries to understand the following question - a) Given a set of environments - when would we expect an ERM solution to be completely bad in the out of distribution (OOD) generalization sense. b) Given a trained model, how can we certify that the model is good in an OOD sense when we dont have access to a new environment. \n\nThis metric might be useful to tune IRM procedures which has been identified as a real issue in some recent work.\n\nIRM optimizes over representation $\\phi$ and a classifier on top of it $\\gamma \\approx E[Y| \\phi(X)]$ is remains the same across all environments. \n\n The metric proposed: Influence function which measures the change in $\\gamma, \\Delta \\gamma $ caused by upweighting an environment $e$'s loss by $\\delta$. The metric assumes that the IRM optimization is done using $\\sum_e L(\\gamma,\\phi, S^{e}) + \\lambda R (f, S^{e})$ where $S^{e}$ represents the dataset for environment $e$.\n\nThe limiting influence function when the weighing factor $\\delta$ tends to $0$ is given by a inverse of a Hessian matrix of the regularized loss over all data  (whose changes are measured with respect to the classifier $\\gamma$) and a gradient vector of the loss (without the regularizer) with respect to the $\\gamma$ on the data of the environment which is up weighted.\n\nNow, log of the spectral norm of the cross covariance matrix when changes are made to the different environments involving these influence measures is used as a metric.\n\nTo test a model: One has to check if this derived measure is low and test accuracy is high (test set made from training envrionments). If so then, that model can be expected to have a high OOD generalization to unseen environments.\n\nTo test environments if they need special OOD optimization frameworks: The authors propose to a) randomly create m environments, and b) use the pooled data and train an ERM model for both. If the metric proposed in b) is larger than the metric in a) across computed using the actual environments, the authors propose that one should check for an algorithm (like IRM) that is specifically tailored towards OOD solutions. \n\nFour experimental results :\n\na) For a simple Bayesian network X_1-> Y->X_2  with linear models and different environments change the Y|X_2 part but not the Y|X_1 part the authors compare REx, IRM and ERM and show that their metric predicts better performance with respect to the causal error and non causal error (measured by presence of just any coefficient on X_2).\n\nb) For Colored MNIST the authors show that when the environments have a varying color-label association , then their metric to test environments is highly correlated with the diversity parameter.\n\nc) Similarly, they compare IRM and REx with different lambda parameters to show that the models with the smallest metric does better in terms of OOD accuracy.\n\nd) For the simpler Linear bayesian network case, they show that for the optimal ERM classifier, only when the environments become identical their metric becomes small. While for IRM, when lambda (regularizer imposing invariance) is large their metric goes to - infty.\n\n\nStrength:\n\nI feel the metric proposed is novel for an important problem of determining the need for IRM and/or tuning IRM based frameworks where new envrionments have not been seen. \n\nThis has been very nicely demonstrated for both simple linear Bayesian networks (also proven for them) and Colored MNIST.\n\nWeaknesses:\n\n1) When authors says we test if the environments are OOD - they dont precisely define it. Only in the colored MNIST experiments this becomes clear. But one has to be very precise about what the authors mean by \"testing if environments are OOD\" - Does it simply mean one is checking if anything else would do better than ERM in terms of OOD accuracy? \n\nI feel this claim is a bit imprecise. However, due to the colored MNIST demonstration of variation spurious factors being correlated with their metric - I am not very inclined to penalize them in the scores.\n\n\n\n2) [My major concern] Another metric to test if a model captures invariance is simply to compute conditional mutual information (CMI) between $e$ ( the environment label) and $Y$ given $\\phi(X)$ (the learnt representation). A good place to get a code that computes a good lower bound to this is https://arxiv.org/abs/1906.01824. Their code is public. \n\nCan the authors plot their metric versus this for example for Figure 3.  Because I would like to understand why just directly computing conditional mutual information would be different from the proposed metric. \n\n\n3) Lots of typos \"..when inversing Hessian..\", \"...unstable inversing...\" in Page 8. \n\nI have conditionally given 7 - However some comments, comparison to conditional mutual information would be more useful (Refer to my point 2 in the weakness)\n\n\n\n", "extra_scores": {}, "extra_text": {}}, {"review_id": "CJEJRxXcIXj", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "**OBJECTIVE**: Develop a method to reliably determine 1) if a model will generalize to out-of-distribution (OOD) samples 2) if it is advantageous to use algorithms that explicitly promote OOD generalization (in place of empirical risk minimization). \n\n**CLAIMS**: \n1)Test error can be a poor indication of OOD generalization. \n2)Model stability across training environments (in the sense of “cross validation” between training environments) is a good indicator of OOD generalization. \n3)The proposed index that uses influence functions to estimate how sensitive the model is to removing training environments does a good job in predicting how robust the model is against OOD samples. \n\n**MAIN CONTRIBUTION**: The authors propose an influence function-based index to determine how well the model will generalize to OOD samples and whether OOD methods are needed in the first place. \n\n(Note: PAIR stands for “please address in rebuttal”)\n\n**STRONG POINTS**:\n**Valuable line of inquiry**: The paper is on a very important topic that will be relevant to many in the ICLR community.\n**Valuable connection**: The idea of utilizing influence functions to analyze OOD properties of machine learning models is, I believe, novel and interesting. \n**More efficient than cross validation across training environments**: Assuming that influence functions are accurate enough to estimate the effect of removing training environments, the proposed method can significantly speed up the process of “running cross validation across different training environments”. Also, running the influence function computation on only the top model (i.e. the classifier) makes sense, and can significantly speed up computation – making the model feasible on even potentially large-scale tasks. \n\n\n**WEAK POINTS**:\n**Experiments**: Given that the main contribution is a heuristic measure of OOD generalization, I believe the experiments section is particularly important to prove that the proposed index is indeed useful for sufficiently complex, realistic problems (if not large scale). I believe the experiments are lacking in several ways: \n1.\t**Lack of different learning objectives**: There are many methods described in the related works section for learning models that generalize to OOD samples. However, only IRM and REX are used in the experiments.   I consider, in particular, the lack of “the robust learning objective” important. This seems particularly important to me, as I expect the proposed index to favor models trained with the robust learning objective that actually might not generalize to OOD samples. (PAIR) \n2.\t **Lack of benchmark OOD generalization diagnostics**: Are there other papers out there that try to quantify how well a model will generalize to OOD samples? How does the model compare against those? For example, the paper “In Search of Lost Domain Generalization” by Gulrajani and Lopez-Paz (2020)[1] (which is not cited), and its related works section, seems quite relevant to this paper and probably should be considered by the authors – both in terms of which benchmark datasets are suitable, and which model selection criteria to use. (PAIR, if you have the time and resources) It would also improve the paper if the authors could try actually running some kind of “cross validation across training environments” (which the proposed approach is, as far as I understand, aiming to more or less capture more efficiently) to see how well it compares with the proposed approach? (PAIR). The authors also note that the proposed approach can be used to see if different training environments are actually different enough to require OOD generalization approaches for training. Couldn’t one simply check this by training a classifier to predict which training environment each example belongs to? If the classifier achieves random accuracy, one probably doesn’t need anything more than ERM. Am I right by saying this? (PAIR)\n3.\t**Lack of more realistic datasets**: All of the experiments in the paper are run on highly synthetic and unrealistic datasets. Although these experiments are still extremely helpful, it would improve the paper drastically if there were experiments run on more realistic datasets. Again, In Search of Lost Domain Generalization” by Gulrajani and Lopez-Paz (2020) seems like a valuable reference.  (PAIR if you have the time and resources)\n\n**Faithfulness of influence functions**: Influence functions, in essence, try to estimate how much the weights would be perturbed if the training set didn’t have particular training examples (or environments in this case). The experiments reported in the paper are small-scale enough that one can likely afford to actually remove particular environments, retrain, then compare the weights with what’s predicted by the influence function. Did the authors run such experiments? (PAIR) I realize that the authors cite Koh (2019) that shows influence functions can still be useful to predict the effect of large number of training examples, but I believe the current setting is different enough that it would be worthwhile to repeat such experiments. \n\n**Lack of justification for the particular form of the index**: It’s not that clear from the paper why the authors picked the particular form of the proposed index. Did the authors consider other forms? For example, I wonder if one could alternatively use the gradient of the excluded training environment with respect to the perturbation (which can be efficiently computed using influence functions) to derive an alternative index to compete with the current one. \n\n**Is test accuracy really that problematic?**: I agree that without plenty of test environments, simply relying on test accuracy is problematic, as the authors point out with a concrete example. However, couldn't a sufficiently rich test set be seen as the gold standard to test OOD generalization, as also expressed in [1]?\n\n**Missing related work**: I believe [1] is important to cite in this paper. \n\n**DECISION**: Weak Reject. \nMain reason: While I find the paper interesting and the line of inquiry promising, I believe it is not yet ready for publication, especially due to the weakness of the experiments section. With a better selection of baselines and benchmarks (both in terms of model selection criteria, more OOD generalization approaches (like robust learning objective) and more justification as to why the current form of the index is selected, the paper, in my opinion, can be ready for publication. \n\n**QUESTIONS TO AUTHORS**: In addition to the sentences I’ve marked with the acronym “PAIR”, I have the additional questions:\n1.\tWhat do you mean by “same model change” in the last paragraph of page 4? Lets say I combine MNIST and CIFAR10 images to create a 20-way classification dataset, and I’m viewing MNIST and CIFAR10 as different training environments. Why should the classification model respond the same way to perturbations made to the different environments in this case?\n2.\tJust to clarify: does “2-norm for matrix” refer to the Frobenius norm, or actually the operator 2-norm?\n\n\n**MINOR POINTS**: (these didn’t play much role in my decision)\nUnpolished writing: I‘d recommend the authors to carefully go through the paper to remove as many grammar errors and typos as possible. That being said, the existence of these errors didn’t make it harder for me to read the paper, hence didn’t contribute to my final decision (at least consciously). \n1. You might also consider restructuring the paper to make the introduction less meaty, and dedicate a section to discuss why test accuracy is limited. \n2. You might want to describe what influence functions are verbally first, before giving a formula for it. \n3. In section 3.2, you use both IF and I to mention influence functions, as far as I can see. \n4. It would be nice if Table 1 refers to Section 5.1. \n\n[1] In Search of Lost Domain Generalization, Ishaan Gulrajani and David Lopez-Paz∗  2020 \n\n", "extra_scores": {}, "extra_text": {}}, {"review_id": "36l64vNnjXK", "reviewer": "AnonReviewer4", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "\n\n=================================\n\n**Update after discussion period**\n\nMy feeling is still that the proposed method has a lot of interesting potential, but that the paper still needs some improvement. I've bulleted my main remaining concerns below:\n\n1. The clarity of the writing still needs to be improved in many places. Notably, the authors' discussion of their metric is mostly discussion-based (rather than providing concrete theoretical claims about their metric and its strengths). While this isn't necessarily a problem, such a presentation really needs precise language and phrasing so that the details of the claims and their supporting arguments can be completely understood. From what I can follow, I think many of the authors' arguments are going in good directions, but the writing should be improved to be sure.\n2. I still don't completely understand why $\\beta$ is being fixed here. The argument in Appendix A.3 seems to say that if $\\hat\\beta$ varies, then the meaning of $\\hat\\gamma$ changes, and so looking at the change in $\\gamma$ would not be meaningful. I agree, but that does not mean we cannot look at the combined change of $\\beta$ and $\\gamma$. It seems like this would be straightforward to experiment with on some smaller problems that only have parameters in the tens of thousands; I think it would be better to include such experiments rather than trying to argue verbally that such an approach will not work well.\n3. I am also not sure I see why we should not be interested in leave-one-domain-out CV. I appreciate the clarification that Figure 1 is actually showing CV's estimate of error, and definitely find this to be a compelling experiment. It is pretty surprising to me that the authors' metric could succeed in this case, given that the influence functions are an approximation to the parameter change as each domain is left out (this is true, whether or not this is the intent of the metric). I think further investigation of this point is needed. Would computing the proposed metric with the parameter changes under leave-one-domain-out CV not detect the OOD issues in Figure 1? Or is it just that feeding these parameter changes into the function measuring test error that is providing a poor assessment of OOD generalization in Figure 1?\n\nOn a different note, I appreciated the increased discussion of \"the shuffle;\" I think this is an important part of the paper that didn't come up much during the discussion period. As a side note, I think it is more common to just call this sort of thing a permutation test. And, along those lines, it would be good to actually perform a permutation test (i.e. run over many shuffles and examine the distribution) so that we know for sure the reported shuffles aren't just unluckily high/low.\n\n===================================\n\n**Original Review**\n\nThis paper is focused on understanding whether a model fitted model is going to have out-of-distribution issues. In particular, the authors consider models trained on multiple \"environments,\" which may fail to generalize to future environments. To detect whether a model will fail in this way, the authors propose to look at the change in model parameters as each environment in the training set is infinitesimally upweighted (i.e., they compute influence functions). If upweighting each environment the same amount changes the model by a similar amount, the authors reason that the model must not be overly fit to any one environment, and so will generalize to future environments well.\n\n\nI think a major issue is the derivation of Eq. (4), which claims to be the \"change of top model g caused by upweighting the environment.\" I'm interpreting this as the change in the parameters $\\gamma$, which are part of the overall parameter vector, $\\theta = (\\gamma, \\beta)$. My understanding of how to calculate influence functions is that the influence function for the overall parameter $\\theta$ would be:\n$$\n    -\n   \\begin{pmatrix}\n            H_{\\gamma\\gamma} & H_{\\beta\\gamma} \\\\\\\\\n            h_{\\gamma\\beta} & H_{\\beta\\beta}\n    \\end{pmatrix}^{-1}\n     \\begin{pmatrix}\n          \\nabla_\\gamma f \\\\\\\\\n          \\nabla_\\beta f\n     \\end{pmatrix}.\n$$\nIf $\\gamma \\in \\mathbb{R}^D$, I would say that the (infinitesimal) change in $\\gamma$ is really the first $D$ entries of this vector. But this is not what is written in Eq. (4), and the derivation of Eq. (4) in Appendix A.2 does not actually derive Eq. (4) (the derivation in the Appendix derives the equation written above and does not refer to $\\gamma$). I think this is especially important, as the reason that Eq. (4) is computable is that the matrix to be inverted is $D \\times D$, rather than the much larger $dim(\\theta) \\times dim(\\theta)$ in the equation I've written above. Can the authors clarify what's going on here?\n\nThe clarity of the writing is also an issue. Many sentences have grammatical issues (e.g. \"...according to detailed derivation in Appendix A.2, the change of top model g...\" is missing the word \"the\" twice). Most of these don't make the paper any harder to understand, but should still be fixed. I did have trouble following some of the more expositional parts of the paper. For example, the following sentences weren't clear to me:\n\n>First, not all OOD problems demand models to learn invariant\nfeatures, e.g. when varying features are always more strongly correlated to the prediction. But to\nour concern, we regard the OOD problem as a bridge to a truly causal model, so these minor ”OOD”\nproblems are out of our consideration. To a large extent, invariant features are still the major target\nand our proposal is still a good criterion to model’s OOD property.\n\nCould the authors provide more details about what their example in the first sentence refers to (is it a counterexample to a specific definition of an \"OOD problem\")? What is a \"truly causal model\" (one that only uses features that have a causal effect on the outcome)? Is this a different goal than building a model that uses the \"invariant features\" discussed in the third sentence?\n\nA final issue is that the authors' method seems close to cross-validation (CV), where each fold of CV leaves out an environment, but this connection isn't discussed. On the surface, it seems like an even better way to understand how much a model is depending on an environment is to actually remove the environment from the training data, rather than just changing the weight of the environment by an infinitesimal amount. Could the authors describe why their method is more desirable than CV in this context? If the issue is computation time, note that influence functions have been used to approximate CV [Giordano et. al 2019].\n\n----------------References---------------------\nR. Giordano, W. T. Stephenson, R. Liu, M. I. Jordan and T. Broderick. A Swiss army infinitesimal jackknife. AISTATS. 2019.\n", "extra_scores": {}, "extra_text": {}}, {"review_id": "2F_tT5RRwvN", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Summary\n\nThe authors study the problem of out-of-distribution (OoD) generalization. The key question authors seek to answer is when given access to data from multiple training environments, can one only rely on test accuracy? or does one have to rely on some new measures to estimate the out-of-distribution performance of the model. The authors develop a metric based on influence functions, which authors claim is a better reflection of OoD accuracy than test accuracy. The metric proposed by the authors measures the variance in the model when the data from each environment is upweighted. The authors show that the proposed metric empirically correlates to the OoD performance of the models.\n\nPros:\nI appreciate the problem that the authors consider. It is important to develop metrics that are good indicators of OoD performance since accuracy on train environments is not a good indicator. \n\nCons: \nI divide my concerns into different subsections below:\n\na)  IRM Arjovsky et al. (2019) already solve the problem that authors want to solve in this work: \nThe authors claim that they show that how test accuracy is not a good metric and highlight the need for a new metric to measure OoD performance. IRM (Arjovsky et al.) have already established this point. Recall the colored MNIST experiment in Arjovsky et al., which has precisely already shown accuracy over any training environment does not suffice in selecting a predictor with good OoD behavior.\n \nIf not accuracy, then what is the right metric? The authors in the current paper propose an influence function-based metric but completely ignore the metric that is already used in IRM to inform the learning of the OoD model. After all IRM is able to learn a model based on training environments itself, which means the IRM loss itself can be thought of as a proxy for the OoD performance. Recall that the loss used in IRMv1 in Arjovsky et al. balances the accuracy over the training environments and invariance achieved across all of them. \n\nLet us now precisely state a metric based on the work of Arjovsky et al.  We perfom the following steps:\n\ni)      Split the data in each environment into a train split and a validation split. Learn the model using the train split from each \nenvironment.\n\nii)     Use the trained model and compute the IRMv1’s penalty value on the validation split for each environment \n\niii)\tCompute the mean and the variance of the penalty value across environments. \n\nWe now discuss why the above metric can capture the OoD performance better than just training accuracy for instance.\nThe penalty in IRMv1 checks if the predictor learned satisfies invariance conditions across all the environments. The penalty by definition is greater than zero. If the penalty is close to zero for all the environments, then the model satisfies invariance across environments. \n\nConsider two models: one trained from ERM and other trained from IRM. Assume that they have a small average validation error across all the environments. Let us analyze the following scenarios:\n\ni)\tSuppose the mean of the penalty value across the environments is close to zero for the IRM model. Suppose the mean of the penalty value across the environments is much larger than zero for the ERM model. In such a case, the IRM model satisfies invariance and will perform better than ERM model on OoD data. \n\nii)\tSuppose the mean of the penalty value across the environments is close to zero for the IRM and the ERM model. In such a case, it is likely that both ERM and IRM models have a good performance on OoD data. \n\niii)\tSuppose the mean of the penalty value across the environments is similar for both the IRM and the ERM model but is greater than zero. In such a case we cannot rely on the mean, we should use the variance in the penalty value across environments. The model with a lower variance is likely to have a better OoD performance. \n\n\nb)\tIncorrect use of IRM loss to compute influence\n\nThe authors define in equation (3) a regularized loss function. Following equation (3) the authors define two types of regularizers from IRM and REx. The regularizer for IRM is incorrectly defined as \\sum_{e}(\\|\\nabla_{w}\\ell(g(w \\Phi), S_e)\\|)^2. The correct regularizer is\n\\sum_{e}(\\|\\nabla_{w}\\ell(w \\Phi, S_e)\\|)^2. There is no function g needed as the scalar w serves the purpose for the classifier that operates on top of the representation. Subsequently everything authors do with computing changes on g is incorrect as g itself is not needed.  \n\n\nc)\tConcern regarding influence computation in REx model\n\nThe authors divide the predictor into a representation and a classifier on top. However, it is not clear why would one do that for REx. REx uses the entire predictor as one. Hence, breaking down the predictor in the context of REx and analyzing the influence on the classifier does not make sense. There needs to be a better justification for doing what authors seem to be doing.  \n\n\n\nd)   Influence metric should be defined properly\n\nThe authors want to analyze influence of all the points in a given environment. The authors suggest upweighting the points but for reasons not very well explained the authors decide against upweighting the points inside the penalty. The whole principle of computing influence of a point is to upweight that  point in the entire loss and then determine how does the trained model change. The current approach decides only to upweight the part concerned with ERM loss and not the regularization. \nThe penalty is a crucial part of IRM and not upweighting it seems to against the spirit of understanding the impact of each environment on the model.\n\ne) What is the correct way of computing influence of a each environment? \n\n\nI would suggest the authors to follow the standard steps used in Koh et al. and upweight the points in an environment in both the ERM part of the loss and the regularizer. Instead of analyzing the impact on some model g, the authors should see the impact on the model phi itself. Analyzing the variance of change in the model phi may be correlated with OoD. However, this remains to be seen.\n\nf) Concern with using influence at all\n\nThe main reason to use influence functions is the computational advantage offered. When we do not want to retrain the model for every point we want to analyze, it makes sense to compute the influence. However, the authors are computing the influence per environment. In general, we do not expect too many environments, thus why not just retrain the model after upweighting the points in an environment. The only argument that can be made in favor of deriving influence in this case is if the authors subsample from the environments and determine the distribution of influence values. However, authors have not discussed any such justifications. \n\nRecall from Koh et al., the analysis of influence requires that the number of points being upweighted be much smaller than the total number of points.  An entire environment can constitute a significant proportion of the total data and changing all the points in an environment can lead to severe miscalculation of influence. The authors should have shown through a plot similar to Figure 2 in Koh et al. that computing influence of an environment vs. actually retraining the model lead to similar values. \n\ng) Concerns regarding experiments\n\nIf the authors believe that influence based metrics do lead to a better estimate of OoD performance, then they should have at least compared with metrics based on IRM penalty that I describe above and also REx penalty. All the IRM based methods rely on some type of signal from the training data to train a good OoD model. \n\nQuality: \nUnfortunately, the paper is of poor quality. The authors have not carefully analyzed how existing works already address this problem. \n\nClarity: \nThe writing of the paper is average. An explanation of how influence is computed for REx vs IRM vs ERM should have been clearly stated and if they used the same technique, then an explanation of why should have been there too. \n\nSignificance:\nThe problem authors are trying, i.e., building metrics that are better indicators of OoD performance  than training accuracy, is very important. However, the approach taken by the authors is incorrect and ignores many important comparisons. \n\n\n\n\n\n\n", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper considers the question of identifying whether a model is bad from an OOD perspective or certifying that it is good. The reviews agree that there are interesting ideas in the paper, however, lack of sufficient experiments and presentation issues were pointed out which make the paper not ready for acceptance at this stage.
Decision: Reject