O UT - OF -D ISTRIBUTION G ENERALIZATION A NALYSIS VIA I NFLUENCE F UNCTION
Anonymous authors
Paper under double-blind review
A BSTRACT
The mismatch between training and target data is one major challenge for current machine learning systems.
When training data is collected from multiple domains and the target domains include all training domains and other new domains, we are facing an Out-of-Distribution (OOD) generalization problem that aims to ﬁnd a model with the best OOD accuracy. One of the deﬁnitions of OOD accuracy is worst-domain accuracy.
In general, the set of target domains is unknown, and the worst over target domains may be unseen when the number of observed domains is limited.
In this paper, we show that the worst accuracy over the observed domains may dramatically fail to identify the OOD accuracy.
To this end, we introduce Inﬂuence Function, a classical tool from robust statistics, into the OOD generalization problem and suggest the variance of inﬂuence function to monitor the stability of a model on training domains.
We show that the accuracy on test domains and the proposed index together can help us discern whether OOD algorithms are needed and whether a model achieves good OOD generalization.
I NTRODUCTION
Most machine learning systems assume both training and test data are independently and identically distributed, which does not always hold in practice (Bengio et al.
Consequently, its performance is often greatly degraded when the test data is from a different domain (distribution).
A classical example is the problem to identify cows and camels (Beery et al.
(2018)), where the empirical risk minimization (ERM, Vapnik (1992)) may classify images by background color instead of object shape.
As a result, when the test domain is “out-of-distribution” (OOD), e.g. when the background color is changed, its performance will drop signiﬁcantly.
The OOD generalization is to obtain a robust predictor against this distribution shift.
Suppose that we have training data collected from m domains:
[Equation]
where P e is the distribution corresponding to domain e , E tr is the set of all available domains , including validation domains, and z ei is a data point.
The OOD problem we considered is to ﬁnd a model f
OOD such that
[Equation]
where E all is the set of all target domains and (cid:96) ( f, P e ) is the expected loss of f on the domain P e .
Recent algorithms address this OOD problem by recovering invariant (causal) features and build the optimal model on top of these features, such as Invariant Risk Minimization (IRM, Arjovsky et al. (2019)), Risk Extrapolation (REx, Krueger et al. Group Distributionally Robust Optimization (gDRO, Sagawa et al.
and Inter-domain Mixup (Mixup, Xu et al.
Yan et al. (2020); Wang et al. Most works evaluate on Colored MNIST (see 5.1 for details) where we can directly obtain the worst domain accuracy over E all .
Gulrajani & Lopez-Paz (2020) has assembled many algorithms and multi-domain datasets, and ﬁnds that OOD algorithms can’t outperform ERM in some domain generalization tasks (Gulrajani & Lopez-Paz (2020)), e.g. VLCS (Torralba & Efros (2011)) and PACS (Li et al.
This is not surprising, since these tasks only require high performance on certain domains, while an OOD algorithm is expected to learn truly invariant features and be excellent on a large set of target domains E all .
This phenomenon is described as “accuracy-vs-invariance trade-off” in Akuzawa et al.
Two questions arise in the min-max problem (2).
First, previous works assume that there are sufﬁcient diversity among the domains in E all .
Thus the supremacy of (cid:96) ( f, P e ) may be much larger than the average, which implies that ERM may fail to discover f OOD .
But in reality, we do not know whether it is true.
If not, the distribution of (cid:96) ( f, P e ) is concentrated on the expectation of (cid:96) ( f, P e ) , and ERM is sufﬁcient to ﬁnd an invariant model for E all . Therefore, we call for a method to judge whether an OOD algorithm is needed.
Second, how to judge a model’s
OOD performance?
Traditionally, we consider test domains E test ⊂ E tr and use the worst-domain accuracy over E test (which we call test accuracy) to approximate the OOD accuracy.
However, test accuracy is a biased estimate of the OOD accuracy unless E tr is closed to E all .
More seriously, It may be irrelevant or even negatively correlated to the OOD accuracy.
This phenomenon is not uncommon, especially when there are features virtually spurious in E all but show a strong correlation to the target in E tr .
We give a toy example in Colored MNIST when the test accuracy fails to approximate the OOD accuracy.
For more details, please refer to Section 5.1 and Appendix A.4.
We choose three domains from Colored MNIST and use cross validation (Gulrajani & Lopez-Paz (2020)) to select models, i.e. we take turns to select a domain S ∈ E tr as the test domain and train on the rest, and select the model with max average test accuracy.
Figure 1 shows the comparison between ERM and IRM.
One can ﬁnd that no matter which domain is the test domain, ERM model uniformly outperforms IRM model on the test domain.
However, IRM model achieves consistently better OOD accuracy.
Shortcomings of the test accuracy here is obvious, regardless of whether cross validation is used.
In short, the naive use of the test accuracy may result in a non-OOD model.
To address this obstacle, we hope to ﬁnd an metric that correlates better with model’s
OOD property, even when E tr is much smaller than E all and the “worst” domain remains unknown . Without any assumption to
E all , our goal is unrealistic. Therefore, we assume that features that are invariant across E tr should also be across E all . This assumption is necessary. Otherwise, the only thing we can do is to collect more domains.
Therefore, we need to focus on what features the model has learnt. Specifically, we want to check whether the model learns invariant features and avoid varying features.
The inﬂuence function (Cook & Weisberg (1980)) can serve our purpose.
Inﬂuence function was proposed to measures the parameter change when a data point is removed or upweighted by a small perturbation (details in 3.2).
When modiﬁed it to domain-level, it measures the inﬂuence of a domain instead of a data point on the model. Note that we are not emulating the changes of the parameter when a domain is removed.
Instead, we are exactly caring about upweighting the domain by δ → 0 + (will be speciﬁed later).
Base on this, the variance of inﬂuence function allows us to measure OOD property and solve the obstacle.
Contributions we summarize our contributions here: (i)
We introduce inﬂuence function to domain-level and propose index V γ | θ (formula 6) based on inﬂuence function of the model f θ .
Our index can measure the OOD extent of available domains, i.e. how different these domains (distributions) are.
This measurement provides a basis for whether to adopt an OOD algorithm and to collect more diverse domains.
See Section 4.1 and Section 5.1.1 for details.
(ii) We point out that the proposed index V γ | θ can solve the weakness of test accuracy.
Speciﬁcally, under most OOD generalization problems, using test accuracy and our index together, we can discern the OOD property of a model.
See Section 4.2 for details. (iii)
We propose to use only a small but important part of the model to calculate the inﬂuence function.
This overcomes the huge computation cost of solving the inverse of Hessian.
It is not merely for calculation efﬁciency and accuracy, but it coincides with our understanding that only these parameters capture what features a model has learnt (Section 4.3).
We organize our paper as follows: Section 2 reviews related works and Section 3 introduces the preliminaries of OOD methods and inﬂuence function.
Section 4 presents our proposal and detailed analysis.
Section 5 shows our experiments.
The conclusion is given in Section 6.
R ELATED WORK
The mismatch between the development dataset and the target domain is one major challenge in machine learning (Castro et al.
(2020); Kuang et al.
Many works assume that the ground truth can be represented by a causal Direct Acyclic Graph (DAG), and they use the DAG structure to discuss the worst-domain performance (Rojas-Carulla et al.
; Peters et al.
(2016); Subbaswamy et al.
; B ¨ uhlmann et al.
(2020); Magliacane et al.
All these works employ multiple domain data and causal assumptions to discover the parents of the target variable.
Rojas-Carulla et al.
(2018) and Magliacane et al.
(2018) also apply this idea to Domain Generalization and Multi-Task Learning setting.
Starting from multiple domain data rather than model assumptions, Arjovsky et al.
(2019) proposes Invariant Risk Minimization (IRM) to extract causal (invariant) features and learn invariant optimal predictor on the top of the causal features.
It analyzes the generalization properties of IRM from the view of sufﬁcient dimension reduction (Cook (2009); Cook et al.
Ahuja et al.
(2020) considers IRM as ﬁnding the Nash equilibrium of an ensemble game among several domains and develops a simple training algorithm.
Krueger et al.
(2020) derives the Risk Extrapolation (REx) to extract invariant features and further derives a practical objective function via variance penalization.
Xie et al.
(2020) employs a framework from distributional robustness to interpret the beneﬁt of REx comparing to robust optimization (Ben-Tal et al.
; Bagnell (2005)).
Besides, Adversarial Domain Adaption (Li et al.
; Koyama & Yamaguchi (2020)) uses discriminator to look for features that are independent of domain and uses these features for further prediction.
Inﬂuence function is a classic method from the robust statistics literature (Robins et al.
Van der Laan et al.
(2003); Tsiatis (2007)).
It can be used to track the impact of a training sample on the prediction.
Koh & Liang (2017) proposes a second-order optimization technique to approximate the inﬂuence function.
They verify their method with different assumptions on the empirical risk ranging from being strictly convex and twice-differentiable to non-convex and non-differentiable losses.
Koh et al.
(2019) also estimates the effect of removing a subgroup of training points via inﬂuence function.
They ﬁnd out that the approximation computed by the inﬂuence function is correlated with the actual effect.
Inﬂuence function has been used in many machine learning tasks.
Cheng et al.
(2019) proposes an explanation method, Fast Inﬂuence Analysis, that employs inﬂuence function on
Latent Factor Model to solve the lack of interpretability of the collaborative ﬁltering approaches for recommender systems.
Cohen et al.
(2020) uses inﬂuence function to detect adversarial attacks.
Ting & Brochu (2018) proposes an asymptotically optimal sampling method via an asymptotically linear estimator and the associated inﬂuence function.
Alaa & Van Der Schaar (2019) develops a model validation procedure that estimates the estimation error of causal inference methods.
Besides,
Fang et al.
(2020) leverages inﬂuence function to select a subset of normal users who are inﬂuential to the recommendations.
P RELIMINARIES 3. ERM, IRM AND RE X
In this section, we give some notations and introduce some recent OOD methods.
Recall the multiple domain setup (1) and OOD problem (2).
For a domain P e and a hypothetical model f , the population loss is (cid:96) ( f, P e ) =
E z ∼ P e
[ L ( f, z )] where L ( f, z ) is the loss function on z .
The empirical loss, which is the objective of ERM, is (cid:96) ( f, S ) = (1 /m ) (cid:80) e ∈E tr (cid:96) ( f, S e ) with (cid:96) ( f, S e ) = (1 /n ) (cid:80) ni
=1 L ( f, z e i ) .
Recent OOD methods propose some novel regularized objective functions in the form:
[Equation]
to discover f OOD in (2).
Here R ( f, S ) is a regularization term and λ is the tuning parameter which controls the degree of penalty.
Note that ERM is a special case by setting λ = 0 .
For simplicity, we will use L ( f, S ) to represent the total loss in case of no ambiguity.
Arjovsky et al.
(2019) focuses on the stability of f OOD and considers the IRM regularization:
[Equation]
where w is a scalar and ﬁxed “dummy” classiﬁer.
Arjovsky et al.
(2019) shows that the scalar ﬁxed classiﬁer w is sufﬁcient to monitor invariance and responds to the idealistic IRM problem which decomposes the entire predictor into data representation and one shared optimal top classiﬁer for all training domains.
On the other hand, Krueger et al.
(2020) encourages the uniform performance of f OOD and proposes the V-REx penalty: [Equation]
Krueger et al.
(2020) derives the invariant prediction by the robustness to spurious features and ﬁgure out that REx is more robust than group distributional robustness (Sagawa et al. In this work, we also decompose the entire predictor into a feature extractor and a classiﬁer on the top of the learnt features. Different to Arjovsky et al. and Krueger et al.
(2020), we directly monitor the invariance of the top model.
3. I NFLUENCE FUNCTION AND GROUP EFFECT
Consider a parametric hypothesis f = f θ and the corresponding solution: ˆ θ = arg min θ L ( f θ , S ) .
By a quadratic approximation of L ( f θ , S ) around ˆ θ , the inﬂuence function takes the form
IF (ˆ θ , z ) = − H − 1 ˆ θ ∇ θ L ( f ˆ θ , z ) with H ˆ θ = ∇ [Equation] L ( f ˆ θ , S ) .
When the sample size of S is sufﬁciently large, the parameter change due to removing a data point z can be approximated by −I ( z ) / (cid:80) e ∈E tr | S e | without retraining the model.
Here | S e | = n e stands for the cardinal of the set S e .
Furthermore, Koh et al.
(2019) shows that the inﬂuence function can also predict the effects of large groups of training points (i.e. Z = { z 1 , ..., z k } ), although there are signiﬁcant changes in the model.
The parameter change due to removing the group can be approximated by
[Equation]
Motivated by the work of Koh et al.
(2019), we introduce inﬂuence function to OOD problem to address our obstacles.
M ETHODOLOGY 4. I NFLUENCE OF DOMAINS
Following Arjovsky et al.
(2019), we decompose a parametric hypothesis f θ ( x ) into a top model g and a feature extractor Φ , i.e. f θ ( x ) =
g (Φ( x , β ) , γ ) and θ = ( γ , β ) .
When upweighting a domain e by a small perturbation δ , we do not upweight the regularized term, i.e.
[Equation]
since the stability across different domains, which is encouraged by the regularization, should not depend on the sample size of a domain.
For a learnt model f ˆ θ , ﬁxing the feature extractor Φ , i.e. ﬁxing β = ˆ β , the change of top model g caused by upweighting the domain is
[Equation]
Here H ˆ γ = ∇ 2ˆ γ L ( f ˆ θ , S ) , and we assume L is twice-differentiable in γ . Please see Appendix A. for detailed derivation and why β should be ﬁxed.
For a regularized method, e.g. IRM and REx, the inﬂuence of their regularized term is reﬂected in H and in learnt model f
ˆ θ .
As mentioned above, IF (ˆ γ , S e | ˆ θ ) measures change of model caused by upweighting domain e .
Therefore, if g (Φ , ˆ γ ) is invariant across domains, the entire model f
ˆ θ treats all domains equally.
As a result, a small perturbation on different domains should cause the same model change .
This leads to our proposal.
4. P ROPOSED I NDEX AND ITS U TILITY
On basis of the domain-level inﬂuence function IF (ˆ γ , S e | ˆ θ ) , we propose our index to measure the ﬂuctuation of the parameter change when different domains are upweighted:
[Equation]
Here (cid:107) · (cid:107) 2 is the 2-norm for matrix, i.e. the largest eigenvalue of the matrix, Cov e ∈E tr ( · ) refers to the covariance matrix of the domain-level inﬂuence function over E tr and ln( · ) is a nonlinear transformation that works well in practice.
OOD Model Under the OOD problem in (2), a good OOD model should (i) learn invariant and useful features; (ii) avoid spurious and varying features.
Learning useful and invariant features means the model should have high accuracy over a set of test domains E test , no matter which test domain it is.
In turn, high accuracy over E test also means the model truly learns some useful features for the test domains.
However, this is not enough, since we do not know whether the useful features are invariant features across E all or just spurious features on E test .
On the other hand, avoiding varying features means that different domains are actually the same to the learnt model, so according to the arguments in Section 4.1, V γ | θ should be small.
Combined this, we derive our proposal:
if a learnt model f
ˆ θ manage to simultaneously achieve small V ˆ γ | ˆ θ and high accuracy over E test , it should have good OOD accuracy.
We prove our proposal in a simple but illuminating case, and we conduct various experiments (Section 5) to support our proposal.
Several issues should be clariﬁed.
First, not all OOD problems demand models to learn invariant features. For example, the set of all target domains is small such that the varying features are always strongly correlated to the labels, or the objective is the mean of the accuracy over E all rather than the worst-domain accuracy.
But to our concern, we regard the OOD problem in (2) as a bridge to causal discover. Thus the set of the target domains is large, and the “weak” OOD problems are out of our consideration.
To a large extent, invariant features are still the major target and our proposal is still a good criterion to model’s
OOD property.
Second, we admit that the gap between being stable in E tr (small V γ | θ ) and avoiding all spurious features on E all does exist.
However, to our knowledge, for features that are varying in E all but are invariant in E tr , demanding a model to avoid them is somehow unrealistic.
Therefore, we make a step forward that we measure whether the learnt model successfully avoids features that vary across E tr .
We leave index about varying features over E all in our future work.
The Shufﬂe V γ | β
As mentioned above, smaller metric V γ | θ means strong stablility across E tr , and hence should have better OOD accuracy. However, the proposed metric depends on the dataset S and the learnt model f ˆ θ .
Therefore, there is no uniform baseline to check whether the metric is “small” enough.
To this end, we propose a baseline value of the proposed metric by shufﬂing the multi-domain data. Consider pooling all data points in S and randomly redistributed to m new synthetic domains { ˜ S 1 , ˜ S 2 , ..., ˜ S m } := ˜ S .
We compute the shufﬂe version of V γ | θ for a learnt model f ˆ θ over the shufﬂed data ˜ S : [Equation]
and denote the standard version and shufﬂe version of the metric as V ˆ γ | ˆ θ and ˜ V
ˆ γ | ˆ θ respectively.
For any algorithm that obtains relatively good test accuracy, if V ˆ γ | ˆ θ is much larger than ˜ V ˆ γ | ˆ θ , f ˆ θ has learnt features that vary across e ∈ E tr , and cannot treat domains in E tr equally. This implies that f ˆ θ may not be an invariant predictor over E all . Otherwise, if the two values are similar, the model has avoided varying features in E tr and maybe invariant across E tr .
Therefore, either the model capture the invariance over the diverse domains, or the domains are not diverse at all.
Note that this process is suitable for any algorithm, hence providing a baseline to see whether V ˆ γ | ˆ θ is small. Here we also obtain a method to judge whether an OOD algorithm is needed. Consider f ˆ θ learnt by
ERM.
If V ˆ γ | ˆ θ is relatively larger than ˜ V γ | θ , then ERM fails to avoid varying features. In this case, one should consider an OOD algorithm to achieve better OOD generalization. Otherwise, ERM is enough, and any attempt to achieve better OOD accuracy should start with ﬁnding more domains instead of using OOD algorithms.
This coincides experiments in Gulrajani & Lopez-Paz (2020) (Section5.2). Our understanding is that domains in ˜ S are similar. Therefore, the difference between shufﬂe and standard version of the metric reﬂects how much varying features a learnt model uses.
We show how to use the two version of V γ | θ in Section 5.1.1 and Section 5.2.
4. I NFLUENCE C ALCULATION
There is a question surrounding the inﬂuence function: how to efﬁciently calculate and inverse Hessian?
Koh & Liang (2017) suggests Conjugate Gradient and Stochastic estimation solve the problem.
However, when ˆ θ is obtained by running SGD, it could hardly arrive at the global minimum.
Although adding a damping term (i.e. let ˆ H ˆ θ =
H ˆ θ + λI ) can moderately alleviate the problem by transforming it into a convex situation, under large neural-network with non-linear activation function like ReLU, this method may still work poorly since the damping term in order to satisfy the transform is so large that it will inﬂuence the performance signiﬁcantly.
Most importantly, the variation of the eigenvalue of Hessian is huge, making the convergence of inﬂuence function calculation quite slow and inaccurate (Basu et al.
In our metric, we circumvent the problem by excluding most parameters β and directly calculate Hessian of γ to get accurate inﬂuence function.
This modiﬁcation not only speed up the calculation, but it also coincides our expectation, that an OOD algorithm should learn invariant features does not mean that the inﬂuence function of all parameters should be identical across domains.
For example, if g (Φ) wants to extract the same features in different domains, the inﬂuence function should be different on Φ( · ) .
Therefore, if we use all parameters to calculate the inﬂuence, given that γ is relatively insigniﬁcant in size compared with β , the information of learnt features provided by γ is hard to be captured.
On the contrary, only considering the inﬂuence of the top model will manifest the inﬂuence of different domains in the aspect of features , thus enabling us to achieve our goal.
As our experiments show, after this modiﬁcation, the inﬂuence function calculation speed can be 2000 times faster, and the utility (correlation with OOD property) could be even higher.
One may not feel surprised given the huge number of parameters in the embedding model Φ( · ) .
They slow down the calculation and overshadow the top model’s inﬂuence value.
E XPERIMENT
In this section, we experimentally show that: (1) A model f
ˆ θ reaches small V ˆ γ | ˆ θ if it has good OOD property, while a non-OOD model won’t.
The metric V γ | θ provides additional information on the stability of a learnt model, which overcomes the weakness of the test accuracy.
The comparison of V γ | β and ˜ V γ | β can check whether a better OOD algorithm is needed.
We consider experiments in Bayesian Network, Colored MNIST and VLCS.
The synthetic data generated by Bayesian Network includes domain-dependent noise and fake associations between features and response.
For Colored MNIST, we already know that the digit is the causal feature and the color is non-causal.
The causal relationships help us to determine the worst domain and obtain the OOD accuracy.
VLCS is a real dataset, in which we show utility of V γ | θ step by step.
Due to the space limitation, we put the experiments in Bayesian Network to the appendix.
Generally, cross-validation (Gulrajani & Lopez-Paz (2020)) is used to judge a model’s
OOD property.
In the introduction, we have already shown that the leave-one-domain-out cross-validation may fail to discern OOD properties.
We also consider another two potential competitors: conditional mutual information and IRM penalty.
The comparison between our metric and the two competitors are postponed into Appendix.
5. C OLORED MNIST
et al troduces a synthetic binary
The images are colored according to their label, making color a spurious feature in predicting the label.
Speciﬁcally, for a domain e , we assign a preliminary binary label ˜
y = 1 digits ≤and randomly ﬂip ˜ y with p = 0 .
Then, we color the image according to ˜ y
but with a ﬂip rate of p e .
Clearly, when p e < 0 .or
p e > 0 .
75 , color is more correlated with ˜
y than real digit.
Therefore, the oracle OOD model
f OOD will attain accuracy 0 .
75 in all domains while an ERM model may attain high training accuracy and low OOD property if p e in training domains is too small or too large.
Throughout the Colored MNIST experiments, we use three-layer MLP with ReLU activation and hidden dimension 256.
Although our MLP model has relatively many parameters and is non-convex due to the activation layer, due to the technique mentioned in Section 4.3, the inﬂuence calculation is still fast and accurate, with directly calculating inﬂuence once spends less than 2 seconds.
5.1. I DENTIFY OOD P ROBLEM
In this section, we show that V γ | θ can discern whether the training domains are sufﬁciently diverse as mentioned in Section 4.2.
Assume E tr has ﬁve training domains with
[Equation]
} , where x ∈ [0 . 0 , 0 . 1] is positively related to the diversity among the training domains.
If x is zero, all data points are generated from the same domain ( p e = 0 . 2 ) and so the learning task on E tr is not an OOD problem.
On the contrary, larger x means that the training domains are more diverse.
We repeat 501 times to learn the model with ERM.
Given the learnt model f ˆ θ and the training data, we compute V ˆ γ | ˆ θ and check the correlation between V ˆ γ | ˆ θ and x .
Figure 2 presents the results.
Our index V γ | θ is highly related to x. The Pearson coefﬁcient is 0.9869, and the Spearman coefﬁcient is 0.9873.
Also, the benchmark of V γ | θ that learns on the same training domains ( ˜ S in 4.2) can be derived from the raw data by pooling and redistributing all data points, and we mark it by the black dashed line.
If
V ˆ γ | ˆ θ is much higher than the benchmark, indicating that x is not small, an OOD algorithm should be considered if better OOD generalization is demanded.
Otherwise, the present algorithm (like ERM) is sufﬁcient.
The results coincide our expectation that V γ | θ can discern whether P e is different.
5.1. R ELATIONSHIP BETWEEN V AND OOD A CCURACY
In this section, we use an experiment to support our proposal in Section 4.2.
As previously proposed, if a model shows high test accuracy and small V γ | θ simultaneously, it captures invariant features and avoids varying features, so it deserves to be an OOD model.
In this experiment, we consider a model with high test accuracy and show that smaller V γ | θ generally corresponds to better OOD accuracy, which supports our proposal.
Consider two setups: p e ∈ { 0 . 0 , 0 . 1 } and p e ∈ { 0 . 1 , 0 . 15 , 0 . 2 , 0 . 25 , 0 . 3 } .
We implement IRM and REx with different penalty (note that ERM is λ = 0 ) to check relationship between V γ | θ and OOD accuracy.
For IRM and REx, we run 190 epochs pre-training with λ = 1 and use early stopping to prevent over-ﬁtting.
With this technique, all models successfully achieve good test accuracy (within 0.1 of the oracle accuracy) and meet our requirement.
Figure 3 presents the results.
We can see that V γ | θ are highly correlated to OOD accuracy in IRM and REx, with the absolute of Pearson Coefﬁcient never less than 0 .
Those models learned with larger λ present better OOD property, learning less varying features, and showing smaller V γ | θ .
The results are consistent with our proposal, except that when λ is large in IRM, V γ | θ is a little bit unstable.
We have carefully examined the phenomenon and found that it is caused by computational instability when inversing Hessian with eigenvalue quite close to 0.
The problem of unstable inversing happens with a low probability and can be addressed by repeating the experiment once or twice. 5. D OMAIN G ENERALIZATION : VLCS In this section, we implement the proposed metric for 4 algorithms: ERM, gDRO, Mixup and
IRM on the VLCS image dataset, which is widely used for doamin generalization.
We emulate a real scenario with E all =
{ V, L, C, S } and E tr = E all \{ S } . As mentioned in Gulrajani & Lopez-Paz (2020), we use “trainingdomain validation set” method, i.e. we split a validation set for each S ∈ E tr and the test accuracy is deﬁned as the average accuracy amount the three validation sets. Note that, our goal is to use the test accuracy and V γ | β to measure the OOD generalization , rather than to tune for the SOTA performance on a unseen domain { S } .
Therefore, we do not apply any model selection method and just use the default hyper-parameters in Gulrajani & LopezPaz (2020).
1: T EST ACCURACY COMPARISON
For each algorithm, we run the naive training process 12 times and show the average of test accuracy of each algorithm in Table 1.
Before calculating V γ | β , the learnt model should at least arrive a good test accuracy.
Otherwise, there is no need to discuss its OOD performance since OOD accuracy is smaller than test accuracy.
In the table, the test accuracy of ERM, Mixup and gDRO is good, but that of IRM is not.
In this case, IRM will be eliminated.
If an algorithm fails to reach high test accuracy ﬁrst, we should ﬁrst change the hyper-parameters until we observe a relatively high test accuracy.
5.2. S TEP 2: SHUFFLE AND STANDARD METRIC COMPARISON
[Table]
Now we are ready to check whether the learnt models are invariant across E tr .
As mentioned in 4.2, the difference of V γ | β and ˜
V γ | β represents whether how much a model is invariant across E tr .
We calculate the value and the results are in Figure 4.
For ERM and Mixup, the two value is nearly the same.
In this case, we expect that ERM and Mixup models are invariant and should have a relatively high OOD accuracy, so no more algorithm is needed.
For gDRO, we can clearly see that ˜ V γ | β is uniformly smaller than V γ | β .
Therefore, gDRO models don’t treat different domains equally, and hence we predict that the OOD accuracy will be relatively low.
In this case, one who starts with gDRO should turn to other algorithms if a better OOD performance is demanded.
Note that, in the whole process, we know nothing about { S } , so the OOD accuracy is unseen.
However, from the above analysis, we know that (1) in this settings, ERM and Mixup is better than gDRO; (2) one who uses gDRO can turn to other algorithms (like Mixup) for better OOD performance; (3) one who uses ERM should consider collecting more environments if he (she) still wants to improve OOD performance.
So far, we ﬁnish the judgement using test accuracy and the proposed metric.
5.2. S TEP 3: OOD ACCURACY RESULTS ( ORACLE )
[Table]
In this step, we fortunately obatin E all and can check whether our judgement is reasonable.
Normally, this step will not happen.
We now show the OOD accuracy of four algorithms in table 2.
Similar to our judgement, ERM and
Mixup models achieve a higher OOD accuracy than gDRO.
The performance of IRM (under this hyper-parameters) is lower than test accuracy.
During the above process, we can also compare the metric of the model from the same algorithm but with different hyper-parameters (as the same in section 5.1.2). Besides, one may notice that even the highest OOD accuracy is just 63 . That is to say, to obtain OOD accuracy larger than 70% , we should consider collecting more environments. In the appendix A.6, we continue our real scenario to see that, if initially E tr is more diverse, what will our metric lead us to.
The whole results in VLCS can also be found in the same appendix, and the comparison of the proposed metric with the IRM penalty in formula 4 can be found there too.
Besides, we show the comparison with Conditional Mutual Information in the appendix A.5.
In summary, we use a realistic task to see how to judge the OOD property of learnt model using the proposed metric and test accuracy. The judgement coincides well with the real OOD performance.  C ONCLUSION In this paper, we focus on two presently unsolved problems, that how can we discern the OOD property of multiple domains and of learnt models.
To this end, we introduce inﬂuence function into OOD problem and propose our metric to help solve these issues.
Our metric can not only discern whether a multi-domains problem is OOD but can also judge a model
’s OOD property when combined with test accuracy.
To make our calculation more meaningful, accurate and efﬁcient, we modify inﬂuence function to domain-level and propose to use only the top model to calculate the inﬂuence.
Our method is proved in simple cases and it works well in experiments.
We sincerely hope that, with the help of this index, our understanding of OOD generalization will become more and more precise and thorough.
A A PPENDIX A. S IMPLE B AYESIAN N ETWORK
In this section we show that the model with better OOD accuracy achieves smaller V γ | θ .
We assume the data is generated from the following Bayesian network:
[Equation]
where x 1 , x 2 ∈ R 5 are the features, y ∈ R 5 is the target vector, W 1 → y ∈ R 5 × 5 and W y → 2 ∈ R 5 ×are the underlying parameters that are invariant across domains.
The variance of gaussian noise is σ 2 e that depends on domain.
For simplicity, we denote e =
σ e to represent a domain.
The goal here is to linearly regress the response y on the input vector ( x 1 , x 2 ) , i.e. ˆ y =
x 1 ˆ W 1 + x 2 ˆ W 2 .
According to the Bayesian network (8), x 1 is the invariant feature, while the correlation between xand y is spurious and unstable since e = σ e varies across domains.
Clearly, the model based only on x 1 is an invariant model.
Any invariant estimator should achieve ˆ W 1 ≈ W 1 → y and ˆ W 2 ≈ 0 .
[Table]
Now consider ﬁve training domains e ∈ E tr = { 0 . 2 , 0 .
7 , 1 . 2 , 1 . 7 , 2 . 2 } , each containing 1000 data points.
We estimate three linear models using ERM, IRM and REx respectively and record the parameter error as well as V γ | θ (note that γ is θ here).
Table 3 presents the results amongrepetitions.
As expected, IRM and REx learn more invariant relationships than ERM (smaller causal error) and better avoid non-causal variables ( ˆ W 2 ≈ 0 ).
Furthermore, the proposed measurement
V γ | θ is highly related to invariance, i.e. model with better OOD property achieves smaller V γ | θ .
This results coincides our understanding.
In this section, we use a simple model to illuminate the validity of V γ | θ proposed in Section 4.
Consider a structural equation model (Wright (1921)):
[Equation]
where P ex is a distribution with a ﬁnite second-order moment, i.e. E x 21 < + ∞ , and σ 2 e is the variance of the noise term in x 2 .
Both P ex and σ 2 e vary across domains.
For simplicity, we assume there are inﬁnite training data points collected from two training domains E tr = { ( P 1 x , σ 21 ) , ( P 2 x , σ 22 ) } .
Our goal is to predict y from x := ( x 1 , x 2 ) (cid:62) using a least-squares predictor ˆ y = x (cid:62) ˆ β := x 1 ˆ β 1 + x 2 ˆ β 2 .
Here we consider two algorithms: ERM and IRM with λ → + ∞ .
According to Arjovsky et al.
(2019), using IRM we obtain β IRM → (1 , 0) (cid:62) .
Intuitively, ERM will exploit both x 1 and x 2 , thus achieving a better regression model.
However, since relationship between y and x 2 varies across domains, our index will be huge in such condition.
Conversely, β IRM only uses invariant features x 1 , thus V γ | θ → −∞ .
Note that we do not have an embedding model here, so V γ | θ = V β .
ERM we denote
[Equation]
Note that in E e , x 1 is sample from P ex .
We then have
[Equation]
To proceed further, we denote ¯
[Equation]
= |E tr | (cid:88) e ∈E tr
[Equation] e [Equation] By solving the following equations: |E tr | (cid:88) e ∈E tr
E
[Equation]
x [Equation] [Equation] =and |E tr | (cid:88)
e ∈E tr E
[Equation]
x
[Equation]
[Equation]
=
[Equation]
[Equation] − [Equation] =we have ˆ β =
(ˆ β 1 , ˆ β 2 ) (cid:62) with ˆ β
= s s
+ 2 , ˆ β 2 = s
+ 2 .
Now we calculate our index.
It is easy to see that ∂(cid:96)
e ( β ) β= − 2 E e [ x 1 ( y − x (cid:62) β )]
= −
E e x 21 (1 − β 1 − β 2 ) ∂(cid:96)
e ( β ) β= − 2 E e [ x 2 ( y − x (cid:62) β )]
= − 2[( E e x 21 + 1)(1 − β 1 − β 2 ) + β 1 − σ 2 e β 2 ] .
Therefore, ∇ (cid:96) 1 ( β ) − ∇ (cid:96) 2 ( β ) = (cid:18) 2 β 2 ( σ 21 − σ 22 ) (cid:19) and ∇ (cid:96) 1 ( ˆ β ) − ∇ (cid:96) 1 ( ˆ β ) = (cid:18)
[Equation]
( σ
[Equation]
−
[Equation]
[Equation]
(cid:19) (9)
On the other hand, calculate the hessian and we have
H ERM = (cid:18)
2 ¯ d 2 ¯ d 2 ¯ d 2 ¯ d
+ s +(cid:19) and H − 1 = 2 ¯ d ( s + 2) (cid:18) 2 ¯ d
+ s + 2 − 2 ¯ d − 2 ¯ d 2 ¯ d (cid:19) .
Then we have (note that IF ( ˆ β , S
[Equation]
where the third equation holds because the rank of matrix is 1 .
Clearly, when | σ 21 − σ 22 | →(means two domains become identical), our index V β → −∞ .
Otherwise, given σ 1 (cid:54) =
σ 2 , we have
V β > −∞ , showing that ERM captures varied features.
IRM We now turn to IRM model and show that V β → −∞ when λ → + ∞ , thus proving IRM learnt model ˆ β IRM does achieve smaller V β compared with ˆ β in ERM.
Under IRM model, assuming the tuning parameter is λ , we have
[Equation]
Then we have the gradient with respect β :
[Equation]
and the Hessian matrix
[Equation]
Denote β λ the solution of IRM algorithm on E tr when penalty is λ .
From Arjovsky et al.
we know β λ → β IRM :=
(1 , 0) (cid:62) .
To show lim λ → + ∞ V β λ =
−∞ , we only need to show that
[Equation]
We prove this by showing that
[Equation] [Equation]
simultaneously.
We add ( λ ) after H − 1 to show that H − 1 is a continuous function of λ .
Rewrite H in formula 10 as
[Equation]
where
F ( β
[Equation]
= |E tr | (cid:88)
[Equation]
[Table]
[Equation]
[Equation]
[Table]
[Equation]
[Equation]
[Table]
[Table]
[Equation]
[Table]
[Equation]
|E
[Equation]
| (cid:88)
[Equation]
tr
[Equation]
1 −
[Equation]
(cid:19) (cid:0)
[Equation]
E
[Equation]
1 − E e x 21 (cid:1) =
F ( β IRM ) exists. Obviously, F ( β IRM ) is positive deﬁnite.
Therefore, we have
[Equation]
The ﬁrst equation holds because lim λ → + ∞ F ( β λ )
=
F ( β IRM ) has the limit and is not 0 , and the last equation holds because the eigenvalue of H goes to
+ ∞ when λ → + ∞ .
Now consider ∇ (cid:96) 1 ( β λ ) − ∇ (cid:96) 2 ( β λ ) .
According to formula 9, we have
[Equation]
Hence we ﬁnish proof of formula 11 and show that V β → −∞ in IRM.
A. F ORMULA (5)
This section shows the derivation of the expression (5).
Recall that the training dataset S
=
{ S 1 , ..., S m } and the objective function
[Equation]
where the second term on the right hand side is the regularization.
As to ERM, the regularization term is zero.
With the feature extractor ( β ) ﬁxed, we upweight a domain S e .
The new objective function is
[Equation]
Notice that when upweight an domain, we only upweight the empirical loss on the corresponding domain.
Further, we denote ˆ γ , ˆ γ + as the optimal solutions before and after upweighting a domain.
It is easy to see that (cid:107) ˆ γ + − ˆ γ (cid:107) → 0 when δ → 0 .
Following the derivation in Koh & Liang (2017), according to the ﬁrst-order Taylor expansion of ∇ γ L + ( θ , S , δ ) with respect to γ on ˆ γ ,
[Equation]
Assume that ∇ 2 [ L (ˆ θ , S ) + δ(cid:96) (ˆ θ , S e )] is invertible, we have
[Equation]
Note that this derivation is not fully rigorous.
Please refer to Van der Vaart (2000) for more rigorous discussions about inﬂuence function. The reason that β should be ﬁxed is as follows.
First, if β can be varied, then the change of θ will become:
[Equation] Therefore, the computational cost is similar to calculate and inverse the whole hessian matrix. Most importantly, without ﬁxing β , the change of γ is somehow useless. Say when upweighting S e , the use of a feature decreases.
It’s possible, however, that the parameter in γ corresponding to the feature increases while β decreases a larger scale.
In this case, the use of the feature decreases but γ increases.
Without ﬁxing β , the change of γ calculated by inﬂuence function may provide no information about the use a feature.
Therefore, we argue that, ﬁxing β is a “double-win” choice.
In Introduction, we have given an example where test accuracy misleads us.
In this section, we will ﬁrst supplement some examples where test accuracy not only misjudge different algorithms, but it also misjudges the OOD property of models learnt with different penalty within the same algorithm.
After that, we will show the universality of these problems and why test accuracy fails.
Consider two training domains
[Equation]
and a test domain with ﬂip rate denoted by p test .
We implement IRM and REx with penalty λ ∈ { 0 , 50 , 100 , 500 , 1000 } to check the relationship between test accuracy and OOD accuracy.
The training process is identical to the experiment in section 5.1.2.
As results showed in Figure 5, when
OOD property of model gradually improves (caused by gradually increasing λ ), its relationship with test accuracy is either completely (when p test is 0.2) or partly (when p test is 0.3) negatively correlated.
This phenomenon reveals the weakness of test accuracy.
If one wants to select a λ when p test is 0.3, judged by test accuracy, λ = 50 may be the best choice, no matter in IRM or REx.
However, the model learnt with λ = 50 has OOD accuracy even less than a random guess model
Whether test accuracy is positively, negatively correlated or irrelevant to model’s OOD property mainly depends on the “distance” between test domain and the “worst” domain for the model.
If test accuracy happens to be the lowest among all the domains, we directly have OOD accuracy equals to test accuracy.
In practice, however, their distance may be huge, and this is precisely the difﬁculty of OOD generalization.
For example, we are accessible to images of cows in grasslands, woods and forests, but cows in desert are rare.
At this point, the “worst” domain is certainly far from what we can get.
If we expect a model to capture the real feature of cows, the model should avoid any usage of background color.
However, a model based on color will perform consistently well (better than any OOD model) no matter in grasslands, woods and forests since all of the available domains are green background in general.
In Colored MNIST, test accuracy fails in the same way.
Such situations are quite common.
Generally, within domains we have, there may be some features that are strongly correlated to the prediction but are slightly varied across domains.
These features are spurious, given that their relationship with prediction is signiﬁcantly disparate in other domains to which we want to generalize.
However, using these features in prediction will easily achieve high test accuracy.
Consequently, it will be extremely risky to judge models merely by test accuracy. A. C ONDITIONAL MUTUAL INFORMATION
A possible alternative of V γ | θ may be Conditional Mutual Information (CMI).
For three continuous random variables X , Y , Z , the CMI is deﬁned as
[Equation]
where p ( · ) is the probability density function.
Consider I ( e ; y | Φ( x )) or I ( e ; y | ˆ y ) , i.e. the mutual information of e and true label y , given the features or the prediction ˆ y of x .
The insight is that, if the model is invariant across different domains, then little information about e should be contained in y given Φ( x ) .
Otherwise, if the prediction ˆ y is highly correlated to e , then the mutual information will be high.
This metric seems to be promising.
However, the numerical estimation of CMI remains a challenge.
To this end, previous works have done a lot to solve this problem, including CCMI proposed in
Mukherjee et al.
and CCIT proposed in Sen et al.
In this part, we will ﬁrst calculate true I ( e ; y | ˆ y ) in a simple Colored MNIST experiment to show that if there is no estimation problem,
CMI could be a potential metric to judge the OOD property of the learnt model, at least in a simple, discrete task.
We then run the code provided by Sen et al.
(2017) ( https://github.com/ rajatsen91/CCIT ) to show that even in this simple task, the estimation of CMI may severely inﬂuence its performance.
Speciﬁcally, the experimental setting is similar to that in subsection 5.1.2, with two OOD algorithm and number of training domains in { 2 , 5 } .
For each algorithm, we consider the penalty weight λ ∈ { 0 , 10 , 100 , 1000 } , run the algorithm 50 times, and record their OOD accuracy as well as true
CMI value or CCIT value.
The results are shown in Figure 6.
We can see that in the case when true CMI can be easily calculated, especially in the case when the number of domains is small and the task is discrete (not continuous), CMI is highly correlated to OOD accuracy.
However, in a regression task or in a task when directly calculating the value of CMI becomes impractical, the estimation process may severely destroy the correlation, and may also result in an inverse correlation.
Therefore, we summarized the estimation of CMI has limited its utility.
We leave the ﬁne-grained analysis of the relationship between CMI, estimated CMI and OOD property to future works.
A. R ESULTS ON VLCS A.6. C ONTINUED S CENARIO
[Table]
This is a continuation of section 5.2.
Say in this task, E all remains the four domains but c E tr = { L, S, V } (empirically we ﬁnd it more diverse).
Similarly, we start with test accuracy shown in table 4.
In this step, the situation is the same, i.e. IRM should be eliminated until proper hyper-parameters are found.
In step 2, we show the comparison between V γ | β and ˜ V γ | β of three algorithms in Figure 7.
As we can see, this time the two value are similar for all three algorithms, including gDRO.
This is different from the case when S is unseen.
In this case, we predict that all of the three algorithms should achieve high OOD accuracy.
In fact, if we act as the oracle and calculate their OOD performance, we will ﬁnd that our judgement is close to the reality: ERM, Mixup and gDRO achieve OOD accuracy from 70 .
55% to 72 .
According to the conﬁdence interval, they difference are not satistically signiﬁcant.
As for IRM, the OOD accuracy is 38 .
One who use ERM, Mixup or gDRO should be satisﬁed for the performance, since higher demand is somehow impractical!
A.6. F ULL RESULTS AND COMPARISON WITH IRM PENALTY
As mentioned in Section 5.2, we consider ERM, gDRO, Mixup and IRM on VLCS image dataset.
We report the full results here, and compare the performance of out metric V γ | θ with IRM penalty in formula 4.
Thorough the whole experiments, E all =
{ V, L, C, S } .
We construct four experimental settings.
In each setting, one domain is removed and the rest consists of E tr .
For each domain in
E tr , we split a validation set, and test accuracy is the average accuracy amount validation sets.
The results are shown in Table 5.
First, our results coincide with Gulrajani & Lopez-Paz (2020) that
ERM nearly outperforms any algorithms.
We can see that the OOD accuracy of ERM is either the highest or only slightly lower than Mixup.
Meanwhile, it has a relatively small V γ | θ .
Second, higher
OOD accuracy corresponds to lower V γ | θ .
In addition, we notice that IRM has a relatively low test accuracy and OOD accuracy.
We explain the phenomenon by an improper hyper-parameters in
IRM, although we didn’t change the default hyper-parameters in the code of Gulrajani & Lopez-Paz (2020) ( https://github.com/facebookresearch/DomainBed ).
No matter what, this phenomenon provides a good example in which we can compare our metric with IRM penalty and discuss their advantages and disadvantages.
domain
[Table]
64) (12.83) Mixup 72.
Mixup 75.
81.02 (2.04) (3.10) (1.57) (3.64) (2.80) (10.54) (11.07) (13.52) gDRO 70.
gDRO
80.03 (1.91) (3.92) (2.56) (2.11) (3.27) (9.41) (11.17) (11.50) IRM 38.
IRM 44.
V γ | θ (our metric)
IRM penalty (e-4)
Domain
C
L
S
V
Domain
C
L
S
V
ERM 2.
ERM 1.
1.63 (0.3474) (0.3231) (0.2887) (0.3955) (1.88) (1.03) (0.75) (2.04) Mixup 2.
Mixup 75.
48.3 (0.1926) (0.2003) (0.1492) (0.2304) (32.6) (26.4) (37.2) (30.6) gDRO 3. gDRO
[Table]
Despite that IRM could be a good OOD algorithm, using IRM penalty as the metric to judge
the
OOD property of a learnt model still has much weakness, and some are severe.
First, in different tasks, the value of λ to obtain an OOD model may be different, so as other hyper-parameters like “anneal steps” in IRM code.
Without exhaustive search on the proper value of the hyper-parameters, it’s easy that IRM overﬁts on the penalty term (which is the situation in VLCS).
When IRM overﬁts, the IRM penalty will become quite small (higher λ often leads to smaller penalty), but absolutely overﬁtting on penalty term will not result in good OOD accuracy.
Therefore, the balance between loss and penalty is important.
However, how to ﬁnd a balanced point?
This is a model selection problem, and Gulrajani & Lopez-Paz (2020) propose that an OOD algorithm without model selection is not complete.
No matter what to be used as the metric, it cannot be IRM penalty since we cannot use what is included in the training process as the metric to select training hyper-parameters.
Second, IRM penalty shows a bias on different algorithms.
In the Table 5, the IRM penalty of
IRM is smaller than most algorithms.
Besides, although the OOD accuracy of Mixup is similar to
ERM, its IRM penalty is signiﬁcantly higher.
This is not strange but will limit the usage of IRM penalty.
As for our metric, we mention that small V γ | θ is better.
However, the understanding of “smallness” is based on the relative value of the shufﬂe version and standard version of V γ | θ .
As mentioned in section 5.2, when E all \E tr = { S } , we can see that shufﬂe section 5.2 is obviously smaller than standard version in gDRO, but in ERM and Mixup, these value are relatively close or indistinguishable.
In this case, we know that gDRO captures less invariant features and is not OOD than the other two algorithms.
During the whole process, we can circumvent the direct comparison of V γ | θ in different algorithms, which is quite important.
In summary, IRM penalty makes IRM a good algorithm, but using it as the general metric of OOD performance is completely another picture.