Out-of-Distribution Generalization Analysis via Influence Function

1 INTRODUCTION . Most machine learning systems assume both training and test data are independently and identically distributed , which does not always hold in practice ( Bengio et al . ( 2019 ) ) . Consequently , its performance is often greatly degraded when the test data is from a different domain ( distribution ) . A classical example is the problem to identify cows and camels ( Beery et al . ( 2018 ) ) , where the empirical risk minimization ( ERM , Vapnik ( 1992 ) ) may classify images by background color instead of object shape . As a result , when the test domain is “ out-of-distribution ” ( OOD ) , e.g . when the background color is changed , its performance will drop significantly . The OOD generalization is to obtain a robust predictor against this distribution shift . Suppose that we have training data collected from m domains : S = { Se : e ∈ Etr , |Etr| = m } , Se = { ze1 , ze2 , . . . , zene } with zei ∼ P e , ( 1 ) where P e is the distribution corresponding to domain e , Etr is the set of all available domains , including validation domains , and zei is a data point . The OOD problem we considered is to find a model fOOD such that fOOD = arg min f sup P e∈Eall ` ( f , P e ) , ( 2 ) where Eall is the set of all target domains and ` ( f , P e ) is the expected loss of f on the domain P e. Recent algorithms address this OOD problem by recovering invariant ( causal ) features and build the optimal model on top of these features , such as Invariant Risk Minimization ( IRM , Arjovsky et al . ( 2019 ) ) , Risk Extrapolation ( REx , Krueger et al . ( 2020 ) ) , Group Distributionally Robust Optimization ( gDRO , Sagawa et al . ( 2019 ) ) and Inter-domain Mixup ( Mixup , Xu et al . ( 2020 ) ; Yan et al . ( 2020 ) ; Wang et al . ( 2020 ) ) . Most works evaluate on Colored MNIST ( see 5.1 for details ) where we can directly obtain the worst domain accuracy over Eall . Gulrajani & Lopez-Paz ( 2020 ) has assembled many algorithms and multi-domain datasets , and finds that OOD algorithms can ’ t outperform ERM in some domain generalization tasks ( Gulrajani & Lopez-Paz ( 2020 ) ) , e.g . VLCS ( Torralba & Efros ( 2011 ) ) and PACS ( Li et al . ( 2017 ) ) . This is not surprising , since these tasks only require high performance on certain domains , while an OOD algorithm is expected to learn truly invariant features and be excellent on a large set of target domains Eall . This phenomenon is described as “ accuracy-vs-invariance trade-off ” in Akuzawa et al . ( 2019 ) . Two questions arise in the min-max problem ( 2 ) . First , previous works assume that there is sufficient diversity among the domains in Eall . Thus the supremacy of ` ( f , P e ) may be much larger than the average , which implies that ERM may fail to discover fOOD . But in reality , we do not know whether it is true . If not , the distribution of ` ( f , P e ) is concentrated on the expectation of ` ( f , P e ) , and ERM is sufficient to find an invariant model for Eall . Therefore , we call for a method to judge whether an OOD algorithm is needed . Second , how to judge a model ’ s OOD performance ? Traditionally , we consider test domains Etest ⊂ Etr and use the worst-domain accuracy over Etest ( which we call test accuracy ) to approximate the OOD accuracy . However , test accuracy is a biased estimate of the OOD accuracy unless Etr is closed to Eall . More seriously , It may be irrelevant or even negatively correlated to the OOD accuracy . This phenomenon is not uncommon , especially when there are features virtually spurious in Eall but show a strong correlation to the target in Etr . We give a toy example in Colored MNIST when the test accuracy fails to approximate the OOD accuracy . For more details , please refer to Section 5.1 and Appendix A.4 . We choose three domains from Colored MNIST and use cross-validation ( Gulrajani & Lopez-Paz ( 2020 ) ) to select models , i.e . we take turns to select a domain S ∈ Etr as the test domain and train on the rest , and select the model with max average test accuracy . Figure 1 shows the comparison between ERM and IRM . One can find that no matter which domain is the test domain , ERM model uniformly outperforms IRM model on the test domain . However , IRM model achieves consistently better OOD accuracy . Shortcomings of the test accuracy here are obvious , regardless of whether cross-validation is used . In short , the naive use of the test accuracy may result in a non-OOD model . To address this obstacle , we hope to find a metric that correlates better with model ’ s OOD property , even when Etr is much smaller than Eall and the “ worst ” domain remains unknown . Without any assumption to Eall , our goal is unrealistic . Therefore , we assume that features that are invariant across Etr should also be across Eall . This assumption is necessary . Otherwise , the only thing we can do is to collect more domains . Therefore , we need to focus on what features the model has learnt . Specifically , we want to check whether the model learns invariant features and avoid varying features . The influence function ( Cook & Weisberg ( 1980 ) ) can serve our purpose . Influence function was proposed to measures the parameter change when a data point is removed or upweighted by a small perturbation ( details in 3.2 ) . When modified it to domain-level , it measures the influence of a domain instead of a data point on the model . Note that we are not emulating the changes of the parameter when a domain is removed . Instead , we are exactly caring about upweighting the domain by δ → 0+ ( will be specified later ) . Base on this , the variance of influence function allows us to measure OOD property and solve the obstacle . Contributions we summarize our contributions here : ( i ) We introduce influence function to domain-level and propose index Vγ|θ ( formula 6 ) based on influence function of the model fθ . Our index can measure the OOD extent of available domains , i.e . how different these domains ( distributions ) are . This measurement provides a basis for whether to adopt an OOD algorithm and to collect more diverse domains . See Section 4.1 and Section 5.1.1 for details . ( ii ) We point out that the proposed index Vγ|θ can solve the weakness of test accuracy . Specifically , under most OOD generalization problems , using test accuracy and our index together , we can discern the OOD property of a model . See Section 4.2 for details . ( iii ) We propose to use only a small but important part of the model to calculate the influence function . This overcomes the huge computation cost of solving the inverse of Hessian . It is not merely for calculation efficiency and accuracy , but it coincides with our understanding that only these parameters capture what features a model has learnt ( Section 4.3 ) . We organize our paper as follows : Section 2 reviews related works and Section 3 introduces the preliminaries of OOD methods and influence function . Section 4 presents our proposal and detailed analysis . Section 5 shows our experiments . The conclusion is given in Section 6 . 2 RELATED WORK . The mismatch between the development dataset and the target domain is one major challenge in machine learning ( Castro et al . ( 2020 ) ; Kuang et al . ( 2020 ) ) . Many works assume that the ground truth can be represented by a causal Direct Acyclic Graph ( DAG ) , and they use the DAG structure to discuss the worst-domain performance ( Rojas-Carulla et al . ( 2018 ) ; Peters et al . ( 2016 ) ; Subbaswamy et al . ( 2019 ) ; Bühlmann et al . ( 2020 ) ; Magliacane et al . ( 2018 ) ) . All these works employ multiple domain data and causal assumptions to discover the parents of the target variable . Rojas-Carulla et al . ( 2018 ) and Magliacane et al . ( 2018 ) also apply this idea to Domain Generalization and Multi-Task Learning setting . Starting from multiple domain data rather than model assumptions , Arjovsky et al . ( 2019 ) proposes Invariant Risk Minimization ( IRM ) to extract causal ( invariant ) features and learn invariant optimal predictor on the top of the causal features . It analyzes the generalization properties of IRM from the view of sufficient dimension reduction ( Cook ( 2009 ) ; Cook et al . ( 2002 ) ) . Ahuja et al . ( 2020 ) considers IRM as finding the Nash equilibrium of an ensemble game among several domains and develops a simple training algorithm . Krueger et al . ( 2020 ) derives the Risk Extrapolation ( REx ) to extract invariant features and further derives a practical objective function via variance penalization . Xie et al . ( 2020 ) employs a framework from distributional robustness to interpret the benefit of REx comparing to robust optimization ( Ben-Tal et al . ( 2009 ) ; Bagnell ( 2005 ) ) . Besides , Adversarial Domain Adaption ( Li et al . ( 2018 ) ; Koyama & Yamaguchi ( 2020 ) ) uses discriminator to look for features that are independent of domains and uses these features for further prediction . Influence function is a classic method from the robust statistics literature ( Robins et al . ( 2008 ; 2017 ) ; Van der Laan et al . ( 2003 ) ; Tsiatis ( 2007 ) ) . It can be used to track the impact of a training sample on the prediction . Koh & Liang ( 2017 ) proposes a second-order optimization technique to approximate the influence function . They verify their method with different assumptions on the empirical risk ranging from being strictly convex and twice-differentiable to non-convex and non-differentiable losses . Koh et al . ( 2019 ) also estimates the effect of removing a subgroup of training points via influence function . They find out that the approximation computed by the influence function is correlated with the actual effect . Influence function has been used in many machine learning tasks . Cheng et al . ( 2019 ) proposes an explanation method , Fast Influence Analysis , that employs influence function on Latent Factor Model to solve the lack of interpretability of the collaborative filtering approaches for recommender systems . Cohen et al . ( 2020 ) uses influence function to detect adversarial attacks . Ting & Brochu ( 2018 ) proposes an asymptotically optimal sampling method via an asymptotically linear estimator and the associated influence function . Alaa & Van Der Schaar ( 2019 ) develops a model validation procedure that estimates the estimation error of causal inference methods . Besides , Fang et al . ( 2020 ) leverages influence function to select a subset of normal users who are influential to the recommendations .

Summary: The authors study the problem of out-of-distribution (OoD) generalization. The key question authors seek to answer is when given access to data from multiple training environments, can one only rely on test accuracy? or does one have to rely on some new measures to estimate the out-of-distribution performance of the model. The authors develop a metric based on influence functions, which authors claim is a better reflection of OoD accuracy than test accuracy. The metric proposed by the authors measures the variance in the model when the data from each environment is upweighted. The authors show that the proposed metric empirically correlates to the OoD performance of the models.