When Does Group Invariant Learning Survive Spurious Correlations?
Abstract
By inferring latent groups in the training data, recent works introduce invariantlearning to the case where environment annotations are unavailable.
Typically,learning group invariance under a majority/minority split is empirically shownto be effective in improving out-of-distribution generalization on many datasets.
However, theoretical guarantee for these methods on learning invariant mechanismsis lacking.
In this paper, we reveal the insufficiency of existing group invariantlearning methods in preventing classifiers from depending on spurious correlationsin the training set.
Specifically, we propose two criteria on judging such sufficiency.
Theoretically and empirically, we show that existing methods can violate bothcriteria and thus fail in generalizing to spurious correlation shifts.
Motivated bythis, we design a new group invariant learning method, which constructs groupswith statistical independence tests, and reweights samples by group label proportionto meet the criteria.
Experiments on both synthetic and real data demonstrate thatthe new method significantly outperforms existing group invariant learning methodsin generalizing to spurious correlation shifts.
Introduction
In many real-world applications, machine learning models inevitably encounter data that rarelyrepresented in its training environment, i.e. being out-of-distribution (OOD).
For example, datacollected under new weather [33], locations [6], or light conditions [9] in vision tasks.
However,machine learning models often fail in generalizing to OOD data, which blocks their deploymentto critical applications [12; 35].
The dependence on spurious correlations that are prone to changeacross environments has been recognized as a major cause of such failure [4; 12; 34].
For example, ithas been shown that models trained on MNLI [36] usually classify sentence pairs with high wordoverlap as the label ‘entailment’, regardless of their semantics [23].
On a new dataset where suchrelation no longer holds, the performance drops over 25% [23; 7].
A notable line of research on improving the robustness of models to distribution shifts is learningfeatures with invariant conditioned label distribution across training environments [25; 4; 15], whichhas been termed invariant learning (IL).
These methods are based on the assumption that causalmechanism keeps invariant across environments [25], while the spurious correlation varies.
Bypenalizing the variance of model prediction across environments, models are then encouraged tocapture causal mechanism instead of spurious correlations.
Recently, invariant learning has been introduced to the scenario where environment labels areunknown [30; 8], which we term the group invariant learning (group-IL).
These methods utilizeprior knowledge on spurious correlations to split the training data into groups.
For example, Teneyet al.
[30] cluster training samples with their predefined spurious features.
A more generic method,
EIIL [8], split training data into the majority/minority sets on which the spurious feature conditioned label distribution varies maximally.
Similar to a priori environments, these groups are supposed toencode variations of spurious information, while hold the causal mechanism.
Though some performance improvements have been gained on several datasets, much uncertaintystill exists on the effectiveness of group-IL methods.
In particular, when these methods can effectivelyaddress spurious correlations remain a question.
Though some theoretical analysis on the successand failure cases of IL with known environments has been proposed [2; 28; 20; 3], they are notsufficient for group-IL.
First, inferred groups may not meet the assumptions on environments inexisting theoretical analysis, thus their conclusions cannot generalize to group-IL.
For example,in each environment, the spurious feature is assumed to have a Gaussian type distribution in [28].
However, the inferred group may not satisfy that condition, for example each group may only containone unique value of spurious feature.
Second, as environments are known and defined with causalstructures, exiting analysis on the effect of environment on IL focus mostly on their number [28; 3]but less on their property or validity.
However, the later are important for group-IL.
Therefore, weneed specific theoretical analysis under the setting of group-IL.
In this study, we discuss necessary conditions for group-IL to survive spurious correlations.
For thispurpose, we first formalize the setting of group-IL and clarify the necessary assumptions requiredfor group-IL, which underlie our theoretical analysis.
We then propose two criteria for group-IL,namely falsity exposure criterion and label balance criterion .
They are respectively for judgingwhether spurious correlations are sufficiently exposed through their variation across groups andwhether group invariance can reach spurious-free conditions.
Based on that, we discuss the successand failure cases of existing methods.
In some synthetic benchmarks (e.g. colored-MNIST in [4; 8]),the majority/minority groups meet the two criteria.
However, in a case when the spurious feature ismultivariate, the majority/minority split violates both criteria according to our theoretical analysisand observation on real datasets.
As a result, existing group-IL methods are insufficient for solvingspurious correlations.
To fix these problems, we propose a new group-IL method guided by the two criteria.
Specifically,this method contains the following two steps to meet the two criteria.
First, groups are defined bystratifying the prediction of a reference predictor which encodes spurious correlations.
The strata isconstructed with statistical tests, such that the spurious prediction is independent of the label on eachgroup.
Second, the label proportion of each inferred group is balanced by attaching weights to eachinstance within the group.
Models are then trained with invariant learning objectives on the definedgroups.
We term this method Spurious-Correlation-Strata Invariant Learning with Label-balancing,abbreviated as SCILL.
We further show that SCILL is provably sufficient in reaching spurious-freewith ideal reference models.
To demonstrate the effectiveness of our proposed strategy, we conduct experiments on both syntheticand real data benchmarks on spurious correlations shifts in image classification and natural languageinference (NLI).
Specifically, we adopt two different invariant learning objectives, IRM (IRMv1) [4]and REx (V-REx) [15], to show the consistency of SCILL.
To show the availability of SCILL, we alsoexperiment with PGI [1] and cMMD [16; 1], which are feature invariance targets used with EIIL [8]in [1].
The experimental results show that SCILL with all the four invariance objectives consistentlyoutperforms the existing state-of-the-art method EIIL in generalizing to spurious correlation shifts.
Ablation study further shows the effectiveness of each component in SCILL.
Our main contributions can be summarized as follows.
• We propose two criteria for group-IL, and theoretically show the insufficiency of existingmethods.• Guided by the two criteria, we propose a new practical group-IL method which is provablysufficient in solving spurious correlations.• Extensive experiments on both synthetic and real-data benchmarks show that SCILL significantly outperforms existing methods on image classification and NLI tasks.
Related works
Combating spurious correlations.
A typical kind of distribution-shift is caused by the shift ofspurious correlations [35], which are correlations between meaningless features (e.g. hospital tokenson a lung scan) and labels (e.g. has pneumonia or not) in the training set.
The existence of spurious
[Figure]
correlations in popular benchmarks have been revealed by many works [13; 26; 23; 29].
For example,predictive models with only incomplete semantic inputs or syntactic statistics can achieve highaccuracy in NLI benchmarks [13; 26].
Geirhos et al.
[11] point out that deep neural networks areprone to take easy-to-fit spurious correlations, i.e. shortcut strategies, in solving problems.
As aresult, resolving model’s dependence on spurious correlation is important for their robustness todistribution shifts.
Invariant learning (IL).
Many works on domain generalization focus on capturing invarianciesacross training environments [12].
Recently a new kind of strategy which has made significantimpacts is to learn features that permit an invariant predictor across environments [25; 4; 15; 34],termed as invariant learning in this paper.
Such strategy is grounded upon the theory of causality [24],where Structural Equation Models [25] or causal graphs [34] are used to describe assumptions on thedata generation process.
The feature-conditioned label distribution invariance is then induced by theinvariance of causal mechanisms in different environments.
Group invariant learning.
In recent works, IL is extended to the scenario without a priori environment labels, but with knowledge on spurious correlations in the training data [30; 8].
Such knowledgeis proven to be necessary in this setting [17].
They are utilized to split the training data into groups,which are supposed to encode variations of spurious information so that they can be avoided bylearning the invariance.
For example, Teney et al.
[30] cluster samples according to their questiontypes.
Liu et al.
[18] construct groups with varying spurious correlations, based on the spuriousfeatures uncovered with feature selection.
A more generic method proposed in EIIL [8] assume theaccess to a reference predictor which encodes spurious correlations, and exploit the outputs of thereference predictor to split training data into two groups, namely the majority and the minority.
Thisstrategy is shown effective, sometimes even outperform an oracle method using true environmentlabels in proving the OOD generalization [8] and also systematic generalization [1].
Formalization of group invariant learning
In this section, we introduce the problem setting and formalization of the scheme of group-IL.
Consider the task of learning a classifier f : X → Y , which maps a value x ∈ X of the input variable X to a value y ∈ Y of the target variable Y .
For example, map an image of horse on grass to thelabel ‘horse’.
We denote x inv as the essential features of input x which define its class label (e.g.the shape of the horse), while x sp as features of x should not inform the label of x (e.g. the grassbackground).
X inv , X sp denote the corresponding random variables.
The target is then to learn aspurious-free predictor whose predictions only depend on the feature X inv , thus is supposed to haveinvariant performance on any dataset.
We first introduce the setting of IL.
IL methods suppose the training data D are collected undermultiple environments E , i.e. D =
{ D e } e ∈E .
D e = { x e i , y e i } n e i =1 contains data i.i.d. sampled fromthe probability distribution P e ( X ×Y ) .
P e ( Y | X inv ) is invariant among different e , while P e ( Y | X sp )varies.
Such idea is based on the invariance of causal mechanisms across environments [25; 4], withassumptions on the causal structure of the data generating process.
Figure 1 demonstrates four kinds ofdifferent assumptions in existing works on invariant learning.
In these causal structures, environmentis treated as a random variable E take values in E all , which satisfies E ∈ E all . P e ( · ) := P ( ·| E = e ) . X inv and X sp are latent feature variables generating the observation X , i.e. X = r ( X inv , X sp ) .
Here r is generally assumed as an injective function so that the latent features can be recovered fromthe observations [28; 2; 34].
Align with the formalization in the former paragraph where X sp , X invare assumed recognizable from X , in this paper we adopt the same assumption.
In all the four kindsof causal graphs, P e ( Y | X inv ) := P ( Y | X inv , E = e ) keeps invariant under different e ∈ E all , while 134 P e ( Y | X sp ) , P e ( X sp ) , P e ( X inv ) , and P e ( X inv | Y ) can vary across different e ∈ E all .
Suppose the predictor f can be decomposed into f = c ◦ Φ , where Φ : X → H denotes a featureencoder which maps the input into a representation space H , c : H → Y is a classifier.
The target ofinvariant learning is then to search for a Φ satisfies the following constraint:
[Equation]
It is termed as Environment Invariance Constraint (EIC).
Note that the EIC stated in [8] is a weakerform of this EIC.
The constraint is incorporated into the training target via a penalty term.
In a genericform, the learning target of invariant learning methods can be written as follows:
[Equation]
where R e ( f ) stands for the expected loss of f on the environment e , weighted by a scalar λ e .
S e ( f )stands for some statistics of f on e , and the penalty is on the variation of S e ( f ) to measure thedeviation degree of EIC.
In IRM [4], the penalty is the summation of S e ( f ) =
∥∇ w R e ( w ◦ f ) ∥ , wherew is a constant scalar multiplier of 1.0 for each output dimension.
In V-REx [15], S e ( f ) =
R e ( f ) ,and the penalty is the variance of S e ( f ) on different environments.
In CLOvE [34], the penalty isdefined as the summation of calibration errors of the model on each environment.
Group invariant learning methods release the dependency of invariant learning on predefined environments by splitting the training data into groups.
Suppose D is sampled from the distribution 149 P ( X × Y ) .
Intuitively, those groups are expected to hold the invariant mechanism P ( Y | X inv ) , whileinform the variation of X sp .
As a result, it is meaningless to divide groups according to X inv .
Ingroup-IL methods [30; 18; 8], group inference algorithms are designed to utilize knowledge on X sp orthe correlation between X sp and Y .
Formally, denote the inferred groups as G := { g 1 , g 2 , . . g m } . Define G : X × Y → I as the function which maps a sample to its group identity, i.e. G ( x, y ) =
i ifand only if ( x, y ) ∈ g i . We have G is ( X sp , Y ) -measurable, i.e. it is a function of X sp , as in [30; 18],or both X sp and Y [8].
In the following sections, we ground our analysis on group-IL with the causal structures in Figure 1 (a)and (b), while without additional assumptions on the causal models.
Our choice of causal structuresis based on the following two observations.
First, in the causal graph (d), X sp is a backdoor variablebetween X inv and Y and confounded with X inv by an unobserved variable.
As a result, whether theinvariant mechanism holds on each group is indeterminate without additional assumptions on themechanisms between X sp and X inv .
Second, as shown by Ahuja et al.
[2], invariance itself can notdeal with spurious feature for causal structure (c).
Additional knowledge or penalty, e.g. informationbottleneck, is needed together with group-IL.
Therefore, we investigate group-IL under the causalstructures depicted by graphs (a) and (b).
Two group criteria
With the above formulation, we are ready to theoretically analyze the ability of existing group-ILmethods in learning spurious-free predictors.
For this purpose, in this section we derive two necessaryconditions for group-IL in surviving spurious correlations, i.e. falsity exposure, and label balance.
Both conditions can be used as criteria to judge the sufficiency of group-IL methods.
We then showthat existing methods can violate the two criteria, thus become insufficient for learning a spurious-freepredictor.
4.1 Falsity exposure criterion
As groups are supposed to expose variance of spurious features so that they can be avoided byinvariant learning, a natural idea is to take into account the sufficiency of such exposure on inferredgroups.
Ideally, if groups are split according to X sp , any variance of P ( Y | X sp ) is then fully exposed.
However, such split is only practical when the value of X sp is accessible and sparse.
On the contrary,we consider the condition when groups provide insufficient exposure.
Intuitively, if some spuriouscorrelation keeps invariant across groups, its variation is then not exposed, thus group invariantpredictor may still depend on such correlation.
Formally, this can be written as the followingcriterion.
Criterion 4.1 (Falsity Exposure) .
For any function h that satisfies ∀ g, g ′ ∈ G , P
( Y | h
( X sp ) , g )
= 182 P ( Y | h
( X sp ) , g ′ ) , it must satisfies P ( Y | h
( X sp ))
=
P ( Y ) .
Intuitively, if the falsity exposure criterion is not satisfied, h ( X sp ) will be an invariant feature acrossenvironments, with predictive ability on Y .
The predictor depending on both X inv and h ( X sp )can satisfy EIC, but fails to be free of spurious features.
The following theorem formalizes thesignificance of the falsity exposure criterion.
Theorem 4.2.
Suppose the falsity exposure criterion is violated, i.e. ∃ h satisfies P ( Y | h ( X sp ) , g ) = 188 P
[Equation]
Then the optimal solution of group-IL is f ( X ) = 189 P
[ Y | X inv , h
( X sp )] , which fails to generalize when P ( Y | X sp ) shifts.
4.2 Label balance criterion
Even if we have sufficient falsity exposure, would the constraint in group-IL, i.e. EIC, guarantee themodel to be free of spurious correlations?
We study this problem by analyzing the relation between
EIC and the constraint for a predictor to be spurious-free.
In our assumed causal structures, X inv ⊥⊥
X sp | Y .
Thus, a spurious-free predictor, which only depends on X inv , satisfies f ( X ) ⊥⊥ X sp | Y .
Infact, it can be proved that this is a sufficient condition for f ( X ) to be invariant to the intervention [24]on X sp .
As a result, we term f ( X ) ⊥⊥ X sp | Y as the spurious-free constraint , i.e.
[Equation]
where B is the image set of X sp .
The following is a necessary condition for EIC to induce the aboveconstraint, which we term as the label balance criterion.
It states that the label proportion in differentenvironments should be the same.
Criterion 4.3 (Label Balance) .
For any g, g ′ ∈ G and y, y ′ ∈ Y with non-zero P ( Y = y | g ) , P ( Y =y ′ | g ) , P ( Y = y | g ′ ) and P ( Y =
y ′ | g ′ ) , the following equation holds.
[Equation]
Mathematically, the following theorem shows the significance of this criterion on the effectivenessgroup-IL.
Theorem 4.4.
With a set of groups G inferred by ( X sp , Y ) , if the label balance criterion is violated,functions satisfying EIC can not satisfy SFC.
4.3 Analysis on existing group invariant learning methods
Now we analyze whether the groups in existing group-IL methods meet the two criteria.
Directly,randomly grouped clusters of X sp as in [30] do not guarantee to meet either criteria, and clusters of 209 P ( Y | X sp ) [18] do not meet the label balance criterion.
In the following discussions, we focus on themajority/minority groups inferred by the EI algorithm in EIIL [8].
We find that in some syntheticdatasets where the spurious feature only has two distinct values, the majority/minority groups satisfythe two criteria.
However, observation on real datasets and theoretical results on the case when thespurious feature is multivariate show that they can violate both criteria.
On some synthetic datasets constructed in existing works [1; 8], the majority/minority groups satisfythese two criteria.
For example, on both colored-MNIST [4] and coloured-MNIST [1], Y has auniform distribution, and the spurious correlation has the same ratio for any spurious features, e.g. 217 P ( Y =
| color = green )
=
P ( Y = 1 | color = red ) on colored-MNIST.
It can be proved that in thiscase the majority/minority groups satisfy both criteria (see the appendix).
However, it no longer holds in general cases.
Empirically, we observe that on MNLI, the labeldistributions of the two groups inferred by EI are significantly different.
Specifically, the ratio ofthe counts of label 0 and label 1 on the two groups are 2.87 and 0.17 respectively.
The followingproposition theoretically provides a case where the majority/minority split satisfies the label balancebut breaks the falsity exposure, and invariant learning objectives fail to find the spurious-free classifier.
Proposition 4.5.
Suppose we have ( X, Y ) ∼ P ( X, Y ) .
Y takes value in { 0 , 1 } .
X is formed withspurious feature variables
X sp =
( B 0 , B 1 ) , and invariant feature variable S , i.e. X = r
( B 0 , B 1 , S ) ,for some injective function r .
B 0 and B 1 are both binary variables, which take values in { b 00 , b 10 } and{ b 01 , b 11 } respectively.
B 0 , B 1 and S are conditionally independent given Y .
Suppose P ( Y = j | B i =
b ji )
=
p i , ∀ i, j ∈ { 0 , 1 } , and p 0 > p 1 .
Then we have 1) the majority/minority split e mar , e minviolates falsity exposure criterion.
2) the optimal classifier under invariant learning objectivesdepends on B 1 .
In this case, we suppose the spurious feature can be decomposed into two variables that are conditionally independent with each other given the label.
Such case can realize when the dataset containsmultiple kinds of spurious features.
For example, in the image classification task, the backgroundpattern and the color of an object can be independent but both correlate with the label spuriously.
SCILL: a new method
According to the above discussion in Section 4, existing group-IL methods may fail to meet thetwo criteria, leading to insufficient training loss for solving spurious correlations.
Faced with thischallenge, we propose a new group-IL method, to satisfy the two criteria.
For the generality, weonly assume the access to a reference model, as in EIIL, instead of spurious features.
Our newmethod includes two steps.
In the group inference step, we define groups as spurious correlation strataconstructed with the reference model, for the falsity exposure criterion.
While in the training objectivestep, we reweight each sample with group label proportion to meet the label balance criterion.
Tohighlight the two parts, our method is named as Spurious-Correlation-strata Invariant Learning with
Label-balance, abbreviated as SCILL.
Theoretically, we prove that this method is provably sufficientwith ideal reference models, i.e. its optimal solution is spurious-free.
5.1 Group inference with statistical split
We first introduce the group inference step in SCILL.
As in EIIL, we assume a reference model f r isavailable, which encodes spurious correlations in the training data.
f r can be a model trained withempirical risk minimization (ERM) [8] or a model designed to capture spurious correlations [19].
We propose to construct groups by stratify the outputs of f r , with the target that Y ⊥⊥ f r | g .
Themotivation of introducing spurious correlation strata comes from the deduction of falsity exposurecriterion, i.e. when groups meet the requirements that X sp ⊥⊥ Y | g , the falsity exposure criterion issatisfied.
That is because with the above two conditions, we have for any function h , h
( X sp ) ⊥⊥ Y | g ,as a result P ( Y | h
( X sp ) , g )
=
P ( Y | g ) .
If P ( Y | h ( X sp ) , g ) =
P ( Y | h
( X sp ) , g ′ ) , ∀ g, g ′ ∈ G , we have 255 P ( Y | g )
=
P ( Y | g ′ )
=
P ( Y ) .
Thus P ( Y | h ( X sp ) , g ) =
P ( Y ) .
As a result the falsity exposure criterionis satisfied.
As different output values of the reference model f r inform the difference in P ( Y | X sp ) ,
Y ⊥⊥ f r | g approximates P ( Y | X sp ) ⊥⊥ Y | g , which is equivalent to X sp ⊥⊥ Y | g .
We propose to construct such environments through the statistical-split algorithm, inspired by thealgorithm proposed in [14] for propensity score estimation.
Specifically, we step-wise split currentgroups into subgroups according to hypothesis-test statistics.
For example for the binary class case, aset B of samples is first divided into two subsets L 0 , L 1 according to the labels of samples.
Thenthe two-sample t-statistics t B of log[ f r ( x ) 0 /f r ( x ) 1 ] are computed on the two sets.
If t B exceedsa fixed threshold thr , the B is then split into two subsets according to the median of f r ( x ) 0 on
B .
In this way, we enhance f r ( X ) ⊥⊥ Y in each group.
Note that thr is a hyper-parameter of thisalgorithm.
Empirical robustness analysis on thr is conducted in our experiments.
More details aboutthe algorithm are provided in the Appendix.
5.2 Training with reweighted loss
Now we introduce the second step.
To guarantee the label balance criterion, we reweight each samplewith group label proportion in the training loss.
Correspondingly, the objective of invariant learningbecomes the following form:
[Equation]
[Table]
where ˜ R g ( f ) =
E [ w g ( Y ) L g ( f ( X ) , Y )] , ω g ( y ) :
= P ( Y = y ) / P ( Y = y | g ) for nonzero P ( Y = y | g ) .
The weight function is defined to balance the label distribution between groups, i.e. P ( Y | g )
= 273 P ( Y | g ′ ) , ∀ g, g ′ , so as to achieve the label balance criterion.
With the above two parts, we can theoretically prove that our method is able to find the spurious-freepredictor, as illustrated in the following theorem.
Theorem 5.1.
If G satisfies f ∗ r ( X ) ⊥⊥ Y on ∀ g ∈ G , where f ∗ r ( x ) minimizes the spurious-onlyprediction loss L rce = E x,y p ( x, y ) log f
( X sp ( x )) , the optimal model minimizing the objective (3)satisfies SFC.
Experiments
In this section, we first conduct experiments to show that SCILL outperforms existing group-ILmethods in generalizing to spurious correlation shifts.
Then we empirically analyze whether theexperimental improvements are consistent with the theoretical findings.
6.1 Experimental settings
Now we describe our experimental settings, including datasets, models and some training details.
More details are provided in the Appendix.
Datasets We conduct experiments on both synthetic and real-world datasets.
The synthetic dataset,
Patched-Colored MNIST (PC-MNIST), is constructed as a realization of the conditions in the Proposition 4.5 to verify the proposed criteria.
It is derived from MNIST, by assigning two conditionallyindependent spurious features given label, namely the color and patch bias to each image.
The designof the patch bias is inspired by [5].
MNLI-HANS is a benchmark widely used in many previousworks on combating spurious correlations, such as [7; 31].
In our experiments, we follow the practiceto utilize MNLI [36] as the training data and HANS [23] as the test data.
Baselines and configurations In our experiments, we compare SCILL with two baselines, i.e. ERMand EIIL [8].
ERM represents the method with traditional empirical risk minimization (ERM)approach.
EIIL is a state-of-the-art group-IL method, where groups are inferred by searching anassignment to make the reference model maximally violates the invariant learning principle.
Weexperiment with three different invariance penalties: IRM [4], REx (V-REx) [15], cMMD [16; 1] and [Table]
PGI [1].
Note that cMMD and PGI target to learn group invariant predictions conditioning on thelabel, different from EIC.
See Appendix for more details of the four penalties.
The training configurations are presented as follows.
For PC-MNIST, we adopt the classifier proposedin [4] for Colored MNIST, which is a MLP with two hidden layers of 390 neurons.
The referencemodel is a MLP with the same structure trained with ERM on the training set, following the settingin EIIL on Colored MNIST.
While for MNLI, we use a BERT-based classifier with the standardsetup for sentence pair classification [10].
The reference model is the same as the biased classifierpropose in [31], which is trained on top of some hand-crafted syntactic features.
For each task, allimplementations of SCILL and EIIL adopt the same model configurations and pretrained referencemodels.
Since models are tested with OOD data, it is important to specify the model selection strategy,as has been revealed by Gulrajani and Lopez-Paz [12] for the case of domain generalization.
In ourexperiments, we report results with 3 different model selection strategies, including ID, Oracle, and TEV.
ID refers to the strategy based on model performance on in-distribution validation set as usedin [31].
Oracle refers to the selection based on data from the test data distribution, as used in [8; 12].
While TEV is a new strategy adapted from the training-domain validation method in [12] to theinferred groups, which alleviates the dependence with the test data as ID.
The detailed algorithm canbe found in the Appendix.
6.2 Experimental results
Now we demonstrate our experimental results, including performance comparison and detailedanalysis.
More empirical results including the robustness analysis on the hyperparameter in SCILLcan be found in the Appendix.
6.2.1 Performance comparison
Table 1 and 2 shows the experimental results on PC-MNIST and MNLI-HANS, respectively.
Themain observation is that all implementations of SCILL consistently outperform the counterpart of EIILacross all model selection strategies, in terms of the performance on OOD data.
Comparing differentmodel selection strategies, Oracle performs the best for both EIIL and SCILL.
However, SCILL withthe TEV strategy has the ability to outperform EIIL with Oracle, demonstrating the superiority ofour new objective.
Additionally, SCILL also gains improvements against some debiasing methodsutilizing the same reference model (see the appendix).
6.2.2 Ablation study and verification of the two criteria
Our main theoretical results in Section 4, i.e. Theorem 4.2 and 4.4 reveal that the two group criteriaare necessary conditions for group-IL to survive spurious correlations.
Now we discuss the empiricalverification of the significance of the two criteria.
Falsity exposure criterion. To show the significance of the falsity exposure criterion, we comparethe performance of methods under the case when label balance criterion is satisfied.
On PC-MNIST,both SCILL and EIIL groups satisfies the label balance criterion 1 , while EIIL groups provably violatethe falsity exposure, according to Proposition 4.5. The significant improvement of SCILL over EIILon Table 1 then shows the importance of falsity exposure criterion. To exclude the effect of thenoise in reference model in the group inference, we further implement SCILL with the ground-truthspurious predictor, obtaining SCILL gt in Table 1. The groups then satisfies the falsity exposurecriterion.
We construct the ground truth majority/minority split and experiment with IL methods,obtaining results in the row maj./min. in Table 1. The significant performance drop of maj./min.compared with SCILL gt verifies the importance of falsity exposure for group-IL. On MNLI, the labelin EIIL groups is unbalanced (see the appendix).
Therefore we attach the instance reweight step asin SCILL to EIIL, obtaining EIIL lb which satisfies the label balance criterion.
As shown in Table 2, EIIL lb fails to achieve improved performance, which verifies the necessity of the falsity exposure.
Label balance criterion. To verify the necessity of label balance criterion, we investigate the caseswhen the falsity exposure is satisfied.
As the SCILL gt on PC-MNIST satisfies the falsity exposure, weconstruct such cases by disturbing the label balancing weights in SCILL. We multiply the estimatedlabel proportion of class 0 by 0.5 to get the unbalanced SCILL ub .
As shown in Table 1, under Oracleselection, the test accuracy of SCILL ub drops approximately 14% compared with SCILL gt , thusverifies the impact of label balance.
More results can be found in the Appendix.
We further show the importance of the instance reweight step in SCILL, which is designed followingthe label balance criterion. For this, we remove the instance reweight step in SCILL, obtaining
SCILL uw .
The experimental results in Table 1 and 2 show that SCILL uw performs worse than SCILL,demonstrating the importance of the instance reweight step in SCILL.
Discussions
The main limitation of the paper is our assumptions on the causal structure.
In fact, our conclusionscan be generalized to more complex structures, e.g. those in [32] (see the appendix).
The most centralassumption is the conditional independence between X inv and X sp given Y , which is adopted inmany existing works on solving spurious correlations [7; 32; 37].
It would be an important directionto find causal structures on which the assumption is not satisfied while group-IL can still be effective.
As this paper focuses on analyzing group invariant learning, comparing SCILL with other algorithmsbesides group-IL is beyond the scope of this paper.
Notably, the form of the objective of SCILLappears to be similar to two recent methods in solving spurious correlations [22; 27], though thepenalties are different.
However, they are only applied for the case when spurious features can beexplicitly defined [27], and are also discrete as assumed in [22].
Besides the setting of group invariant learning, the two criteria may also bring benefits to the study ofdomain generalization.
Furthermore, our empirical results show SCILL can achieve good performancewith other kinds of invariances besides that in invariant learning, e.g. the invariance of the distributionof model outputs conditioned on the label.
Discussing the effect of the two group criteria on otherkinds of group invariance is a potential research direction.
Conclusion
This paper is concerned with when group invariant learning (group-IL) can survive spurious correlations.
We first formulate the setting of group-IL and necessary assumptions.
Then we theoreticallyanalyze the necessary conditions for group-IL in learning spurious-free predictors, and obtain twogroup criteria, i.e. falsity exposure and label balance.
Considering the limitations of previous group-ILmethods, we propose a new method SCILL to satisfy the two criteria.
Furthermore, we theoreticallyprove that SCILL has the ability to learn a spurious-free predictor.
Finally, we conduct extensiveexperiments on both synthetic and real data to evaluate the proposed new method.
Experimentalresults show that SCILL significantly outperforms existing SOTA group-IL methods, owing to itsability to satisfy the two criteria.
The empirical studies validates our theoretical findings.
Checklist
For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper’s
contributions and scope?
[Yes]
(b) Did you describe the limitations of your work?
[Yes] See Section 7.
(c) Did you discuss any potential negative societal impacts of your work?
[No]
(d) Have you read the ethics review guidelines and ensured that your paper conforms to
them?
[Yes]
If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results?
[Yes] See Section
(b) Did you include complete proofs of all theoretical results?
[Yes] In the appendix.
If you ran experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main ex-
perimental results (either in the supplemental material or as a URL)?
[Yes]
In the
supplemental material.
(b) Did you specify all the training details (e.g., data splits, hyperparameters,
how they
were chosen)?
[Yes] See the appendix.
(c) Did you report error bars (e g., with respect to the random seed after running experiments multiple times)?
[Yes]
(d) Did you include the total amount of compute and the type of resources used (e.g., typeof GPUs, internal cluster, or cloud provider)?
[No]
If you are using existing assets (e g., code, data, models) or curating/releasing new assets...
(a) If your work uses existing assets, did you cite the creators?
[Yes]
(b) Did you mention the license of the assets?
[N/A]
(c) Did you include any new assets either in the supplemental material or as a URL?
[N/A]
(d) Did you discuss whether and how consent was obtained from people whose data you’re
using/curating?
[N/A]
(e) Did you discuss whether the data you are using/curating contains personally identifiable
information or offensive content?
[N/A]
If you used crowdsourcing or conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, ifapplicable?
[N/A]
(b) Did you describe any potential participant risks, with links to Institutional Review
Board (IRB) approvals, if applicable?
[N/A]
(c) Did you include the estimated hourly wage paid to participants and the total amount
spent on participant compensation?
[N/A]