Original: P REDICTIVE M ODELING IN THE P RESENCE OF N UISANCE -I NDUCED S PURIOUS C ORRELATIONS
Revised: P REDICTIVE M ODELING IN THE P RESENCE OF N UISANCE -I NDUCED S PURIOUS C ORRELATIONS

Original: Anonymous authors

Original: Paper under double-blind review

Original: A BSTRACT
Revised: A BSTRACT

Original: In many prediction problems, spurious correlations are induced by a changing relationship between the label and a nuisance variable that is also correlated with the covariates.
Revised: In many prediction problems, spurious correlations are induced by a changing relationship between the label and a nuisance variable that is also correlated with the covariates.

Original: For example, in classifying animals in natural images, the background, which is the nuisance, can predict the type of animal.
Revised: For example, in classifying animals in natural images, the background, which is a nuisance, can predict the type of animal.

Original: This nuisance-label relationship does not always hold, and the performance of a model trained under one such relationship may be poor on data with a different nuisance-label relationship.
Revised: This nuisance-label relationship does not always hold, and the performance of a model trained under one such relationship may be poor on data with a different nuisance-label relationship.

Original: To build predictive models that perform well regardless of the nuisance-label relationship, we develop Nuisance-Randomized Distillation (N U RD).
Revised: To build predictive models that perform well regardless of the nuisance-label relationship, we develop Nuisance-Randomized Distillation (N U RD).

Original: We ﬁrst deﬁne the nuisance-varying family , a set of distributions that differ only in the nuisance-label relationship.

Original: Wethen introduce the nuisance-randomized distribution, a distribution where the nuisance and the label are independent.
Revised: We introduce the nuisance-randomized distribution, a distribution where the nuisance and the label are independent.

Original: Under this distribution, we deﬁne the set of representations such that conditioning on any member, the nuisance and the label remain independent.
Revised: Under this distribution, we deﬁne the set of representations such that conditioning on any member, the nuisance and the label remain independent.

Original: We prove that the representations in this set always perform better than chance, while representations outside of this set may not.
Revised: We prove that the representations in this set always perform better than chance, while representations outside of this set may not.

Original: N U RD ﬁnds a representation from this set that is most informative of the label under the nuisance-randomized distribution, and we prove that this representation achieves the highest performance within the set on every distribution in the nuisance-varying family.
Revised: N U RD ﬁnds a representation from this set that is most informative of the label under the nuisancerandomized distribution, and we prove that this representation achieves the highest performance regardless of the nuisance-label relationship.

Original: We evaluate N U RD on several tasks including chest X-ray classiﬁcation where, using non-lung patches as the nuisance, N U RD produces models that predict pneumonia under strong spurious correlations.
Revised: We evaluate N U RD on several tasks including chest X-ray classiﬁcation where, using non-lung patches as the nuisance, N U RD produces models that predict pneumonia under strong spurious correlations.

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: Spurious correlations are relationships between the label and the covariates that are prone to change between training and test distributions (Gulrajani and Lopez-Paz, 2020).
Revised: Spurious correlations are relationships between the label and the covariates that are prone to change between training and test distributions (Gulrajani and Lopez-Paz, 2020).

Original: Predictive models that exploit spurious correlations can perform worse than even predicting without covariates on the test distribution (Arjovsky et al., 2019).
Revised: Predictive models that exploit spurious correlations can perform worse than even predicting without covariates on the test distribution (Arjovsky et al., 2019).

Original: Discovering spurious correlations requires more than the training distribution because any single distribution has a ﬁxed label-covariate relationship.
Revised: Discovering spurious correlations requires more than the training distribution because any single distribution has a ﬁxed label-covariate relationship.

Original: Often, spurious correlations are discovered by noticing different relationships across multiple distributions between the label and nuisance factors correlated with the covariates.
Revised: Often, spurious correlations are discovered by noticing different relationships across multiple distributions between the label and nuisance factors correlated with the covariates.

Original: We call these nuisance-induced spurious correlations.
Revised: We call these nuisance-induced spurious correlations .

Original: For example, in classifying cows vs. penguins, typical images have cows appear on grasslands and penguins appear near snow due to natural habitats (Arjovsky et al., 2019; Geirhos et al., 2020), but these animals can be photographed outside their habitats.
Revised: For example, in classifying cows vs. penguins, typical images have cows appear on grasslands and penguins appear near snow, their respective natural habitats (Arjovsky et al., 2019; Geirhos et al., 2020), but these animals can be photographed outside their habitats.

Original: In classifying hair color from celebrity faces on CelebA (Liu et al., 2015), gender is correlated with the hair color.
Revised: In classifying hair color from celebrity faces on CelebA (Liu et al., 2015), gender is correlated with the hair color.

Original: This relationship may not hold in different countries (Sagawa et al., 2020).
Revised: This relationship may not hold in different countries (Sagawa et al., 2020).

Original: In language, sentiment of a movie review determines the types of words used in the review to convey attitudes and opinions.
Revised: In language, sentiment of a movie review determines the types of words used in the review to convey attitudes and opinions.

Original: However, directors’ names appear in the reviews and are correlated with positive sentiment in time periods where directors make movies that are well-liked (Wang and Culotta, 2020a).
Revised: However, directors’ names appear in the reviews and are correlated with positive sentiment in time periods where directors make movies that are well-liked (Wang and Culotta, 2020a).

Original: In X-ray classiﬁcation, conditions like pneumonia are spuriously correlated with non-physiological traits of X-ray images due to the association between the label and hospital X-ray collection protocols (Zech et al., 2018).
Revised: In X-ray classiﬁcation, conditions like pneumonia are spuriously correlated with non-physiological traits of X-ray images due to the association between the label and hospital X-ray collection protocols (Zech et al., 2018).

Original: Such factors are rarely recorded in datasets but produce subtle differences in X-ray images that convolutional networks easily learn (Badgeley et al., 2019).
Revised: Such factors are rarely recorded in datasets but produce subtle differences in X-ray images that convolutional networks easily learn (Badgeley et al., 2019).

Original: We formalize nuisance-induced spurious correlations in a nuisance-varying family of distributions where any two distributions are different only due to the differences in the nuisance-label relationship.
Revised: We formalize nuisance-induced spurious correlations in a nuisance-varying family of distributions where any two distributions are different only due to the differences in the nuisance-label relationship.

Original: As the nuisance is informative of the label, predictive models exploit the nuisance-label relationship to achieve the best performance on any single member of the family.
Revised: As the nuisance is informative of the label, predictive models exploit the nuisance-label relationship to achieve the best performance on any single member of the family.

Original: However, predictive models that perform best on one member can perform even worse than predicting without any covariates on another member.
Revised: However, predictive models that perform best on one member can perform even worse than predicting without any covariates on another member, which may be out-of-distribution ( OOD ).

Original: We develop N U RD to use data collected under one nuisance-label relationship to build predictive models that do well on every member of the family regardless of the nuisance-label relationship in that member.
Revised: We develop N U RD to use data collected under one nuisance-label relationship to build predictive models that perform well on other members of the family regardless of the nuisance-label relationship in that member.

Original: In section 2, we motivate and develop ideas that help guarantee performance on every member of the family.
Revised: In section 2, we motivate and develop ideas that help guarantee performance on every member of the family.

Original: The ﬁrst is the nuisance-randomized distribution : a distribution where the nuisance is independent of the label.
Revised: The ﬁrst is the nuisance-randomized distribution : a distribution where the nuisance is independent of the label.

Original: An example is the distribution where cows and penguins have equal chances of appearing on backgrounds of grass or snow.
Revised: An example is the distribution where cows and penguins have equal chances of appearing on backgrounds of grass or snow.

Original: The second is an uncorrelating representation : a representation of the covariates such that under the nuisance-randomized distribution, the nuisance remains independent of the label after conditioning on the representation.
Revised: The second is an uncorrelating representation : a representation of the covariates such that under the nuisance-randomized distribution, the nuisance remains independent of the label after conditioning on the representation.

Original: The set of such representations is the uncorrelating set .
Revised: The set of such representations is the uncorrelating set .

Original: We show that the nuisance-randomized conditional of the label given an uncorrelating representation dominates predicting without covariatesin performance on every member in the family, whereas conditionalsnot of this form may not.
Revised: We show that the nuisance-randomized conditional of the label given an uncorrelating representation has performance guarantees: such conditionals perform as well or better than predicting without covariates on every member in the family while other conditionals may not.

Original: Within the uncorrelating set, we characterize one that is optimal on every member of the nuisance-varying family simultaneously and prove  the same optimal performance can be realized by uncorrelating representations that are most informative of the label under the nuisance-randomized distribution.
Revised: Within the uncorrelating set, we characterize one that is optimal on every member of the nuisance-varying family simultaneously . We then prove that the same optimal performance can be realized by uncorrelating representations that are most informative of the label under the nuisance-randomized distribution.

Original: Following the insights in section 2, we develop an algorithm to maximize the information between an uncorrelating representation and the label under the nuisance-randomized distribution, called Nuisance-Randomized Distillation (N U RD).
Revised: Following the insights in section 2, we develop Nuisance-Randomized Distillation (N U RD) in section 3. N U RD ﬁnds an uncorrelating representation that is maximally informative of the label under the nuisance-randomized distribution.

Original: N U RD’s
Revised: N U RD’s

Original: ﬁrst step, nuisance-randomization , breaks the nuisance-label dependence to produce nuisance-randomized data.
Revised: ﬁrst step, nuisance-randomization , breaks the nuisance-label dependence to produce nuisance-randomized data.

Original: We provide two nuisance randomization methods based on generative models and reweighting.
Revised: We provide two nuisance randomization methods based on generative models and reweighting.

Original: The second step, distillation , maximizes the information a representation has with the label on the nusiance-randomized data over a set of uncorrelating representations.
Revised: The second step, distillation , maximizes the information a representation has with the label on the nuisance-randomized data over the uncorrelating set.

Original: We evaluate N U RD on class-conditional Gaussians, labeling colored MNIST images (Arjovsky et al., 2019), detecting waterbirds, and classifying chest X-rays.
Revised: We evaluate N U RD on class-conditional Gaussians, labeling colored MNIST images (Arjovsky et al., 2019), distinguishing waterbirds from landbirds, and classifying chest Xrays.

Original: In the latter, using the non-lung patches as the nuisance, N U RD produces models that predict pneumonia under strong spurious correlations.
Revised: In the latter, using the non-lung patches as the nuisance, N U RD produces models that predict pneumonia under strong spurious correlations.

Original:  N UISANCE -R ANDOMIZATION AND U NCORRELATING S ETS
Revised:  N UISANCE -R ANDOMIZATION AND U NCORRELATING S ETS

Original: We formalize nuisance-induced spurious correlations via a family of data generating processes.
Revised: We formalize nuisance-induced spurious correlations via a family of data generating processes.

Original: Let y be the label, z be the nuisance, and x be the covariates (features).
Revised: Let y be the label, z be the nuisance, and x be the covariates (i.e. features).

Original: The family consists of distributions where the only difference in the members of the family comes from the difference in their nuisance-label relationships.
Revised: The family consists of distributions where the only difference in the members of the family comes from the difference in their nuisance-label relationships.

Original: Let D index a family of distributions F = { p D } D ; a member p D in the nuisance-varying family of distributions F takes the form
Revised: Let D index a family of distributions F = { p D } D ; a member p D in the nuisance-varying family of distributions F takes the following form:

Original: [Equation]
Revised: [Equation]

Original: where p D ( z | y ) is positive and bounded for any y where p ( y ) > 0 and any z in the family’s nuisance space S F .
Revised: where p D ( z | y ) is positive and bounded for any y where p ( y ) > 0 and any z in the family’s nuisance space S F .

Original: This family is called the nuisance-varying family.
Revised: This family is called the nuisance-varying family.

Original: Due to changing nuisancelabel relationships in this family, the conditional distribution of the label y given the covariates x in one member, e.g. the training distribution, can perform worse than predicting without covariates on another member of the family, e.g. a test distribution with a different nuisance-label relationship.
Revised: Due to changing nuisancelabel relationships in this family, the conditional distribution of the label y given the covariates x in one member, e.g. the training distribution, can perform worse than predicting without covariates on another member of the family, e.g. a test distribution with a different nuisance-label relationship.

Original: We deﬁne performance of a model ˆ p ( y | x ) on a distribution p te as the negative expected KL-divergence from the true conditional p te ( y | x ) :
Revised: We deﬁne performance of a model ˆ p ( y | x ) on a distribution p te as the negative expected KL-divergence from the true conditional p te ( y | x ) :

Original: Perf p te
Revised: Perf p te

Original: [Equation]
Revised: [Equation]

Original: ˆ p 
Revised: p [Equation] =

Original: [Equation]
Revised: [Equation]

Original: p
Revised: x

Original: [Equation]
Revised: [Equation]

Original:  y | x )] .
Revised: ˆ p ( y | x )] .

Original: Higher is better.
Revised: Higher is better.

Revised: This performance equals the expected log-likelihood up to a constant, C p te =

Revised: H p te ( y | x ) , that only depends on the p te : Perf p te (ˆ p ( y | x )) =

Revised: E p te ( y , x ) log ˆ p ( y | x ) + C p te .

Original: For example, consider a family { q a } a ∈ R :
Revised: Consider the following example family { q a } a ∈ R :

Original: [Equation]
Revised: [Equation]

Original: Giventhe training distributionis p tr = q 1 anda test distributionis p te = q − 1 ,  p tr ( y | x ) performs even worse than predicting without covariates, Perf p te ( p ( y )) ≥ Perf p te ( p tr ( y | x )) ; see appendix A.9 for the proof.
Revised: Given training distribution p tr = q 1 and test distribution p te = q − 1 , the conditional p tr ( y | x ) performs even worse than predicting without covariates, Perf p te ( p ( y )) ≥ Perf p te ( p tr ( y | x )) ; see appendix A.9 for the proof.

Original: The problem is that p tr ( y | x ) utilizes label-covariate relationships that do not hold when the label-nuisance relationships changes.
Revised: The problem is that p tr ( y | x ) utilizes label-covariate relationships that do not hold when the nuisance-label relationships change.

Original: When the changing nuisance-label relationship makes the conditional p D ( y | x ) of one member unsuitable for another p (cid:48) D ∈ F , the family exhibits nuisance-induced spurious correlations .
Revised: When the changing nuisance-label relationship makes the conditional p D ( y | x ) of one member unsuitable for another p (cid:48) D ∈ F , the family exhibits nuisance-induced spurious correlations .

Original: Next, we identify a conditional distribution with performance guarantees across all members of the family.
Revised: Next, we identify a conditional distribution with performance guarantees across all members of the family.

Original: We develop two concepts to guarantee performance on every member of the nuisancevarying family: the nuisance-randomized distribution and uncorrelating representations.
Revised: We develop two concepts to guarantee performance on every member of the nuisancevarying family: the nuisance-randomized distribution and uncorrelating representations.

Original: Deﬁnition 1.
Revised: Deﬁnition 1.

Original: The nuisance-randomized distribution is p | =
Revised: The nuisance-randomized distribution is p | =

Original: ( x , y , z ) =
Revised: ( x , y , z ) =

Original: p ( x | y , z ) p tr ( z ) p ( y ) .produce
Revised: p ( x | y , z ) p tr ( z ) p ( y ) .

Original: Appendix In the cows vs. penguins example, p | = is the distribution where either animal has an equal chance to appear on backgrounds of grass or snow.
Revised: In the cows vs. penguins example, p | = is the distribution where either animal has an equal chance to appear on backgrounds of grass or snow.

Original: The motivation behind the nuisance-randomized distribution is that when the nuisance is independent of the label, (noisy 2 ) functions of only the nuisance are not predictive of the label.
Revised: The motivation behind the nuisance-randomized distribution is that when the nuisance is independent of the label, (noisy 2 ) functions of only the nuisance are not predictive of the label.

Original: Then, if the covariates only consist of (noisy) functions of either the nuisance or the label but never a mix of the two (an example of mixing is x 1 =
Revised: If the covariates only consist of (noisy) functions of either the nuisance or the label but never a mix of the two (an example of mixing is x 1 =

Original: y − z + noise ),
Revised: y − z + noise ),

Original: then the conditional p | =
Revised: then the conditional p | =

Original: ( y | x ) does not vary with the parts of x that are (noisy) functions of just the nuisance.
Revised: ( y | x ) does not vary with the parts of x that are (noisy) functions of just the nuisance.

Original: Thus p | = ( y | x ) ignoresone type of relationship between the covariates and the label that changes with the nuisance-label relationship.
Revised: Thus, p | = ( y | x ) ignores the features which have changing relationships with the label.

Original: How about nuisance-varying families where the covariates contain functions that mix the label and the nuisance?
Revised: How about nuisance-varying families where the covariates contain functions that mix the label and the nuisance?

Original: Equation (2) is one such family, where the covariates x 1 and x 2 are functions of both the label and the nuisance.
Revised: Equation (2) is one such family, where the covariates x 1 and x 2 are functions of both the label and the nuisance.

Original: In such nuisance-varying families, the conditional p | =
Revised: In such nuisance-varying families, the conditional p | =

Original: ( y | x ) can use functions that mix the label and the nuisance even though the nuisance is not predictive of the label by itself.
Revised: ( y | x ) can use functions that mix the label and the nuisance even though the nuisance is not predictive of the label by itself.

Original: These mixed functions have relationships with the label which change across the family; for example in eq.
Revised: These mixed functions have relationships with the label which change across the family; for example in eq.

Original: (2), the coordinate x 1 is correlated positively with the label under qbut negatively under q − 2 .
Revised: (2), the coordinate x 1 is correlated positively with the label under qbut negatively under q − 2 .

Original: Then, under changes in the nuisance-label relationship, the conditional p | =
Revised: Then, under changes in the nuisance-label relationship, the conditional p | =

Original: ( y | x ) can perform worse than predicting without covariates because it utilizes a relationship, via these mixed features, that no longer holds.
Revised: ( y | x ) can perform worse than predicting without covariates because it utilizes a relationship, via these mixed features, that no longer holds.

Original: See appendix A.9 for details.
Revised: See appendix A.9 for details.

Original: We address this performance degradation  by avoiding reliance on functions that mix the label and the nuisance.
Revised: We address this performance degradation of p | = ( y | x ) by introducing representations that help avoid reliance on functions that mix the label and the nuisance.

Original: To do so, we note that if the conditional p | = ( y | x ) uses functions that mix the label and the nuisance, knowing the exact value of the nuisance should improve the prediction of the label.
Revised: We note that when the conditional p | = ( y | x ) uses functions that mix the label and the nuisance, knowing the exact value of the nuisance should improve the prediction of the label, i.e. y (cid:54) | =

Original: This is because the nuisance is independent of the label under the nuisance-randomized distribution and can be thought of a source of noise in p |=
Revised: p |

Original: ( x |y ) ; consequently, conditional on x containing these mixed functions, knowing z provides extra information about the label by decreasing noise.
Revised: = z | x .

Original: This motivates us to  deﬁne uncorrelating representations r ( x ) , where the nuisance does not provide any extra information about the label givenknowledge of the representation:
Revised: Therefore, to avoid reliance on mixed functions, we deﬁne uncorrelating representations r ( x ) , where the nuisance does not provide any extra information about the label given the representation:

Original: Deﬁnition 2.
Revised: Deﬁnition 2.

Original: An uncorrelating set of representations is R ( p | = )
Revised: An uncorrelating set of representations is R ( p | = )

Original: s.t. [Equation] r ∈ R 
Revised: s.t. ∀ r ∈ R ( p | = ) ,

Original: [Equation]
Revised: [Equation]

Original: p |

Original: [Equation]

Original: [Equation]

Original: x ) .

Original: In the example in eq.
Revised: In the example in eq.

Original: (2), r ( x ) =
Revised: (2), r ( x ) =

Original: x 1 + x 2 is an uncorrelating representation because it is purely a function of the label and the noise.
Revised: x 1 + x 2 is an uncorrelating representation because it is purely a function of the label and the noise.

Original: Conditional distributions p | =
Revised: Conditional distributions p | =

Original: ( y |  ( x )) for any uncorrelating r ( x ) only depend on properties that are shared across all distributions in the nuisance-varying family.
Revised: ( y | r ( x )) for any uncorrelating r ( x ) only depend on properties that are shared across all distributions in the nuisance-varying family.

Original: Speciﬁcally, for r ∈ R ( p | = ) , the conditional distribution p | =
Revised: Speciﬁcally, for r ∈ R ( p | = ) , the conditional distribution p | =

Original: ( y | r ( x )) uses p ( r ( x ) | y , z ) and p ( y ) which are both shared across all members of the family F .
Revised: ( y | r ( x )) uses p ( r ( x ) | y , z ) and p ( y ) which are both shared across all members of the family F .

Original: For z (cid:48) such that p | = ( z (cid:48) | r ( x )) > 0 ,
Revised: For z (cid:48) such that p | = ( z (cid:48) | r ( x )) > 0 ,

Original: [Equation]
Revised: [Equation]

Original: This fact helps characterize the performance of p |
Revised: This fact helps characterize the performance of p |

Original: = ( y | r ( x )) on any member p te ∈ F :
Revised: = ( y | r ( x )) on any member p te ∈ F :

Original: [Equation]
Revised: [Equation]

Original: As KL-divergence is non-negative, for any uncorrelating representation r , the conditional p | = ( y | r ( x )) does at least as well as predicting without covariates for all members p te ∈ F :
Revised: As KL-divergence is non-negative, for any uncorrelating representation r , the conditional p | = ( y | r ( x )) does at least as well as predicting without covariates for all members p te ∈ F :

Original: Perf p te ( p | = ( y | r ( x ))) ≥ Perf p te ( p ( y )) .
Revised: Perf p te ( p | = ( y | r ( x ))) ≥ Perf p te ( p ( y )) .

Original: See appendix A.3 for the formal derivation.
Revised: See appendix A.3 for the formal derivation.

Original: In fact, we show in appendix A.5 that when the identity representation r ( x ) = x is uncorrelating, then p | =
Revised: In fact, we show in appendix A.5 that when the identity representation r ( x ) = x is uncorrelating, then p | =

Original: ( y | x ) is minimax optimal for a family with sufﬁciently diverse nuisance-label relationships.
Revised: ( y | x ) is minimax optimal for a family with sufﬁciently diverse nuisance-label relationships.

Original: Equation (4) lower bounds the performance of p | =
Revised: Equation (4) lower bounds the performance of p | =

Original: ( y | r ( x )) for any representation in the uncorrelating set across all p te ∈ F .
Revised: ( y | r ( x )) for any representation in the uncorrelating set across all p te ∈ F .

Original: However, it does not specify which of these representations leads to the best performing conditional.
Revised: However, it does not specify which of these representations leads to the best performing conditional.

Original: For example, between two uncorrelating representations like theanimal shape  and whether the animal has horns, which predicts better?
Revised: For example, between two uncorrelating representations like the shape of the animal and whether the animal has horns, which predicts better?

Original: Next, we characterize uncorrelating representations that are simultaneously optimal for all test distributions p te ∈ F .
Revised: Next, we characterize uncorrelating representations that are simultaneously optimal for all test distributions p te ∈ F .

Original: Optimal uncorrelating representations.
Revised: Optimal uncorrelating representations.

Original: As we focus on conditionals of the nuisancerandomized distribution, henceforth, by performance of r ( x ) , we mean the performance of p | =
Revised: As we focus on nuisance-randomized conditionals, henceforth, by performance of r ( x ) , we mean the performance of p | =

Original: ( y | r ( x )) :
Revised: ( y | r ( x )) :

Original: Perf p te ( r ( x )) =
Revised: Perf p te ( r ( x )) =

Original: Perf p te ( p | = ( y | r ( x ))) .
Revised: Perf p te ( p | = ( y | r ( x ))) .

Original: Consider two uncorrelating representations r, r 2 , where the pair ( r, r 2 ) is also uncorrelating.
Revised: Consider two uncorrelating representations r, r 2 , where the pair ( r, r 2 ) is also uncorrelating.

Original: How can r 2 ( x ) dominate r ( x ) in performance across the nuisance-varying family?
Revised: How can r 2 ( x ) dominate r ( x ) in performance across the nuisance-varying family?

Original: Equation (3) shows that
Revised: Equation (3) shows that

Original: [Equation]
Revised: [Equation]

Original: of it. If r 2 ( x ) blocks the dependence between the label and r ( x ) , i.e. r ( x ) | = p | = y | r 2 ( x ) , z , then  r does not change the performance when r 2 is known, suggesting that blocking relates to performance.
Revised: If r 2 ( x ) blocks the dependence between the label and r ( x ) , i.e. r ( x ) | = p | = y | r 2 ( x ) , z , then knowing r does not change the performance when r 2 is known, suggesting that blocking relates to performance.

Original: In Theorem 1, we show that the maximally blocking uncorrelating representation is simultaneously optimal: its performance is as good or better than every other uncorrelating representation on every distribution in the nuisance-varying family.
Revised: In theorem 1, we show that the maximally blocking uncorrelating representation is simultaneously optimal: its performance is as good or better than every other uncorrelating representation on every distribution in the nuisance-varying family.

Original: We state the theorem and discuss each part after.
Revised: We state the theorem ﬁrst :

Original: Theorem 1.
Revised: Theorem 1.

Original: Let r ∗ ∈ R
Revised: Let r ∗ ∈ R

Original: ( p | = ) be maximally blocking : ∀ r ∈ R ( p | = ) , y | = p | = r ( x ) | z , r ∗( x ) .
Revised: ( p | = ) be maximally blocking : ∀ r ∈ R ( p | = ) , y | = p | = r ( x ) | z , r ∗ x ) .

Original: Then,
Revised: Then,

Original: (cid:0) Simultaneous optimality (cid:1) ∀ p te ∈ F , 
Revised: (cid:0) Simultaneous optimality (cid:1) ∀ p te ∈ F , ∀ r ∈ R ( p | = ) ,

Original: [Equation]
Revised: [Equation]

Original:  r  ∈ R ( p 
Revised: (cid:0) Information maximality (cid:1) ∀ r ( x ) ∈ R ( p | = ) , [Equation] r

Original: [Equation]
Revised: [Equation]

Original: (cid:0) Information maximality (cid:1) ∀
Revised: x ))

Original: [Equation]
Revised: [Equation]

Revised: p

Original: [Equation]
Revised: [Equation]

Revised: y

Original: [Equation]
Revised: [Equation]

Revised: r ( x )) .

Original: (cid:0) Information maximality to simultaneous optimality (cid:1) ∀ 
Revised: (cid:0) Information maximality implies simultaneous optimality (cid:1) ∀ r (cid:48)

Revised: ∈ R ( p | = ) ,

Original: [Equation]
Revised: [Equation]

Revised: ( y ; r (cid:48)

Revised: ( x )

Revised: [Equation]

Revised: = I p

Revised: [Equation]

Revised: ( y ; r ∗ ( x ))

Original: [Equation]
Revised: [Equation]

Original: The proof is in appendix A.4.
Revised: The proof is in appendix A.4.

Original: The ﬁrst part of theorem 1 says that a maximally blocking uncorrelating representation r ∗ dominates every other r ∈ R ( p | = ) in performance on every test distribution in the family; this is simultaneous optimality.
Revised: The ﬁrst part of theorem 1, simultaneous optimality , says that a maximally blocking uncorrelating representation r ∗ dominates every other r ∈ R ( p | = ) in performance on every test distribution in the family.

Original: In the cows vs. penguins example, the segmented foreground that contains only the animal is a maximally blocking representation because the animal blocks the dependence between the label and any other semantic feature of the animal.
Revised: In the cows vs. penguins example, the segmented foreground that contains only the animal is a maximally blocking representation because the animal blocks the dependence between the label and any other semantic feature of the animal.

Original: The second and third parts of theorem 1 are useful for algorithm building.
Revised: The second and third parts of theorem 1 are useful for algorithm building.

Original: The second part proves that a maximally blocking r ∗ is also maximally informative of the label under p | = , indicating how to ﬁnd a simultaneously optimal uncorrelating representation.
Revised: The second part proves that a maximally blocking r ∗ is also maximally informative of the label under p | = , indicating how to ﬁnd a simultaneously optimal uncorrelating representation.

Original: What about other information-maximal uncorrelating representations?
Revised: What about other information-maximal uncorrelating representations?

Original: The third part shows that if an uncorrelating representation r (cid:48) has the samemaximal mutual information with the label (under p | = ) as the maximally blocking r ∗ , then r (cid:48) achieves the  performance of r ∗ .
Revised: The third part shows that if an uncorrelating representation r (cid:48) has the same mutual information with the label (under p | = ) as the maximally blocking r ∗ , then r (cid:48) achieves the same simultaneously optimal performance as r ∗ .

Original: In the cows vs. penguins example, an example of a maximally informative uncorrelating representation is the number of legs of the animal because the rest of the body does not give more information about the label.
Revised: In the cows vs. penguins example, an example of a maximally informative uncorrelating representation is the number of legs of the animal because the rest of the body does not give more information about the label.

Original: The secondpart and third part of theorem 1 together show that ﬁnding a uncorrelating representation that maximizes information under the nuisancerandomized distribution ﬁnds a simultaneously optimal uncorrelating representation.
Revised: The second and third parts of theorem 1 together show that ﬁnding an uncorrelating representation that maximizes information under the nuisance-randomized distribution ﬁnds a simultaneously optimal uncorrelating r ( x ) .

Original:  N UISANCE -R ANDOMIZED D ISTILLATION (N U RD)
Revised:  N UISANCE -R ANDOMIZED D ISTILLATION (N U RD)

Original: Theorem 1 says a representation that maximizes information with the label under the nuisancerandomized distribution has the best performance within the uncorrelating set.
Revised: Theorem 1 says a representation that maximizes information with the label under the nuisancerandomized distribution has the best performance within the uncorrelating set.

Original: We develop a representation learning algorithm to maximize the mutual information between the label and a representation in the uncorrelating set under the nuisance-randomized distribution.
Revised: We develop a representation learning algorithm to maximize the mutual information between the label and a representation in the uncorrelating set under the nuisance-randomized distribution.

Original: We call this algorithm Nuisance-Randomized Distillation (N U RD).
Revised: We call this algorithm Nuisance-Randomized Distillation (N U RD).

Original: N U RD has two steps.
Revised: N U RD has two steps.

Original: The ﬁrst step called nuisance randomization creates an estimate of the nuisance-randomized distribution.
Revised: The ﬁrst step, called nuisance randomization, creates an estimate of the nuisance-randomized distribution.

Original: The second step called distillation ﬁnds a representation in the uncorrelating set with the maximum information with the label under the estimate of the nuisance-randomized distribution from step one.
Revised: The second step, called distillation, ﬁnds a representation in the uncorrelating set with the maximum information with the label under the estimate of the nuisance-randomized distribution from step one.

Original: Nuisance Randomization.
Revised: Nuisance Randomization.

Original: We estimate the nuisance-randomized distribution with generative models or by reweighting existing data.
Revised: We estimate the nuisance-randomized distribution with generative models or by reweighting existing data.

Original: Generative-N U RD uses the fact that p ( x | y , z ) is the same for each member of the nuisance-varying family F .
Revised: Generative-N U RD uses the fact that p ( x | y , z ) is the same for each member of the nuisance-varying family F .

Original: With an estimate of this conditional denoted ˆ p ( x | y , z ) , generative-N U
Revised: With an estimate of this conditional denoted ˆ p ( x | y , z ) , generative-N U

Original: RD’s estimate of the nuisance-randomized distribution is z ∼ p tr ( z ) , y ∼ p ( y ) , x ∼ ˆ p ( x | y , z ) .
Revised: RD’s estimate of the nuisance-randomized distribution is z ∼ p tr ( z ) , y ∼ p ( y ) , x ∼ ˆ p ( x | y , z ) .

Original: For high dimensional x , the estimate ˆ p ( x | y , z ) can be constructed with deep generative models.
Revised: For high dimensional x , the estimate ˆ p ( x | y , z ) can be constructed with deep generative models.

Original: Reweighting-N U RD importance weights the data from p tr by p ( y ) / p tr ( y | z ) , making it match the nuisance-randomized distribution:
Revised: Reweighting-N U RD importance weights the data from p tr by p ( y ) / p tr ( y | z ) , making it match the nuisance-randomized distribution:

Original: [Equation]
Revised: [Equation]

Original: Reweighting-N U RD uses a model trained on samples from p tr to estimate
Revised: Reweighting-N U RD uses a model trained on samples from p tr to estimate

Original: [Equation]
Revised: [Equation]

Original: Distillation.
Revised: Distillation.

Original: Distillation seeks to ﬁnd the representation in the uncorrelating set that maximizes the information with the label under ˆ p | = , the estimate of the nuisance-randomized distribution.
Revised: Distillation seeks to ﬁnd the representation in the uncorrelating set that maximizes the information with the label under ˆ p | = , the estimate of the nuisance-randomized distribution.

Original: Maximizing the information translates to maximizing likelihood because the entropy H ˆ p | =
Revised: Maximizing the information translates to maximizing likelihood because the entropy H ˆ p | =

Original: ( y ) is constant with respect to the representation r γ  :
Revised: ( y ) is constant with respect to the representation r γ parameterized by γ :

Original: [Equation]
Revised: [Equation]

Original: Theorem 1 requires the representations be in the uncorrelating set.
Revised: Theorem 1 requires the representations be in the uncorrelating set.

Original: When conditioning on representations in the uncorrelating set, the nuisance has zero mutual information with the label
Revised: When conditioning on representations in the uncorrelating set, the nuisance has zero mutual information with the label:

Original: I p | =
Revised: I p | =

Original: ( y ; z | r γ ( x )) = 0 .
Revised: ( y ; z | r γ ( x )) = 0 .

Original: We operationalize this constraint by adding a conditional mutual information penalty to the maximum likelihood objective with a tunable scalar parameter λ
Revised: We operationalize this constraint by adding a conditional mutual information penalty to the maximum likelihood objective with a tunable scalar parameter λ

Original: [Equation]
Revised: [Equation]

Original: The objective in eq.
Revised: The objective in eq.

Original: (5) can have local optima when the representation is a function of the nuisance andthe exogenous noise in the generation of the covariates given the nuisance and the label.
Revised: (5) can have local optima when the representation is a function of the nuisance and exogenous noise (noise that generates the covariates given the nuisance and the label).

Original: The intuition behind these local optima is that the value of introducing information that predicts the label does not exceed the cost of the introduced conditional dependence.
Revised: The intuition behind these local optima is that the value of introducing information that predicts the label does not exceed the cost of the introduced conditional dependence.

Original: Appendix A.6 gives a formal discussion and an example with such local optima.
Revised: Appendix A.6 gives a formal discussion and an example with such local optima.

Original: Annealing λ , which controls the cost of conditional dependence, can mitigate the local optima issue at the cost of setting annealing schedules.
Revised: Annealing λ , which controls the cost of conditional dependence, can mitigate the local optima issue at the cost of setting annealing schedules.

Original: Instead, we restrict the distillation step to search over representations r γ ( x ) that are also marginally independent of the nuisance z under p | = , i.e. z | = p | = r γ ( x ) .
Revised: Instead, we restrict the distillation step to search over representations r γ ( x ) that are also marginally independent of the nuisance z under p | = , i.e. z | = p | = r γ ( x ) .

Original: This additional independence removes representations that depend on the nuisance but are not predictive of the label; in turn, this removes local optima that correspond to functions of the nuisance and exogenous noise.
Revised: This additional independence removes representations that depend on the nuisance but are not predictive of the label; in turn, this removes local optima that correspond to functions of the nuisance and exogenous noise.

Original: In the cows vs. penguins example, representations that are functions of the background only, like the presence of snow, are uncorrelating but do not satisfy the marginal independence.
Revised: In the cows vs. penguins example, representations that are functions of the background only, like the presence of snow, are uncorrelating but do not satisfy the marginal independence.

Original: Together, the conditional independence y | = p | =
Revised: Together, the conditional independence y | = p | =

Original: z | r γ ( x ) and marginal independence z | = p | =
Revised: z | r γ ( x ) and marginal independence z | = p | =

Original: r γ ( x ) hold if and only if the representation and the label are jointly independent of the nuisance : ( y , r γ ( x ))
Revised: r γ ( x ) hold if and only if the representation and the label are jointly independent of the nuisance : ( y , r γ ( x ))

Original: |
Revised: |

Original: =
Revised: =

Original: p |
Revised: p |

Original: = z .
Revised: = z .

Original: Using mutual information to penalize joint dependence , the distillation step in N U RD is
Revised: Using mutual information to penalize joint dependence (instead of the penalty in eq. (5)), the distillation step in N U RD is

Original: [Equation]
Revised: [Equation]

Original:  We show in lemma 4 that within the set of representations that satisfy joint independence, N U RD learns a representation that is simultaneously optimal in performance on all members of the nuisance-varying family.
Revised: [Equation] We show in lemma 4 that within the set of representations that satisfy joint independence, N U RD learns a representation that is simultaneously optimal in performance on all members of the nuisance-varying family.

Original: To learn representations using gradients, the mutual information needs to be estimated in a way that is amenable to gradient optimization.
Revised: To learn representations using gradients, the mutual information needs to be estimated in a way that is amenable to gradient optimization.

Original:  N U RD makes use of the classiﬁcation-based density-ratio estimation trick (Sugiyama et al., 2012).
Revised: To achieve this, we estimate the mutual information in N U RD via the classiﬁcation-based density-ratio estimation trick (Sugiyama et al., 2012).

Original: We use a critic model p φ to estimate said density ratio.
Revised: We use a critic model p φ to estimate said density ratio.

Original: We describe this technique in appendix A.1 for completeness.
Revised: We describe this technique in appendix A.1 for completeness.

Original: We implement the distillation step as a bi-level optimization where the outer loop optimizes the predictive model p θ ( y | r γ ( x )) and the inner loop optimizes the critic model p φ which helps estimate the mutual information.
Revised: We implement the distillation step as a bi-level optimization where the outer loop optimizes the predictive model p θ ( y | r γ ( x )) and the inner loop optimizes the critic model p φ which helps estimate the mutual information.

Original: Algorithm.
Revised: Algorithm.

Original: We give the full algorithm boxes for both reweighting-N U RD and generative-N U RD in appendix A.1.
Revised: We give the full algorithm boxes for both reweighting-N U RD and generative-N U RD in appendix A.1.

Revised: In reweighting-N U RD, to avoid poor weight estimation due to models memorizing the training data, we use cross-ﬁtting; see algorithm 1.

Original: The setup of nuisance-induced spurious correlations in eq.
Revised: The setup of nuisance-induced spurious correlations in eq.

Original: (1) assumes p ( y ) is ﬁxed across distributions within the nuisance-varying family F .
Revised: (1) assumes p ( y ) is ﬁxed across distributions within the nuisance-varying family F .

Original: This condition can be relaxed when p te ( y ) is known. See appendix A.1 for a procedure to build models when p tr ( y ) (cid:54) = p te ( y ) .
Revised: This condition can be relaxed when p te ( y ) is known; see appendix A.1.

Original:  R ELATED W ORK
Revised:  R ELATED W ORK

Original: In table 1, we summarize key differences between N U RD and the related work which we elaborate on in the rest of this section. Table 1 points out differences with invariant learning (Arjovsky et al., 2019; Krueger et al., 2020), distribution matching (Mahajan et al., 2020;
Revised: In table 1, we summarize key differences between N U RD and the related work: invariant learning (Arjovsky et al., 2019; Krueger et al., 2020), distribution matching (Mahajan et al., 2020;

Original: Guo et al., 2021), shift-stable prediction (Subbaswamy et al., 2019a), group-DRO (Sagawa et al., 2019), and causal regularization (Veitch et al., 2021; Makar et al., 2021). 
Revised: Guo et al., 2021), shift-stable prediction (Subbaswamy et al., 2019a), group-DRO (Sagawa et al., 2019), and causal regularization (Veitch et al., 2021; Makar et al., 2021). We detail the differences here.

Original: Nuisance versus Environment.
Revised: Nuisance versus Environment.

Original: In general, an environment is a distribution with a speciﬁc spurious correlation (Sagawa et al., 2019).
Revised: In general, an environment is a distribution with a speciﬁc spurious correlation (Sagawa et al., 2019).

Original: When the training and test distributions are members of the same nuisance-varying family, environments denote speciﬁc nuisance-label relationships.
Revised: When the training and test distributions are members of the same nuisance-varying family, environments denote speciﬁc nuisance-label relationships.

Original: In contrast, nuisances are variables whose changing relationship with the label induces spurious correlations.
Revised: In contrast, nuisances are variables whose changing relationship with the label induces spurious correlations.

Original: While obtaining data from diverse environments requires  collection from sufﬁciently different sources, nuisances can be speciﬁed from a single source of data using domain knowledge.
Revised: While obtaining data from diverse environments requires data collection from sufﬁciently different sources, one can specify nuisances from a single source of data via domain knowledge.

Original: Domain generalization and subgroup-robustness We brieﬂy mention existing methods that have performance guarantees on unseen test data and focus on how these methods can suffer in the presence of nuisance-induced spurious correlations; for a more detailed presentation, see appendix A.2.
Revised: Domain generalization, domain-invariant learning, and subgroup robustness We brieﬂy mention existing methods that aim to generalize to unseen test data and focus on how these methods can suffer in the presence of nuisance-induced spurious correlations; for a more detailed presentation, see appendix A.2.

Original: Domain generalization and domain-invariant learning methods assume the training data consists of multiple sufﬁciently different environments to generalize to unseen test data that lies in the support of the given environments or subgroups (Li et al., 2018a;
Revised: Domain generalization and domain-invariant learning methods assume the training data consists of multiple sufﬁciently different environments to generalize to unseen test data that is related to the given environments or subgroups (Li et al., 2018a;

Original: Arjovsky et al., 2019; Akuzawa et al., 2019; Mahajan et al., 2020; Krueger et al., 2020; Bellot and van der Schaar, 2020; Li et al., 2018b; 
Revised: Arjovsky et al., 2019; Akuzawa et al., 2019; Mahajan et al., 2020; Krueger et al., 2020; Bellot and van der Schaar, 2020; Li et al., 2018b; Goel et al., 2020; Wald et al., 2021;

Original: Goel et al., 2020; Wald et al., 2021; Ganin et al., 2016; Xie et al., 2017; Zhang et al., 2018; Ghimire 
Revised: Ganin et al., 2016; Xie et al., 2017; Zhang et al., 2018; Ghimire et al., 2020; Adeli et al., 2021; Zhang et al., 2021).

Revised: Due to its focus on nuisances, N U RD works with data from a single environment.

Revised: Taking a distributional robustness (Duchi et al., 2021) approach, Sagawa et al.

Revised: (2019) used group-DRO to build models that perform well on every one of a ﬁnite set of known subgroups.

Revised: Other work also aims to minimize worst subgroup error with a ﬁnite number of ﬁxed but unknown subgroups (Lahoti et al., 2020;

Original:  [Table] 
Revised: Martinez et al., 2021) ; as subgroups [Table] are unknown ﬁnd approximate of the worst subgroup error in even with inﬁnite data

Revised: While methods (Lahoti et al., 2020; Martinez et al., 2021) developed to enforce fairness with respect to a sensitive attribute, they can be applied to generalization with the nuisance treated as the sensitive attribute

Revised: (Creager et al

Original:  Nuisance as the environment label for domain generalization.
Revised: Given nuisance existence of a ﬁnite number of subgroups maps to an additional discreteness assumption on the nuisance variable; in contrast N works with general nuisances Given a high dimensional as in our experiments, deﬁning groups based on the value of the nuisance like in (Sagawa et typically results in groups with at most one sample; with the groups, worst subgroup error will encourage memorizing the Nuisance as the environment label for domain generalization.

Original: Domain generalization methods are inapplicable when the training data consists only of a single environment.
Revised: Domain generalization methods are inapplicable when the training data consists only of a single environment.

Original: In this work, the training data comes from only one member of the nuisance-varying family, i.e. from a single environment.
Revised: In this work, the training data comes from only one member of the nuisance-varying family, i.e. from a single environment.

Original: Instead, groupsof samples deﬁned by nuisance valuescan be treated as environments.
Revised: What if one treats groups deﬁned by nuisance values as environments?

Original: Using the nuisance as the environment label will produce non-overlapping supports over images between environments in some of our experiments.
Revised: Using the nuisance as the environment label can produce non-overlapping supports (over the covariates) between environments.

Original: In Colored-MNIST for example, splitting based on color produces an environment of green images and an environment of red images.
Revised: In Colored-MNIST for example, splitting based on color produces an environment of green images and an environment of red images.

Original: When the covariates do not overlap between environments, methods such as Arjovsky et al.
Revised: When the covariates do not overlap between environments, methods such as Arjovsky et al.

Original: Krueger et al.
Revised: Krueger et al.

Original: (2020) will not produce invariant representations because the model can segment out the covariate space and learn separate functions for each environment.
Revised: (2020) will not produce invariant representations because the model can segment out the covariate space and learn separate functions for each environment.

Original: Methods based on conditional distribution matching (Mahajan et al., 2020; Guo et al., 2021) build representations that are conditionally independent of the environment variable given the label.
Revised: Methods based on conditional distribution matching (Mahajan et al., 2020; Guo et al., 2021) build representations that are conditionally independent of the environment variable given the label.

Original: When the training data is split into groups based on the nuisance value, representations built by these methods are independent of the nuisance given the label.
Revised: When the training data is split into groups based on the nuisance value, representations built by these methods are independent of the nuisance given the label.

Original: However, splitting on a high-dimensional nuisance like image patches tends to yield many groups with only a single image.
Revised: However, splitting on a high-dimensional nuisance like image patches tends to yield many groups with only a single image.

Original: Matching distributions of representations for the same label across all environments is not possible when some environments only have one label.
Revised: Matching distributions of representations for the same label across all environments is not possible when some environments only have one label.

Original: Causal learning and shift-stable prediction.
Revised: Causal learning and shift-stable prediction.

Original: Anticausal learning (Sch ¨ olkopf et al., 2012) assumes a causal generative process for a class of distributions like the nuisance-varying family in eq.
Revised: Anticausal learning (Sch ¨ olkopf et al., 2012) assumes a causal generative process for a class of distributions like the nuisance-varying family in eq.

Original: In such an interpretation, the label y and nuisance z cause the image x , and under independence of cause and mechanism, p ( x | y , z ) is ﬁxed — in other words, independent — regardless of the distribution p D ( y , z ) .
Revised: In such an interpretation, the label y and nuisance z cause the image x , and under independence of cause and mechanism, p ( x | y , z ) is ﬁxed — in other words, independent — regardless of the distribution p D ( y , z ) .

Original: A closely related idea to N U RD is that of Shift-Stable Prediction (Subbaswamy et al., 2019a;b).
Revised: A closely related idea to N U RD is that of Shift-Stable Prediction (Subbaswamy et al., 2019a;b).

Original: Subbaswamy et al.
Revised: Subbaswamy et al.

Original: (2019a) perform graph surgery to learn p |
Revised: (2019a) perform graph surgery to learn p |

Original: = ( y | x , z ) assuming access to z during test time.
Revised: = ( y | x , z ) assuming access to z during test time.

Original: Shift-stable models are not applicable without nuisances at test time but N U RD only requires nuisances during training.
Revised: Shift-stable models are not applicable without nuisances at test time while N U RD only requires nuisances during training.

Original: However, if the nuisance is available at test time, the combined covariate-set [ x , z ] is  uncorrelating because y | =
Revised: However, if the nuisance is available at test time, the combined covariate-set [ x , z ] is 1) uncorrelating because y | =

Original: p |
Revised: p |

Original: =
Revised: =

Original: z | [ x , z ] and  maximally blocking because r ([ x , z ]) | =
Revised: z | [ x , z ] and 2) maximally blocking because r ([ x , z ]) | =

Original: y | z , [ x , z ] ; then, theorem 1 shows p | =
Revised: y | z , [ x , z ] ; then, theorem 1 implies that p | =

Original: ( y | [ x , z ]) is optimal.
Revised: ( y | [ x , z ]) is optimal.

Original: Two concurrent works also build models using the idea of a nuisance variable.
Revised: Two concurrent works also build models using the idea of a nuisance variable.

Original: Makar et al.
Revised: Makar et al.

Original: (2021) assume that there exists a stochastic function of y but not z , called x ∗ , such that y | =
Revised: (2021) assume that there exists a stochastic function of y but not z , called x ∗ , such that y | =

Original: p D x | x ∗ , z ; they use a marginal independence penalty r ( x ) | =
Revised: p D x | x ∗ , z ; they use a marginal independence penalty r ( x ) | =

Original: p |
Revised: p |

Original: = z .
Revised: = z .

Original: Veitch et al.
Revised: Veitch et al.

Original: (2021) use counterfactual invariance to derive a conditional independence penalty r ( x ) |
Revised: (2021) use counterfactual invariance to derive a conditional independence penalty r ( x ) |

Original: = p tr z | y .
Revised: = p tr z | y .

Original: The theory in these works requires the nuisance to be discrete and their algorithms require the nuisance to be both discrete and low-cardinality, while N U RD and its theory work with general high-dimensional nuisances.
Revised: The theory in these works requires the nuisance to be discrete, and their algorithms require the nuisance to be both discrete and lowcardinality; N U RD and its theory work with general high-dimensional nuisances.

Original: Counterfactual invariance promises that a representation will not vary with the nuisance; but it does not produce optimal models in general because it rejects models that depend on functions of the nuisance.
Revised: Counterfactual invariance promises that a representation will not vary with the nuisance but it does not produce optimal models in general because it rejects models that depend on functions of the nuisance.

Original: On the other hand,  uncorrelating representations allow using functions of only the nuisance that are in the covariates to extract further information about the label from rest of the covariates; this leads to better performance in some nuisance-varying families as we show using the theory of minimal sufﬁcient statistics (Lehmann and Casella, 2006) in appendix A.7.
Revised: On the other hand, the uncorrelating property allows using functions of only the nuisance that are in the covariates to extract further information about the label from rest of the covariates; this leads to better performance in some nuisance-varying families, as we show using the theory of minimal sufﬁcient statistics (Lehmann and Casella, 2006) in appendix A.7.

Original:  E XPERIMENTS
Revised:  E XPERIMENTS

Original: We evaluate the implementations of N U RD on class-conditional Gaussians, Colored MNIST (Arjovsky et al., 2019; Gulrajani and Lopez-Paz, 2020), Waterbirds (Sagawa et al., 2019), and chest X-rays (Irvin et al., 2019; Johnson et al., 2019).
Revised: We evaluate the implementations of N U RD on class-conditional Gaussians, Colored-MNIST (Arjovsky et al., 2019), Waterbirds (Sagawa et al., 2019), and chest X-rays (Irvin et al., 2019; Johnson et al., 2019).

Original: See appendix B for implementation details and further evaluations of N U RD.
Revised: See appendix B for implementation details and further evaluations of N U RD.

Original: We released the code here .

Original: Model selection, baselines, and metrics.
Revised: Model selection, baselines, and metrics.

Original: Models in both steps of N U RD are selected using heldout subsets of the training data.
Revised: Models in both steps of N U RD are selected using heldout subsets of the training data.

Revised: We split the training data into training and validation datasets with an 80 − 20 split.

Original: For nuisance-randomization, this selection uses standard measures of held-out performance.
Revised: For nuisance-randomization, this selection uses standard measures of held-out performance.

Original: Selection in the distillation step picks models that give the best value of the distillation objective on a held out subset of the nuisance-randomized data from N U RD’s
Revised: Selection in the distillation step picks models that give the best value of the distillation objective on a held out subset of the nuisance-randomized data from N U RD’s

Original: ﬁrst step.
Revised: ﬁrst step.

Original: We compare against Empirical Risk minimization (ERM) because, as discussed in section 4, existing methods that aim to generalize under spurious correlations require assumptions, such as access to multiple environments or discrete nuisance of small cardinality, that do not hold in the experiments.
Revised: We compare against Empirical Risk minimization (ERM) because, as discussed in section 4, existing methods that aim to generalize under spurious correlations require assumptions, such as access to multiple environments or discrete nuisance of small cardinality, that do not hold in the experiments.

Original: When possible, we report the oracle accuracy or build gold standard models that make use of additional labels at both training and test times.
Revised: When possible, we report the oracle accuracy or build gold standard models using data without spurious correlations.

Original: For every method, we report the average accuracy and standard errorof mean across 10 runs each with a different random seed.
Revised: For every method, we report the average accuracy and standard error across 10 runs each with a different random seed.

Original: We report the accuracy of each model for each experiment on the test data ( p te ) and on heldout subsets of the original training data ( p tr ) and the estimate of the nuisance-randomized distribution ( ˆ p | = ).
Revised: We report the accuracy of each model for each experiment on the test data ( p te ) and on heldout subsets of the original training data ( p tr ) and the estimate of the nuisance-randomized distribution ( ˆ p | = ).

Revised: For all experiments, we use λ = 1 and one or two epoch’s worth of critic model updates for every predictive model update.

Original: 5. C LASS -C ONDITIONAL G AUSSIANS
Revised: 5. C LASS -C ONDITIONAL G AUSSIANS

Original: We generate data as follows: with B (0 . 5) as the uniform Bernoulli distribution, q a ( y , z , x ) is 
Revised: We generate data as follows: with B (0 . 5) as the uniform Bernoulli distribution, q a ( y , z , x ) is y ∼ B (0 . 5)

Original: [Equation] , x 2 ∼ N ( y + z , 0 . 01)] .
Revised: z ∼ N ( a (2 y − 1) , 1) x = [ x 1 ∼ N ( y − z , 9) , x 2 ∼ N ( y + z , 0 . 01)] .

Original: The training and validation sets consist of 10000and 2000 samples from p tr = q 0 .
Revised: The training and test sets consist of 10000 samples from p tr = q 0 .

Original: 5respectively, andthe test set consists of 2000 samples from p te = q − 0 . 
Revised: 5 and 2000 samples from p te = q − 0 . respectively.

Original: We parameterize all models in reweightingN U RD and generative-N U RD using neural networks.
Revised: All models in both N U RD methods are parameterized with neural networks.

Original: [Table]
Revised: [Table]

Original: Results.
Revised: Results.

Original: Table 2 reports results.
Revised: Table 2 reports results.

Original: The analytically computed accuracy of p | = ( y | r ∗ ( x )) , with the optimal linear uncorrelating representation r ∗ ( x ) = x 1 + x 2 , is 63% ; appendix B gives the optimality proof.
Revised: The test accuracy of predicting with the optimal linear uncorrelating representation r ∗ ( x ) = x 1 + x 2 , is 62% ; appendix B gives the optimality proof.

Original: Both generative-N U RD and reweighting-N U
Revised: Both generative-N U RD and reweighting-N U

Original: RD achieve close to this accuracy.
Revised: RD achieve close to this accuracy.

Original: 5. C OLORED -MNIST
Revised: 5. C OLORED -MNIST

Original: We construct a colored MNIST dataset (Arjovsky et al., 2019; Gulrajani and Lopez-Paz, 2020) with images of 0 s and 1 s where the values in each channel for every pixel are either 0 or 1.
Revised: We construct a colored-MNIST dataset (Arjovsky et al., 2019; Gulrajani and Lopez-Paz, 2020) with images of 0 s and 1 s. In this dataset, the values in each channel for every pixel are either 0 or 1 .

Original: We construct two environments and use one as the training distribution and the other as the test.
Revised: We construct two environments and use one as the training distribution and the other as the test.

Original: In training, 90% of of red images have label 0 ; 90% of green images have label 1 .
Revised: In training, 90% of of red images have label 0 ; 90% of green images have label 1 .

Original: In test, the relationship is ﬂipped: 90% of the 0 s are green, and 90% of the 1 s are red.
Revised: In test, the relationship is ﬂipped: 90% of the 0 s are green, and 90% of the 1 s are red.

Original: In both training and test, the digit determines the label only 75% of the time, meaning that exploiting the nuisance-label relationship produces better training accuracies.
Revised: In both training and test, the digit determines the label only 75% of the time, meaning that exploiting the nuisance-label relationship produces better training accuracies.

Original: The training, validation, and test data consist of 3941 , 986 and 4939 samples.
Revised: The training and test data consist of 4851 andsamples respectively.

Original: The nuisance is the color which is given by the most intense pixel.
Revised: We run N U RD with the most intense pixel as the nuisance.

Original: [Table]
Revised: [Table]

Original: Results.
Revised: Results.

Original: See table 3 for the results.
Revised: See table 3 for the results.

Original: ERM learns to use color, evidenced by the fact that it achieves a test accuracy of only 10% .
Revised: ERM learns to use color, evidenced by the fact that it achieves a test accuracy of only 10% .

Original: The oracle accuracy of 75% is the highest achievable by models that do not use color because the digit only predicts the label with 75% accuracy.
Revised: The oracle accuracy of 75% is the highest achievable by models that do not use color because the digit only predicts the label with 75% accuracy.

Original: While generative-N U RD has an average accuracy close to the oracle, reweighting-N U RD matches the oracle at 75% .
Revised: While generative-N U RD has an average accuracy close to the oracle, reweighting-N U RD matches the oracle at 75% .

Original: 5. L EARNING TO CLASSIFY W ATERBIRDS AND L ANDBIRDS
Revised: 5. L EARNING TO CLASSIFY W ATERBIRDS AND L ANDBIRDS

Original: Sagawa et al.
Revised: Sagawa et al.

Original: (2019) consider the task of detecting the type of bird (water or land) from images where the background is a nuisance.
Revised: (2019) consider the task of detecting the type of bird (water or land) from images where the background is a nuisance.

Original: Unlike Sagawa et al.
Revised: Unlike Sagawa et al.

Original: (2019), we do not assume access to validation and test sets without background-label relationships.
Revised: (2019), we do not assume access to validation and test sets with independence between the background and the label.

Original: So, we ﬁlter from their dataset  to create our own training and test datasets with substantially different nuisance-label relationships.
Revised: So, we split their dataset differently to create our own training and test datasets with substantially different nuisance-label relationships.

Original: The training data has 90% waterbirds on backgrounds with water and 90% landbirds on backgrounds with land.
Revised: The training data has 90% waterbirds on backgrounds with water and 90% landbirds on backgrounds with land.

Original: The test data has this relationship ﬂipped.
Revised: The test data has this relationship ﬂipped.

Original: We use the original image size of 224 × 224 × 3 .
Revised: We use the original image size of 224 × 224 × 3 .

Original: The training, validation, and test sets consist of 1936 , 484 and 500 samples respectively.
Revised: The training and test sets consist of 3510 and 400 samples respectively.

Original: We ensure that p ( y = 1) = 0 .
Revised: We ensure that p ( y = 1) = 0 .

Original: 5 in train, validation, and test data.
Revised: 5 in train, validation, and test data.

Original: Thus, predicting the most-frequent-class achieves an accuracy of 0 . 
Revised: Thus, predicting the most frequent class achieves an accuracy of 0 . 5 .

Original: The nuisance is the  background of the image, which we represent as the pixels outside the central patch of 196 × 196 pixels.
Revised: Cropping out the whole background requires manual segmentation. Instead, we use the pixels outside the central patch of 196 × 196 pixels as a nuisance in N U RD.

Original: This is a high-dimensional nuisance which impacts many existing methods negatively; see section 4.
Revised: This is a high-dimensional nuisance which impacts many existing methods negatively; see section 4.

Original: The covariates are the whole image; see ﬁg. 1.
Revised: The covariates are the whole image; see ﬁg. 1.

Original: Method

Original: [Equation]
Revised: [Equation]

Revised: the corresponding border nuisance.

Original:  both images during training and only the left imageis given at test time.
Revised: N U RD has both images during training and only the left image at test time.

Original: Results.
Revised: Results.

Original: Figure 1 reports results.
Revised: Figure 1 reports results.

Original: We construct a gold standard with additional labels of what the background is in the image in both the training and the test data. Using these labels, we build separate models for each background and predict on the corresponding subsets of the test data, achieving a gold standard accuracy of 82% .
Revised: We construct a gold standard model on data where waterbirds and landbirds have equal chances of appearing on backgrounds with water and land; this models achieves a test accuracy of 90% .

Original: ERM uses the background to predict the label, as evidenced by its test accuracy of 58% .
Revised: ERM uses the background to predict the label, as evidenced by its test accuracy of 66% .

Original: Reweighting-N U RD uses the background patches to adjust for the spurious correlation to achieve an average accuracy close to the gold standard, 78% .
Revised: Reweighting-N U RD uses the background patches to adjust for the spurious correlation to achieve an average accuracy close to the gold standard, 83% .

Original: We do not report generative-N U RD’s performance as training on the generated images resulted in classiﬁers that predict as poorly as chance on real images.
Revised: We do not report generative-N U RD’s performance as training on the generated images resulted in classiﬁers that predict as poorly as chance on real images.

Original: This may be due to the small training data.
Revised: This may be due to the small training dataset.

Original: 5. L EARNING TO LABEL PNEUMONIA FROM X- RAYS
Revised: 5. L EARNING TO LABEL PNEUMONIA FROM X- RAYS

Original: In many problems such as classifying cows versus penguins in natural images, the background predicts the label; the background is  the nuisance.
Revised: In many problems such as classifying cows versus penguins in natural images, the background, which is a nuisance, predicts the label.

Original: Medical imaging datasets have a similar property, where factors like the device used to take the measurement are predictive of the label but also leave a signature on the whole image, including the region of the X-ray where the physiological signal of pneumonia is absent (Zech et al., 2018; Badgeley et al., 2019).
Revised: Medical imaging datasets have a similar property, where factors like the device used to take the measurement are predictive of the label but also leave a signature on the whole image.

Original: For this experiment, we construct a dataset by mixing two chest x-ray datasets, CheXpert and MIMIC, that have different factorslike the device type that affect the whole image, with or without pneumonia.
Revised: Here, we construct a dataset by mixing two chest x-ray datasets, CheXpert and MIMIC, that have different factors that affect the whole image, with or without pneumonia.

Original: The training data has 90% pneumonia images from MIMIC and 90% healthy images from CheXpert.
Revised: The training data has 90% pneumonia images from MIMIC and 90% healthy images from CheXpert.

Original: The test data has the ﬂipped relationship with 90% of the pneumonia images from CheXpert and 90% of the healthy images from MIMIC.
Revised: The test data has the ﬂipped relationship, with 90% of the pneumonia images from CheXpert and 90% of the healthy images from MIMIC.

Original: We resize the X-ray images to 32 × 32 .
Revised: We resize the X-ray images to 32 × 32 .

Original: Pneumonia cases are upsampled to make sure that in the training, validation, and test sets, healthy and pneumonia cases are equally probable.
Revised: Healthy cases are downsampled to make sure that in the training and test sets, healthy and pneumonia cases are equally probable.

Original: Thus, predicting the most-frequent-class achieves an accuracy of 0 . 5 .
Revised: Thus, predicting the most frequent class achieves an accuracy of 0 . 5 .

Original: See appendix B for additional experiments with downsampling to balance the label.

Original: The training, validation, and test datasets consist of 34200 , 3804 , and 5602 samples respectively.
Revised: The training and test datasets consist of 12446 and 400 samples respectively.

Original: Specifying a nuisance as all the parts of the X-ray where physiological signals of pneumonia are absent requires manual segmentation.
Revised: In chest X-rays, image segmentation cannot remove all the nuisances because nuisances like scanners alter the entire image (Pooch et al., 2019;

Revised: Zech et al., 2018; Badgeley et al., 2019).

Original: However, non-lung patches,  pixels outside the central patches which contain the lungs, are a nuisance because they do not contain physiological signals of pneumonia.
Revised: However, non-lung patches, i.e. pixels outside the central patches which contain the lungs, are a nuisance because they do not contain physiological signals of pneumonia.

Original: We use the non-lung patches  as a nuisance.
Revised: We use the non-lung patches ( 4 -pixel border) as a nuisance in N U RD.

Original: This high-dimensional nuisance which impacts existing methods negatively; see section 4.
Revised: This is a highdimensional nuisance which impacts existing methods negatively; see section 4.

Original: The covariates are the whole image; see ﬁg. 2 for an example.
Revised: The covariates are the whole image; see ﬁg. 2 for an example.

Original: Results.
Revised: Results.

Original: Figure 2 reports results.
Revised: Figure 2 reports results.

Original: Building an oracle model in this experiment requires knowledge of all factors that correlate the label with all the parts of the X-ray without pneumonia.
Revised: Building an oracle model in this experiment requires knowledge of all factors that correlate the label with all the parts of the X-ray.

Original: Such factors 
Revised: Such factors also exist within each hospital but are not recorded in MIMIC and CheXpert; for example, different departments in the same hospital can have different scanners which correlate the non-lung patches of the X-ray

Original: [Figure]
Revised: [Figure]

Revised: with the label (Zech et al., 2018).

Revised: ERM uses the nuisance to predict pneumonia, as evidenced by its test accuracy of 37% .

Revised: Reweighting-N U RD uses the non-lung patches to adjust for the spurious correlation and achieves an accuracy of 61% , a large improvement over ERM.

Original: Generative-N U RD also outperforms ERM’s performance on average.
Revised: Generative-N U RD also outperforms ERM’s performance on average.

Original: Unlike reweighting-N U RD which outperforms predicting without covariates, generative-N U
Revised: Unlike reweighting-N U RD which outperforms predicting without covariates, generative-N U

Original: RD performs similar to predicting without covariates on average.
Revised: RD performs similar to predicting without covariates on average.

Original: We see a few seeds do not produce good test accuracieswhich may be due to a couple of ways generative nuisance-randomization may be imperfect: 1) little reliance of x on z with y ﬁxed, 2) insufﬁcient quality of generation which leads to poor generalization from generatedimages to real images.
Revised: The few poor test accuracies may be due to two ways generative nuisance-randomization can be imperfect: 1) little reliance of x on z with y ﬁxed, 2) insufﬁcient quality of generation which leads to poor generalization from generated to real images.

Original:  D ISCUSSION
Revised:  D ISCUSSION

Original: We formulate the  nuisance-varying family. Changing nuisance-label relationshipsin this family make predictive models builton top of the covariates from samples of one memberof the family unsuitable for other members.We say such a family manifests nuisance-induced spurious correlations.
Revised: We develop an algorithm for OOD generalization in the presence of spurious correlations induced by a nuisance variable. We formalize nuisance-induced spurious correlations in a nuisance-varying family, where changing nuisance-label relationships make predictive models built from samples of one member unsuitable for other members.

Original: To identify conditional distributions that have performance guarantees on all members of the nuisance-varying family, we introduce the nuisance-randomized distribution and uncorrelating representations.
Revised: To identify conditional distributions that have performance guarantees on all members of the nuisance-varying family, we introduce the nuisancerandomized distribution and uncorrelating representations.

Original: We characterize one uncorrelating representation that is simultaneously optimal for all members in the family.
Revised: We characterize one uncorrelating representation that is simultaneously optimal for all members.

Original: We show that uncorrelating representations most informative of the label under the nuisance-randomized distribution also achieve the same optimal performance.
Revised: Then, we show that uncorrelating representations most informative of the label under the nuisance-randomized distribution also achieve the same optimal performance.

Original: Following this result, we propose to estimate the nuisance-randomized distribution and, under this distribution, construct the uncorrelating representation that is most informative of the label.
Revised: Following this result, we propose to estimate the nuisance-randomized distribution and, under this distribution, construct the uncorrelating representation that is most informative of the label.

Original: We develop an algorithm called N U RD and show  it outperforms ERM on synthetic and real data by adjusting for nuisance-induced spurious correlations.
Revised: We develop an algorithm called N U RD and show that it outperforms ERM on synthetic and real data by adjusting for nuisance-induced spurious correlations.

Revised: Our experiments show that N U RD can use easy-to-acquire nuisances (like the border of an image) to do this adjustment; therefore, our work suggests that the need for expensive manual segmentation, even if it does help exclude all the nuisances, could be mitigated.

Original: Limitations and the future.
Revised: Limitations and the future.

Original: Given groups based on pairs of nuisance-label values, Sagawa et al.
Revised: Given groups based on pairs of nuisance-label values, Sagawa et al.

Original: (2020) suggest that subsampling equally from each group produces models more robust to spurious correlations than reweighting (Byrd and Lipton, 2019; Sagawa et al., 2019); however, subsampling is ineffective when the nuisance is high-dimensional.
Revised: (2020) suggest that subsampling equally from each group produces models more robust to spurious correlations than reweighting (Byrd and Lipton, 2019; Sagawa et al., 2019); however, subsampling is ineffective when the nuisance is high-dimensional.

Original: Instead, as sufﬁcient statistics of the conditional p tr ( y | z ) render y , z independent, grouping based on values of sufﬁcient statistics could be promising.
Revised: Instead, as sufﬁcient statistics of the conditional p tr ( y | z ) render y , z independent, grouping based on values of sufﬁcient statistics could be promising.

Original: The nuisance-randomization steps in generative-N U RD and reweighting-N U
Revised: The nuisance-randomization steps in generative-N U RD and reweighting-N U

Original: RD model different distributions in the training distribution: p tr ( x | y , z ) and p tr ( y | z ) respectively.
Revised: RD model different distributions in the training distribution: p tr ( x | y , z ) and p tr ( y | z ) respectively.

Original: Methods that combine the two approaches to produce better estimates of the nuisance-randomized distribution would be interesting.
Revised: Methods that combine the two approaches to produce better estimates of the nuisance-randomized distribution would be interesting.

Original: The ﬁrst step in reweighting-N U RD is to estimate p tr ( y | z ) .
Revised: The ﬁrst step in reweighting-N U RD is to estimate p tr ( y | z ) .

Original: As deep networks tend to produce inﬂated probabilities (Guo et al., 2017), one must take care to build calibrated models for p ( y | z ) .
Revised: As deep networks tend to produce inﬂated probabilities (Guo et al., 2017), one must take care to build calibrated models for p ( y | z ) .

Original: Adapting either calibration-focused losses (Kumar et al., 2018; Goldstein et al., 2020) or ensembling (Lakshminarayanan et al., 2016) may produce calibrated probabilities.
Revised: Adapting either calibration-focused losses (Kumar et al., 2018; Goldstein et al., 2020) or ensembling (Lakshminarayanan et al., 2016) may produce calibrated probabilities.

Original: In our experiments, the training data contains a single environment.
Revised: In our experiments, the training data contains a single environment.

Original: Methods for invariant representation learning (Arjovsky et al., 2019; Krueger et al., 2020; Mahajan et al., 2020; Guo et al., 2021) typically require data from multiple different environments.
Revised: Methods for invariant representation learning (Arjovsky et al., 2019; Krueger et al., 2020; Mahajan et al., 2020; Guo et al., 2021) typically require data from multiple different environments.

Original: Nuisance-randomized data has a different nuisance-label relationship from the training data, meaning it is a different environment from the training data.
Revised: Nuisance-randomized data has a different nuisance-label relationship from the training data, meaning it is a different environment from the training data.

Original: Following this insight, using nuisance-randomization to produce samples fromother different environments using data from only a single environment would a fruitful direction.
Revised: Following this insight, using nuisance-randomization to produce samples from different environments using data from only a single environment would a fruitful direction.

Original: The absolute performance for both ERM which exploits spurious correlations and N U RD which does not, is too low to be of use in the clinic.
Revised: The absolute performance for both ERM which exploits spurious correlations and N U RD which does not, is too low to be of use in the clinic.

Original: Absolute performance could be improved with larger models, more data, using pretrained models, and multi-task learning over multiple lung conditions, all techniques that could be incorporated into learning procedures in general, including N U RD. 
Revised: Absolute performance could be improved with larger models, more data, using pretrained models, and multi-task learning over multiple lung conditions, all techniques that could be incorporated into learning procedures in general, including N U RD. A CKNOWLEDGEMENTS

Revised: The authors were partly supported by NIH/NHLBI Award R01HL148248, and by NSF Award 1922658 NRT-HDR: FUTURE Foundations, Translation, and Responsibility for Data Science.

Revised: The authors would like to thank Mark Goldstein for helpful comments.

Original: A T HEORETICAL D ETAILS A. T HE FULL N U RDALGORITHM
Revised: A F URTHER DETAILS ABOUT N U RD AND PROOFS A. D ETAILS ABOUT N U RD

Original: Density-ratio trick in Distillation.
Revised: Estimating and using the weights in reweighting-N U RD. In learning p tr ( y | z ) for a highdimensional z , ﬂexible models like deep neural networks can have zero training loss when the model memorizes the training data. For a discrete y , such a model would output ˆ p tr ( y | z ) = 1 for every sample in the training data. Then, the model’s weight estimates on the training data are p ( y ) /

Revised: ˆ p tr ( y | z ) = p ( y ) .

Revised: Weighting the training data with such estimates fails to break the nuisancelabel relationship because p tr ( y , z , x )

Revised: [Equation]

Revised: p

Revised: [Equation]

Revised: x

Revised: [Equation]

Revised: [Equation]

Revised: To avoid such poor weight estimation, we employ a cross-ﬁtting procedure: split the data into K disjoint folds, and the weights for each fold are produced by a model trained and validated on the rest of the folds.

Revised: See algorithm 1 for details.

Revised: In estimating loss for each batch under ˆ p | = during training, one can either weight the per-sample loss or produce the batches themselves via weighted sampling from the data with replacement.

Revised: Density-ratio trick in Distillation.

Original: The density-ratio trick for estimating mutual information (Sugiyama et al., 2012) involves Monte Carlo estimating the mutual information using a binary classiﬁer.
Revised: The density-ratio trick for estimating mutual information (Sugiyama et al., 2012) involves Monte Carlo estimating the mutual information using a binary classiﬁer.

Original: Let (cid:96) =
Revised: Let (cid:96) =

Original: 1 be the pseudolabel for samples from p | =
Revised: 1 be the pseudolabel for samples from p | =

Original: ( y , r γ ( x ) , z ) , and (cid:96) = 0 for samples from p | =
Revised: ( y , r γ ( x ) , z ) , and (cid:96) = 0 for samples from p | =

Original: ( y , r γ ( x ))
Revised: ( y , r γ ( x ))

Original: p | =
Revised: p | =

Original: ( z ) .
Revised: ( z ) .

Original: Then,
Revised: Then,

Original: I ˆ p | =
Revised: I ˆ p | =

Original: ([ r γ ( x ) , y ]; z ) = 
Revised: ([ r γ ( x ) , y ]; z ) = E

Original: E ˆ p | =
Revised: [Equation]

Original: ( y , z , x ) log ˆ p| = ( y , r γ ( x ) , z ) ˆ p | = ( y , r γ ( x ))
Revised: p

Original: [-ˆ p | = ( z ) = E ˆ p | = ( y , z , x ) log p ( (cid:96) = 
 | y , z , r γ ( x )) 1 − p ( (cid:96) = 
-]
Revised: [-ˆ p | = ( z ) = E ˆ p | = ( y , z , x ) log p ( (cid:96) = 
 | y , z , r γ ( x )) 1 − p ( (cid:96) = 
-][Equation]

Original: | y , z ,r γ ( x )) .
Revised: y , z , x ) log [Equation]

Original: With parameters φ , we estimate the conditional probability with a  model  p φ which we call the critic.
Revised: With parameters φ , we estimate the conditional probability with a critic model , denoted p φ .

Original: Accounting for shifts in the marginal label distribution.
Revised: Accounting for shifts in the marginal label distribution.

Original: N U RD relies on the assumption in eq.
Revised: N U RD relies on the assumption in eq.

Original: (1) that distributions in the nuisance-varying family F have the same marginal p ( y ) .
Revised: (1) that distributions in the nuisance-varying family F have the same marginal p ( y ) .

Original: What happens if p te comes from a nuisance-varying family with a different marginal?
Revised: What happens if p te comes from a nuisance-varying family with a different marginal?

Original: Formally, with p tr ∈ F , let p te belong to a nuisance-varying family F (cid:48)
Revised: Formally, with p tr ∈ F , let p te belong to a nuisance-varying family F (cid:48)

Original: = { p te ( y ) / p tr ( y ) p D ( y , z , x ) =
Revised: = { p te ( y ) / p tr ( y ) p D ( y , z , x ) =

Original: p te ( y ) p D ( z | y ) p ( x | y , z ) } where p D ∈ F .
Revised: p te ( y ) p D ( z | y ) p ( x | y , z ) } where p D ∈ F .

Original: Given knowledge of the marginal distribution p te ( y ) , note that the weighted training distribution p tr (cid:48)
Revised: Given knowledge of the marginal distribution p te ( y ) , note that the weighted training distribution p tr (cid:48)

Original: =
Revised: =

Original: p te ( y ) / p tr ( y ) p tr ( y , z , x ) lives in F (cid:48) .
Revised: p te ( y ) / p tr ( y ) p tr ( y , z , x ) lives in F (cid:48) .

Original: Running N U
Revised: Running N U

Original: RD on p tr (cid:48) produces predictive models that generalize to p te .
Revised: RD on p tr (cid:48) produces predictive models that generalize to p te .

Original: To see this, note p (cid:48)
Revised: To see this, note p (cid:48)

Original: | =
Revised: | =

Original: ( y , z , x ) = p (cid:48)
Revised: ( y , z , x ) = p (cid:48)

Original: tr ( y ) p (cid:48)
Revised: tr ( y ) p (cid:48)

Original: tr ( z ) p ( x | y , z ) is a nuisance-randomized distribution in F (cid:48) .
Revised: tr ( z ) p ( x | y , z ) is a nuisance-randomized distribution in F (cid:48) .

Original: With R ( p (cid:48) | = ) as the uncorrelating set of representations deﬁned with respect to p (cid:48)
Revised: With R ( p (cid:48) | = ) as the uncorrelating set of representations deﬁned with respect to p (cid:48)

Original: | = , i.e. r (cid:48) ( x ) ∈ R ( p (cid:48) |
Revised: | = , i.e. r (cid:48) ( x ) ∈ R ( p (cid:48) |

Original: = ) =
Revised: = ) =

Original: ⇒ y
Revised: ⇒ y

Original: | =
Revised: | =

Original: p (cid:48)
Revised: p (cid:48)

Original: | =
Revised: | =

Original: z | r (cid:48) , lemmaand theorem 1 hold.
Revised: z | r (cid:48) , lemmaand theorem 1 hold.

Original: It follows that running N U RD on samples from p te ( y ) / p tr ( y ) p tr ( y , z , x ) produces an estimate of p (cid:48)
Revised: It follows that running N U RD on samples from p te ( y ) / p tr ( y ) p tr ( y , z , x ) produces an estimate of p (cid:48)

Original: | =
Revised: | =

Original: ( y | r (cid:48) ( x ))( r (cid:48) ( x ) ∈ R ( p (cid:48) | = ) ) with the maximal performance on every p te ∈ F (cid:48) if a maximally blocking r ∗ ( x ) ∈ R ( p (cid:48) | = ) .
Revised: ( y | r (cid:48) ( x ))

Original: Result

Original: Nuisance Randomization step ; for N w iterations do nuisance

Original: Sample training batch B ∼ D ;

Original: Compute likelihood (cid:80) ( y i , z i , xi ) ∈ B log p β ( x i | z
Revised: ( r (cid:48) ( x ) ∈ R ( p (cid:48)

Original: i , y i ) (or some generative objective);

Original: Update β to maximize objective above; end

Original: Estimate the marginal distribution over the label ˆ p ( y ) ;

Original: Sample independent label and nuisance y i ∼ D, z j ∼ D , and then sample ˜ x ∼ p

Original: β

Original: ( x | y i , z j ) ;

Original: Construct dataset ˆ D using triples { y k

Original: = y i , z k =

Original: z j , x k =

Original: ˆ x } ;

Original: Distillation step ;

Original: Initialize r γ , p θ , p φ ; for N p iterations do for N c iterations do

Original: Sample batch from generated training data B ∼ ˆ D ;

Original: Sample independent copies of the nuisance marginally:

Original: ˜ z i ∼ D ;

Original: Construct batch ˜ B

Original:  = { y i , ˜ z i , x i } using B ; Compute likelihood (cid:80) [Equation]
Revised: | = ) ) with the maximal performance on every p te ∈ F (cid:48)

Original: Update φ to maximize likelihood (via Adam for example); end

Original: Sample batch from generated training data B ∼ ˜ D ;

Original: Compute distillation objective using the density-ratio trick | [Equation] | (cid:88) [Equation]
Revised: if a maximally blocking r ∗ ( x ) ∈ R ( p (cid:48) | = ) .

Original: A. E XTENDED RELATED WORK
Revised: A. E XTENDED RELATED WORK

Original: Domain generalization methods aim to build models with the goal of generalizing to unseen test data different from the training data (Gulrajani and Lopez-Paz, 2020).
Revised: Domain generalization methods aim to build models with the goal of generalizing to unseen test data different from the training data (Gulrajani and Lopez-Paz, 2020).

Original: Recent work uses multiple sufﬁciently different environments to generalize to unseen test data that lies in the support of the given environments or subgroups (Li et al., 2018a;Arjovsky et al., 2019; Akuzawa et al., 2019;
Revised: Recent work uses multiple sufﬁciently different environments to generalize to unseen test data that lies in the support of the given environments or subgroups (Li et al., 2018a;

Original:  Mahajan et al., 2020; Krueger et al., 2020; Bellot and van der Schaar, 2020; Li et al., 2018b; Goel et al., 2020; Wald et al., 2021).
Revised: Arjovsky et al., 2019; Akuzawa et al., 2019; Mahajan et al., 2020; Krueger et al., 2020; Bellot and van der Schaar, 2020; Li et al., 2018b; Goel et al., 2020; Wald et al., 2021).

Original: Chang et al.
Revised: Chang et al.

Original: (2020) develop a multi-environment objective to interpret neural network predictions that are robust to spurious correlations.
Revised: (2020) develop a multi-environment objective to interpret neural network predictions that are robust to spurious correlations.

Original: Similarly, domain-invariant learning and related methods build representations that are independent of the domain (or some spe- cial variable)(Ganin et al., 2016; Xie et al., 2017; Zhang et al., 2018; Ghimire et al., 2020; Adeli et al., 2021;
Revised: Similarly, domain-invariant learning and related methods build representations that are independent of the domain (Ganin et al., 2016; Xie et al., 2017; Zhang et al., 2018; Ghimire et al., 2020; Adeli et al., 2021;

Original: Zhou et al., 2020).
Revised: Zhou et al., 2020).

Original: Due to its focus on nuisances, N U RD works with data from a single environment.
Revised: Due to its focus on nuisances, N U RD works with data from a single environment.

Original: As in section 4, to split the data into multiple environments, one can split the data into groups based on the value of the nuisance.
Revised: As in section 4, to split the data into multiple environments, one can split the data into groups based on the value of the nuisance.

Original: Then, domain-invariant methods build representations that are independent of the nuisance and under nuisance-induced spurious correlations these representations may ignore semantic features because they are correlated with the nuisance.
Revised: Then, domain-invariant methods build representations that are independent of the nuisance and under nuisance-induced spurious correlations these representations may ignore semantic features because they are correlated with the nuisance.

Original: Domain adaptation Daum´e III (2009); Ben-David et al. Farahani et al. (2021) methods assume access to unlabelled test data which N U RD does not require.
Revised: Domain adaptation (Daum´e III, 2009; BenDavid et al., 2007; Farahani et al., 2021) methods assume access to unlabelled test data which N U RD does not require.

Original: We do not assume access to the test data because nuisance-label relationships can change over time or geography which, in turn, changes the the target distribution.
Revised: We do not assume access to the test data because nuisance-label relationships can change over time or geography which, in turn, changes the the target distribution.

Original: Taking a distributional robustness (Duchi et al., 2021) approach, Sagawa et al.
Revised: Taking a distributional robustness (Duchi et al., 2021) approach, Sagawa et al.

Original: (2019) applied groupDRO to training data where the relative size of certain groups in the training data results in spurious correlations.
Revised: (2019) applied groupDRO to training data where the relative size of certain groups in the training data results in spurious correlations.

Original: Given these groups, group-DRO optimizes the worst error across distributions formed by weighted combinations of the groups. 
Revised: Given these groups, group-DRO optimizes the worst error across distributions formed by weighted combinations of the groups. With high dimensional z as in our experiments, deﬁning B Construct batch

Original: With high dimensional z as in our experiments, deﬁning groups based on the value of the nuisance typically results in groups with at most one sample; with such groups, group-DRO will encourage memorizing the training data.
Revised: ˜ likelihood (cid:88) ( y i , z i , x i ) ∈ B w i log p φ ( (cid:96) = 1 | y i , z i , r γ ( x i )) + (cid:88) ( y i ,

Revised: ˜

Revised: z i , x i ) ∈

Original: {+˜ B w i log p φ ( (cid:96) = 
+}
Revised: {+˜ B w i log p φ ( (cid:96) = 
+}

Revised: | y i , ˜

Revised: z

Revised: i , r γ ( x i )) .

Revised: | B | (cid:88) ( x i , y i , z i ) ∈

Revised: B w i (cid:20) log p θ ( y i |

Original: {+r γ ( x i )) − λ log p φ ( (cid:96) = 
+}
Revised: {+r γ ( x i )) − λ log p φ ( (cid:96) = 
+}

Revised: | y i , z i , r γ ( x i ))

Revised: 1 − p φ ( (cid:96) = 1 | y i , z i , r γ ( x i )) (cid:21) .

Revised: groups based on the value of the nuisance typically results in groups with at most one sample; with such groups, group-DRO will encourage memorizing the training data.

Original: Other work aims to minimize worst subgroup error with a ﬁnite number of ﬁxed but unknown subgroups (Lahoti et al., 2020;
Revised: Other work aims to minimize worst subgroup error with a ﬁnite number of ﬁxed but unknown subgroups (Lahoti et al., 2020;

Original: Martinez et al., 2021); as subgroups are unknown, they only ﬁnd an approximate minimizer of the worst subgroup error in general even with inﬁnite data.
Revised: Martinez et al., 2021); as subgroups are unknown, they only ﬁnd an approximate minimizer of the worst subgroup error in general even with inﬁnite data.

Original: In contrast, N U RD builds predictive models with performance guarantees across all test distributions (that factorize as eq.
Revised: In contrast, N U RD builds predictive models with performance guarantees across all test distributions (that factorize as eq.

Original: (1)) using knowledge of the nuisance.
Revised: (1)) using knowledge of the nuisance.

Original: Given the nuisance, existence of a ﬁnite number of subgroups maps to an additional discreteness assumption on the nuisance variable; N U RD works with general highdimensional nuisances.
Revised: Given the nuisance, existence of a ﬁnite number of subgroups maps to an additional discreteness assumption on the nuisance variable; N U RD works with general high-dimensional nuisances.

Original: Wang and Culotta (2020b) focus on sentiment analysis of reviews and build a dataset where the nuisance label relationship is destroyed by swapping words known to be associated with sentiment of the review, with their antonyms.
Revised: Wang and Culotta (2020b) focus on sentiment analysis of reviews and build a dataset where the nuisance label relationship is destroyed by swapping words known to be associated with sentiment of the review, with their antonyms.

Original: This is equivalent to using domain-speciﬁc knowledge to sample from p ( x | y , z ) in generative N U RD.
Revised: This is equivalent to using domain-speciﬁc knowledge to sample from p ( x | y , z ) in generative N U RD.

Original: N U RD requires no domain-speciﬁc knowledge about the generative model p ( x
Revised: N U RD requires no domain-speciﬁc knowledge about the generative model p ( x

Original: | y , z ) . 
Revised: | y , z ) . z

Revised: ∈ B log β do likelihood (cid:88) ( y i , z i , x i )

Revised: ∈

Revised: B w i log p φ ( (cid:96)

Original: A. K EY LEMMAS FOR UNCORRELATING REPRESENTATIONS r ∈ R ( p | = ) 
Revised: = 1 | y i , z i , r γ ( x i )) + (cid:88) ( y i , ˜ z i , x i ) ∈

Original: {+˜ B w i log p φ ( (cid:96) = 
+}
Revised: {+˜ B w i log p φ ( (cid:96) = 
+}

Revised: | y i , ˜

Revised: z

Revised: i , r γ ( x i )) .

Revised: | B | (cid:88) ( x k , y k , z k ) ∈ B (cid:20) log p θ ( y k | r γ ( x k )) − λ log p φ ( (cid:96)

Revised: = 1 | y k , z k , r γ ( x k ))

Revised: 1 − p φ ( (cid:96) = 1 | y k , z k , r γ ( x k ))

Revised: (cid:21) ; end

Revised: Return p θ ( y | r γ ( x )) .

Revised: A. K EY LEMMAS FOR UNCORRELATING REPRESENTATIONS r ∈ R ( p | = )

Original: In this lemma, we derive the performance of the nuisance-randomized conditional p | =
Revised: In this lemma, we derive the performance of the nuisance-randomized conditional p | =

Original: ( y | r ( x )) for any r ∈ R ( p | = ) and show that it is at least as good as predicting without covariates on any p te ∈ F .
Revised: ( y | r ( x )) for any r ∈ R ( p | = ) and show that it is at least as good as predicting without covariates on any p te ∈ F .

Original: Lemma 1.
Revised: Lemma 1.

Original: Let F be a nuisance-varying family (eq.
Revised: Let F be a nuisance-varying family (eq.

Original: (1) ) and p | =
Revised: (1) ) and p | =

Original: = p ( y ) p tr ( z ) p ( x | y , z ) for some p tr ∈ F .
Revised: = p ( y ) p tr ( z ) p ( x | y , z ) for some p tr ∈ F .

Original: Assume ∀ p D ∈ F , p D ( z | y ) is bounded.
Revised: Assume ∀ p D ∈ F , p D ( z | y ) is bounded.

Original: If r ( x ) ∈ R ( p | = ) , then ∀ p te ∈ F , the performance of p | = ( y | r ( x )) is 
Revised: If r ( x ) ∈ R ( p | = ) , then ∀ p te ∈ F , the performance of p | = ( y | r ( x )) is Perf p te ( p | = ( y | r ( x ))) =

Original: [Equation]
Revised: [Equation]

Original: As the KL-divergence is non-negative, Perf pte ( p | = ( y | r ( x ))) ≥ Perf p te ( p ( y )) . Proof. (of lemma 1) Note that the identity E p ( x ) g ◦ f ( x ) = E p ( f ( x ))
Revised: p

Original: g ◦ f ( x ) implies that

Original: [Equation]
Revised: [Equation]

Original: p
Revised: p

Original: [Equation]
Revised: [Equation]

Revised: +

Revised: E

Revised: [Equation]

Revised: (cid:2)

Revised: [Equation]

Original: As p | = ( z | y ) =
Revised: [Equation] ( x [Equation] y , z

Revised: [Equation]

Revised: ( y ) p ( r ( x ) | y , z ) (cid:3) .

Revised: As the KL-divergence is non-negative, Perf p te ( p | = ( y | r ( x ))) ≥ Perf p te ( p ( y )) .

Revised: Proof.

Original:  p tr ( z ) > 0 on z ∈ S F and y s.t. p
Revised: (of lemma [Equation] As p | = ( z | y ) = p tr ( z ) > 0 on z ∈ S F and y s.t. p

Original: ( y ) > 0 is bounded, lemma 3 implies that p | =
Revised: ( y ) > 0 is bounded, lemma 3 implies that p | =

Original: ( y , z , x ) > 0 ⇔ p te ( y , z , x ) > 0 .
Revised: ( y , z , x ) > 0 ⇔ p te ( y , z , x ) > 0 .

Original: This fact implies the following KL terms and expectations of log-ratios are all well-deﬁned:
Revised: This fact implies the following KL terms and expectations of log-ratios are all well-deﬁned:

Original: [Equation]
Revised: [Equation]

Original: [Equation]
Revised: Here, p | = ( r ( x ) | y , z ) = p ( r ( x ) | y , z ) as p | =

Original:  y , z ) = p ( x | y , z ) by deﬁnition of the nuisancevarying family.
Revised: ( x | y , z ) = p ( x | y , z ) by deﬁnition of the nuisancevarying family.

Original: The proof follows by noting that the gap in performance of p | =
Revised: The proof follows by noting that the gap in performance of p | =

Original: ( y | r ( x )) and p ( y ) equals an expected KL term:
Revised: ( y | r ( x )) and p ( y ) equals an expected KL term:

Original: [Equation]
Revised: [Equation]

Original: Rearranging these terms completes the proof.
Revised: Rearranging these terms completes the proof.

Original: Lemma 2 shows that uncorrelating sets are the same for any nuisance-randomized distribution and that the conditional distribution of the label given an uncorrelating representations is the same for all nuisance-randomized distribution.
Revised: Lemma 2 shows that uncorrelating sets are the same for any nuisance-randomized distribution and that the conditional distribution of the label given an uncorrelating representations is the same for all nuisance-randomized distributions.

Original: Lemma 2.
Revised: Lemma 2.

Original: Let F be a nuisance-varying family with p ( y ) and p ( x | y , z ) and nuisance space S F .
Revised: Let F be a nuisance-varying family with p ( y ) and p ( x | y , z ) and nuisance space S F .

Original: Consider distributions p | = , 1 ( y , z , x )
Revised: Consider distributions p | = , 1 ( y , z , x )

Original: = p ( y ) p | = , 1 ( z ) p ( x | y , z ) and p | = , 2 ( y , z , x ) = p ( y ) p | = , 2 ( z ) p ( x
Revised: = p ( y ) p | = , 1 ( z ) p ( x | y , z ) and p | = , 2 ( y , z , x ) = p ( y ) p | = , 2 ( z ) p ( x

Original: | y , z ) such that p | = , 1 ( z ) > 0 , p | = , 2 ( z ) > 0 for z ∈ S F , and p | = , 1 ( y , z , x ) > 

Revised: | y , z ) such that p | = , 1 ( z ) > 0 , p | = , 2 ( z ) > 0 for z ∈ S F , and p | = , 1 ( y , z , x ) >

Original: ⇐⇒ p | = , 2 ( y , z , x ) > 0 .
Revised: ⇐⇒ p | = , 2 ( y , z , x ) > 0 .

Original: Then, the uncorrelating sets are equal R ( p | = , 1 ) =
Revised: Then, the uncorrelating sets are equal R ( p | = , 1 ) =

Original: R ( p | = , 2 ) and for any r ( x ) ∈ R ( p | = , 1 ) , [Equation]
Revised: R ( p | = , 2 ) and for any r ( x ) ∈ R ( p | = , 1 ) , p | = , 1 ( y | r ( x )) = p | = , 2 ( y | r ( x )) .

Original: Proof.
Revised: Proof.

Original: By the assumption that p | = , 1 ( y , z , x ) > 0 ⇔ p | = , 2 ( y , z , x ) > 0 , there exist some z such that p | = , 1 ( z | r ( x )) > 0 and p | = , 2 ( z | r ( x )) > 0 . 
Revised: By the assumption that p | = , 1 ( y , z , x ) > 0 ⇔ p | = , 2 ( y , z , x ) > 0 , there exist some z such that p | = , 1 ( z | r ( x )) > 0 and p | = , 2 ( z | r ( x )) > 0 . With such z , for any r ∈ R ( p | = , 1 ) , p | = , 1 ( y | r ( x ))

Original: With such z , for any r∈ R (  p | = , 1  )  , 
Revised: = p | = , 1 ( y | r ( x ) , z ) = p ( y ) p ( r ( x ) | y , z ) p | = , 1 ( r ( x ) | z ) = p ( y ) p ( r ( x ) | y , z ) [Equation]

Original: [Equation]
Revised: [Equation]

Original: Taking expectation on both sides with respect to p | = , 2 ( z | r ( x )) ,
Revised: Taking expectation on both sides with respect to p | = , 2 ( z | r ( x )) ,

Original: [Equation]
Revised: [Equation]

Original: Note that E p | = , 2 ( z | r ( x )) p | = , 1 ( y | r ( x ))
Revised: Note that E p | = , 2 ( z | r ( x )) p | = , 1 ( y | r ( x ))

Original: =
Revised: =

Original: p | = , 1 ( y | r ( x )) , which implies [Equation]
Revised: p | = , 1 ( y | r ( x )) , which implies p | = , 1 ( y | r ( x )) =

Original:  y | r ( x )) , completing one part of the proof, p | = , 1 ( y | r ( x ))
Revised: p | = , 1 ( y | r ( x ) , z ) = p | = , 2 ( y | r ( x ) , z ) = p | = , 2 ( y | r ( x )) , completing one part of the proof, p | = , 1 ( y | r ( x ))

Original: = p | = , 2 ( y | r ( x )) .
Revised: = p | = , 2 ( y | r ( x )) .

Original: Further, we showed y | =
Revised: Further, we showed y | =

Original: p | = , 1 z | r ( x ) =
Revised: p | = , 1 z | r ( x ) =

Original: ⇒ y
Revised: ⇒ y

Original: | =
Revised: | =

Original: p | = , 2 z | r ( x ) which means r ( x ) ∈ R ( p | = , 2 ) .
Revised: p | = , 2 z | r ( x ) which means r ( x ) ∈ R ( p | = , 2 ) .

Original: As the above proof holds with p | = , 1 , p | = , 2 swapped with each other,
Revised: As the above proof holds with p | = , 1 , p | = , 2 swapped with each other,

Original: r ( x ) ∈ R ( p | = , 1 ) ⇐⇒ r ( x ) ∈ R ( p | = , 2 ) .
Revised: r ( x ) ∈ R ( p | = , 1 ) ⇐⇒ r ( x ) ∈ R ( p | = , 2 ) .

Original: The next lemma shows that every member of the nuisance-varying family is positive over the same set of y , z , x and is used in proposition 1 and lemma 1.
Revised: The next lemma shows that every member of the nuisance-varying family is positive over the same set of y , z , x and is used in proposition 1 and lemma 1.

Original: Lemma 3.
Revised: Lemma 3.

Original: Let the nuisance-varying family F be deﬁned with p ( y ) , p ( x | y , z ) and nuisance space
Revised: Let the nuisance-varying family F be deﬁned with p ( y ) , p ( x | y , z ) and nuisance space

Original: S F .
Revised: S F .

Original: Let distributions p D = p ( y )
Revised: Let distributions p D = p ( y )

Original: p D ( z | y ) p ( x | z , y ) and p (cid:48) D = p ( y ) p (cid:48)
Revised: p D ( z | y ) p ( x | z , y ) and p (cid:48) D = p ( y ) p (cid:48)

Original: D ( z | y ) p ( x | z , y ) be such that p D ( z | y ) , p (cid:48) D
Revised: D ( z | y ) p ( x | z , y ) be such that p D ( z | y ) , p (cid:48) D

Original: ( z | y ) > 0 for all y such that p ( y ) > 0 and z ∈ S F .
Revised: ( z | y ) > 0 for all y such that p ( y ) > 0 and z ∈ S F .

Original: Further assume p D ( z | y ) , p (cid:48) D ( z | y ) are bounded.
Revised: Further assume p D ( z | y ) , p (cid:48) D ( z | y ) are bounded.

Original: Then, p D ( y , z , x ) > 0 ⇔ p (cid:48) D ( y , z , x ) > 0 .
Revised: Then, p D ( y , z , x ) > 0 ⇔ p (cid:48) D ( y , z , x ) > 0 .

Original: Proof.
Revised: Proof.

Original: For any z ∈ S F and any y such that p ( y ) > 0 , p D ( z | y ) > 0 and p (cid:48) D ( z | y ) p D ( z | y ) > 0 ,p (cid:48)
Revised: For any z ∈ S F and any y such that p ( y ) > 0 , p D ( z | y ) > 0 and p (cid:48) D ( z | y ) p D ( z | y ) > 0 ,

Original: D ( y , z , x ) = p ( x | y , z ) p (cid:48) D ( z | y ) p ( y ) = p ( x | y , z ) p D ( z | y ) p ( y ) p (cid:48) D ( z | y )
Revised: [Equation]

Original: [Equation]
Revised: [Equation]

Original: Thus, for all z in the nuisance space S F and any y such that p ( y ) > 0 ,
Revised: Thus, for all z in the nuisance space S F and any y such that p ( y ) > 0 ,

Original: [Equation]
Revised: [Equation]

Original: As z only takes values in the nuisance space S F , when p ( y ) = 0 ,p
Revised: As z only takes values in the nuisance space S F , when p ( y ) = 0 ,

Original: [Equation]
Revised: [Equation]

Original: D ( y , z , x ) =

Original: p D ( y , z , x ) = 0 .

Original: Together, the two statements above imply
Revised: Together, the two statements above imply

Original: [Equation]
Revised: [Equation]

Revised: x ) > 0 .

Original: A. O PTIMAL UNCORRELATING REPRESENTATIONS
Revised: A. O PTIMAL UNCORRELATING REPRESENTATIONS

Original: Theorem 1.
Revised: Theorem 1.

Original: Let r ∗ ∈ R
Revised: Let r ∗ ∈ R

Original: ( p | = ) be maximally blocking : ∀ r ∈ R ( p | = ) , y | = p | = r ( x ) | z , r ∗ ( x ) .
Revised: ( p | = ) be maximally blocking : ∀ r ∈ R ( p | = ) , y | = p | = r ( x ) | z , r ∗ ( x ) .

Original: Then,
Revised: Then,

Original: (cid:0) Simultaneous optimality (cid:1) ∀ p te ∈ F , ∀ [Equation] ( p 
Revised: (cid:0) Simultaneous optimality (cid:1) ∀ p te ∈ F , ∀ r ∈ R ( p | = ) ,

Original: [Equation]
Revised: [Equation]

Original: = ),
Revised: x )

Original: [Equation]
Revised: [Equation]

Original: (cid:0) Information maximality (cid:1) ∀ r ( x ) ∈ R ( p | = ) 
Revised: (cid:0) Information maximality (cid:1) ∀ r ( x ) ∈ R ( p | = ) ,

Original: [Equation]
Revised: [Equation]

Original: x
Revised: r

Original: [Equation]
Revised: [Equation]

Original: (cid:0) Information maximality to simultaneous optimality (cid:1) ∀ r
Revised: ( x )) ≥ I p

Original: [Equation]
Revised: [Equation]

Revised: ( y ; r ( x )) .

Revised: (cid:0) Information maximality implies simultaneous optimality (cid:1) ∀ r (cid:48)

Original: ∈ R ( p [Equation]
Revised: ∈ R ( p | = ) ,

Original: [Equation]
Revised: [Equation]

Original: Proof.
Revised: Proof.

Original: (proof for theorem 1)
Revised: (proof for theorem 1)

Original: We ﬁrst prove that for any pair r, r 2 ∈ R ( p | = ) such that r 2 blocks r , r ( x ) | =
Revised: We ﬁrst prove that for any pair r, r 2 ∈ R ( p | = ) such that r 2 blocks r , r ( x ) | =

Original: p | =
Revised: p | =

Original: y | z , r 2 ( x ) , r 2 dominates the performance of r on every p te ∈ F .
Revised: y | z , r 2 ( x ) , r 2 dominates the performance of r on every p te ∈ F .

Original: The simultaneously optimality of the maximally blocking representation will follow.
Revised: The simultaneously optimality of the maximally blocking representation will follow.

Original: For readability, let (cid:96) ( r 2 ) =
Revised: For readability, let (cid:96) ( r 2 ) =

Original: E p te ( x ) KL [ p te ( y | x ) (cid:107) p | =
Revised: E p te ( x ) KL [ p te ( y | x ) (cid:107) p | =

Original: ( y | r 2 ( x ))] .
Revised: ( y | r 2 ( x ))] .

Original: We will show that
Revised: We will show that

Original: [Equation]
Revised: [Equation]

Original: We will use the following identity which follows from the fact that p ( x | y , z ) does not change between distributions in the data generating process eq.
Revised: We will use the following identity which follows from the fact that p ( x | y , z ) does not change between distributions in the data generating process eq.

Original: [Equation]
Revised: [Equation]

Original: Next, we will show that
Revised: Next, we will show that

Original: [Equation]

Original: [Equation]
Revised: [Equation]

Original: The steps are similar to lemma 1’s proof
Revised: The steps are similar to lemma 1’s proof

Original: [Equation]
Revised: [Equation]

Revised: | =

Revised: ( y | r 2 ( x ))

Original: Noting that KL is non-negative and that Perf is negative- KL proves the theorem:
Revised: + E p te ( y , x ) log p | = ( y | r 2 ( x )) p | = ( y | r ( x ))

Revised: = (cid:96) ( r 2 ) + E p te ( y ,r ( x ) ,r 2 ( x )) log p |

Revised: =

Revised: ( y | r 2 ( x ))

Revised: p | =

Revised: ( y | r ( x ))

Revised: = (cid:96) ( r 2 ) + E p te ( y , z ,r ( x ) ,r 2 ( x )) log p |

Revised: =

Revised: ( y | r 2 ( x ) , z ) p | =

Revised: ( y | r ( x ) , z ) { as r, r 2 ∈ R ( p | = ) } = (cid:96) ( r 2 ) + E p te ( y , z ,r ( x ) ,r 2 ( x )) log p |

Revised: =

Revised: ( y | r 2 ( x ) , r ( x ) , z ) p | =

Revised: ( y | r ( x ) , z ) { y | = p | = r ( x ) | z , r 2 ( x ) } = (cid:96) ( r 2 ) + E p te ( y , z ,r ( x ) ,r 2 ( x )) log p | =

Revised: ( y , r 2 ( x ) | r ( x ) , z ) p | =

Revised: ( y | r ( x ) , z ) p | = ( r 2 ( x ) | r ( x ) , z ) =

Original: [Equation]
Revised: (cid:96) ( r 2 ) + E p te ( y , z ,r ( x ) ,r 2 ( x )) log p | = ( r 2 ( x ) | y , r ( x ) , z ) p | = ( r 2 ( x ) | r ( x ) , z ) = (cid:96) ( r 2 ) + E p te ( y , z ,r ( x )) E p te ( r 2 ( x ) | y , z ,r ( x )) log p | =

Revised: ( r 2 ( x ) | y , r ( x ) , z ) p | = ( r 2 ( x ) | r ( x ) , z ) = (cid:96) ( r 2 ) + E p te

Original: [Equation]
Revised: [Equation]

Original: It follows that for a maximally blocking r ∗
Revised: Noting that KL is non-negative and that Perf is negative- KL proves the theorem:

Original: [Equation]
Revised: [Equation]

Original: p
Revised: It follows that for a maximally blocking r ∗

Original: [Equation]
Revised: [Equation]

Original: As performance is negative KL , the proof follows that r ∗ dominates r in performance.
Revised: As performance is negative KL , the proof follows that r ∗ dominates r in performance.

Original: This concludes the ﬁrst part of the proof.
Revised: This concludes the ﬁrst part of the proof.

Original: For the second part, we prove information maximality of a maximally blocking r ∗ ( x ) ∈ R ( p | = ) .
Revised: For the second part, we prove information maximality of a maximally blocking r ∗ ( x ) ∈ R ( p | = ) .

Original: The proof above shows that the model p | =
Revised: The proof above shows that the model p | =

Original: ( y | r ∗ ( x )) performs at least as well as p |
Revised: ( y | r ∗ ( x )) performs at least as well as p |

Original: = ( y | r ( x )) for any r ( x ) ∈ R ( p | = ) on any p te ∈ F .
Revised: = ( y | r ( x )) for any r ( x ) ∈ R ( p | = ) on any p te ∈ F .

Original: We characterize the gap in performance between p | =
Revised: We characterize the gap in performance between p | =

Original: ( y | r ∗ ( x )) and p | =
Revised: ( y | r ∗ ( x )) and p | =

Original: ( y | r ( x )) for any r ( x ) ∈ R ( p | = ) as the following conditional mutual information term:
Revised: ( y | r ( x )) for any r ( x ) ∈ R ( p | = ) as the following conditional mutual information term:

Original: [Equation]
Revised: [Equation]

Original: The entropy decomposition of conditional mutual information (with H q ( · ) as the entropy under a distribution q ) gives two mutual information terms. 
Revised: The entropy decomposition of conditional mutual information (with H q ( · ) as the entropy under a distribution q ) gives two mutual information terms. [Equation]

Original: [Equation]
Revised: [Equation]

Original: This difference is non-negative for any r ∈ R ( p | = ) which proves the second part of the theorem:
Revised: This difference is non-negative for any r ∈ R ( p | = ) which proves the second part of the theorem:

Original: [Equation]
Revised: [Equation]

Original: z [Equation] r 
Revised: For the third part, note that any representation r (cid:48) which satisﬁes

Original: [Equation]
Revised: [Equation]

Original: x
Revised: x

Original: [Equation]
Revised: [Equation]

Original: For the third part, note that any representation r (cid:48) which satisﬁes I p | =

Original: ( y ; r ∗ ( x ))

Original: =

Original: I p | =

Original: ( y , r (cid:48)

Original: ( x ))

Original: (information-equivalence) also satisﬁes
Revised: (information-equivalence) also satisﬁes

Original: [Equation]
Revised: [Equation]

Original: Under this condition, eq.
Revised: Under this condition [Equation] eq.

Original: (12) implies
Revised: (12) implies

Original: [Equation]
Revised: [Equation]

Original: However, as r (cid:48) ∈ R ( p | = ) and that r ∗ ( x ) is maximally blocking, which (by the proof above) implies
Revised: However, as r (cid:48) ∈ R ( p | = ) and that r ∗ ( x ) is maximally blocking, which (by the proof above) implies

Original: [Equation]
Revised: [Equation]

Original: The only way both these conditions hold is if
Revised: The only way both these conditions hold is if

Original: [Equation]
Revised: [Equation]

Original: This completes the proof that for any r (cid:

Revised: This completes the proof that for any r (cid:

Original: ) ∈ R ( p | = ) that is information-equivalent to r ∗ ( x ) under p | = , the model p | =
Revised: ) ∈ R ( p | = ) that is information-equivalent to r ∗ ( x ) under p | = , the model p | =

Original: ( y | r (cid:48) ( x )) has the same performance as p |
Revised: ( y | r (cid:48) ( x )) has the same performance as p |

Original: = ( y | r ∗ ( x )) for every p te ∈ F , and consequently, r (cid:48) is also optimal.
Revised: = ( y | r ∗ ( x )) for every p te ∈ F , and consequently, r (cid:48) is also optimal.

Original: A. M INIMAX OPTIMALITY
Revised: A. M INIMAX OPTIMALITY

Original: Proposition 1.
Revised: Proposition 1.

Original: Consider a nuisance-varying family F (eq.
Revised: Consider a nuisance-varying family F (eq.

Original: such that for some p tr ∈ F there exists a distribution p | = ∈ F such that p |
Revised: such that for some p tr ∈ F there exists a distribution p | = ∈ F such that p |

Original: = =
Revised: = =

Original: p ( y ) p
Revised: p ( y ) p

Original: tr ( z ) p ( x | y , z ) ∈ F .
Revised: tr ( z ) p ( x | y , z ) ∈ F .

Original: Let F satisfy
Revised: Let F satisfy

Original: [Equation]
Revised: [Equation]

Original: If y | = p | =
Revised: If y | = p | =

Original: z | x
Revised: z | x

Original: [Equation]
Revised: [Equation]

Original: then p | =
Revised: then p | =

Original: ( y | x ) is minimax optimal :
Revised: ( y | x ) is minimax optimal :

Original: [Equation]
Revised: [Equation]

Original: Proof.
Revised: Proof.

Original: (of proposition 1)
Revised: (of proposition 1)

Original: By lemma 3, as p D ( y , z , x ) > 0 ⇔ p (cid:48) D ( y , z , x ) > 0 , performance is well deﬁned for any p D ( y | x ) on any p (cid:48)
Revised: By lemma 3, as p D ( y , z , x ) > 0 ⇔ p (cid:48) D ( y , z , x ) > 0 , performance is well deﬁned for any p D ( y | x ) on any p (cid:48)

Original: D ∈ F .
Revised: D ∈ F .

Original: First,note that lemma 1 with p te = p (cid:48) D with r ( x ) = x gives 
Revised: First, lemma 1 with p te = p (cid:48) D and r ( x ) = x gives I p

Original: [Equation]
Revised: [Equation]

Original: Thus, unlike any p D ∈ F such that y (cid:54) | = p D z , 
Revised: Thus, unlike any p D ∈ F such that y (cid:54) | = p D z , [Equation]

Original: [Equation]
Revised: [Equation]

Original: For any p D such that y (cid:54) | = p D z , let p (cid:48)
Revised: For any p D such that y (cid:54) | = p D z , let p (cid:48)

Original: D be such that E p (cid:48) D ( x ) KL [ p (cid:48) D
Revised: D be such that E p (cid:48) D ( x ) KL [ p (cid:48) D

Original: ( y | x ) (cid:107) p D ( y | x )] − I p (cid:48) D ( x ; y ) > 0 .
Revised: ( y | x ) (cid:107) p D ( y | x )] − I p (cid:48) D ( x ; y ) > 0 .

Original: As eq.
Revised: As eq.

Original: (15) implies E p (cid:48)
Revised: (15) implies E p (cid:48)

Original: D ( x ) KL [ p (cid:48) D ( y | x ) (cid:107) p | =
Revised: D ( x ) KL [ p (cid:48) D ( y | x ) (cid:107) p | =

Original: ( y | x )] − I p (cid:

Revised: ( y | x )] − I p (cid:

Original: ) D ( x ; y ) ≤ 0 , it follows that ∀ p D such that y (cid:54) | =
Revised: ) D ( x ; y ) ≤ 0 , it follows that ∀ p D such that y (cid:54) | =

Original: p D z ,
Revised: p D z ,

Original: [Equation]
Revised: [Equation]

Original: By lemma 3, any p D ∈ F is positive over the same set of y , z , x and if y | = p D z , then p D
Revised: By lemma 3, any p D ∈ F is positive over the same set of y , z , x and if y | = p D z , then p D

Original: ( y | x ) = p |
Revised: ( y | x ) = p |

Original: = ( y | x ) (see lemma 2 for proof with instantiation r ( x ) = x ).
Revised: = ( y | x ) (see lemma 2 for proof with instantiation r ( x ) = x ).

Original: This means
Revised: This means

Original: [Equation]
Revised: [Equation]

Revised: p D ( y | x )] .

Revised: See proposition 3 for an example nuisance-varying family where the information criterion in eq.

Revised: (13) holds.

Original: A. D ISTILLATION DETAILS AND A LOCAL OPTIMA EXAMPLE FOR EQ . (5)
Revised: A. D ISTILLATION DETAILS AND A LOCAL OPTIMA EXAMPLE FOR EQ . (5)

Original: The objective in eq.
Revised: The objective in eq.

Original: (5) can have local optima when the representation is a function of the nuisance and the exogenous noise in the generation of the covariates given the nuisance and the label.
Revised: (5) can have local optima when the representation is a function of the nuisance and the exogenous noise in the generation of the covariates given the nuisance and the label.

Original: Formally, let the exogenous noise (cid:15) satisfy ( (cid:15) , z ) | =
Revised: Formally, let the exogenous noise (cid:15) satisfy ( (cid:15) , z ) | =

Original: p | = y .
Revised: p | = y .

Original: Then,( (cid:15) , z ) | = p | =
Revised: Then,

Original: y = ⇒ ( f ( (cid:15) , z ) , z ) | = p | = y = ⇒ z | = p | = y | f ( z , (cid:15) ) .
Revised: [Equation]

Original: Such a representation r ( x ) = f
Revised: Such a representation r ( x ) = f

Original: ( (cid:15) , z ) is both in the uncorrelating set and independent of the label y under the nuisance-randomized distribution p |
Revised: ( (cid:15) , z ) is both in the uncorrelating set and independent of the label y under the nuisance-randomized distribution p |

Original: = meaning it does not predict the label.
Revised: = meaning it does not predict the label.

Original: Local optima example for conditional information regularization eq.
Revised: Local optima example for conditional information regularization eq.

Original: Figure 3 plots the value of the objective in eq.
Revised: Figure 3 plots the value of the objective in eq.

Original: (5) computed analytically for λ = 20 , over the class of linear representations indexed by u, v ∈ R , r u,v ( x ) =
Revised: (5) computed analytically for λ = 20 , over the class of linear representations indexed by u, v ∈ R , r u,v ( x ) =

Original: u x 1 + v x 2 , under the data generating process in eq.
Revised: u x 1 + v x 2 , under the data generating process in eq.

Original: Representations of the kind r − u,u ( x ) =
Revised: Representations of the kind r − u,u ( x ) =

Original: u ( x 2 − x 1 ) are functions of z and some noise independent of the label and, as ﬁg. 3 shows, are local maxima on the landscape of the maximization objective in eq.
Revised: u ( x 2 − x 1 ) are functions of z and some noise independent of the label and, as ﬁg. 3 shows, are local maxima on the landscape of the maximization objective in eq.

Original: Global maxima correspond to representations r u,u .
Revised: Global maxima correspond to representations r u,u .

Original: Performance characterization for jointly independent representations
Revised: Performance characterization for jointly independent representations

Original: Lemma 4.
Revised: Lemma 4.

Original: Let F be a nuisance varying family.
Revised: Let F be a nuisance varying family.

Original: For any jointly independent representation r , i.e.
Revised: For any jointly independent representation r , i.e.

Revised: [ r ( x ) , y ] | = p | =

Revised: z ,

Revised: [Equation]

Revised: p te

Original: [Equation]
Revised: [Equation]

Revised: Perf p te ( p | = ( y | r ( x )))

Revised: [Equation]

Original:  where C p te is a p te -dependent constant that does not vary with r ( x ) .
Revised: C p te + I p | = ( r ( x ); y ) , where C p te is a p te -dependent constant that does not vary with r ( x ) .

Original: N U RD maximizes the information term I p | =
Revised: N U RD maximizes the information term I p | =

Original: ( y ; r γ ( x )) and, therefore, maximizes performance on every member of F simultaneously.
Revised: ( y ; r γ ( x )) and, therefore, maximizes performance on every member of F simultaneously.

Original: It follows that within the set of jointly independent representations, N U RD , at optimality, produces a representation that is simultaneously optimal on every p te ∈ F .
Revised: It follows that within the set of jointly independent representations, N U RD , at optimality, produces a representation that is simultaneously optimal on every p te ∈ F .

Original: Proof.
Revised: Proof.

Original: Lemma 1 says that for any uncorrelating representation r ∈ R ( p | = ) and ∀ p te ∈ F ,
Revised: Lemma 1 says that for any uncorrelating representation r ∈ R ( p | = ) and ∀ p te ∈ F ,

Original: [Equation]
Revised: [Equation]

Original: p ( r ( x ) | y , z ) (cid:107) E p ( y ) p ( r ( x ) | y , z ) (cid:3) .

Original: However, as the joint independence [ r ( x ) , y ] | =
Revised: However, as the joint independence [ r ( x ) , y ] | =

Original: p | =
Revised: p | =

Original: z implies both the uncorrelating property and r ( x ) | =
Revised: z implies both the uncorrelating property and r ( x ) | =

Original: p |
Revised: p |

Original: =
Revised: =

Original: z | y , the second term in the RHS above can be expressed as I p | =
Revised: z | y , the second term in the RHS above can be expressed as I p | =

Original: ( r ( x ); y ) :
Revised: ( r ( x ); y ) :

Original: [Equation]
Revised: [Equation]

Original: =

Original: [Equation]
Revised: [Equation]

Original: Noting C p te =
Revised: Noting C p te =

Original: Perf p te ( p ( y )) does not vary with r ( x ) completes the proof.
Revised: Perf p te ( p ( y )) does not vary with r ( x ) completes the proof.

Original: Performance gaps between jointly independent representations and uncorrelating representations.
Revised: Performance gaps between jointly independent representations and uncorrelating representations.

Original: The joint independence [ r ( x ) , y ] | =
Revised: The joint independence [ r ( x ) , y ] | =

Original: p | = z implies the uncorrelating property but uncorrelating representations only satisfy this joint independence when they are independent of the nuisance.
Revised: p | = z implies the uncorrelating property but uncorrelating representations only satisfy this joint independence when they are independent of the nuisance.

Original: Thus, representations that satisfy joint independence form a subset of uncorrelating representations.
Revised: Thus, representations that satisfy joint independence form a subset of uncorrelating representations.

Original: This begs a question: is there a loss in performance by restricting N U RD to representations that satisfy said joint independence?
Revised: This begs a question: is there a loss in performance by restricting N U RD to representations that satisfy said joint independence?

Original: In appendix A.7.1, we use the theory of minimal sufﬁcient statistics (Lehmann and Casella, 2006) to show that there exists a nuisance-varying family where the best uncorrelating representation dominates every representation that satisﬁes joint independence on every member distribution, and is strictly better in at least one.
Revised: In appendix A.7.1, we use the theory of minimal sufﬁcient statistics (Lehmann and Casella, 2006) to show that there exists a nuisance-varying family where the best uncorrelating representation dominates every representation that satisﬁes joint independence on every member distribution, and is strictly better in at least one.

Original: A. C OUNTERFACTUAL INVARIANCE VS . THE UNCORRELATING PROPERTY
Revised: A. C OUNTERFACTUAL INVARIANCE VS . THE UNCORRELATING PROPERTY

Original: We A) show that counterfactually invariant representations are a subset of uncorrelating representations by reducing counterfactual invariance to the joint independence [ r ( x ) , y ] | = p | =
Revised: We A) show that counterfactually invariant representations are a subset of uncorrelating representations by reducing counterfactual invariance to the joint independence [ r ( x ) , y ] | = p | =

Original: z and B) give an example nuisance-varying family F where the best uncorrelating representation strictly dominates every jointly independent representation in performance on every test distribution p te ∈ F : at least as good on all p te ∈ F and strictly better on at least one.
Revised: z and B) give an example nuisance-varying family F where the best uncorrelating representation strictly dominates every jointly independent representation in performance on every test distribution p te ∈ F : at least as good on all p te ∈ F and strictly better on at least one.

Original: We show A by proving counterfactually invariant representations satisfy joint independence [ r ( x ) , y ] | = p | =
Revised: We show A by proving counterfactually invariant representations satisfy joint independence [ r ( x ) , y ] | = p | =

Original: z which implies the uncorrelating property, but not vice versa.
Revised: z which implies the uncorrelating property, but not vice versa.

Original: Counterfactual invariance implies that for all p D ∈ F , the conditional independence r ( x ) | =
Revised: Counterfactual invariance implies that for all p D ∈ F , the conditional independence r ( x ) | =

Original: p D z | y holds by theorem 3.
Revised: p D z | y holds by theorem 3.

Original: in Veitch et al.
Revised: in Veitch et al.

Original: As y | =
Revised: As y | =

Original: p |
Revised: p |

Original: = z , it follows that
Revised: = z , it follows that

Original: [ r ( x ) , y ] | =
Revised: [ r ( x ) , y ] | =

Original: p |
Revised: p |

Original: = z ; this joint independence implies the uncorrelating property, y | =
Revised: = z ; this joint independence implies the uncorrelating property, y | =

Original: p |
Revised: p |

Original: =
Revised: =

Original: z | r ( x ) .
Revised: z | r ( x ) .

Original: But, uncorrelating representations only satisfy the said joint independence when they are independent of the nuisance.
Revised: But, uncorrelating representations only satisfy the said joint independence when they are independent of the nuisance.

Original: We show B in appendix A.7.1 by constructing a nuisance-varying family where the optimal performance is achieved by an uncorrelating representation that is dependent on the nuisance.
Revised: We show B in appendix A.7.1 by constructing a nuisance-varying family where the optimal performance is achieved by an uncorrelating representation that is dependent on the nuisance.

Original: A.7. J OINT INDEPENDENCE VS . THE UNCORRELATING PROPERTY
Revised: A.7. J OINT INDEPENDENCE VS . THE UNCORRELATING PROPERTY

Original: Here, we discuss the performance gap between representations that are uncorrelating ( y | = p | =
Revised: Here, we discuss the performance gap between representations that are uncorrelating ( y | = p | =

Original: z | r ( x ) ) and those that satisfy the joint independence ( y , r ( x ))
Revised: z | r ( x ) ) and those that satisfy the joint independence ( y , r ( x ))

Original: |
Revised: |

Original: =
Revised: =

Original: p |
Revised: p |

Original: = z .
Revised: = z .

Original: We construct a data generating process where optimal performance on every member of F is achieved only by uncorrelating representations that do not satisfy joint independence.
Revised: We construct a data generating process where optimal performance on every member of F is achieved only by uncorrelating representations that do not satisfy joint independence.

Original: Theorem 2.
Revised: Theorem 2.

Original: Deﬁne a nuisance-varying family F = { p D
Revised: Deﬁne a nuisance-varying family F = { p D

Original: ( y , z , x ) =
Revised: ( y , z , x ) =

Original: p ( y ) p D ( z | y ) p ( x | y , z ) } .
Revised: p ( y ) p D ( z | y ) p ( x | y , z ) } .

Original: Let R J =
Revised: Let R J =

Original: { r ( x ); [ r ( x ) , y ] | =
Revised: { r ( x ); [ r ( x ) , y ] | =

Original: p |
Revised: p |

Original: =
Revised: =

Original: z } and R C = { r ( x );
Revised: z } and R C = { r ( x );

Original: y | = p | =
Revised: y | = p | =

Original: z | r ( x ) } be the set of representations that, under the nuisance-randomized distribution, satisfy joint independence and conditional independence respectively.
Revised: z | r ( x ) } be the set of representations that, under the nuisance-randomized distribution, satisfy joint independence and conditional independence respectively.

Original: Then there exists a nuisance-varying family F such that
Revised: Then there exists a nuisance-varying family F such that

Original: [Equation]
Revised: [Equation]

Original: (16) and ∃ p te ∈ F for which the inequality is strict
Revised: and ∃ p te ∈ F for which the inequality is strict

Original: [Equation]
Revised: [Equation]

Original: Proof.
Revised: Proof.

Original: In this proof we will build a nuisance-varying family F such that p | = ∈ F and y | =
Revised: In this proof we will build a nuisance-varying family F such that p | = ∈ F and y | =

Original: p |
Revised: p |

Original: = z | x .
Revised: = z | x .

Original: This makes x a maximally blocking uncorrelating representation because r ( x ) | =
Revised: This makes x a maximally blocking uncorrelating representation because r ( x ) | =

Original: p | =
Revised: p | =

Original: y | x , z .
Revised: y | x , z .

Original: Thus it has optimal performance on every p te ∈ F within the class of uncorrelating representations.
Revised: Thus it has optimal performance on every p te ∈ F within the class of uncorrelating representations.

Original: We let y be binary, p | = ∈ F .
Revised: We let y be binary, p | = ∈ F .

Original: The structure of the rest of the proof is as follows:
Revised: The structure of the rest of the proof is as follows:

Original: The representation f ( x ) = p | =
Revised: The representation f ( x ) = p | =

Original: ( y = 1 | x ) is optimal in that it performs exactly as well as x on every member of the family F . 2.
Revised: ( y = 1 | x ) is optimal in that it performs exactly as well as x on every member of the family F . 2.

Original: Any representation T ( x ) that matches the performance of x on every p te ∈ F satisﬁes y | =
Revised: Any representation T ( x ) that matches the performance of x on every p te ∈ F satisﬁes y | =

Original: p | = x | T ( x ) .
Revised: p | = x | T ( x ) .

Original: All functions T ( x ) such that y | = p | = x | T ( x ) determine f ( x ) .
Revised: All functions T ( x ) such that y | = p | = x | T ( x ) determine f ( x ) .

Original: This is shown in lemma 5.
Revised: This is shown in lemma 5.

Original: We construct a family where f ( x ) (cid:54) | =
Revised: We construct a family where f ( x ) (cid:54) | =

Original: p | =
Revised: p | =

Original: z which, by the point above, means that every optimal representation T ( x ) is dependent on z : T ( x ) (cid:54)
Revised: z which, by the point above, means that every optimal representation T ( x ) is dependent on z : T ( x ) (cid:54)

Original: | =
Revised: | =

Original: p |
Revised: p |

Original: = z .
Revised: = z .

Original: But every representation r ∈ R J satisﬁes r ( x ) | =
Revised: But every representation r ∈ R J satisﬁes r ( x ) | =

Original: p |
Revised: p |

Original: =
Revised: =

Original: z and, therefore, is strictly worse in performance than f ( x ) on p | = , meaning that they perform also strictly worse than x (because Perf p te ( f ( x ))
Revised: z and, therefore, is strictly worse in performance than f ( x ) on p | = , meaning that they perform also strictly worse than x (because Perf p te ( f ( x ))

Original: =
Revised: =

Original: Perf p te ( x ) ).
Revised: Perf p te ( x ) ).

Original: Noting x ∈ R C completes the proof.
Revised: Noting x ∈ R C completes the proof.

Original: For 1, let f ( x ) =
Revised: For 1, let f ( x ) =

Original: p |
Revised: p |

Original: =
Revised: =

Original: ( y = 1 | x ) .
Revised: ( y = 1 | x ) .

Original: However, we show here that p | =
Revised: However, we show here that p | =

Original: ( y | f ( x ))
Revised: ( y | f ( x ))

Original: =
Revised: =

Original: p |
Revised: p |

Original: = ( y | x ) .
Revised: = ( y | x ) .

Original: [Equation]
Revised: [Equation]

Original:  This means f ( x ) performs exactly as well as x on every p te ∈ F and x | = p | =
Revised: = [Equation] This means f ( x ) performs exactly as well as x on every p te ∈ F and x | = p | =

Original: y | f ( x ) .
Revised: y | f ( x ) .

Original: For 2, recall Perfp te ( r ( x )) = − E p te ( x ) KL [ p te ( y | x ) (cid:107) p | = ( y | r ( x ))] and note that 1 implies Perf p | = ( x
Revised: For 2, recall Perf

Original: [Equation]
Revised: [Equation]

Original: = Perf p | =( f ( x
Revised: p | =

Original: [Equation] = 0 . Let T ( x ) be any function that performs as well as f ( x ) on every p te ∈ F .
Revised: ( y | r ( x ))] and note that 1 implies

Original: As p | = ∈ F ,
Revised: Perf p | = ( x ) =

Original: [Equation]
Revised: [Equation]

Original: = Perf p |[Equation] ( f ( x )) =Perf p
Revised: p | =

Original: [Equation] ( T ( x ))[Equation] = ( y
Revised: ( f ( x )) = 0 .

Original: [Equation]

Original:  x [Equation] = [Equation]
Revised: Let T ( x ) be any function that performs as well as f ( x ) on every p te ∈ F .

Original:  | = [Equation] T
Revised: As p | = ∈ F ,

Original: [Equation]
Revised: [Equation]

Original: We leave 3 to lemma 5 and show 4 here.
Revised: We leave 3 to lemma 5 and show 4 here.

Original: The example data generating process.
Revised: The example data generating process.

Original: We give a data generating process where f ( x ) = p ( y = 1 | x ) is dependent on z : f ( x ) (cid:54) | = p |
Revised: We give a data generating process where f ( x ) = p ( y = 1 | x ) is dependent on z : f ( x ) (cid:54) | = p |

Original: = z .
Revised: = z .

Original: We assume p | = ∈ F .
Revised: We assume p | = ∈ F .

Original: With a binary y and a normal z , let p | =
Revised: With a binary y and a normal z , let p | =

Original: ( y , z , x ) =
Revised: ( y , z , x ) =

Original: p ( y ) p ( z ) p ( x | y , z ) be generated as follows: with ρ : { 0 , 1 } × { 0 , 1 } → (0 , 1) , let p ( y = 1) = 0 . 
Revised: p ( y ) p ( z ) p ( x | y , z ) be generated as follows: with ρ : { 0 , 1 } × { 0 , 1 } → (0 , 1) , let p ( y = 1) = 0 . 5 , z ∼ N (0 , 1) , p ( b = 1 | y

Original: [Equation]
Revised: =

Original: ∼ N (
Revised: y, z = z ) =

Original: [Equation]
Revised: ρ ( y, 1 [ z ≥ 0]) , x = [ b , 1 [ z ≥ 0]] .

Original: We will drop the subscript in | = p for readability next.
Revised: We will drop the subscript in | = p for readability next.

Original: Throughout the next part, we use a key property of independence: [ a, b ] | =
Revised: Throughout the next part, we use a key property of independence: [ a, b ] | =

Original: c ⇐⇒ b | = c | a, a | = c .
Revised: c ⇐⇒ b | = c | a, a | = c .

Original: As z is a standard normal random variable 1 [ z ≥ 0] | = | z | | y , meaning we can write ( y , 1 [ z ≥ 0]) | = | z | because z is generated independently of y .
Revised: As z is a standard normal random variable 1 [ z ≥ 0] | = | z | | y , meaning we can write ( y , 1 [ z ≥ 0]) | = | z | because z is generated independently of y .

Original: Thus, as the distribution of b only depends on 1 [ z ≥ 0] and y due to the data generating process, it holds that ( b , y , 1 [ z ≥ 0]) | = | z | .
Revised: Thus, as the distribution of b only depends on 1 [ z ≥ 0] and y due to the data generating process, it holds that ( b , y , 1 [ z ≥ 0]) | = | z | .

Original: Then ( b , y , 1 [ z ≥ 0]) | = | z | = ⇒ y
Revised: Then ( b , y , 1 [ z ≥ 0]) | = | z | = ⇒ y

Original: | = | z | | b , 1 [ z ≥ 0] = ⇒ y
Revised: | = | z | | b , 1 [ z ≥ 0] = ⇒ y

Original: | =
Revised: | =

Original: z | b , 1 [ z ≥ 0] = ⇒ y
Revised: z | b , 1 [ z ≥ 0] = ⇒ y

Original: | =
Revised: | =

Original: z | x
Revised: z | x

Original: As x only depends on 1 [ z ≥ 0] and y , for readability, we deﬁne a = 1 [ z ≥ 0] .
Revised: As x only depends on 1 [ z ≥ 0] and b , for readability, we deﬁne a = 1 [ z ≥ 0] .

Original: Then p | =
Revised: Then p | =

Original: ( y , a , b ) =
Revised: ( y , a , b ) =

Original: p | = ( y ) p | = ( a ) p ( b | y , a ) , where p ( b = 1 | y
Revised: p | = ( y ) p | = ( a ) p ( b | y , a ) , where p ( b = 1 | y

Original: =
Revised: =

Original: y, a = a ) =
Revised: y, a = a ) =

Original: ρ ( y, a ) and x = [ b , a ] .
Revised: ρ ( y, a ) and x = [ b , a ] .

Original: We overload the notation for f : expanding x = [ b, a ] , we let f ( x ) =
Revised: We overload the notation for f : expanding x = [ b, a ] , we let f ( x ) =

Original: f ( b, a ) = p ( y = 1 | x = [ b, a ]) .
Revised: f ( b, a ) = p ( y = 1 | x = [ b, a ]) .

Original: We write f ( b, a ) for different values of b here,
Revised: We write f ( b, a ) for different values of b here,

Original: [Equation]
Revised: [Equation]

Original: [Equation]
Revised: [Equation]

Original: We let ρ ( y, 1) = 0 .
Revised: We let ρ ( y, 1) = 0 .

Original: 5 for y ∈ { 0 , 1 } , ρ (0 , 0)
Revised: 5 for y ∈ { 0 , 1 } , ρ (0 , 0)

Original: = 0 .
Revised: = 0 .

Original: 1 , and ρ (1 , 0)
Revised: 1 , and ρ (1 , 0)

Original: = 0 .
Revised: = 0 .

Original: Then, with a = 1 , f (1 , a ) =
Revised: Then, with a = 1 , f (1 , a ) =

Original: ρ (1 , 1) ρ (0 , 1) + ρ (1 , 1) = 0 .0 .
Revised: ρ (1 , 1) ρ (0 , 1) + ρ (1 , 1) = 0 .0 .

Original: 5 + 0 .
Revised: 5 + 0 .

Original: 5 = 0 .
Revised: 5 = 0 .

Original: 5 , (18) f (0 , a ) =
Revised: 5 , (18) f (0 , a ) =

Original: 1 − ρ (1 , 1) 2 − ρ (0 , 1) − ρ (1 , a ) = 1 − 0 .2 − 0 .
Revised: 1 − ρ (1 , 1) 2 − ρ (0 , 1) − ρ (1 , a ) = 1 − 0 .2 − 0 .

Original: 5 − 0 .
Revised: 5 − 0 .

Original: 5 = 0 .
Revised: 5 = 0 .

Original: 5 , (19) and with a = 0 , f (1 , a 
Revised: 5 , (19) and with a = 0 , f (1 , a )

Original: [Equation]

Original: =
Revised: =

Original: [Equation] [Equation] [Equation] [Equation]
Revised: ρ (1 , 0) ρ (0 , 0)

Original: [Equation] 1 + 0 .
Revised: + ρ (1 , 0)

Original: 9 = 0  .
Revised: = 0 .0 .

Original: 9 , (20) f (0 , [Equation] [Equation] [Equation]
Revised: 1 + 0 .

Original: [Equation]

Original: [Equation]
Revised: [Equation]

Original: 1 − 0 .2 − 0 . 1 − 0 . 9 = 0 . Thus, the distribution f ( x ) | a = a changes with a meaning that f ( x ) (cid:54) | = a which implies f ( x ) (cid:54) | =
Revised: Thus, the distribution f ( x ) | a = a changes with a meaning that f ( x ) (cid:54) | = a which implies f ( x ) (cid:54) | =

Original: z as a = 1 [ z ≥ a ] is a function of z .
Revised: z as a = 1 [ z ≥ a ] is a function of z .

Original: Note that f ( x ) (cid:54) |
Revised: Note that f ( x ) (cid:54) |

Original: =
Revised: =

Original: z , then f ( x ) (cid:54)∈ R J as f ( x ) |
Revised: z , then f ( x ) (cid:54)∈ R J as f ( x ) |

Original: =
Revised: =

Original: z is an implication of joint independence.
Revised: z is an implication of joint independence.

Original: Any function T ( x ) that achieves the same performance as p | =
Revised: Any function T ( x ) that achieves the same performance as p | =

Original: ( y | f ( x ))
Revised: ( y | f ( x ))

Original: (by 3 and lemma 5), determines f ( x ) .
Revised: (by 3 and lemma 5), determines f ( x ) .

Original: It follows that, T ( x ) (cid:54)∈ R J because
Revised: It follows that, T ( x ) (cid:54)∈ R J because

Original: [Equation]
Revised: [Equation]

Original: So every r ∈ R J must perform worse than f ( x ) , and consequently x , on p | = .
Revised: So every r ∈ R J must perform worse than f ( x ) , and consequently x , on p | = .

Original: Finally, the independence x (cid:54) | = p | =
Revised: Finally, the independence x (cid:54) | = p | =

Original: z implies x (cid:54)∈ R J but y | = p | =
Revised: z implies x (cid:54)∈ R J but y | = p | =

Original: z | x and so x ∈ R C .
Revised: z | x and so x ∈ R C .

Original: In this example we constructed, x is the maximally blocking uncorrelating representation which means that it is optimal in R C on every p te ∈ F .
Revised: In this example we constructed, x is the maximally blocking uncorrelating representation which means that it is optimal in R C on every p te ∈ F .

Original: As R J is a subset, any r ∈ R J can at best match the performance of x and we already showed that every r ∈ R J is worse than f ( x ) and
Revised: As R J is a subset, any r ∈ R J can at best match the performance of x and we already showed that every r ∈ R J is worse than f ( x ) and

Original: consequently x on p | = .
Revised: consequently x on p | = .

Original: This completes the proof.
Revised: This completes the proof.

Original: Lemma 5.
Revised: Lemma 5.

Original: Consider a joint distribution p ( y , x ) with binary y .
Revised: Consider a joint distribution p ( y , x ) with binary y .

Original: Assume that p ( x | y = y ) has the same support for y ∈ { 0 , 1 } .
Revised: Assume that p ( x | y = y ) has the same support for y ∈ { 0 , 1 } .

Original: Then, for any function T ( x ) such that y | = x | T ( x ) , the function f ( x ) = p ( y = 1 | x ) is T ( x ) -measurable
Revised: Then, for any function T ( x ) such that y | = x | T ( x ) , the function f ( x ) = p ( y = 1 | x ) is T ( x ) -measurable

Original: ( T ( x ) determines f ( x ) ).
Revised: ( T ( x ) determines f ( x ) ).

Original: Proof.
Revised: Proof.

Original: We use the notion of sufﬁcient statistics from estimation theory, which are deﬁned for a family of distributions, to deﬁne the set of functions T ( x ) for the joint distribution p ( y , x ) .
Revised: We use the notion of sufﬁcient statistics from estimation theory, which are deﬁned for a family of distributions, to deﬁne the set of functions T ( x ) for the joint distribution p ( y , x ) .

Original: Sufﬁcient statistics in estimation theory.
Revised: Sufﬁcient statistics in estimation theory.

Original: Consider a family of distributions P = { p θ ( x ); θ ∈ Ω } .
Revised: Consider a family of distributions P = { p θ ( x ); θ ∈ Ω } .

Original: Assume θ is discrete and that Ω is ﬁnite.
Revised: Assume θ is discrete and that Ω is ﬁnite.

Original: A function T ( x ) is a sufﬁcient statistic of a family of distributions P
Revised: A function T ( x ) is a sufﬁcient statistic of a family of distributions P

Original: if the conditional distribution p θ ( x | T ( x ) = t ) does not vary with θ for (almost) any value of t .
Revised: if the conditional distribution p θ ( x | T ( x ) = t ) does not vary with θ for (almost) any value of t .

Original: A minimal sufﬁcient statistic is a sufﬁcient statistic M ( x ) such that for any sufﬁcient statistic T ( X ) T ( x )
Revised: A minimal sufﬁcient statistic is a sufﬁcient statistic M ( x ) such that for any sufﬁcient statistic T ( X ) T ( x )

Original: =
Revised: =

Original: T ( x (cid:48) )
Revised: T ( x (cid:48) )

Original: =
Revised: =

Original: ⇒ M ( x ) =
Revised: ⇒ M ( x ) =

Original: M ( x (cid:48) ) .
Revised: M ( x (cid:48) ) .

Original: Any bijective transform of M ( x ) is also a minimal sufﬁcient statistic.
Revised: Any bijective transform of M ( x ) is also a minimal sufﬁcient statistic.

Original: The rest of the proof will follow from relying on theorem 6.12 from Lehmann and Casella (2006) which constructs a minimal sufﬁcient statistic for a ﬁnite family of distributions P
Revised: The rest of the proof will follow from relying on theorem 6.12 from Lehmann and Casella (2006) which constructs a minimal sufﬁcient statistic for a ﬁnite family of distributions P

Original: = { p i ; i ∈ { 0 , K − 1 }} as
Revised: = { p i ; i ∈ { 0 , K − 1 }} as

Original: [Equation]
Revised: [Equation]

Original: Deﬁning the family with conditionals.
Revised: Deﬁning the family with conditionals.

Original: Now let the family P = { p y ( x )); y ∈ { 0 , 1 }} where p y ( x ) = p ( x | y = y ) which are conditionals of the joint distribution p ( x , y ) we were given in the theorem statement.
Revised: Now let the family P = { p y ( x )); y ∈ { 0 , 1 }} where p y ( x ) = p ( x | y = y ) which are conditionals of the joint distribution p ( x , y ) we were given in the theorem statement.

Original: Next, we show that the set of functions T ( x ) such that y | =
Revised: Next, we show that the set of functions T ( x ) such that y | =

Original: p x | T ( x ) is exactly the set of the sufﬁcient statistics for the family { p ( x | y =
Revised: p x | T ( x ) is exactly the set of the sufﬁcient statistics for the family { p ( x | y =

Original: y ); y ∈ { 0 , 1 }} .
Revised: y ); y ∈ { 0 , 1 }} .

Original: By deﬁnition of sufﬁciency where p y ( x | T ( x ) = t ) does not vary with y for any value of t , ∀ t,
Revised: By deﬁnition of sufﬁciency where p y ( x | T ( x ) = t ) does not vary with y for any value of t , ∀ t,

Original: [Equation]
Revised: [Equation]

Original: where the last statement is equivalent to the conditional independence x |
Revised: where the last statement is equivalent to the conditional independence x |

Original: = y | T ( x ) .
Revised: = y | T ( x ) .

Original: Minimality of p ( y = 1 | x ) .
Revised: Minimality of p ( y = 1 | x ) .

Original: By deﬁnition p y ( x ) = p ( x | y = y ) .
Revised: By deﬁnition p y ( x ) = p ( x | y = y ) .

Original: As this family contains only two elements, the minimal sufﬁcient statistic is
Revised: As this family contains only two elements, the minimal sufﬁcient statistic is

Original: [Equation]
Revised: [Equation]

Original: Thus, M ( x ) is a bijective transformation of the function p ( y = 1 | x ) (when p ( y = 1 | x ) ∈ (0 , 1) ) which in turn implies that p ( y = 1 | x ) is a minimal sufﬁcient statistic for the family P .
Revised: Thus, M ( x ) is a bijective transformation of the function p ( y = 1 | x ) (when p ( y = 1 | x ) ∈ (0 , 1) ) which in turn implies that p ( y = 1 | x ) is a minimal sufﬁcient statistic for the family P .

Original: Conclusion.
Revised: Conclusion.

Original: We showed that the set of functions that satisfy y | = x | T ( x ) are sufﬁcient statistics for the family P .
Revised: We showed that the set of functions that satisfy y | = x | T ( x ) are sufﬁcient statistics for the family P .

Original: In turn, because only sufﬁcient statistics T ( x ) of the family P satisfy y | =
Revised: In turn, because only sufﬁcient statistics T ( x ) of the family P satisfy y | =

Original: p x | T ( x ) , it follows by deﬁnition that p ( y = 1 | x ) is determined by every T ( x ) , completing the proof.
Revised: p x | T ( x ) , it follows by deﬁnition that p ( y = 1 | x ) is determined by every T ( x ) , completing the proof.

Original: A. G AUSSIAN EXAMPLE OF THE INFORMATION CRITERION
Revised: A. G AUSSIAN EXAMPLE OF THE INFORMATION CRITERION

Original: Proposition 3.
Revised: Proposition 3.

Original: Consider the following family of distributions q a indexed by a ∈ R ,
Revised: Consider the following family of distributions q a indexed by a ∈ R ,

Original: [Equation]
Revised: [Equation]

Original: In this family, for any p D = q b ( y | z ) where y (cid:54) | =
Revised: In this family, for any p D = q b ( y | z ) where y (cid:54) | =

Original: p D z , there exists a p (cid:48)
Revised: p D z , there exists a p (cid:48)

Original: D =
Revised: D =

Original: q a ( y | z ) such that
Revised: q a ( y | z ) such that

Original: [Equation]
Revised: [Equation]

Revised: >

Revised: [Equation]

Original: Proof.
Revised: Proof.

Original: (of proposition 3)
Revised: (of proposition 3)

Original: First, write z = a y + (cid:112) 1 / 2 δ where δ ∼ N (0 , 1) .
Revised: First, write z = a y + (cid:112) 1 / 2 δ where δ ∼ N (0 , 1) .

Original: Let (cid:15) =
Revised: Let (cid:15) =

Original: (cid:112) 1 / 

Revised: (cid:112) 1 /

Original: ( δ + (cid:15) z ) ; this is a normal variable with mean 0 and variance 1.
Revised: ( δ + (cid:15) z ) ; this is a normal variable with mean 0 and variance 1.

Original: Then, write x =
Revised: Then, write x =

Original: [ y + (cid:15) y , a y + (cid:15) ] where (cid:15)
Revised: [ y + (cid:15) y , a y + (cid:15) ] where (cid:15)

Original: y , (cid:15) are
Revised: y , (cid:15) are

Original: Gaussian random variables with joint distribution q ( (cid:15) y )
Revised: Gaussian random variables with joint distribution q ( (cid:15) y )

Original: q ( (cid:15) ) .
Revised: q ( (cid:15) ) .

Original: Therefore, q a ( y , x ) is a multivariate
Revised: Therefore, q a ( y , x ) is a multivariate

Original: Gaussian distribution, with the following covariance matrix (over y , x 1 , x 2 ):
Revised: Gaussian distribution, with the following covariance matrix (over y , x 1 , x 2 ):

Original: [Equation]
Revised: [Equation]

Original: The conditional mean and variance are:
Revised: The conditional mean and variance are:

Original: [Equation]
Revised: [Equation]

Original: Rewrite the quantity in the theorem statement as a single expression:
Revised: Rewrite the quantity in the theorem statement as a single expression:

Original: [Equation]
Revised: [Equation]

Original: Expand (log q ( y ) − log q b ( y | x )) in terms of quantities that vary with y , x and those that do not:
Revised: Expand (log q ( y ) − log q b ( y | x )) in terms of quantities that vary with y , x and those that do not:

Original: [Equation]
Revised: [Equation]

Original: [Equation]

Original: As only the ﬁrst two terms vary with y
Revised: As only the ﬁrst two terms vary with y

Revised: [Equation]

Revised: x

Original: [Equation]
Revised: [Equation]

Original: x , compute the expectations E q a over these: 
Revised: compute the expectations E q a over these: E

Original: [Equation]
Revised: [Equation]

Original: The proof follows for any a such that
Revised: The proof follows for any a such that

Original: [Equation]
Revised: [Equation]

Original: Let a = b + 1+ νb for some scalar ν .
Revised: Let a = b + 1+ νb for some scalar ν .

Original: Then
Revised: Then, if | ν | > 1 + ( b 2 + 2) log( b 2 + 2) ,

Original: [Equation]
Revised: [Equation]

Original: if | ν | > 1 + ( b 2 + 2) log( b2 + 2) ,
Revised: b 2 + 1 − ab [Equation] [Equation] b

Original: [Equation] 
Revised: [Equation] [Equation]

Revised: b

Original: [Equation]
Revised: [Equation]

Original: (cid:1) > 0 .
Revised: = [Equation] [Equation] [Equation]

Original: A. E XAMPLE WHERE p tr ( y | x ) AND p | = ( y | x ) PERFORM WORSE THAN p ( y ) UNDER p te
Revised: b [Equation] [Equation] [Equation]

Original: In this section
Revised: b

Original: [Equation]
Revised: [Equation]

Revised: [Equation]

Revised: > 0 .

Revised: | = ( y | x ) PERFORM WORSE THAN p ( y ) UNDER p te

Original:  we motivate nuisance-randomization and the uncorrelating property.
Revised: In this section, we motivate nuisance-randomization and the uncorrelating property.

Original: Consider the following data generating process for a family { q a } a ∈ R and ﬁxed positive scalar σ 2 :
Revised: Consider the following data generating process for a family { q a } a ∈ R and ﬁxed positive scalar σ 2 :

Original: [Equation]
Revised: [Equation]

Original: . 5) , x 2 ∼ N ( y + z , 0 . 5) (cid:3) .

Original: Letting σ 2 = 2 recovers the example in eq.
Revised: Letting σ 2 = 2 recovers the example in eq.

Original: We keep σ 2 for ease of readability.
Revised: We keep σ 2 for ease of readability.

Original: We ﬁrst derive the performance of p ( y ) =
Revised: We ﬁrst derive the performance of p ( y ) =

Original: q b ( y ) =
Revised: q b ( y ) =

Original: q ( y ) relative to q b ( y | x ) under q a .
Revised: q ( y ) relative to q b ( y | x ) under q a .

Original: Performance of q ( y ) relative to q b ( y | x ) under q a
Revised: Performance of q ( y ) relative to q b ( y | x ) under q a

Revised: Rewrite x = (cid:104) (1 − a ) ∗

Original: Rewrite x = (cid:104) (1 − a ) ∗ y + √ σ 2 (cid:15) 1 , (1 + a ) ∗ y + (cid:15) 2 ) (cid:105) , where (cid:15) 1 , (cid:15) 2 ∼ N (0 , 1) .
Revised: [Equation] + √ [Equation] 2 (cid:15) 1 , (1 + a ) ∗ y + (cid:15) 2 ) (cid:105) , where (cid:15) 1 , (cid:15) 2 ∼ N (0 , 1) .

Original: Let the joint distribution over y , (cid:15) 1 , (cid:15) 2 be q ( y ) q
Revised: Let the joint distribution over y , (cid:15) 1 , (cid:15) 2 be q ( y ) q

Original: ( (cid:15) 1 ) q ( (cid:15) 2 ) .
Revised: ( (cid:15) 1 ) q ( (cid:15) 2 ) .

Original: Then, q a ( y , x ) is a multivariate Gaussian distribution with the following covariance matrix (over y , x 1 , x 2 ):
Revised: Then, q a ( y , x ) is a multivariate Gaussian distribution with the following covariance matrix (over y , x 1 , x 2 ):

Original: [Equation]
Revised: [Equation]

Original: The performance of q ( y ) (recall Perf is negative KL ) relative to q b ( y | x ) on q a ( y , x ) can be written as
Revised: The performance of q ( y ) (recall Perf is negative KL ) relative to q b ( y | x ) on q a ( y , x ) can be written as

Original: [Equation]
Revised: [Equation]

Original:  qa (  x 
Revised: Expand (log q ( y ) − log q b ( y | x )) in terms that vary with y , x and those that do not:

Original: [Equation]
Revised: [Equation]

Original: y ) (log q ( y ) − log q b ( y | x)) . Expand (log q ( y ) − log q b ( y | x )) in terms that vary withy , x and those that do not: log
Revised: As σ 2 q b ( y | x ) does not vary with x , y

Original: [Equation]
Revised: [Equation]

Original: As σ 2 q b ( y | x ) does not vary with x , y , we will compute the expectation over the ﬁrst two terms: 
Revised: we will compute the expectation over the ﬁrst two terms: E [Equation] log

Original: [Equation]
Revised: [Equation]

Revised: y

Original: 
Revised: [Equation] log q [Equation] y | x ))

Original: [Equation]
Revised: [Equation]

Original:  − y (cid:19) + ( σ 2 (1 + b ) 2 + (1 − b ) 2 + σ 2 )E q ( y ) q a ( x | y ) (cid:16) y − 1 σ 2 (1+ b ) 2 +(1 − b ) 2 + σ 2 [(1{+
 (1+} − b ) , σ 2 (1 + b )] x (cid:17) 2 σ= − 1 2 + ( σ 2 (1 + b ) 2 + (1 − b ) 2 + σ 2 ) E q ( y ) q a ( x | y ) (cid:16) ( σ 2 (1 + b ) 2 + (1 − b ) 2 + σ 2 ) y − [(1 − b ) , σ 2 (1 + b )]{+
+}
Revised: | x )   − y2 + ( σ 2 (1 + b ) 2 + (1 − b ) 2 + σ 2 ) (cid:16) y − 1 σ 2 (1+ b ) 2 +(1 − b ) 2 + σ {+
 (1+} − b ) , σ 2 (1 + b )] x (cid:17) 2 σ  = E q ( y ) (cid:18) − y (cid:19) + ( σ 2 (1 + b ) 2 + (1 − b ) 2 + σ 2 ) E q ( y ) q a ( x | y ) (cid:16) y − 1 σ 2 (1+ b ) 2 +(1 − b ) 2 + σ {+
+}

Original: x (cid:17)2 σ 2  ( σ 2 (1 + b ) 2 + (1 − b ) 2 + σ 2 )
Revised: [(1 − b ) , σ 2 (1 + b )] x (cid:17) 2 σ= − 1 2 + ( σ 2 (1 + b ) 2 + (1 − b ) 2 + σ 2 )

Original:  = − 1 2 + E q ( y ) q a ( x | y ) (cid:16) ( σ 2 (1 + b ) 2 + (1 − b ) 2 + σ 2 ) y − [(1 − b ) , σ 2 (1 + b )]
Revised: E q ( y ) q a ( x | y ) (cid:16) ( σ 2 (1 + b ) 2 + (1 − b ) 2 + σ 2 ) y − [(1 − b ) , σ 2 (1 + b )] x (cid:17)2 σ 2 ( σ 2 (1 + b ) 2 + (1 − b ) 2 + σ 2 ) = − 1 2 + E q ( y ) q a ( x | y ) (cid:16) ( σ 2 (1 + b ) 2 + (1 − b ) 2 + σ 2 ) y − [(1 − b ) , σ 2 (1 + b )]

Original: x (cid:17)2 σ 2 ( σ 2 (1 + b ) 2 + (1 − b ) 2 + σ 2 ) (cid:16) recall x =
Revised: x (cid:17)2 σ 2 ( σ 2 (1 + b ) 2 + (1 − b ) 2 + σ 2 ) (cid:16) recall x =

Original: [(1 − a ) y + √ σ 2 (cid:15) 1 , (1 + a ) y
Revised: [(1 − a ) y + √ σ 2 (cid:15) 1 , (1 + a ) y

Original: + (cid:15) 2 ] (cid:17)
Revised: + (cid:15) 2 ] (cid:17)

Original: = − 1 2+
Revised: = − 1 2+

Original: E q ( y ) q
Revised: E q ( y ) q

Original: ( (cid:15) y )
Revised: ( (cid:15) y )

Original: q ( (cid:15) ) (cid:16) ( σ 2 (1 + b ) 2 + (1 − b ) 2 + σ 2 ) y − (1 − b ) √ σ 2 (cid:15) 1 − (1 − a )(1 − b ) y − σ 2 (1 + a )(1 + b ) y − σ 2 (1 + b ) (cid:15) 2 (cid:17)2 σ 2 ( σ 2 (1 + b ) 2 + (1 − b ) 2 + σ 2 ) = − 1 2 + E q ( y ) q ( (cid:15) [Table]{+
+}
Revised: q ( (cid:15) ) (cid:16) ( σ 2 (1 + b ) 2 + (1 − b ) {+
+}

Original: [Equation] [Table] [Equation] [Table]
Revised: + σ

Original: [Equation]
Revised: [Equation]

Original: [Table]

Original: [Equation][Table] [Equation] [Table]
Revised: [Equation]

Original: [Equation]
Revised: [Equation]

Original: [Table]
Revised: [Table]

Original: [Equation]
Revised: [Equation]

Original: [Table]
Revised: [Table]

Original: [Equation]
Revised: [Equation]

Original: [Table]
Revised: [Table]

Original: [Equation]
Revised: [Equation]

Original: [Table]
Revised: [Table]

Original: [Equation]
Revised: [Equation]

Original: [Table] [Equation] [Table]
Revised: = 12 log ( σ 2 (1+ b

Original: [Equation]
Revised: [Equation]

Original: [Table] [Equation] [Table] [Equation]
Revised: +(

Original: [Equation]
Revised: [Equation]

Original: Note that log (cid:113)
Revised: b

Original: [Equation]
Revised: [Equation]

Original: log (cid:113)
Revised: 2 +

Original: [Equation]
Revised: [Equation]

Original:  With this, we have the ability to bound the performance of q ( y ) relative to q b ( y | x ) under q a .
Revised: σ. With this, we have the ability to bound the performance of q ( y ) relative to q b ( y | x ) under q a .

Original: The conditional q 1 ( y | x ) performs worse than q 1 ( y )
Revised: The conditional q 1 ( y | x ) performs worse than q 1 ( y )

Original: =
Revised: =

Original: p ( y ) on q − 1 .
Revised: p ( y ) on q − 1 .

Original: We show here that p tr ( y | x ) =
Revised: We show here that p tr ( y | x ) =

Original: q 1 ( y | x ) does worse than q 1 ( y ) =
Revised: q 1 ( y | x ) does worse than q 1 ( y ) =

Original: p ( y ) on p te =
Revised: p ( y ) on p te =

Original: q − 1 .
Revised: q − 1 .

Original: Letting a = − 1 =
Revised: Letting a = − 1 =

Original: ⇒ (1 + a ) = 0 , (1 − a )
Revised: ⇒ (1 + a ) = 0 , (1 − a )

Original: = 2 and b = 1 = ⇒ (1 + b ) = 2 , (1 − b ) = 

Revised: = 2 and b = 1 = ⇒ (1 + b ) = 2 , (1 − b ) =

Original: then
Revised: then

Revised: C ( a, b )

Original: [Equation]
Revised: [Equation]

Original: Predictive failure when x is not uncorrelating Note that q 0 ( y | x ) =
Revised: Predictive failure when x is not uncorrelating Note that q 0 ( y | x ) =

Original: p |
Revised: p |

Original: = ( y | x ) corresponds to the nuisance randomized conditional of the label given the covariates.
Revised: = ( y | x ) corresponds to the nuisance randomized conditional of the label given the covariates.

Original: We show that q 

Revised: We show that q

Original: ( y | x ) performs worse than p ( y ) on inﬁnitely many q a .
Revised: ( y | x ) performs worse than p ( y ) on inﬁnitely many q a .

Original: Letting b = 0 in C ( a, b ) helps compute the performance of q ( y ) over q 0 ( y | x ) for every q a :C
Revised: Letting b = 0 in C ( a, b ) helps compute the performance of q ( y ) over q 0 ( y | x ) for every q a :

Original: [Equation]
Revised: [Equation]

Original: a, b

Original: [Equation]
Revised: [Equation]

Original: 1 − a ) = σ 2 + a (1 − σ 2 ) . Then, the performance of q ( y ) relative to p | =
Revised: Then, the performance of q ( y ) relative to p | =

Original: ( y | x ) is
Revised: ( y | x ) is

Original: [Equation]
Revised: [Equation]

Original: a

Original: [Equation]

Original: (1 − σ 2 )

Original: [Equation]

Original: + ( σ 2 ) [Equation] [Equation] (cid:1)
Revised: σ 2 =

Original: [Equation]

Original: [Equation]

Original: [Equation]

Original: [Equation]

Original: log (cid:0)

Original: [Equation]
Revised: [Equation]

Original: σ

Original: [Equation]
Revised: [Equation]

Original: + 1 (cid:1) { letting σ 2 = 2 . }

Original: = a 2 − 4 a −2 log 2 .

Original: This performance difference between q ( y ) and p | =
Revised: This performance difference between q ( y ) and p | =

Original: ( y | x ) is positive for any a > 5 or a < − 1 .
Revised: ( y | x ) is positive for any a > 5 or a < − 1 .

Original: Thus, for every test distribution q a such that a > 5 or a < − 1 , p | = ( y | x ) performs worse than marginal prediction.
Revised: Thus, for every test distribution q a such that a > 5 or a < − 1 , p | = ( y | x ) performs worse than marginal prediction.

Original: B F URTHER EXPERIMENTAL DETAILS
Revised: B F URTHER EXPERIMENTAL DETAILS

Original: Implementation details In section 5, the label y is a binary variable and, consequently, we use the
Revised: Implementation details In section 5, the label y is a binary variable and, consequently, we use the

Original: Bernoulli likelihood in the predictive model and the weight model.
Revised: Bernoulli likelihood in the predictive model and the weight model.

Original: In reweighting-N U RD in practice, the estimate of the nuisance-randomized distribution ˆ p | =
Revised: In reweighting-N U RD in practice, the estimate of the nuisance-randomized distribution ˆ p | =

Original: ( y , z , x ) ∝ p tr ( y ) /
Revised: ( y , z , x ) ∝ p tr ( y ) /

Original: ˆ p tr ( y | z ) p tr ( y , z , x ) with an estimated ˆ p tr ( y | z ) may have a different marginal distribution ˆ p | =
Revised: ˆ p tr ( y | z ) p tr ( y , z , x ) with an estimated ˆ p tr ( y | z ) may have a different marginal distribution ˆ p | =

Original: ( y ) (cid:54) =
Revised: ( y ) (cid:54) =

Original: p tr ( y ) .
Revised: p tr ( y ) .

Original: To correct for this in the estimate ˆ p | = ( y, z , x ) , we weight it again, p tr ( y ) ˆ p | =
Revised: To ensure that p tr ( y ) = ˆ p | = ( y ) , we weight our preliminary estimate ˆ p | = again as p tr ( y ) ˆ p | =

Original: ( y ) ˆ p | =
Revised: ( y ) ˆ p | =

Original: ( y , z , x ) , to ensure that p tr ( y ) =
Revised: ( y , z , x ) .

Original: ˆ p | = ( y ) . In all the experiments, the distribution p θ ( y | r γ ( x )) is a Bernoulli distribution parameterized by r γ and a scaling parameter θ .
Revised: In all the experiments, the distribution p θ ( y | r γ ( x )) is a Bernoulli distribution parameterized by r γ and a scaling parameter θ .

Original: In general, when the family of p | = ( y | r γ ( x )) is unknown, learning predictive models requires a parameterization p θ ( y | r γ ( x )) .
Revised: In general, when the family of p | = ( y | r γ ( x )) is unknown, learning predictive models requires a parameterization p θ ( y | r γ ( x )) .

Original: When the family is known, for example when y is categorical, the parameters θ are not needed because the distribution p ( y | r γ ( x )) can be parameterized by the representation itself.
Revised: When the family is known, for example when y is categorical, the parameters θ are not needed because the distribution p ( y | r γ ( x )) can be parameterized by the representation itself.

Original: For the critic model p φ ( (cid:96) | y , z , r γ ( x )) in the distillation step, we use a two layer neural network with 16 hidden units and ReLU activations that takes as input y , r γ ( x ) , and a scalar representation s ψ ( z )  .
Revised: For the critic model p φ ( (cid:96) | y , z , r γ ( x )) in the distillation step, we use a two layer neural network with 16 hidden units and ReLU activations that takes as input y , r γ ( x ) , and a scalar representation s ψ ( z ) ; the critic model’s parameters are φ, ψ .

Original: The representation s ψ ( z ) is different in the different experiments and we give these details below.
Revised: The representation s ψ ( z ) is different in the different experiments and we give these details below.

Original: In all experiments, we reported results with the distillation step optimized with a ﬁxed λ = 1 and one epoch’s worth of gradient steps for the critic model for each gradient step of the predictive model.

Original: In generative-N U RD, we select models for p ( x | y , z ) by using the generative objective’s value on a heldout subset of the training data.
Revised: In generative-N U RD, we select models for p ( x | y , z ) by using the generative objective’s value on a heldout subset of the training data.

Original: For model selection, we use Gaussian likelihood in the classconditional Gaussian experiment, binary likelihood in the colored-MNIST experiment, and squaredloss reconstruction error in the Waterbirds and chest X-ray experiments.
Revised: For model selection, we use Gaussian likelihood in the class-conditional Gaussian experiment, binary likelihood in the colored-MNIST experiment, and squared-loss reconstruction error in the Waterbirds and chest X-ray experiments.

Original: Similarly, in reweightingN U RD,  the  model  for p tr ( y | z ) is  evaluated on a heldout  subset of the training data.
Revised: In reweightingN U RD, we use a cross-ﬁtting procedure where the training data is split into K folds, and K models are trained: for each fold, we produce weights using a model trained and validated on the other K − 1 folds. Hyperparameter selection for the distillation step is done using the distillation loss from eq. (6) evaluated on a heldout validation subset of the nuisance-randomized data from the ﬁrst step.

Revised: In all experiments, we report results with the distillation step optimized with a ﬁxed λ = 1 and with 1 or 2 epochs worth of critic model updates per every representation update.

Revised: In setting the hyperparameter λ , a practitioner should choose the largest λ such that optimization is still stable for different seeds and the validation loss is bounded away from that of marginal prediction.

Original: Next, we give details about each individual experiment.
Revised: Next, we give details about each individual experiment.

Original: Theoretical details for Class Conditional Gaussians.
Revised: Optimal linear uncorrelating representations in Class Conditional Gaussians.

Original: Here, we show that r ∗ ( x ) =
Revised: Here, we show that r ∗ ( x ) =

Original: x 1 + x 2 is the best linear uncorrelating representation in terms of performance.
Revised: x 1 + x 2 is the best linear uncorrelating representation in terms of performance.

Original: First let the Gaussian noises in the two coordinates of x (given y , z ) be (cid:15) 1 ∼ N (0 , 9) and (cid:15) 2 ∼ N (0 , 0 . 01) respectively.
Revised: First let the Gaussian noises in the two coordinates of x (given y , z ) be (cid:15) 1 ∼ N (0 , 9) and (cid:15) 2 ∼ N (0 , 0 . 01) respectively.

Original: Deﬁne r u,v ( x ) =
Revised: Deﬁne r u,v ( x ) =

Original: u x 1 + v x 2 = ( u + v )
Revised: u x 1 + v x 2 = ( u + v )

Original: y
Revised: y

Original: + ( v − u ) z
Revised: + ( v − u ) z

Original: + u (cid:15) 1 + v (cid:15) 2 .
Revised: + u (cid:15) 1 + v (cid:15) 2 .

Original: We will show that q 0 ( z | r u,v ( x ) , y = 1) (cid:54) =
Revised: We will show that q 0 ( z | r u,v ( x ) , y = 1) (cid:54) =

Original: q 0 ( z | r u,v ( x ) , y = 0) when u (cid:54) = v and u (cid:54) = − v .
Revised: q 0 ( z | r u,v ( x ) , y = 0) when u (cid:54) = v and u (cid:54) = − v .

Original: First, q 0 ( z , r u,v ( x ) | y
Revised: First, q 0 ( z , r u,v ( x ) | y

Original: =
Revised: =

Original: y ) is a bivariate Gaussian with the following covariance matrix:
Revised: y ) is a bivariate Gaussian with the following covariance matrix:

Original: [Equation]
Revised: [Equation]

Original: The conditional mean is:
Revised: The conditional mean is:

Revised: [Equation]

Original: [Equation]
Revised: [Equation]

Original: which is independent of y if and only if u + v = 0 or u − v = 0 .
Revised: which is independent of y if and only if u + v = 0 or u − v = 0 .

Original: The conditional variance does not change with y because it is determined by Σ y which does not change with y .
Revised: The conditional variance does not change with y because it is determined by Σ y which does not change with y .

Original: Thus q 0 ( z | r u,v ( x ) , y )
Revised: Thus q 0 ( z | r u,v ( x ) , y )

Original: =
Revised: =

Original: q 0 ( z | r u,v ( x ))
Revised: q 0 ( z | r u,v ( x ))

Original: if and only if u = v or u = − v .
Revised: if and only if u = v or u = − v .

Original: When u = v , r u,v = 2 u y + noise and r u,u (cid:54) | = q 

Revised: When u = v , r u,v = 2 u y + noise and r u,u (cid:54) | = q

Original: y meaning that r u,u helps predict y .
Revised: y meaning that r u,u helps predict y .

Original: In contrast, when u = − v , r u,v = 2 v z + noise and so r − v,v | = q 0 y = ⇒ q 0 ( y | r − v,v ) =
Revised: In contrast, when u = − v , r u,v = 2 v z + noise and so r − v,v | = q 0 y = ⇒ q 0 ( y | r − v,v ) =

Original: q 0 ( y ) , meaning that q 0 ( y | r − v,v ) has the same performance as the marginal.
Revised: q 0 ( y ) , meaning that q 0 ( y | r − v,v ) has the same performance as the marginal.

Original: However, for all u (cid:54) = 0 , r u,u = ur 1 , 1 is a bijective transform of r 1 , 1 and, therefore, q 0 ( y | r u,u ( x ))
Revised: However, for all u (cid:54) = 0 , r u,u = ur 1 , 1 is a bijective transform of r 1 , 1 and, therefore, q 0 ( y | r u,u ( x ))

Original: =
Revised: =

Original: q 

Revised: q

Original: ( y | r 1 , 1 ( x )) .
Revised: ( y | r 1 , 1 ( x )) .

Original: Thus, within the set of linear uncorrelating representations, r 1 , 1 is the best because its performance dominates all others on every p te ∈ F .
Revised: Thus, within the set of linear uncorrelating representations, r 1 , 1 is the best because its performance dominates all others on every p te ∈ F .

Original: Implementation details for Class Conditional Gaussians.
Revised: Implementation details for Class Conditional Gaussians.

Original: In reweighting-N U RD, the model for p tr ( y | z ) is a Bernoulli distribution parameterized by a neural network with 1 hidden layer withunits and ReLU activations.
Revised: In reweighting-N U RD, the model for p tr ( y | z ) is a Bernoulli distribution parameterized by a neural network with 1 hidden layer withunits and ReLU activations.

Original: In generative-N U RD, the model for p ( x | y , z ) is an isotropic Gaussian whose mean and covariance are parameterized with a neural network with one layer with 16 units and ReLU activations. 
Revised: In generative-N U RD, the model for p ( x | y , z ) is an isotropic Gaussian whose mean and covariance are parameterized with a neural network with one layer with 16 units and ReLU activations. We use 5 cross-ﬁtting folds in estimating the weights in reweighting-N U RD.

Revised: We use weighted sampling with replacement in computing the distillation objective.

Original: In the distillation step in both reweighting and generative-N U RD, the representation r γ ( x ) is a neural network with one hidden layer with 16 units and ReLU activations.
Revised: In the distillation step in both reweighting and generative-N U RD, the representation r γ ( x ) is a neural network with one hidden layer with 16 units and ReLU activations.

Original: The critic model p φ ( (cid:96) | y , z , r γ ( x )) consists of a neural network with 2 hidden layers with 16 units each and ReLU activations that takes as input y , r γ ( x ) , and a scalar representation s ψ ( z ) which is again a neural network with a single hidden layer of 16 units and ReLU activations.
Revised: The critic model p φ ( (cid:96) | y , z , r γ ( x )) consists of a neural network with 2 hidden layers with 16 units each and ReLU activations that takes as input y , r γ ( x ) , and a scalar representation s ψ ( z ) which is again a neural network with a single hidden layer of 16 units and ReLU activations.

Original: We use cross entropy to train ˆ p tr ( y | z ) , ˆ p | =
Revised: We use cross entropy to train ˆ p tr ( y | z ) , ˆ p | =

Original: ( y | r γ ( x )) , and p φ ( (cid:96) | y , z , r γ ( x )) using the Adam (Kingma and Ba, 2014) optimizer with a learning rate of 10 − 3 .
Revised: ( y | r γ ( x )) , and p φ ( (cid:96) | y , z , r γ ( x )) using the Adam (Kingma and Ba, 2014) optimizer with a learning rate of 10 − 2 .

Original: To train p tr ( y | z )we use a batch size of 64 and a batch size of 256 for  the distillation step to train p θ ( y | r γ ( x)) and p φ ( (cid:96) | y , z , r γ ( x )) .
Revised: We optimized the model for ˆ p tr ( y | z ) for 100 epochs and the model for ˆ p tr ( x | y , z ) for 300 epochs.

Original: We optimized for a 100 epochs and chose the model based on the value of the objective in both stages of N U RD on a heldout subset of the training data.
Revised: We ran the distillation step for 150 epochs with the Adam optimizer with the default learning rate. We use a batch size of 1000 in both stages of N U RD.

Original: We run the distillation step with a ﬁxed λ = 1 and one epoch’s worth of gradient steps (40) for the critic model for each gradient step of the predictive model.
Revised: We run the distillation step with a ﬁxed λ = 1 and two epoch’s worth of gradient steps ( 16 ) for the critic model for each gradient step of the predictive model and the representation.

Revised: In this experiment, we do not re-initialize p φ after a predictive model update.

Original: Implementation details for Colored-MNIST.
Revised: Implementation details for Colored-MNIST.

Original: For reweighting-N U RD, to use the same architecture for the representation r γ ( x ) and for p tr ( y | z ) , we construct the nuisance as a 28 × 28 image with each pixel being equal to the most intense pixel in the original image.
Revised: For reweighting-N U RD, to use the same architecture for the representation r γ ( x ) and for p tr ( y | z ) , we construct the nuisance as a 28 × 28 image with each pixel being equal to the most intense pixel in the original image.

Original: In generative-N U RD, we use a PixelCNN model for p ( x | y , z ) with 10 masked convolutional layers each with 64 ﬁlters.
Revised: In generative-N U RD, we use a PixelCNN model for p ( x | y , z ) with 10 masked convolutional layers each with 64 ﬁlters.

Original: The model was trained  with the Adam optimizer and a ﬁxed learning rate of 10 − 3 and batch size 128 .
Revised: The model was trained using a Bernoulli likelihood with the Adam optimizer and a ﬁxed learning rate of 10 − 3 and batch size 128 .

Original: We parameterize multiple models in this experiment with the following neural network: 4 convolutional layers (with 32 , 64 , 128 , 256 channels respectively) with ReLU activations followed by a fully connected linear layer into a single unit.
Revised: We parameterize multiple models in this experiment with the following neural network: 4 convolutional layers (with 32 , 64 , 128 , 256 channels respectively) with ReLU activations followed by a fully connected linear layer into a single unit.

Original: Both r γ ( x ) , s ψ ( z ) are parameterized by this network.
Revised: Both r γ ( x ) , s ψ ( z ) are parameterized by this network.

Original: Both ˆ p tr ( y | z ) in reweighting-N U RD and ˆ p tr ( y | x ) for ERM are Bernoulli distributions parameterized by the network described above.
Revised: Both ˆ p tr ( y | z ) in reweighting-N U RD and ˆ p tr ( y | x ) for ERM are Bernoulli distributions parameterized by the network described above.

Original:  For the critic model p φ ( (cid:96) | y , z , r γ ( x )) in the distillation step, we use a two-hidden-layer neural network with 16 hidden units and ReLU activations that takes as input y , r γ ( x ) , and the scalar representation s ψ ( z ) ; the parameters φ contain ψ and the parameters for the two hidden-layer neural network.
Revised: We use 5 cross-ﬁtting folds in estimating the weights in reweighting-N U RD. For the critic model p φ ( (cid:96) | y , z , r γ ( x )) in the distillation step, we use a two-hidden-layer neural network with 16 hidden units and ReLU activations that takes as input y , r γ ( x ) , and the scalar representation s ψ ( z ) ; the parameters φ contain ψ and the parameters for the two hidden-layer neural network.

Original: The predictive model p θ ( y | r γ ( x )) is a Bernoulli distribution parameterized by r γ ( x ) multiplied by a scalar θ .
Revised: The predictive model p θ ( y | r γ ( x )) is a Bernoulli distribution parameterized by r γ ( x ) multiplied by a scalar θ .

Original: We use cross entropy to train ˆ p tr ( y | z ) , ˆ p | =
Revised: We use cross entropy to train ˆ p tr ( y | z ) , ˆ p | =

Original: ( y | r γ ( x )) , and p φ ( (cid:96) | y , z , r γ ( x )) using the
Revised: ( y | r γ ( x )) , and p φ ( (cid:96) | y , z , r γ ( x )) using the

Original: Adam (Kingma and Ba, 2014) optimizer with a learning rate of 10 − 3 .
Revised: Adam (Kingma and Ba, 2014) optimizer with a learning rate of 10 − 3 .

Original: To train p tr ( y | z )we use a batch size of 64 and a batch size of 256 for  the distillation stepto train p θ ( y | r γ ( x )) and p φ ( (cid:96) | y , z , r γ ( x )) . We optimized for 10 epochs and chose the model based on the value of the objective in both stages of N U RD on a heldout subset of the training data.
Revised: We optimized the model for ˆ p tr ( y | z ) for 20 epochs and ran the distillation step for 20 epochs with the Adam optimizer with the default learning rate. We use a batch size of 300 in both stages of N U RD.

Original: We run the distillation step with a ﬁxed λ = 1 and one epoch’s worth of gradient steps ( 16 ) for the critic model for each gradient step of the predictive model and the representation.
Revised: We run the distillation step with a ﬁxed λ = 1 and one epoch’s worth of gradient steps ( 14 ) for the critic model for each gradient step of the predictive model and the representation.

Original: To prevent the critic model from overﬁtting, we re-initialize p φ after every gradient step of the predictive model.
Revised: To prevent the critic model from overﬁtting, we re-initialize p φ after every gradient step of the predictive model.

Original: Implementation details for the Waterbirds experiment.
Revised: Implementation details for the Waterbirds experiment.

Original: For generative-N U RD, we use VQVAE 2 (Razavi et al., 2019) to model p tr ( x | y , z ) .
Revised: For generative-N U RD, we use VQVAE 2 (Razavi et al., 2019) to model p tr ( x | y , z ) .

Original: For multiple latent sizes and channels in the encoder and the decoder, we saw that the resulting generated images were insufﬁcient to build classiﬁers that predict better than chance on real data.
Revised: For multiple latent sizes and channels in the encoder and the decoder, we saw that the resulting generated images were insufﬁcient to build classiﬁers that predict better than chance on real data.

Original: This may be because of the small training dataset that consists of only 1936 samples.
Revised: This may be because of the small training dataset that consists of only 3000 samples.

Original: The model for ˆ p | =
Revised: The model for ˆ p | =

Original: ( y | r γ ( x )) is two feedforward layers stacked on top of the representation r γ ( x ) .
Revised: ( y | r γ ( x )) is two feedforward layers stacked on top of the representation r γ ( x ) .

Original: The model
Revised: The model

Original: ˆ p tr ( y | z ) in reweighting-N U RD is the same model as ˆ p | =
Revised: ˆ p tr ( y | z ) in reweighting-N U RD is the same model as ˆ p | =

Original: ( y | r γ ( x )) as a function of x .
Revised: ( y | r γ ( x )) as a function of x .

Original: The model for p φ ( (cid:96) | y , z , r γ ( x )) consists of a neural network with two feedforward layers that takes as input y , r γ ( x ) , and a representation s ψ ( z ) .
Revised: The model for p φ ( (cid:96) | y , z , r γ ( x )) consists of a neural network with two feedforward layers that takes as input y , r γ ( x ) , and a representation s ψ ( z ) .

Original: Both r γ and s ψ are Resnet-18 models initialized with weights pretrained on Imagenet; the parameters φ contain ψ and the parameters for the two hidden-layer neural network.
Revised: Both r γ and s ψ are Resnet-18 models initialized with weights pretrained on Imagenet; the parameters φ contain ψ and the parameters for the two hidden-layer neural network.

Original: The model in ERM for p tr ( y | x ) uses the same architecture as ˆ p | =
Revised: The model in ERM for p tr ( y | x ) uses the same architecture as ˆ p | =

Original: ( y | r γ ( x )) as a function of x .
Revised: ( y | r γ ( x )) as a function of x .

Revised: We use 5 cross-ﬁtting folds in estimating the weights in reweighting-N U RD.

Original: We use binary cross entropy as the loss in training ˆ p tr ( y | z ) , p θ ( y | r γ ( x )) , and p φ ( (cid:96) | y , z , r γ ( x ))) using the Adam (Kingma and Ba, 2014) optimizer with a learning rate of 10 − 3 .
Revised: We use binary cross entropy as the loss in training ˆ p tr ( y | z ) , p θ ( y | r γ ( x )) , and p φ ( (cid:96) | y , z , r γ ( x ))) using the Adam (Kingma and Ba, 2014) optimizer with a learning rate of 10 − 3 .

Original: To train ˆ p tr ( y | z )we use a batch size of 64 and a batch size of 256 for  the distillation step to train p θ ( y | r γ( x )) and p φ ( (cid:96) | y , z ,  r γ( x )) . 
Revised: We optimized the model for ˆ p tr ( y | z ) for 10 epochs and ran the distillation step for 5 epochs with the Adam optimizer with the default learning rate for all parameters except γ , which parameterizes the representation r γ . The predictive model, the critic model, and the weight model are all optimized with a weight decay of 0 . We use a batch size of 300 for both stages of N U RD.

Original: We run the distillation step with a ﬁxed λ = 1 and one epoch’s worth of gradient steps ( 12 ) for the critic model for each gradient step of the predictive model and the representation. 
Revised: We run the distillation step with a ﬁxed λ = 1 and two epoch’s worth of gradient steps ( 16 ) for the critic model for each gradient step of the predictive model and the representation. To prevent the critic model from overﬁtting, we re-initialize [Table]

Original: Implementation details for the chest X-ray experiment.
Revised: Implementation details for the chest X-ray experiment.

Revised: To help with generative modeling, when creating the dataset, we remove X-ray samples from MIMIC that had all white or all black borders.

Original: We use a VQ-VAE2 (Razavi et al., 2019) to model p ( x | y , z ) using code from here to both train and sample.
Revised: We use a VQ-VAE2 (Razavi et al., 2019) to model p ( x | y , z ) using code from here to both train and sample.

Original: The encoder takes the lung patch as input, and the decoder takes the quantized embeddings and the non-lung patch as input.
Revised: The encoder takes the lung patch as input, and the decoder takes the quantized embeddings and the non-lung patch as input.

Original: VQ-VAE2 is hierarchical with a top latent code and a bottom latent code which are both vector-quantized and fed into the decoder to reconstruct the image.
Revised: VQ-VAE2 is hierarchical with a top latent code and a bottom latent code which are both vector-quantized and fed into the decoder to reconstruct the image.

Original: Both latents consist of 8 ×embeddings each of dimension 64 .
Revised: Both latents consist of 8 × 8 embeddings each of dimension 64 .

Original: The VQ-VAE is trained for 200 epochs with Adam (Kingma and Ba, 2014) with a batch size of 256 and dropout rate of 0 . 1 .
Revised: The VQ-VAE is trained for 200 epochs with Adam (Kingma and Ba, 2014) with a batch size of 256 and dropout rate of 0 . 1 .

Original: Generating samples from the VQ-VAE involves sampling the top latent code conditioned on the label, followed by sampling the bottom latent code conditioned on the label and the top latent code, and passing both latent codes to the decoder.
Revised: Generating samples from the VQ-VAE2 involves sampling the top latent code conditioned on the label, followed by sampling the bottom latent code conditioned on the label and the top latent code, and passing both latent codes to the decoder.

Original: To generate from the latent codes, we build a PixelSNAIL to generate the top latent code given the label and a PixelCNN to generate the bottom latent code given the label and the top latent code.
Revised: To generate from the latent codes, we build a PixelSNAIL to generate the top latent code given the label and a PixelCNN to generate the bottom latent code given the label and the top latent code.

Original: These models have 5 residual layers with 128 convolutional channels.
Revised: These models have 5 residual layers withconvolutional channels.

Original: All other details were default as in here.
Revised: All other details were default as in here.

Original: We train these models for 450 epochs with a batch size of 256 with a learning rate of 5 × 10 − 5 .
Revised: We train these models forepochs with a batch size of 256 with a learning rate of 5 × 10 − 5 .

Original: For reweighting-N U RD, the model ˆ p | =
Revised: For reweighting-N U RD, the model ˆ p | =

Original: ( y | r γ ( x )) is two feedforward layers stacked on top of the representation r γ ( x ) .
Revised: ( y | r γ ( x )) is two feedforward layers stacked on top of the representation r γ ( x ) .

Original: The model in ERM for p tr ( y | x ) uses the same architecture as ˆ p | =
Revised: The model in ERM for p tr ( y | x ) uses the same architecture as ˆ p | =

Original: ( y | r γ ( x )) as a function of x .
Revised: ( y | r γ ( x )) as a function of x .

Original: Next we use a single architecture to parameterize multiple parts in this experiment: 3 convolutional layers (each 64 channels) each followed by batch norm, and dropout with a rate of 0 . 5 and followed by a linear fully-connected layer into a single unit.
Revised: Next we use a single architecture to parameterize multiple parts in this experiment: 3 convolutional layers (each 64 channels) each followed by batch norm, and dropout with a rate of 0 . 5 and followed by a linear fully-connected layer into a single unit.

Original: We parameterize the two representations r γ ( x ) , s ψ ( z ) with this network.
Revised: We parameterize the two representations r γ ( x ) , s ψ ( z ) with this network.

Original: To build p φ ( (cid:96) | y , z , r γ ( x )) , we stack two feedforward layers of 16 hidden units with ReLU activations on top of a concatenation of y , r γ ( x ) ,and the scalar representation s ψ ( z ) as described above; the parameters φ contain ψ and the parameters for the two hidden-layer neural network.
Revised: To build p φ ( (cid:96) | y , z , r γ ( x )) , we stack two feedforward layers of 16 hidden units with ReLU activations on top of a concatenation of y , r γ ( x ) ,and the scalar representation s ψ ( z ) as described above; the parameters φ contain ψ and the parameters for the two hidden-layer neural network.

Revised: We use 5 cross-ﬁtting folds in estimating the weights in reweighting-N U RD.

Original: We use binary cross entropy as the loss in training ˆ p tr ( y | z ) , p θ ( y | r γ ( x )) , and p φ ( (cid:96) | y , z , r γ ( x ))) using the Adam (Kingma and Ba, 2014) optimizer with a learning rate of 10 − 3 .
Revised: We use binary cross entropy as the loss in training ˆ p tr ( y | z ) , p θ ( y | r γ ( x )) , and p φ ( (cid:96) | y , z , r γ ( x ))) using the Adam (Kingma and Ba, 2014) optimizer with a learning rate of 10 − 3 .

Original: To train p tr ( y | z ) we use a batch size of 64 and a batch size of 256 for  the distillation step to train p θ ( y |r γ ( x )) and p φ ( (cid:96) | y , z , r γ ( x )) . We optimized for 10 epochs and chose the model based on the value of the objective in both stages of N U RD on a heldout subset of the training data.
Revised: We use a batch size of 1000 for both stages of N U RD. We optimized the model for ˆ p tr ( y | z ) for 150 epochs and ran the distillation step for 100 epochs with the Adam optimizer with the default learning rate. Only the optimization for ˆ p tr ( y | z ) has a weight decay of 1 e − 2 .

Original: The distillation stepis run with  λ = 1 and one epoch’s worth of gradient steps ( 133 ) for the critic model for each gradient step of the predictive model and the representation.
Revised: We run the distillation step with a ﬁxed λ = 1 and two epoch’s worth of gradient steps ( 20 ) for the critic model for each gradient step of the predictive model and the representation.

Original: To prevent the critic model from overﬁtting, we re-initialize p φ after every gradient step of the predictive model.
Revised: To prevent the critic model from overﬁtting, we re-initialize p φ after every gradient step of the predictive model.

Original: B. A DDITIONAL EXPERIMENTSWITH CLASSIFYING W ATERBIRDS AND P NEUMONIA
Revised: B. A DDITIONAL EXPERIMENTS

Revised: Excluding the boundary from the images (covariates) does not improve ERM in general.

Original: We evaluate reweighting-N U RD further on classifying Waterbirds and detecting Pneumonia.
Revised: In both Waterbirds and chest X-rays, we use the easy-to-acquire border as a nuisance in N U RD.

Original: We report accuracies for the following:
Revised: Models trained on the central (non-border) regions of the image can exploit the nuisances in the center and consequently fail to generalize when the nuisance-label relationship changes. In fact, classiﬁers produced by ERM on border-less images do not generalize well to the test data, producing test accuracies of 39 ± 0 . 5% on chest X-rays and 65 ± 2 . 3% on Waterbirds averaged over 10 seeds.

Revised: However, as independence properties that hold for the border also hold for nuisances in the central region that are determined by the border, N U RD can use the border to control for certain nuisances in the center of the image.

Revised: Additional experiments with N U RD.

Revised: We evaluate reweighting-N U RD further in the following ways:

Original: Run N U RD on data from
Revised: Run N U RD on data from

Original: the
Revised: the

Original: training data distribution deﬁned
Revised: training data distribution deﬁned

Original: in section5 and
Revised: in section and

Original: evaluate on data fromdifferent test distributions p te with different nuisance-label relationships.
Revised: evaluate on data from test distributions p te with different nuisance-label relationships.

Original: Train N U RD with nuisances that are borders of the image (all but the central patches) of sizes different from those in section 5. Run N U RD in the chest X-ray experiment where the labels are balanced in the datasets is by downsampling healthy cases instead of upsampling Pneumonia cases.
Revised: Train N U RD with different-sized borders as nuisances.

Original: Train N U RD without a nuisance where the training and the test data have the same nuisance-label relationship; we implement this by setting the nuisance z = 0 wherever it is passed as input in the weight model or critic model. 
Revised: Train N U RD without a nuisance where the training and the test data have the same nuisance-label relationship; we implement this by setting the nuisance z = 0 wherever it is passed as input in the weight model or critic model. Run the distillation step with different λ .

Original: Multiple test distributions For this experiment, we use the same training data as in section 5.
Revised: Different test distributions. For this experiment, we compute the test accuracies of the models trained in the experiments in section 5 on data with different nuisance-label relationships.

Original: For both classifying Waterbirds and Pneumonia, a scalar parameter ρ controls nuisance-label relationships in the data generating process.
Revised: For both classifying Waterbirds and Pneumonia, a scalar parameter ρ controls nuisance-label relationships in the data generating process.

Original: In waterbirds, ρ = p ( y = waterbird | background = land ) = p ( y = landbird | background = water ) .
Revised: In waterbirds, ρ = p ( y = waterbird | background = land ) = p ( y = landbird | background = water ) .

Original: In chest X-rays, ρ corresponds to the fraction of Pneumonia cases  that come from CheXpert and normal cases that come from MIMIC in the data; in this task, hospital differences are one source of nuisance-induced spurious correlations.
Revised: In chest X-rays, ρ corresponds to the fraction of Pneumonia cases    7HVW VWUHQJWKRIWKHVSXULRXVFRUUHODWLRQ      $ Y J  D FF X U D F\ 1X5'YV(50RQFODVVLI\LQJ:DWHUELUGV 1X5' (5    7HVW VWUHQJWKRIWKHVSXULRXVFRUUHODWLRQ     $ Y J  D FF X U D F\ 1X5'YV(50RQ3QHXPRQLDGHWHFWLRQ 1X5' (5 quickly that come from CheXpert and normal cases that come from MIMIC in the data; in this task, hospital differences are one source of nuisance-induced spurious correlations.

Original: In both tasks, ρ = 0 .in
Revised: In both tasks, ρ = 0 . 1 in the training data; as test ρ increases, the nuisance-label relationship changes and becomes more different from the training data. We plot the average and standard error of accuracies aggregated over 10 seeds for different test ρ ∈ { 0 . 5 , 0 . 7 , 0 . 9 } in ﬁg. 4. Nuisance speciﬁcation with different borders.

Revised: For Waterbirds, we ran NuRD with the pixels outside the central 168x168 patch (a 56 pixel border) as the nuisance.