Title: A hierarchical decomposition for explaining ML performance discrepancies
Venue: NeurIPS 2024
Accepted: yes
Keywords: explainability

distribution shift

double machine learning
Abstract: Machine learning (ML) algorithms can often differ in performance across domains. Understanding why their performance differs is crucial for determining what types of interventions (e.g., algorithmic or operational) are most effective at closing the performance gaps. Aggregate decompositions express the total performance gap as the gap due to a shift in the feature distribution $p(X)$ plus the gap due to a shift in the outcome's conditional  distribution $p(Y|X)$. While this coarse explanation is helpful for guiding root cause analyses, it provides limited details and can only suggest coarse fixes involving all variables in an ML system. Detailed decompositions quantify the importance of each variable to each term in the aggregate decomposition, which can provide a deeper understanding and suggest more targeted interventions. Although parametric methods exist for conducting a full hierarchical decomposition of an algorithm's performance gap at the aggregate and detailed levels, current nonparametric methods only cover parts of the hierarchy; many also require knowledge of the entire causal graph. We introduce a nonparametric hierarchical framework for explaining why the performance of an ML algorithm differs across domains, without requiring causal knowledge. Furthermore, we derive debiased, computationally-efficient estimators and statistical inference procedures to construct confidence intervals for the explanations.

A hierarchical decomposition for explaining ML performance discrepancies

Abstract

Machine learning (ML) algorithms can often differ in performance across domains. Understanding why their performance differs is crucial for determining what types of interventions (e.g., algorithmic or operational) are most effective at closing the performance gaps. Aggregate decompositions express the total performance gap as the gap due to a shift in the feature distribution p ( X ) plus the gap due to a shift in the outcome's conditional distribution p ( Y | X ) . While this coarse explanation is helpful for guiding root cause analyses, it provides limited details and can only suggest coarse fixes involving all variables in an ML system. Detailed decompositions quantify the importance of each variable to each term in the aggregate decomposition, which can provide a deeper understanding and suggest more targeted interventions. Although parametric methods exist for conducting a full hierarchical decomposition of an algorithm's performance gap at the aggregate and detailed levels, current nonparametric methods only cover parts of the hierarchy; many also require knowledge of the entire causal graph. We introduce a nonparametric hierarchical framework for explaining why the performance of an ML algorithm differs across domains, without requiring causal knowledge. Furthermore, we derive debiased, computationally-efficient estimators and statistical inference procedures to construct confidence intervals for the explanations.

1 Introduction

The performance of an ML algorithm can differ across domains due to shifts in the data distribution. Understanding what contributed to this performance gap can help teams choose the most effective corrective action(s), ranging from algorithmic modifications (e.g. model retraining) to operational fixes (e.g. updating data pipelines). Prior works have focused primarily on aggregate decompositions, which decompose the performance gap into that due to a shift in the marginal distribution of the input features p ( X ) (covariate shift [37]) and that due to a shift in the conditional distribution of the outcome p ( Y | X ) (concept shift or conditional outcome shift) [5, 50, 30, 36, 16]. However, coarse decompositions can only suggest coarse corrective actions, such as investigating data pipelines for all features. The goal of this work is to provide a hierarchical nonparametric framework that first decomposes a performance gap into aggregate terms and then each aggregate term into detailed terms. This helps narrow down the features to investigate and understand how they affect the gap.

If one is willing to make the strong assumption that the expected loss of a model is a linear function of some feature set X , the problem of obtaining aggregate and detailed decompositions drastically simplifies. This is the key assumption underlying the Oaxaca-Blinder (OB) decomposition, one of the most widely used frameworks in the (income and health) disparities literature [32, 3]. Given an ML

AGGREGATE

DETAILED-Covariate

AGGREGATE

DETAILED

Z

0.95

Covariateshift

Covariate shift

5%

W,Z,Z distr.

Due to shift in

(±0.1%)

0.05H

M

W

Outcome shift

Due to shift in

15%

DETAILED-Outcome

Y|W,Z,Z distr.

(±0.8%)

Z

0.5

Outcomeshift

W

W

5+15=20%

Z

0.5

Accuracydrop

model with average loss E D [ ℓ ] in domains D = 0 and 1 and assuming the linear loss relationship E D [ ℓ | X ] = β ⊤ D X , the OB framework decomposes the performance gap at the aggregate level into that due to a covariate shift ( β ⊤ 0 ( E 1 [ X ] -E 0 [ X ]) ) and that due to a conditional outcome shift ( ( β 1 -β 0 ) ⊤ E 1 [ X ] ). That is, the former is due to a shift in the feature means and the latter is due to a shift in the coefficients. At the detailed level, the aggregate terms corresponding to covariate and conditional outcome shifts are further broken down into the contributions from each feature, i.e. β 0 ,j ( E 1 [ X j ] -E 0 [ X j ]) and ( β 1 ,j -β 0 ,j ) E 1 [ X j ] , respectively. Although the highly intuitive nature of the OB framework has led to its widespread popularity, the terms are difficult to interpret under model misspecification. As such, this work aims to define a similar hierarchical decomposition framework for explaining ML performance disparities, without making strong parametric assumptions .

There is currently no unified, nonparametric framework that obtains aggregate and detailed decompositions. Instead, solutions have been proposed for parts of the hierarchy (see Table 1): nonparametric methods exist for the aggregate decomposition [5, 30, 48] and, assuming the causal graph is known, detailed decompositions of the covariate shift [40, 44, 50, 38, 4, 23]. However, the causal graph is unlikely to be known in high-dimensional settings and, more importantly, there are no methods for simultaneously obtaining a detailed decomposition of the conditional outcome shift. There are also methods that do not decompose the performance gap and instead describe distribution shifts in the variables [29] or model explanations [12, 31, 23]. However, such approaches do not quantify how such shifts ultimately contribute to an ML performance gap. We make the following contributions.

We introduce a unified hierarchical nonparametric framework for decomposing the performance gap of an ML algorithm (Fig 1 left). Using the concept of partial distribution shifts, we generalize shifts with respect to variable subgroups to encompass not only covariate shifts but also conditional outcome shifts. We then introduce a unified scoring rule for (candidate) partial shifts, which can be used even when the causal graph is not known.

We derive novel debiased and asymptotically normal estimators for terms in the decomposition, which allow us to construct confidence intervals (CIs) with asymptotically valid coverage rates .

We demonstrate the utility of our framework in real-world examples of prediction models for hospital readmission and insurance coverage. Code for reproducing experiments is available at https://github.com/jjfeng/HDPD .

2 A unifying framework for explaining performance gaps

̸

Notation. Consider a prediction algorithm f : X ⊆ R m → [0 , 1] for binary outcomes Y across source and target domains, denoted by D = 0 and D = 1 , respectively. Let the performance of f be quantified in terms of a loss function ℓ : X × { 0 , 1 } → R , such as the 0-1 misclassification loss ✶ { f ( X ) = Y } . Suppose variables X can be partitioned into disjoint sets W ∈ R m 1 and Z = X \ W ∈ R m 2 , where m = m 1 + m 2 . Although our framework does not require knowing the causal ordering between variables, the interpretation is more intuitive when W is causally upstream

of Z and Y (Fig 1 right). Variables Z can be chosen to be mediators or modifiers of the effect of the domain shift D on Y . For instance, if Z are treatment variables and W are baseline variables, one can interpret a covariate shift as a change in the treatment policy and an outcome shift as a change in the treatment effect across the two environments. In absence of any causal knowledge, another option is to choose W as the variables for which one would like the expected loss given W to be invariant across the two environments; this can be useful to promote fairness of ML algorithms across environments. When this invariance does not hold, the framework explains how variables Z contribute to these differences. We refer to W as baseline variables and Z as conditional covariates. Please refer to Appendix C for more discussion on choosing W and Z as well as a summary of notation (Table 2).

Our proposed hierarchical decomposition of an ML performance gap is based on a stratification of distribution shifts into aggregate and partial shifts. At the aggregate level, the joint distribution of ( W,Z,Y ) can be factorized with respect to the aggregate variable groups W,Z, and Y , i.e.

where subscripts D W , D Z and D Y indicate the domain of that factor. An aggregate shift substitutes a factor from the source domain in (1) with that from the target domain, i.e. we swap the factor from p 0 to p 1 . A partial shift with respect to variable subset s (or an s -partial shift) shifts a factor from the source domain in (1) only with respect to variable subset s ; we denote this by swapping a factor from p 0 to p s . (We keep the precise definition of s -partial shifts purposely vague until Section 2.2.) We denote expectations with respect to the joint distribution (1) as E D W D Z D Y .

The overall performance gap between domains, Λ = E 111 [ ℓ ( W,Z,Y )] -E 000 [ ℓ ( W,Z,Y )] , can be decomposed hierarchically as follows. In the Appendix D, we also discuss how this decomposition can be interpreted causally under certain conditions.

Aggregate. At the first level of the hierarchy, the framework quantifies how aggregate shifts contribute to the performance gap individually. This leads to the decomposition Λ = Λ W +Λ Z +Λ Y , where Λ W quantifies the impact of a shift in the baseline distribution p ( W ) , Λ Z quantifies the impact of a shift in the conditional covariate distribution p ( Z | W ) , and Λ Y quantifies the impact of a shift in the outcome distribution p ( Y | W,Z ) . More concretely,

The same (or similar) aggregate decompositions have also appeared in prior works [5, 30, 15, 50, 38].

Detailed. At the detailed level, each aggregate term is further broken down into variable-level attributions. The effect of each variable can be isolated using partial shifts. However, because variables can interact to induce complex partial distribution shifts, we define variable importance (VI) using the Shapley attribution framework [41, 6, 31, 17], which has the benefits of satisfying axiomatic properties such as fairness, monotonicity, and full attribution. Thus, given a real-valued value function v that quantifies the contribution of an s -partial shift to an aggregate shift for all

s ⊆ { 1 , · · · , m } , the attribution to variable j is the average gain in value when additionally shifting with respect to j , i.e.

Interpretation. Such VI values can help ML teams identify the underlying cause(s) for a performance gap and design targeted operational and/or algorithmic interventions. For instance, a variable with high importance to the conditional covariate shift term Λ Z may indicate differences in the variable's missingness rates, prevalence, or selection bias across domains. If instead the variable is highly important to the conditional outcome shift term Λ Y , it may indicate inherent differences in the conditional distribution (i.e. effect modification), differences in measurement error or the way outcome is defined between domains, or omission of variables predictive of the outcome. Finally, note that variable importances should be viewed as relative to the variables included in the framework rather than absolute importances, as one cannot include all possible explanatory variables.

To define VI values, the key question is how to define a value function v that is applicable to different types of s -partial shifts, even when the causal graph is not known. It turns out that the answer is far from straightforward. The next section discusses how the value function and candidate s -partial shifts must be defined with care.

2.1 Value of partial distribution shifts

When the true causal graph is known, prior works define an s -partial covariate shift as the substitution of nodes s with mechanisms from the target domain and its value v ( s ) as the difference in the average loss, e.g. E 1 s 0 [ ℓ ] -E 100 [ ℓ ] [50, 38]. However, this has a number of limitations: (i) knowing the entire causal graph is often impractical, (ii) in the absence of such a graph, this value function is not a proper scoring rule and can assign high values to partial shifts that contradict the true causal graph (see Example E.1 for details), and (iii) v ( s ) can be high even if the shift does not induce similar shifts in the loss as the aggregate shift.

Instead, we propose to evaluate candidate s -partial shifts by how closely they approximate aggregate shifts, using a nonparametric extension of the traditional R 2 measure. In the case of conditional covariate shifts, an aggregate shift induces a performance difference of ∆ · 10 ( W ) in strata W while a candidate s -partial shift induces a performance difference of ∆ · s 0 ( W ) = E · s 0 [ ℓ | W ] -E · 00 [ ℓ | W ] . The value of this s -partial shift is then the percent variation of ∆ · 10 explained by ∆ · s 0 , i.e.

Likewise, for conditional outcome shifts, an aggregate shift induces a performance difference of ∆ ·· 1 ( W,Z ) in strata ( W,Z ) while a candidate s -partial shift induces a performance difference of ∆ ·· s ( W,Z ) = E ·· s [ ℓ | W,Z ] -E ·· 0 [ ℓ | W,Z ] . The value of this s -partial conditional outcome shift is then defined as the percent variation of ∆ ·· 1 explained by ∆ ·· s , i.e.

This formulation of the value function in terms of R 2 provides a unified way to score partial conditional covariate and outcome shifts, does not require knowledge of the true causal graph, and is a strictly proper scoring rule under certain conditions (see Appendix E). In general, we expect the highest scoring candidate s -partial shifts to be those that are close to the true causal graph and induce large shifts in the ML algorithm's loss. Finally, we acknowledge one caveat with this framework: because some variables must be held out to define the R 2 measure, we cannot score partial shifts in the baseline variables W . We hope to close this gap in future work.

2.2 Candidate partial distribution shifts

We now present the set of candidate partial shifts considered in this work. High-level illustrations for the candidate partial shifts are given in Fig 1 right top; more detailed illustrations are given in

Fig 4 of the Appendix. We emphasize that these are candidates , as the true causal graph is not known. While there are certainly other partial shifts that one may consider, many have various disadvantages. As such, we leave the investigation of other partial shifts to future work.

s -partial conditional covariate shift : Suppose Z -s is downstream of Z s . Then p s ( z | w ) := p 1 ( z s | w ) p 0 ( z -s | z s , w ) . Wu et al. [48] considered a similar proposal.

s -partial conditional outcome shift : Shifting the conditional distribution of Y only with respect to a variable subset Z s but not Z -s requires care. We cannot simply define p s ( Y | W,Z ) as a function of only W and Z s . Such a definition would imply that an s -partial shift has a non-zero effect, even in settings with no shift in the conditional outcome distribution (i.e. p 1 ( Y | W,Z ) ≡ p 0 ( Y | W,Z ) ).

Instead, we define an s -partial outcome shift based on models commonly used in model recalibration/revision [42, 34], where the modified risk (conditional probability of Y ) is a function of the risk in the source domain Q := q ( W,Z ) := p 0 ( Y = 1 | W,Z ) , W , and Z s . That is, we define the shift as

By defining the shifted outcome distribution solely as a function of Q,W , and Z s , any direct effect from Z -s to Y is eliminated and p s has the desired behavior in the setting where there is no conditional outcome shift.

3 Estimation and statistical inference

Here we discuss estimation and statistical inference for the aggregate terms ( Λ W , Λ Z , and Λ Y ), the value functions v Z ( s ) and v Y ( s ) , and the Shapley-based detailed terms ϕ Z ,j and ϕ Y ,j for j ∈ (0 , · · · , m 2 ) . One approach is to rely on plug-in estimators, which plug in estimates of conditional means (also called outcome models) or density ratios [43], which we collectively refer to as nuisance parameters. For instance, one can estimate the conditional means µ · 10 ( w ) = E · 10 [ ℓ | W ] and µ · 00 ( w ) = E · 00 [ ℓ | W ] using ML and take the empirical mean of ˆ µ · 10 -ˆ µ · 00 with respect to the target domain to get a plug-in estimator for Λ Z = E 1 ·· [ µ · 10 -µ · 00 ] . However, because estimation of the true nuisance parameters using ML typically converge at a rate slower than n -1 / 2 , plug-in estimators generally fail to be consistent at a rate of n -1 / 2 and cannot be used to construct CIs [25].

To this end, we use the method of one-step correction from semiparametric inference to derive debiased ML estimators [45, 7]. The core idea is to subtract the first-order bias of a plug-in estimator, which requires characterizing the canonical gradient (or efficient influence function) of the estimand [25]. The primary technical contribution in this section is the derivation of debiased estimators for the detailed decompositions. (Estimation and inference for the aggregate decomposition is well-studied, as the aggregate terms can be formulated as average causal effects.) Due to space limitations, this section only presents estimators for the detailed decomposition of the conditional outcome shift. This estimand is particularly interesting, as its unique structure is not amenable to standard techniques for debiasing ML estimators. We refer the reader to the Appendix for derivations, pseudocode, and proofs for all the estimators.

Notation. Let P D denote the expectation with respect to domain D . For ease of exposition, suppose the number of IID observations from each domain is the same, denoted by n . We present split-sample estimators, though the results can be readily extended using cross-fitting [7, 25]. Let the data be randomly split into 'training' and 'evaluation' partitions. Let P D,n denote the empirical average in the evaluation partition for domain D . All estimated quantities are denoted using hat notation.

3.1 Value of s -partial conditional outcome shifts

Here we describe the high-level steps for deriving a debiased estimator for v Y ( s ) , the value of a candidate s -partial conditional outcome shift. The following section describes a computationally efficient procedure for combining such estimates to obtain Shapley values.

Standard recipes for deriving asymptotically normal, nonparametric-efficient estimators rely on pathwise differentiability of the estimand and analyzing its efficient influence function [25]. However, v Y ( s ) is not pathwise differentiable because it is a function of (5), which conditions on the source risk q ( w,z ) equalling some value r . Taking the pathwise derivative of v Y ( s ) requires taking a derivative

of the indicator function ✶ { q ( w,z ) = r } , which generally does not exist. Given the difficulties in deriving an asymptotically normal estimator for v Y ( s ) , we propose estimating a close alternative that is pathwise differentiable.

Condition 3.1. Let Ξ be the set of ( W,Z ) such that q ( W,Z ) falls precisely on some bin edge and is not equal to zero or one. The set Ξ is measure zero.

The idea is to replace q in (5) with its binned variant q bin ( w,z ) = 1 B ⌊ q ( w,z ) B + 1 2 ⌋ for some B ∈ Z + , which discretizes outputs from q into B disjoint bins. As long as B is sufficiently high, the binned version of the estimand, denoted v Y , bin ( s ) , is a close approximation to v Y ( s ) . (We use B = 20 in the empirical analyses, which we believe to be sufficient in practice.) The benefit of this binned variant is that the derivative of the indicator function ✶ { q bin ( w,z )= r } is zero almost everywhere as long as observations with source risks exactly equal to a bin edge have measure zero. More formally, we require the following:

Under this condition, v Y , bin ( s ) is pathwise differentiable and, using one-step correction, we derive a debiased ML estimator that has the unique form of a V-statistic (this follows from the integration over 'phantom' ˜ z -s in (5)). We represent V-statistics using the operator P 1 ,n ˜ P 1 ,n , which takes the average over all pairs of observations O i with replacement, i.e. 1 n 2 ∑ n i =1 ∑ n j =1 g ( O i , O j ) for some function g . Calculation of this estimator and its theoretical properties are as follows.

Estimation. Using the training partition, estimate the outcome models µ ·· D ( W,Z ) = E ·· D [ ℓ | W,Z ] for D = 0 , 1 , the shifted outcome model µ ·· s ( W,Z ) = E ·· s [ ℓ | W,Z ] ; and the density ratio models π 110 ( W,Z ) = p 1 ( W,Z ) /p 0 ( W,Z ) and π ( W,Z s , Z -s , Q bin ) = p 1 ( Z -s | W,Z s , q bin ( W,Z ) = Q bin ) /p 1 ( Z -s ) . The outcome and density ratio models can be fit using ML-based regression models and probabilistic classifiers [43], respectively (see Section H for details). The estimator for v Y , bin ( s ) is the ratio ˆ v Y , bin ( s ) = ˆ v num Y ,n ( s ) / ˆ v den Y ,n , where the numerator and denominator are estimated using the evaluation partition as

where ˆ ξ s ( W,Z ) = ˆ µ ·· 1 ( W,Z ) -ˆ µ ·· s ( W,Z ) . Note that the first terms in (6) and (7) are the plug-in estimates, followed by additional terms that correct its bias.

Inference. This estimator is asymptotically normal assuming the estimators for the nuisance parameters converge at a fast enough rate, per the following theorem.

Theorem 3.2. Suppose Condition 3.1 holds. For variable subset s , suppose the density ratios π ( W,Z s , Z -s , Q bin ) and π 110 ( W,Z ) are bounded; denominator in case of no shift v den Y ( ∅ ) > 0 ; estimator ˆ π is consistent; estimators ˆ µ ·· 0 , ˆ µ ·· 1 and ˆ µ ·· s converge at an o p ( n -1 / 4 ) rate, and

Then the estimator ˆ v Y , bin ( s ) is asymptotically normal centered at the estimand v Y , bin ( s ) .

Note that the product terms in (9) mean that the estimator converges to normal at n -1 / 2 -rate even if one of the nuisance parameters is estimated at a rate slower than n -1 / 2 . Hence, it is multiply-robust to nuisance model misspecification. A convergence rate of o p ( n -1 / 4 ) can be achieved by ML estimators in a wide variety of conditions, and such assumptions are commonly used to construct debiased ML estimators. The additional requirement in (8) that ˆ q bin converges at a o p ( n -1 ) rate is new, but fast or even super-fast convergence rates of binned risks is achievable under suitable margin conditions [2] such as Condition G.7 in the Appendix.

3.2 Shapley values

Calculating the exact Shapley value is computationally intractable as it involves an exponential number of terms. However, Williamson and Feng [47] showed that calculating the exact Shapley

0.0

0.5

1.0

Coverage

Subset = {Z1}

0.0

0.5

1.0

Coverage

Subset = {Z2}

2000

4000

8000

16000

n

0.0

0.5

1.0

Coverage

Subset = {Z3}

Plug-in

Debiased ML

0.0

0.5

1.0

Coverage

Subset = {Z1}

0.0

0.5

1.0

Coverage

Subset = {Z2}

2000

4000

8000

16000

n

0.0

0.5

1.0

Coverage

Subset = {Z3}

(a)

(i) Conditional covariate

(ii) Conditional outcome

(b)

HDPD

Z1

WuShift

Variable

MeanChange

Oaxaca-Blinder

0.0

0.2

0.4

0.6

0.8

1.0

Attribution

Z1

HDPD

ParametricChange

Variable

ParamietricAcc

RandomForestAcc

Z3

Oaxaca-Blinder

Z4

F

0.0

0.2

0.4

0.6

0.8

1.0

Attribution

value is unnecessary for the purposes of statistical inference. Because there is inherent uncertainty in estimates of the value functions v ( s ) , one only needs to sample and estimate the values for enough variable subsets such that the uncertainty due to estimation dominates that due to subset sampling. This leads to a drastic reduction in computation time: Williamson and Feng [47] proves that the number of subsets one needs to sample only needs to be linear or super-linear in the total number of observations n . Using this result, Algorithm 4 outlines a computationally efficient procedure for estimation and inference of the detailed decomposition.

4 Simulation

We now present simulations to show that the proposed procedure achieves the desired coverage rates (Section 4.1) and illustrate how the HDPD framework provides more intuitive explanations of performance gaps (Section 4.2). In all empirical analyses, performance of the ML algorithm is quantified in terms of 0-1 accuracy. Below, we briefly describe the simulation settings; full details are provided in Section I in the Appendix.

4.1 Verifying theoretical properties

We first verify that the inference procedures for the decomposition terms have CIs with coverage close to their nominal rate. We check the coverage of the aggregate decomposition as well as the value of s -partial conditional covariate and partial conditional outcome shifts for s = { Z 1 } , { Z 2 } , { Z 3 } . ( W,Z 1 , Z 2 , Z 3 ) are sampled from independent normal distributions with different means in source and target, while Y is simulated from logistic regression models with different coefficients. CIs for the debiased ML estimator converge to the nominal 90% coverage rate with increasing sample size, whereas those for the naïve plug-in estimator do not (Fig 2a and Fig 6).

4.2 Comparing explanations

We now compare the proposed definitions for the detailed decomposition with existing methods. For the detailed decomposition due to conditional covariate shift, the comparators are:

MeanChange Tests for a difference in means for each feature. Defines importance as 1 -p-value.

Oaxaca-Blinder : Fits a linear model of the logit-transformed expected loss with respect to Z in the source domain. Defines importance of Z i as its coefficient multiplied by the difference in the means of Z i [32, 3].

WuShift [48]: Defines importance of subset s as change in overall performance due to s -partial conditional covariate shifts. Applies Shapley framework to obtain VIs.

(a) Readmission risk (General → Heart Failure)

(b) Insurance coverage (NE

→

LA)

Aggregate

Aggregate

Aggregate Variable

Aggregate Variable

W: Demographics

W: Demographics

HDPD debiased

HDPD debiased

HDPD plug-in

Z: Diagnosis codes

HDPD plug-in

X: Health, work, education and other

Y: Readmission *

Y: Public health insurance?

0.15

0.10

0.05

0.00

0.05

0.14-0.12-0.10-0.08-0.060.04-0.02

0.00

0.02

Decomposition Value

Decomposition Value

Detailed:Conditionalcovariate

Detailed:Conditionaloutcome

Drug induced/toxic related condition

xxxxxxxxxx

Total person's income

Mental & substance use disorders in remission

Married -

XF

Variable

Diabetes mellitus with complication

Respiratory signs & symptoms

Fluid & electrolyte disorders

+

HDPD debiased

Variable

Never married or under 15 years

Citizenship status not a citizen

Employment status employed -

Diabetes mellitus without complication

HDPD plug-in

Citizenship status born in US

HDPD debiased

Abdominal pain, digestive signs & symptoms

Educationalattainment

Conditions due to neoplasm or its treatment

Musculoskeletal pain, not low back pain

x

+

WuShift

MeanChange

Employment status partial armed-

Employment status unemployed

Employment status armed -

X

HDPD plug-in

RandomForestAcc

Exposure to infectious disease

Ancestry

0.0

0.2

0.4

0.6

0.8

1.0

0.00

0.05

0.10

0.15

0.20

0.25

Attribution Value

Attribution Value

For the detailed decomposition due to conditional outcome shifts, we compare against:

ParametricChange : Fits a logistic model for Y with interaction terms between domain and Z . Defines importance of Z i as the coefficient of its interaction term.

ParametricAcc : Same as ParametricChange but models the 0-1 loss rather than Y .

RandomForestAcc : Compares VI of random forest models trained on data from both domains with input features D , Z , and W to predict the 0-1 loss.

Oaxaca-Blinder : Fits linear models for the logit-transformed expected loss in each domain. Defines importance of Z i as its mean in the target domain multiplied by the difference in its coefficients across domains.

Although the proposed method may agree with these other methods on the top features in certain data settings, we highlight important situations where the methods differ.

Conditional covariate. (Fig 2b(i)) We simulate ( W,Z 1 ) from a standard normal distribution, Z 2 from a mixture of two Gaussians whose means depend on the value of Z 1 (i.e. Z 1 → Z 2 ), and Y from a logistic regression model depending on ( W,Z 1 , Z 2 ) . We induce a shift from the source domain to the target domain by shifting only the distribution of Z 1 , so that p 1 ( Z | W ) = p 0 ( Z 2 | Z 1 , W ) p 1 ( Z 1 | W ) . Only the proposed estimator correctly recovers that Z 1 is more important than Z 2 , as the { 1 } -partial conditional covariate shift explains all the variation in performance gaps across strata W (i.e. the corresponding R 2 -based value function v Z ( { 1 } ) is equal to 1). The other methods incorrectly assign higher importance to Z 2 . MeanChange only measures shifts but not loss due to shifts, Oaxaca-Blinder uses a misspecified linear model, and WuShift estimates the performance change due to hypothesized s -partial shifts but does not check if the partial shifts are good explanations in the first place.

Conditional outcome. (Fig 2b(ii)) W and Z ∈ R 4 are simulated from the same distribution in both domains. Y is generated from a logistic regression model with coefficients for ( W,Z 1 , · · · , Z 4 ) as (0 . 5 , 0 . 5 , 1 , 0 . 3 , 0 . 3) in the source and (0 . 5 , 0 . 3 , 1 , 1 . 3 , -0 . 1) in the target. Interestingly, none of the methods have the same ranking of the features. ParametricChange identifies Z 1 as having the largest shift on the logit scale, but this does not mean that it is the most important explanation for changes in the loss . According to our decomposition framework, Z 3 is actually the most important for explaining changes in model performance due to outcome shifts. Oaxaca-Blinder , ParametricAcc , and RandomForestAcc have odd behavior. Oaxaca-Blinder assigns Z 3 second to the lowest importance and ParametricAcc assigns Z 2 the highest importance), likely because they misspecify the outcome models. RandomForestAcc likely ranks Z 2 highly because its VI values quantify which variables are good predictors of performance, not performance shift.

Amore objective evaluation is to compare the performance of fixes based on the different explanations. To this end, we re-fit the ML algorithm in the target domain with respect to input features Q,W , and the top variables Z s from each explanation. We find that model revisions based on the proposed method achieve the highest performance gain (Table 3 in Appendix).

5 Real-world data case studies

We now demonstrate applicability of the framework on two datasets with naturally-occurring shifts.

Hospital readmission. Using electronic health record data from the Zuckerberg San Francisco General Hospital, we analyzed performance of a Gradient Boosted Tree (GBT) trained on the general patient population (source) to predict 30-day readmission risk but applied to patients diagnosed with heart failure (HF, target). Features include 4 demographic variables ( W ) and 16 diagnosis codes ( Z ). Each domain supplied n = 3750 observations from which we keep 20% in the evaluation partition.

Model accuracy drops from 70% to 53% in HF population. From the aggregate decompositions (Fig 3a), we observe that the drop is mainly due to covariate shift. If one performed the standard check to see which variables significantly changed in their mean value ( MeanChange ), then one would find a significant shift in nearly every variable. Little support is offered to identify main drivers of the performance drop. In contrast, the detailed decomposition from the proposed framework estimates diagnoses 'Drug-induced or toxic-related condition' and 'Mental & substance use disorder in remission' as having the highest estimated contributions to the conditional covariate shift, and most other variables having little to no contribution. Upon discussion with clinicians from this hospital, differences in the top two diagnoses may be explained by (i) substance use being a major cause of HF at this hospital, with over eighty percent of its HF patients reporting current or prior substance use, and (ii) substance use and mental health disorders often occurring simultaneously in this HF patient population. Based on these findings, closing the performance gap may require a mixture of both operational (e.g. care programs centered around substance use) and algorithmic interventions (e.g. reweighting data with respect to the top two features). Finally, CIs from the debiased ML procedure provide valuable information on the uncertainty of the estimates and highlight, for instance, that more data is necessary to determine the true ordering between the top two features. In contrast, existing methods do not provide (asymptotically valid) CIs.

ACS Public Coverage. We analyze a neural network trained to predict whether a person has public health insurance using data from Nebraska in the American Community Survey (source, n = 3000 ), applied to data from Louisiana (target, n = 6000 ). Baseline variables include 3 demographics (sex, age, race), and covariates Z include 31 variables related to health conditions, employment, marital status, citizenship status, and education.

Model accuracy drops from 84% to 66% across the two states. The main driver is the shift in the outcome distribution per the aggregate decomposition (Fig 3b) and the most important contributor to the outcome shift is annual income, perhaps due to differences in cost of living across the two states. Income is significantly more important than all the other variables; the ranking between the remaining variables is unclear. In comparing the performance of targeted model revisions, we find that revising the model based on top variables identified by the proposed procedure leads to AUCs that are better or as good as those based on RandomForestAcc (Table 4 in the Appendix).

6 Prior work

Describing distribution shifts. This line of work focuses on detecting and localizing which distributions shift between datasets [29, 39]. Budhathoki et al. [4] identify the main variables contributing to a distribution shift via a Shapley framework, Kulinski and Inouye [28] fits interpretable optimal transport maps, and Liu et al. [30] finds the region with the largest shift in the conditional outcome distribution. However, these works do not quantify how these shifts contribute to changes in performance , the metric of practical importance.

Explaining loss differences across subpopulations. Understanding differences in model performance across subpopulations in a single dataset is similar to understanding differences in model performance across datasets, but the focus is typically to find subpopulations with poor performance rather than to explain how distribution shifts contributed to the performance change. Existing approaches include slice discovery methods [35, 22, 10, 13] and structured representations of the subpopulation using e.g. Euclidean balls [1].

Attributing performance changes. Prior works have described similar aggregate decompositions of the performance change into covariate and conditional outcome shift components [5, 36]. To provide more granular explanations of performance shifts, existing works on causal attribution [50, 38] and

mediation analysis [44] quantify the importance of shifts in each variable assuming the causal graph is correctly specified; covariate shifts restricted to variable subsets assuming that the partial shifts follow a particular structure [48]; and conditional shifts in each variable assuming a parametric model [11]. However, the strong assumptions made by these methods make them difficult to apply in practice, and model misspecification can lead to unintuitive interpretations. Furthermore, such methods do not provide hierarchical decompositions, i.e. VIs for each type of shift. Decomposition methods such as Oaxaca-Blinder similarly make strong parametric assumptions [32, 3, 16, 14, 49, 15], which is inappropriate for the complex data settings in ML. In addition, there is no unifying nonparametric framework for decomposing both covariate and outcome shifts, and many methods do not output CIs, which is important when the amount of labeled data from a given domain is limited. A summary of how the proposed framework compares against prior works is shown in Table 1.

7 Discussion

MLalgorithms regularly encounter distribution shifts in practice, leading to drops in performance. We present a novel framework that helps ML developers and deployment teams build a more nuanced understanding of the shifts. Compared to past work, the approach provides a nonparametric hierarchical framework for decomposing both conditional covariate and outcome shifts, does not require fine-grained knowledge of the causal relationship between variables, and quantifies the uncertainty of the estimates by constructing confidence intervals. We present real-world case studies to demonstrate how this framework can help diagnose performance drops and guide corrective actions. This framework requires overlapping support of the covariates, which may not always be applicable in practice. In such cases, one solution is to restrict to the common support [5].

Important extensions of this work include decompositions of more complex measures of model performance such as AUC and analyzing other factorizations of the data distribution (e.g. label/prior shifts [27]). For unstructured data (e.g. image and text), the current framework can be applied to low-dimensional embeddings or by extracting interpretable concepts [26]; more work is needed to directly analyze unstructured data. Finally, while the focus of this work is to interpret performance gaps, future work may take this work one step further to design optimal interventions for closing the performance gap.

Acknowledgments and Disclosure of Funding

We would like to thank Lucas Zier for supplying the dataset from the Zuckerberg San Francisco General Hospital and providing their clinical expertise to interpret results. We are grateful to Nicholas Petrick, Berkman Sahiner, Gene Pennello, Mi-Ok Kim, Romain Pirracchio, Julian Hong, Avni Kothari, and the anonymous reviewers for helpful feedback. This work was funded through a Patient-Centered Outcomes Research Institute® (PCORI®) Award (ME-2022C1-25619). The views presented in this work are solely the responsibility of the author(s) and do not necessarily represent the views of the PCORI®, its Board of Governors or Methodology Committee. The contents are those of the author(s) and do not necessarily represent the official views of, nor an endorsement, by FDA/HHS, or the U.S. Government.

References

Alnur Ali, Maxime Cauchois, and John C. Duchi. The lifecycle of a statistical model: Model failure detection, identification, and refitting, 2022. URL https://arxiv.org/abs/2202. 04166 .

Jean-Yves Audibert and Alexandre B Tsybakov. Fast learning rates for plug-in classifiers. Ann. Stat. , 35(2):608-633, April 2007.

Alan S. Blinder. Wage discrimination: Reduced form and structural estimates. The Journal of Human Resources , 8(4):436-455, 1973. ISSN 0022166X. URL http://www.jstor.org/ stable/144855 .

Kailash Budhathoki, Dominik Janzing, Patrick Bloebaum, and Hoiyi Ng. Why did the distribution change? In Arindam Banerjee and Kenji Fukumizu, editors, Proceedings of The

24th International Conference on Artificial Intelligence and Statistics , volume 130 of Proceedings of Machine Learning Research , pages 1666-1674. PMLR, 13-15 Apr 2021. URL https://proceedings.mlr.press/v130/budhathoki21a.html .

Tiffany Tianhui Cai, Hongseok Namkoong, and Steve Yadlowsky. Diagnosing model performance under distribution shift. March 2023. URL http://arxiv.org/abs/2303.02011 .

A Charnes, B Golany, M Keane, and J Rousseau. Extremal principle solutions of games in characteristic function form: Core, chebychev and shapley value generalizations. In Jati K Sengupta and Gopal K Kadekodi, editors, Econometrics of Planning and Efficiency , pages 123-133. Springer Netherlands, Dordrecht, 1988.

Victor Chernozhukov, Denis Chetverikov, Mert Demirer, Esther Duflo, Christian Hansen, Whitney Newey, and James Robins. Double/debiased machine learning for treatment and structural parameters. Econom. J. , 21(1):C1-C68, February 2018.

Juan Correa and Elias Bareinboim. A calculus for stochastic Interventions:Causal effect identification and surrogate experiments. AAAI , 34(06):10093-10100, April 2020. URL https://ojs.aaai.org/index.php/AAAI/article/view/6567 .

A P Dawid. Influence diagrams for causal modelling and inference. Int. Stat. Rev. , 70 (2):161-189, August 2002. URL https://onlinelibrary.wiley.com/doi/10.1111/ j.1751-5823.2002.tb00354.x .

Greg d'Eon, Jason d'Eon, James R. Wright, and Kevin Leyton-Brown. The spotlight: A general method for discovering systematic errors in deep learning models. In 2022 ACM Conference on Fairness, Accountability, and Transparency , FAccT '22, page 1962-1981, New York, NY, USA, 2022. Association for Computing Machinery. ISBN 9781450393522. doi: 10.1145/3531146.3533240. URL https://doi.org/10.1145/3531146.3533240 .

Lori E Dodd and Margaret Sullivan Pepe. Semiparametric regression for the area under the receiver operating characteristic curve. J. Am. Stat. Assoc. , 98(462):409-417, 2003.

Christopher Duckworth, Francis P Chmiel, Dan K Burns, Zlatko D Zlatev, Neil M White, Thomas W V Daniels, Michael Kiuber, and Michael J Boniface. Using explainable machine learning to characterise data drift and detect emergent health risks for emergency department admissions during COVID-19. Sci. Rep. , 11(1):23017, November 2021.

Sabri Eyuboglu, Maya Varma, Khaled Kamal Saab, Jean-Benoit Delbrouck, Christopher LeeMesser, Jared Dunnmon, James Zou, and Christopher Re. Domino: Discovering systematic errors with cross-modal embeddings. In International Conference on Learning Representations , 2022. URL https://openreview.net/forum?id=FPCMqjI0jXN .

Robert W Fairlie. An extension of the blinder-oaxaca decomposition technique to logit and probit models. Journal of economic and social measurement , 30(4):305-316, 2005.

Sergio P. Firpo, Nicole M. Fortin, and Thomas Lemieux. Decomposing wage distributions using recentered influence function regressions. Econometrics , 6(2), 2018. ISSN 2225-1146. doi: 10.3390/econometrics6020028. URL https://www.mdpi.com/2225-1146/6/2/28 .

Nicole Fortin, Thomas Lemieux, and Sergio Firpo. Chapter 1 - decomposition methods in economics. volume 4 of Handbook of Labor Economics , pages 1-102. Elsevier, 2011. doi: https://doi.org/10.1016/S0169-7218(11)00407-2. URL https://www.sciencedirect.com/ science/article/pii/S0169721811004072 .

Amirata Ghorbani and James Zou. Data shapley: Equitable valuation of data for machine learning. In Kamalika Chaudhuri and Ruslan Salakhutdinov, editors, Proceedings of the 36th International Conference on Machine Learning , volume 97 of Proceedings of Machine Learning Research , pages 2242-2251. PMLR, 09-15 Jun 2019. URL https://proceedings.mlr. press/v97/ghorbani19c.html .

Tilmann Gneiting and Adrian E Raftery. Strictly proper scoring rules, prediction, and estimation. Journal of the American Statistical Association , 102(477):359-378, 2007. doi: 10.1198/ 016214506000001437. URL https://doi.org/10.1198/016214506000001437 .

Chuan Guo, Geoff Pleiss, Yu Sun, and Kilian Q Weinberger. On calibration of modern neural networks. International Conference on Machine Learning , 70:1321-1330, 2017.

Oliver Hines, Karla Diaz-Ordaz, and Stijn Vansteelandt. Variable importance measures for heterogeneous causal effects. April 2022.

John W Jackson. Meaningful causal decompositions in health equity research: definition, identification, and estimation through a weighting framework. Epidemiology , 32(2):282-290, 2021.

Saachi Jain, Hannah Lawrence, Ankur Moitra, and Aleksander Madry. Distilling model failures as directions in latent space. In The Eleventh International Conference on Learning Representations , 2023. URL https://openreview.net/forum?id=99RpBVpLiX .

Yonghan Jung, Shiva Kasiviswanathan, Jin Tian, Dominik Janzing, Patrick Bloebaum, and Elias Bareinboim. On measuring causal contributions via do-interventions. In Kamalika Chaudhuri, Stefanie Jegelka, Le Song, Csaba Szepesvari, Gang Niu, and Sivan Sabato, editors, Proceedings of the 39th International Conference on Machine Learning , volume 162 of Proceedings of Machine Learning Research , pages 10476-10501. PMLR, 17-23 Jul 2022. URL https: //proceedings.mlr.press/v162/jung22a.html .

Joseph D. Y. Kang and Joseph L. Schafer. Demystifying Double Robustness: A Comparison of Alternative Strategies for Estimating a Population Mean from Incomplete Data. Statistical Science , 22(4):523 - 539, 2007. doi: 10.1214/07-STS227. URL https://doi.org/10.1214/ 07-STS227 .

Edward H Kennedy. Semiparametric doubly robust targeted double machine learning: a review. March 2022.

Been Kim, Martin Wattenberg, Justin Gilmer, Carrie Cai, James Wexler, Fernanda Viegas, and Rory sayres. Interpretability beyond feature attribution: Quantitative testing with concept activation vectors (TCAV). In Jennifer Dy and Andreas Krause, editors, Proceedings of the 35th International Conference on Machine Learning , volume 80 of Proceedings of Machine Learning Research , pages 2668-2677. PMLR, 10-15 Jul 2018. URL https://proceedings. mlr.press/v80/kim18d.html .

Wouter M. Kouw and Marco Loog. An introduction to domain adaptation and transfer learning, 2019.

Sean Kulinski and David I. Inouye. Towards explaining distribution shifts. In Andreas Krause, Emma Brunskill, Kyunghyun Cho, Barbara Engelhardt, Sivan Sabato, and Jonathan Scarlett, editors, Proceedings of the 40th International Conference on Machine Learning , volume 202 of Proceedings of Machine Learning Research , pages 17931-17952. PMLR, 23-29 Jul 2023. URL https://proceedings.mlr.press/v202/kulinski23a.html .

Sean Kulinski, Saurabh Bagchi, and David I Inouye. Feature shift detection: Localizing which features have shifted via conditional distribution tests. In H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin, editors, Advances in Neural Information Processing Systems , volume 33, pages 19523-19533. Curran Associates, Inc., 2020. URL https://proceedings. neurips.cc/paper/2020/file/e2d52448d36918c575fa79d88647ba66-Paper.pdf .

Jiashuo Liu, Tianyu Wang, Peng Cui, and Hongseok Namkoong. On the need for a language describing distribution shifts: Illustrations on tabular datasets. July 2023.

Scott M Lundberg and Su-In Lee. A unified approach to interpreting model predictions. In I. Guyon, U. Von Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett, editors, Advances in Neural Information Processing Systems , volume 30. Curran Associates, Inc., 2017. URL https://proceedings.neurips.cc/paper_files/paper/2017/file/ 8a20a8621978632d76c43dfd28b67767-Paper.pdf .

Ronald Oaxaca. Male-female wage differentials in urban labor markets. International Economic Review , 14(3):693-709, 1973. ISSN 00206598, 14682354. URL http://www.jstor.org/ stable/2525981 .

F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel, P. Prettenhofer, R. Weiss, V. Dubourg, J. Vanderplas, A. Passos, D. Cournapeau, M. Brucher, M. Perrot, and E. Duchesnay. Scikit-learn: Machine learning in Python. Journal of Machine Learning Research , 12:2825-2830, 2011.

John Platt. Probabilistic outputs for support vector machines and comparisons to regularized likelihood methods. Advances in large margin classifiers , 10(3):61-74, 1999.

Gregory Plumb, Nari Johnson, Angel Cabrera, and Ameet Talwalkar. Towards a more rigorous science of blindspot discovery in image classification models. Transactions on Machine Learning Research , 2023. ISSN 2835-8856. URL https://openreview.net/forum?id= MaDvbLaBiF . Expert Certification.

Hongxiang Qiu, Eric Tchetgen Tchetgen, and Edgar Dobriban. Efficient and multiply robust risk estimation under general forms of dataset shift. June 2023.

Joaquin Quiñonero-Candela, Masashi Sugiyama, Anton Schwaighofer, and Neil D Lawrence. Dataset shift in machine learning . Mit Press, 2022.

Victor Quintas-Martinez, Mohammad Taha Bahadori, Eduardo Santiago, Jeff Mu, Dominik Janzing, and David Heckerman. Multiply-robust causal change attribution, 2024.

Stephan Rabanser, Stephan Günnemann, and Zachary Lipton. Failing loudly: An empirical study of methods for detecting dataset shift. In H. Wallach, H. Larochelle, A. Beygelzimer, F. d'Alché-Buc, E. Fox, and R. Garnett, editors, Advances in Neural Information Processing Systems , volume 32. Curran Associates, Inc., 2019. URL https://proceedings.neurips. cc/paper/2019/file/846c260d715e5b854ffad5f70a516c88-Paper.pdf .

James M Robins, Thomas S Richardson, and Ilya Shpitser. An interventionist approach to mediation analysis. In Probabilistic and Causal Inference: The Works of Judea Pearl . Association for Computing Machinery, August 2020. URL http://arxiv.org/abs/2008. 06019 .

L S Shapley. 17. a value for n-person games. In Harold William Kuhn and Albert William Tucker, editors, Contributions to the Theory of Games (AM-28), Volume II , pages 307-318. Princeton University Press, Princeton, December 1953.

Ewout W Steyerberg. Clinical Prediction Models: A Practical Approach to Development, Validation, and Updating . Springer, New York, NY, 2009.

Masash Sugiyama, Shinichi Nakajima, Hisashi Kashima, Paul Buenau, and Motoaki Kawanabe. Direct importance estimation with model selection and its application to covariate shift adaptation. Adv. Neural Inf. Process. Syst. , 2007.

Eric J. Tchetgen Tchetgen and Ilya Shpitser. Semiparametric theory for causal mediation analysis: Efficiency bounds, multiple robustness and sensitivity analysis. The Annals of Statistics , 40(3):1816 - 1845, 2012. doi: 10.1214/12-AOS990. URL https://doi.org/10. 1214/12-AOS990 .

Anastasios A Tsiatis. Semiparametric Theory and Missing Data . Springer New York, 2006.

A W van der Vaart. Asymptotic Statistics . Cambridge University Press, October 1998.

Brian D Williamson and Jean Feng. Efficient nonparametric statistical inference on population feature importance using shapley values. International Conference on Machine Learning , 2020. URL https://proceedings.icml.cc/static/paper_files/icml/2020/3042-Paper. pdf .

Eric Wu, Kevin Wu, and James Zou. Explaining medical AI performance disparities across sites with confounder shapley value analysis. November 2021. URL http://arxiv.org/ abs/2111.08168 .

Myeong-Su Yun. Decomposing differences in the first moment. Economics Letters , 82(2): 275-280, 2004. ISSN 0165-1765. doi: https://doi.org/10.1016/j.econlet.2003.09.008. URL https://www.sciencedirect.com/science/article/pii/S0165176503002866 .

Haoran Zhang, Harvineet Singh, Marzyeh Ghassemi, and Shalmali Joshi. "Why did the model fail?": Attributing model performance changes to distribution shifts. In Andreas Krause, Emma Brunskill, Kyunghyun Cho, Barbara Engelhardt, Sivan Sabato, and Jonathan Scarlett, editors, Proceedings of the 40th International Conference on Machine Learning , volume 202 of Proceedings of Machine Learning Research , pages 41550-41578. PMLR, 23-29 Jul 2023. URL https://proceedings.mlr.press/v202/zhang23ai.html .

A Appendix

Contents of the Appendix are as follows.

Table 2 collects all the notation used for reference.

Section B discusses broader impacts of the work.

Algorithms 1 and 4 provide the steps required for computing the aggregate and detailed decomposition respectively. Detailed decompositions require computing the value of s -partial conditional outcome and conditional covariate shifts which is described in Algorithms 2 and 3.

Section C discusses considerations for choosing baseline variables W and conditional covariates Z .

Section D describes a causal interpretation of the aggregate and detailed decompositions as effects of stochastic (or shift) interventions on a structural causal model.

Section E explain why value functions in prior work give unintuitive attribution and that the R 2 -based value functions are proper scoring rules.

Section F describes the estimation and inference for aggregate decomposition and detailed decomposition of conditional covariate shift.

Section G provides the derivations of the results.

Sections H and I describe the implementation and simulation details.

Section J provides additional details on the two real world datasets and results.

B Broader impacts

This work presents a method for understanding failures of ML algorithms when they are deployed in settings or populations different from the ones in development datasets. Therefore, the work can be used to suggest ways of improving the algorithms or mitigating their harms. The method is generally applicable to tabular data settings for any classification algorithm, hence, it can potentially be applied across multiple domains where ML is used including medicine, finance, and online commerce.

Care must be taken while interpreting the results. As usual, assumptions underlying the decompositions such as the coarse causal ordering between the variables W,Z , and Y should be validated through domain knowledge.

Algorithm 1 Aggregate decompositions into baseline, conditional covariate, and conditional outcome shifts

Input: Source and target data { ( W ( d ) i , Z ( d ) i , Y ( d ) i ) } n d i =1 for d ∈ { 0 , 1 } , loss function ℓ ( W,Z,Y ; f ) . Output: Performance change due to baseline, conditional covariate, and conditional outcome shifts Λ W , Λ Z , Λ Y .

1 Split source and target data into training Tr and evaluation Ev partitions. Let n Ev be the total number of data points in the Ev partition.

2 Fit nuisance parameters η W , η Z , η Y , defined in Section G.1, on the Tr partition as outlined in Section H.1.

3 Estimate Λ W , Λ Z , Λ Y using fitted nuisance parameters on the Ev partitions following the equations in Section F.1.

4 Estimate variance of influence functions ψ W ( d, w, z, y ; ˆ η N ) , ψ Z ( d, w, z, y ; ˆ η N ) , and ψ Y ( d, w, z, y ; ˆ η N ) as defined in (23), (24), and (25), respectively.

5 Compute α -level confidence intervals as ˆ Λ N ± z 1 -α/ 2 √ ̂ var ( ψ N ( d, w, z, y ; ˆ η N )) /n Ev for N ∈ { W , Z , Y } , where z is the inverse CDF of the standard normal distribution.

6 return ˆ Λ W , ˆ Λ Z , ˆ Λ Y and confidence intervals

Algorithm 2 VALUECONDITIONALOUTCOME(S): Value for s -partial conditional outcome shift for a subset s

Input: Training Tr and evaluation Ev partitions of source and target data, subset of variables s . Output: Value for s -partial conditional outcome shift for subset s .

1 Fit nuisance parameters η num Y ,s , η den Y , defined in Sections G.3, on the Tr partitions as outlined in H.2.

2 Estimate v Y ( s ) by ˆ v num Y ( s ) / ˆ v den Y where ˆ v num Y ( s ) is estimated using (6) and ˆ v den Y is estimated using (7) on the Ev partition.

3 Estimate variance of influence function ψ Y , bin ,s ( d, w, z, y ; ˆ η num Y ,s , ˆ η den Y ) as defined in (74).

4 Compute α -level confidence interval as ˆ v Y ( s ) ± z 1 -α/ 2 √ ̂ var ( ψ Y , bin ,s ( d, w, z, y ; ˆ η num Y ,s , ˆ η den Y )) /n Ev .

5 return ˆ v Y ( s ) and confidence interval

Algorithm 3 VALUECONDITIONALCOVARIATE(S): Value for s -partial conditional covariate shift for a subset s

Input: Training Tr and evaluation Ev partitions of source and target data, subset of variables s . Output: Value for s -partial conditional covariate shift for subset s .

1 Fit nuisance parameters η num Z ,s , defined in Sections G.2, on the Tr partition, as outlined in H.3.

2 Estimate v Z ( s ) by ˆ v num Z ( s ) / ˆ v num Z ( ∅ ) using (21) on the Ev partition.

3 Estimate variance of influence function ψ Z ,s ( d, w, z, y ; ˆ η num Z ,s ) as defined in (41).

4

Compute

α

-level confidence interval as

5 return ˆ v Z ( s ) and confidence interval

ˆ

v

Z

(

s

)

±

z

-

1

α/

2

var

(

ψ

Z

,s

(

d, w, z, y

; ˆ

η

num

Z

,s

))

/n

Ev

.

Algorithm 4 Detailed decomposition for conditional outcome and covariate shift

Input: Source and target data { ( W ( d ) i , Z ( d ) i , Y ( d ) i ) } n d i =1 for d ∈ { 0 , 1 } , loss function ℓ ( W,Z,Y ; f ) , γ ∈ R + .

Output: Detailed decomposition for conditional outcome or covariate shift, { ϕ Y ,j : j = 0 , · · · , m 2 } or { ϕ Z ,j : j = 1 , · · · , m 2 } .

1 Split source and target data into training Tr and evaluation Ev partitions. Let n Ev be the total number of data points in the Ev partition.

2 Subsample ⌊ γn Ev ⌋ subsets from Z = { 1 , · · · , m 2 } with respect to Shapley weights, including ∅ and Z , denoted s 1 , · · · , s k .

3 Estimate v Y ( s ) ← VALUECONDITIONALOUTCOME(S) and v Z ( s ) ← VALUECONDITIONALCOVARIATE(S) for s ∈ s 1 , · · · , s k .

4 Get estimated Shapley values { ϕ Y ,j } and { ϕ Z ,j } by solving constrained linear regression problems in (7) in Williamson and Feng [47] with value functions v Y ( s ) and v Z ( s ) , respectively.

5 Compute confidence intervals based on the influence functions defined in Theorem 1 in Williamson and Feng [47].

6 return Shapley values { ϕ Y ,j : j = 0 , . . . , m 2 } and { ϕ Z ,j : j = 1 , . . . , m 2 } and confidence intervals

C Considerations for the choice of W and Z variables

We suppose that variables X are partitioned into baseline variables W and conditional covariates Z . We suggest selecting Z to be the variables that may act as mediators of the environment shift and/or variables whose associations with Y are likely to be modified by the environment shift (i.e. effect modification). This selection can be chosen based on a high-level causal graph, where W are variables known to be upstream of Z . For instance, if Z are treatment variables and W are baseline variables, one can interpret a covariate shifts as a change in the treatment policy and an outcome shift as a change in the treatment effect across the two environments.

In the absence of any prior knowledge, another option is to choose W as the variables for which one would like the expected loss given W to be invariant across the two environments; this can be useful to promote fairness of ML algorithms across environments. When this invariance does not hold, the proposed framework explains how variables Z contribute to these differences, which can inform efforts to eliminate performance gaps. This last option is similar to how variables are typically chosen in disparity analyses [21]. For instance, to understand why income differs between males and females controlling for age, one would set domain to D = gender, W = age, and Z as variables that may explain this disparity (e.g. marital status, employment status).

In general, including more variables in W and Z to explain the performance difference is preferable. Nevertheless, there are tradeoffs. For instance, including more variables in W leads to higher variance of ∆ · 10 ( W ) , so it allows one to better distinguish the relative importance of variables in Z for explaining its variability. On the other hand, when more variables are assigned to W , the performance gap with respect to W ( ∆ · 10 ( W ) ) is a more complex function. Thus we may have more uncertainty in our estimate of ∆ · 10 ( W ) , which may lead to wider confidence intervals for the variable importance values.

D Causal interpretation of aggregate and detailed decompositions

Partial distribution shifts that we define in the framework can be equivalently described as stochastic (or shift) interventions in a structural causal model (SCM) respecting a causal directed acyclic graph (DAG) [8]. To represent an intervention on variable X , we use regime indicator σ X which means that the conditional probability distribution for X in the SCM has been updated to a new one [9].

Suppose that the source and target data ( W,Z,Y ) are generated by SCMs respecting the same DAG G in Figure 4 with no unmeasured confounders. Intervening on a variable in source SCM sets its

Z

Y

W

Zs

Y

W

Z

-

s

Zs

Y

W

Z

-

s

Q

Z

Y

W

Z

Y

W

AGGREGATE

DETAILED

σW

σW

σ

Z

σW

σ

Z

σ

Y

σW

σ

Zs

σW

σ

Z

σ

Y

,

Zs

conditional distribution to the corresponding one in the target SCM. Under the assumption of no unmeasured confounders, we have

for any variable set V where pa G denotes parents in G . Therefore, the target data is obtained by intervening on all the variables p 0 ( W,Z,Y ; σ W , σ Z , σ Y ) = p 1 ( W,Z,Y ) . Expectation E is taken over the source distribution or its shifted version based on the specified regime indicator.

Aggregate decompositions can then be written as causal effect of intervening on W,Z,Y incrementally as follows

Due to the factorization (10), the above are equivalent to aggregate decompositions presented in Section 2.

For detailed decomposition of the conditional covariate shift, assume the causal DAG in Figure 4 with Z s → Z -s for variable subset s . The s -partial conditional covariate shift is represented by p 0 ( V ; σ Z s ) and, under the factorization (10), is equivalent to the one considered in Section 2.2. Thus, the performance difference ∆ · s 0 ( W ) can be equivalently written as E [ ℓ | W ; σ Z s ] -E [ ℓ | W ] and ∆ · 10 ( W ) as E [ ℓ | W ; σ Z ] -E [ ℓ | W ] . The value function is then

The s -partial conditional outcome shift corresponds to a stochastic intervention that changes the distribution of Y to (5). Denoting it as p 0 ( V ; σ Y , Z s ) , the performance difference ∆ ·· s ( W,Z ) can be equivalently written as E [ ℓ | W,Z ; σ Y , Z s ] -E [ ℓ | W,Z ] and ∆ ·· 1 ( W,Z ) as E [ ℓ | W,Z ; σ Y ] -E [ ℓ | W,Z ] . The value function (4) is then

E Proper scoring rules for partial distribution shift

Prior works have considered scoring rules for partial distribution shifts in terms of their change in the average loss [48]. For instance, for conditional covariate shifts, prior works have considered scoring rules of the form

However, in the absence of a detailed causal graph, we show that this can lead to unintuitive attributions. Consider the following counterexample. We drop W from the data example for simplicity of exposition.

Example E.1 . Consider the following data-generating process with random variables Z 1 and Z 2 , a real-valued outcome Y , and loss function as squared error ℓ = ( f ( Z 1 , Z 2 ) -Y ) 2 :

Y = Z 1 + Z 2 + ϵ ✶ { Z 1 ≤ 0 } (16) where D = 0 and D = 1 correspond to the source and target domains, respectively. Suppose the ML model is the optimal model for minimizing the expected loss in the source domain, i.e. f ( Z 1 , Z 2 ) = Z 1 + Z 2 . Consider the following candidate partial distribution shifts for explaining the performance change, where Option 1 hypothesizes Z 1 causes Z 2 and Option 2 hypothesizes Z 2 causes Z 1 .

Per the given data-generating process, the partial shift in Option 1 exactly corresponds to the true (aggregate) dataset shift; thus we would expect it to have a higher value than Option 2. Nevertheless, the MSE resulting from Option 2's shift is 0.5 (the marginal distribution of Z 1 under Option 2 is symmetric around 0). In contrast, the MSE resulting from Option 1's shift is the probability that a standard normal variable is less than -1, which is 0.159.

In Example E.1, scoring rule (12) assigns a higher value to a partial shift that contradicts the true causal graph than even the true aggregate shift, because the rule assumes the hypothesized dataset shift is true. Consequently, (12) is not a proper scoring rule. In contrast, the proposed value functions described in Section 2.1 are.

More formally, we extend the traditional definition for a proper scoring rule [18] to the context of explaining performance changes as follows. Let a scoring function χ : O 0 × O 1 × Q be defined as a function of source observation O 0 = ( W 0 , Z 0 , Y 0 ) , target observation O 1 = ( W 1 , Z 1 , Y 1 ) , and candidate shift probability model Q . A scoring rule χ is proper with respect to the set of distribution shift models F if the following holds: for any true model of the post-shift data distribution p ∗ ( W,Z,Y ) ∈ F , the expectation of the scoring function χ ( O 0 , O 1 , p ∗ ) with respect to O 0 ∼ p 0 and O 1 ∼ p ∗ is always no smaller than the expectation of χ ( O 0 , O 1 , p ) for any p ∈ F , i.e.

Moreover, we say that χ is strictly proper if (17) holds with equality iff p = p ∗ .

Let F Z be the set of candidate s -partial conditional covariate shifts p · s 0 and F Y be the set of candidate s -partial conditional outcome shifts p ·· s , i.e.

Then, v Z defined in (3) is a proper scoring rule with respect to F Z , as v Z attains its largest value when ∆ · s 0 ≡ ∆ · 10 , which holds when p · s 0 matches the true covariate-only distribution shift model p · 10 . Similarly, v Y defined in (4) is a proper scoring rule with respect to F Y . Moreover, these are strictly proper scoring rules as long as the conditional expectation of the losses of candidate distribution shifts are not adversarially aligned. That is, under the assumptions that

v Z and v Y are strictly proper scoring rules with respect to F Z and F Y , respectively. As the candidate distribution shifts considered in this work are functionals of the observed distribution shift and observed distribution shifts are unlikely to be adversarial in many real-world situations, Assumptions (A.1) and (A.2) are likely reasonable in practice.

F Estimation and Inference

Let { ( W ( d ) i , Z ( d ) i , Y ( d ) i ) : i = 1 , · · · , n d } denote n d independent and identically distributed (IID) observations from the source and target domains d = 0 and 1 , respectively. Let a fixed fraction of the data be partitioned towards 'training' ( Tr ) and the remaining to 'evaluation' ( Ev ); let n Ev be the number of observations in the evaluation partition. Let P d denote the expectation with respect to domain d and P d,n denote the empirical average over observations in partition Ev from domain d = { 0 , 1 } .

F.1 Aggregate decomposition

The aggregate decomposition terms can be formulated as an average treatment effect, a well-studied estimand in causal inference, where domain D corresponds to treatment. As such, one can use augmented inverse probability weighting (AIPW) to define debiased ML estimators of the aggregate decomposition terms (e.g. Kang and Schafer [24]). We review estimation and inference for these terms below.

Estimation. Using the training data, estimate outcome models µ · 00 ( W ) = E · 00 [ ℓ | W = w ] and µ ·· 0 ( W,Z ) = E [ ℓ | W,Z ] and density ratio models π 100 ( W ) = p 1 ( W ) /p 0 ( W ) and π 110 ( W,Z ) = p 1 ( W,Z ) /p 0 ( W,Z ) . The debiased ML estimators for Λ W , Λ Z , Λ Y are

Inference. Assuming the estimators for the outcome and density ratio models converge at a fast enough rate, the AIPW estimators for the aggregate decomposition terms are asymptotically linear and, thus, facilitate the construction of CIs.

Theorem F.1. Suppose π 100 and π 110 are bounded; estimators ˆ µ · 00 , ˆ π ·· 0 , ˆ π 100 , and ˆ π 110 are consistent; and

Then ˆ Λ W , ˆ Λ Z , and ˆ Λ Y are asymptotically linear estimators of their respective estimands.

F.2 Value of s -partial conditional covariate shifts

Estimation. Using the training partition, estimate the density ratio π 1 s 0 ( z s , w ) = p 1 ( z s , w ) /p 0 ( z s , w ) and the outcome models

in addition to the other nuisance models previously mentioned. We propose the estimator ˆ v Z ( s ) = ˆ v num Z ( s ) / ˆ v den Z , where

and ˆ v den Z := ˆ v num Z ( ∅ ) .

Inference. The estimator is asymptotically normal as long as the outcome and density ratio models are estimated at a fast enough rate defined formally as follows.

Condition F.2. For variable subset s , suppose the following holds

P 0 ( µ 001 ( W ) -µ 01 ( W )) 2 is bounded

P 1 (ˆ µ · s 0 ( W ) -µ · s 0 ( W )) 2 = o p ( n -1 / 2 )

P 1 (ˆ µ · 10 ( W ) -µ · 10 ( W )) 2 = o p ( n -1 / 2 )

(Positivity) p 0 ( z s , w ) > 0 and p 0 ( w,z ) > 0 almost everywhere, such that the density ratios π 1 s 0 ( w,z s ) and π 110 ( w,z ) are well-defined and between (0 , 1) .

Theorem F.3. For variable subset s , suppose v den Z ( s ) > 0 and Condition F.2 hold. Then the estimator ˆ v Z ( s ) is asymptotically normal.

G Proofs

Notation. For all proofs, we will write P to mean expectation on the evaluation partition (and likewise for the empirical version) for notational simplicity.

Overview of derivation strategy. We first present the general strategy for proving asymptotic normality of the estimators for the decompositions. Details on nonparametric debiased estimation can be found in texts such as Tsiatis [45] and Kennedy [25].

Let v ( P ) be a pathwise differentiable quantity that is a function of the true regular (differentiable in quadratic mean) probability distribution P over random variable O . For instance, v in the case of mean is defined as v ( P ) := E o ∼ P ( O ) [ o ] . Let ˆ P denote an arbitrary regular estimator of P , such as the maximum likelihood estimator. The plug-in estimator is then defined as v ( ˆ P ) .

The von-Mises expansion of the functional v (which linearizes v in analogy to the first-order Taylor expansion), given it is pathwise differentiable, gives

Here, the function ψ is called an influence function (or a functional gradient of v at ˆ P ). R ( ˆ P , P ) is a second-order remainder term. The one-step corrected estimators we consider have the form of v ( ˆ P ) + P n ψ ( o ; ˆ P ) where P n denotes a sample average. Following the expansion above, the one-step corrected estimator can be analyzed as follows,

Our goal will be to analyze each of the three terms and to show that they are asymptotically negligible at √ n -rate, such that the one-step corrected estimator satisfies

where we used the property of influence functions that they have zero mean. Thus the one-step corrected estimator is asymptotically normal with mean v ( P ) and variance var ( ψ ( o ; P )) /n , which allows for the construction of CIs. In the following proofs, we present the influence functions without derivations; see Kennedy [25] and Hines et al. [20] for strategies for deriving influence functions.

G.1 Aggregate decompositions

Let the nuisance parameters in the one-step estimators Λ W , Λ Z , Λ Y be denoted by η W = ( µ · 00 , π 100 ) , η Z = ( µ ·· 0 , µ · 00 , π 110 ) , η Y = ( µ ·· 0 , π 110 ) respectively. Denote the estimated nuisances by ˆ η W , ˆ η Z , ˆ η Y . The canonical gradients for the three estimands are

Theorem G.1 (Theorem F.1) . Under conditions outlined in Theorem F.1, the one-step corrected estimators for the aggregate decomposition terms, baseline, conditional covariate, and conditional outcome ˆ Λ W , ˆ Λ Z , and ˆ Λ Y , are asymptotically linear, i.e.

Proof. The estimands Λ W , Λ Z , Λ Y have similarities to the standard average treatment effect (ATE) in the causal inference literature (see [25, Example 2]. Hence, the estimators and their asymptotic properties directly follow. For treatment T , outcome O , and confounders C , the mean outcome under T = 1 among the population with T = 0 is identified as

and its one-step corrected estimator can be derived from the canonical gradient of ϕ , which takes the following form after plugging in the estimates of the nuisance models:

satisfies

where µ 1 ( c ) = E [ o | c, t = 1] and π ( c ) = p ( c | t = 0) /p ( c | t = 1) as long as the following conditions hold:

p ( c | t = 1) > 0 almost everywhere such that the density ratios π ( c ) are well-defined and bounded,

We establish the estimators and their influence functions by showing that they can all be viewed as mean outcomes of the form (27).

Baseline term Λ W . The first term E 100 [ ℓ ( w,z, y )] is a mean outcome with respect to p ( ℓ ( w,z, y ) | w,d = 0) p ( w | d = 1) , which is the same as that in (27) but with ℓ ( w,z, y ) as the outcome, w as the confounder, and d as the (flipped) treatment. The second term E 000 [ ℓ ( w,z, y )] is a simple average over D = 0 population whose influence function is the ℓ ( w,z, y ) itself.

Conditional covariate term Λ Z . First term E 110 [ ℓ ( w,z, y )] is the mean outcome with respect to p ( ℓ ( w,z, y ) | w,z, d = 0) p ( w,z | d = 1) , where the chief difference is ( w,z ) is the confounder. Second term E 100 [ ℓ ( w,z, y )] is also a mean outcome, as discussed above.

Conditional outcome term Λ Y . First term E 111 [ ℓ ( w,z, y )] is a simple average over the D = 1 population.

G.2 Value of s -partial conditional covariate shifts

Let nuisance parameters in the one-step estimator v num Z ,s be denoted η num Z ,s = ( µ · s 0 , µ · 10 , µ · 0 -s 0 , µ 001 , µ ·· 0 , π 1 s 0 , π 110 ) and the set of estimated nuisances by ˆ η num Z ,s . The canonical gradient of v num Z ( s ) is

Lemma G.2. Under Condition F.2, ˆ v num Z ( s ) satisfies

Proof. Consider the following decomposition

We note that (32) converges to a normal distribution per CLT assuming the variance of ψ num Z ,s is finite. The empirical process term (32) is asymptotically negligible, as the nuisance parameters η num Z ,s are estimated using a separate training data split from the evaluation data and [25, Lemma 1] states that

as long as estimators for all nuisance parameters are consistent. We now establish that the remainder term (33) is also asymptotically negligible. Integrating with respect to Y , we have that

(34)

From convergence conditions in Condition F.2, this simplifies to

Given the true density ratios, we can further simplify the expectations over D = 0 weighted by the density ratios in the expression above to expectations over D = 1 . By definition of µ · 0 -s 0 ( Z s , W ) in

(18) and µ · s 0 ( W ) in (20) and the definition of µ ·· 0 ( W,Z ) and µ · 10 ( W ) in Section F.1, (33) simplifies to

which is o p ( n -1 / 2 ) as long as the convergence conditions in Condition F.2 hold.

As the denominator v den Z is equal to the numerator v num Z ( ∅ ) , it follows that the one-step estimator for the denominator ˆ v den Z is asymptotically linear with influence function ψ num Z , ∅ .

Proof for Theorem F.3. Combining Lemma G.2 and the Delta method [46, Theorem 3.1], the estimator ˆ v Z ( s ) = ˆ v num Z ( s ) / ˆ v den Z is asymptotically linear with influence function

where ψ num Z ,s ( D,W,Z,Y ; η num Z ,s ) is defined in (28) and ψ den Z ( D,W,Z,Y ; η den Z ) = ψ num Z , ∅ ( D,W,Z,Y ; η num Z , ∅ ) .

Accordingly, the estimator asymptotically follows the normal distribution,

G.3 Value of s -partial conditional outcome shifts

Let the nuisance parameters in v num Y , bin be denoted η num Y ,s = ( Q bin , µ ·· 1 , µ ·· s , π ) and its estimate as ˆ η num Y ,s . We represent the one-step corrected estimator for v num Y , bin ( s ) as the V-statistic

In more detail, the conditions in Theorem 3.2 are as follows.

Condition G.3. For variable subset s , suppose the following hold

π ( W,Z s , Z -s , Q bin ) is bounded

ˆ π is consistent

Lemma G.4. Assuming Condition G.3 holds, ˆ v num Y , bin is an asymptotically linear estimator for v num Y , bin , i.e.

with influence function

Proof. Defining the symmetrized version of h in (46) as h sym ( W,Z,Y, ˜ W, ˜ Z, ˜ Y ) = h ( W,Z,Y, ˜ W, ˜ Z, ˜ Y )+ h ( ˜ W, ˜ Z, ˜ Y ,W,Z,Y ) 2 , we rewrite the estimator as

Per Theorem 12.3 in [46], the Hájek projection of ˆ v num Y , bin ( s ) is

where ¯ ˆ v num Y ( s ) = P 1 ˜ P 1 h sym ( W,Z,Y, ˜ W, ˜ Z, ˜ Y ; ˆ η num Y ,s ) .

Consider the decomposition

We analyze each term in turn.

Term (49): Suppose P 1 h 2 sym ( W,Z,Y, ˜ W, ˜ Z, ˜ Y ; ˆ η num Y ,s ) < ∞ . Via a straightforward extension of the proof in Theorem 12.3 in van der Vaart [46], one can show that

Then by Theorem 11.2 in van der Vaart [46] and Slutsky's lemma, we have

Term (50): We perform sample splitting to estimate the nuisance parameters and calculate the estimator for ˆ v num Y ( s ) . Then by Lemma 1 in Kennedy [25], we have that

as long as the estimators for the nuisance parameters are consistent.

Term (52): We will show that this bias term is asymptotically negligible. For notational simplicity, let ˆ ξ ( W,Z s , Z -s ) = ˆ µ ·· 1 ( W,Z ) -ˆ µ ·· s ( W,Z ) .

Note that (56) is o p ( n -1 / 2 ) under the assumed convergence rates for ˆ q bin . In addition, (54) is o p ( n -1 / 2 ) , under the assumed convergence rates for ˆ µ ·· 1 and ˆ µ ·· s .

Analyzing the remaining summands (53) + (55), we note that it simplifies as follows:

which is o p ( n -1 / 2 ) , under the assumed convergence rates for ˆ µ ·· s , ˆ µ ·· 1 , and ˆ π .

Condition G.5 (Convergence conditions for ˆ v den Y ) . Suppose the following holds

P 0 ( µ ·· 1 -µ ·· 0 ) 2 is bounded

(Positivity) p ( w,z | d = 0) > 0 almost everywhere, such that the density ratios π 110 ( w,z ) are well-defined and bounded.

Let the nuisance parameters in the one-step estimator v den Y be denoted by η den Y = ( µ ·· 0 , µ ·· 1 , π 110 ) and the set of estimated nuisances by ˆ η den Y .

Lemma G.6. Assuming Condition G.5 holds, then ˆ v den Y is an asymptotically linear estimator for v den Y , i.e.

with influence function

Proof. Consider the following decomposition of bias in the one-step corrected estimate

We observe that (61) converges to a normal distribution per CLT assuming that the variance of ψ den Y is finite. The empirical process term (62) is asymptotically negligible since the nuisance parameters η den Y are evaluated on an separate evaluation data split from the training data used for estimation.

In addition assuming that the estimators for the nuisance parameters are consistent, Kennedy [25, Lemma 1] states that

Wenow show that the remainder term (63) is also asymptotically negligible. Substituting the influence function and integrating with respect to Y , (63) becomes

Thus the remainder term is o p ( n -1 / 2 ) if Condition G.5 holds.

Proof for Theorem 3.2. Combining Lemmas G.4, G.6, and the Delta method [46, Theorem 3.1], the estimator ˆ v Y , bin ( s ) = ˆ v num Y , bin ( s ) / ˆ v den Y is asymptotically linear with influence function

where ψ num Y ,s and ψ den Y are defined in (48) and (60).

Accordingly, the estimator follows a normal distribution asymptotically,

Remark on super-fast convergence of ˆ q bin . One of the conditions in Condition G.3 states that the ˆ q bin converges at o p ( n -1 ) rate. While this seems restrictive, binned risk converges exponentially under suitable margin conditions presented in Audibert and Tsybakov [2]. In particular, consider the following margin condition which is a relaxation of Condition 3.1.

Condition G.7. For all bin edges b except b ∈ { 0 , 1 } , the set Ξ ϵ = { x : | q ( x ) -b | ≤ ϵ } is measure zero for some ϵ > 0 .

Suppose the function q belongs to the Hölder class Σ( β, L, R d ) for positive integer β and L > 0 (see definition in Audibert and Tsybakov [2]), the marginal law of X satisfies the strong density assumption, and the margin condition G.7 is satisfied. Then the rate of convergence of ˆ q bin is exponential, i.e.

for constants C 1 , C 2 > 0 that do not depend on n .

Remark on value of s -partial conditional outcome shift. Unlike the case of s -partial conditional covariate shift, note that v Y ( ∅ ) may be non-zero when the risk in the target domain is a recalibration (i.e. temperature-scaling) of the risk in the source domain [34, 19]. For instance, there may be general environmental factors such that readmission risks in the target domain are uniformly lower.

H Implementation details

Here we describe how the nuisance parameters can be estimated in each of the decompositions. In general, density ratio models can be estimated via a standard reduction to a classification problem where a probabilistic classifier is trained to discriminate between source and target domains [43].

Note on computation time . Shapley value computation can be parallelized over the subsets. For high-dimensional tabular data, grouping together variables can further reduce computation time (and increase interpretability).

H.1 Aggregate decompositions

Density ratio models. Using direct importance estimation [43], density ratio models π 100 ( W ) and π 110 ( W,Z ) can be estimated by fitting classifiers on the combined source and target data to predict D = 0 or 1 from features W and ( W,Z ) , respectively.

Outcome models. The outcome models µ · 00 ( W ) and µ ·· 0 ( W,Z ) can be fit in a number of ways. One option is to estimate the conditional distribution of the outcome (i.e. p 0 ( Y | W ) or p 0 ( Y | W,Z ) ) using binary classifiers, from which one can obtain an estimate of the conditional expectation of the loss. Alternatively, one can estimate the conditional expectations of the loss directly by fitting regression models.

H.2 Detailed decomposition for s -partial outcome shift

Density ratio models. The density ratio π ( W,Z s , Z -s , Q ) = p 1 ( Z -s | W,Z s , q ( W,Z ) = Q ) /p 1 ( Z -s ) in (6) can be estimated as follows. Create a second ('phantom') dataset of the target domain in which Z -s is independent of Z s by permuting the original Z -s in the target domain. Compute q bin for all observations in the original dataset and the permuted dataset. Concatenate the original dataset from the target domain with the permuted dataset. Train a classifier to predict if an observation is from the original versus the permuted dataset.

Outcome models. The outcome models µ ·· 1 and µ ·· s ( W,Z ) can be similarly fit by estimating the conditional distributions p 0 ( Y | W,Z ) and p s ( Y | W,Z,q bin ( W,Z )) on the target domain, and then taking expectation of the loss.

Computing U-statistics. Calculating the double average P 1 ,n ˜ P 1 ,n in the estimator requires evaluating all n 2 pairs of data points in target domain. This can be computationally expensive, so a good approximation is to subsample the inner average. We take 2000 subsamples. We did not see large changes in the bias of the estimates compared to calculating the exact U-statistics.

H.3 Detailed decomposition for s -partial conditional outcome shift

Density ratio models. The ratio π 1 s 0 ( W,Z s ) = p 1 ( W,Z s ) /p 0 ( W,Z s ) can be similarly fit using direct importance estimation.

Outcome models. We require the following models.

µ · 0 -s 0 ( z s , w ) = E · 00 [ ℓ | z s , w ] , defined in (18), can be estimated by regressing loss against w,z s on the source domain.

µ · 10 ( w ) = E · 10 [ ℓ | w ] , defined in (19), can be estimated by regressing µ ·· 0 ( w,z ) against w in the target domain.

µ · s 0 ( w ) = E · s 0 [ ℓ | z s , w ] , defined in (20), can be estimated by regressing µ · 0 -s 0 ( z s , w ) against w in the target domain.

For all models, we use cross-validation to select among model types and hyperparameters. Model selection is important so that the convergence rate conditions for the asymptotic normality results are met.

I Simulation details

Data generation : We generate synthetic data under two settings. For the coverage checks in Section 4.1, all features are sampled independently from a multivariate normal distribution. The mean of the ( W,Z ) in the source and target domains are (0 , 2 , 0 . 7 , 3) and (0 , 0 , 0 , 0) , respectively. The outcome in the source and target domains are simulated from a logistic regression model with coefficients (0 . 3 , 1 , 0 . 5 , 1) and (0 . 3 , 0 . 1 , 0 . 5 , 1 . 4) .

In the second setting for baseline comparisons in Figure 2b(ii), each feature in W ∈ R and Z ∈ R 5 is sampled independently from the rest from a uniform distribution over [ -1 , 1) . The binary outcome Y is sampled from a logistic regression model with coefficients (0 . 2 , 0 . 4 , 2 , 0 . 25 , 0 . 1 , 0 . 1) in source and (0 . 2 , -0 . 4 , 0 . 8 , 0 . 1 , 0 . 1 , 0 . 1) in target.

In both the settings, we analyze performance gap of logistic regression models fit on a held-out source dataset.

Sample-splitting : We fit all models on 80% of the data points from both source and target datasets which is the Tr partition, and keep the remaining 20% for computing the estimators which is the Ev partition.

Model types : We use scikit-learn implementations for all models [33]. We use 3-fold cross validation to select models. For density models, we fit random forest classifiers and logistic regression models with polynomial features of degree 3. We clip the predicted probabilities from the density model for π at 10 -6 to avoid very large density weights. Depending on whether the target outcome in outcome models is binary or real-valued, we fit random forest classifiers or regressors, and logistic regression or linear regression models with ridge penalty. Specific hyperparameter ranges for the grid search are provided in the code.

Computing time and resources : Computation for the VI estimates can be quite fast, as Shapley value computation can be parallelized over the subsets and the number of unique variable subsets sampled in the Shapley value approximation is often quite small. For instance, for the ACS Public Coverage case study with 34 features, the unique subsets is 131 even when the number of sampled subsets is 3000 , and it takes around 160 seconds to estimate the value of a single variable subset. All experiments are run on a 2.60 GHz processor with 8 CPU cores.

J Data analysis details

Synthetic. We describe accuracy of the ML algorithm after it is retrained with the top k features and predictions from the original model (Table 3). Proposed method results in the revised model with highest gain in accuracy and AUC. Figure 5 shows the aggregate and detailed decompositions for the simulation setup in Section 4.1. Figure 6 shows the coverage rates of 90% CIs for the aggregate decompositions.

Hospital readmission. Using data from the electronic health records of a large safety-net hospital in the US, we analyzed the transferability of performance measures of a Gradient Boosted Tree (GBT) trained to predict 30-day readmission risk for the general patient population (source) but applied to patients diagnosed with heart failure (target). Each of the source and target datasets have 3750 observations for analyzing the performance gap. The GBT is trained on a held-out sample of 18,873 points from the general population. Features include 4 demographic variables ( W ) and 16 diagnosis

Aggregate

Aggregate Variable

W

Proposed

Z

Plugin

x

Oracle

0.150

0.125

0.100

0.075

0.050

0.025

0.000

0.025

Decomposition Value

Detailed:Conditional covariate

(x1)

Proposed

Plugin

{X2}

x

Oracle

0.0

0.2

0.4

0.6

0.8

1.0

Detailed:Conditionaloutcome

{X1}

Proposed

{X2}

Plugin

x

Oracle

0.0

0.2

0.4

0.6

0.8

1.0

Attribution Value

Subset=W

Coverage

1.0

0.5

Plug-in

Debiased ML

0.0

Subset=Z

Coverage

1.0

0.5

0.0

Subset=Y

Coverage

1.0

0.5

0.0

8000

16000

n

Detailed:Conditional covariate

Druginduced/toxicrelatedcondition

xxXXXXXXXXXxx

Mental&substanceusedisordersinremission

Respiratorysigns&symptoms

Diabetesmellituswithcomplication

Fluid&electrolytedisorders

Diabetesmellituswithoutcomplication

HDPDdebiased

Variable

Abdominalpain,digestivesigns&symptoms

HDPD plug-in

Conditionsduetoneoplasmoritstreatment

Musculoskeletalpain,not lowbackpain

×

MeanChange

Exposuretoinfectiousdisease

Tobacco-relateddisorders

WuShift

Nervoussystemsigns&symptoms

Personal/familyhistoryofdisease

Injury,sequela

×

Essentialhypertension

xx

Abnormalfindingswithoutdiagnosis

0.0

0.2

0.4

0.6

0.8

1.0

AttributionValue

codes ( Z ). While training, we reweigh samples by class weights to address class imbalance. Figure 7 shows the detailed decomposition of conditional covariate shift for the dataset.

ACS Public Coverage. We extract data from the American Community Survey (ACS) to predict whether a person has public health insurance. The data only contains persons of age less than 65 and having an income of less than $30,000. We analyze a neural network (MLP) trained on data from Nebraska (source) to data from Louisiana (target) given 3000 and 6000 observations from the source and target domains, respectively. Another 3300 from source for training the model. Figure 8 shows the detailed decomposition of conditional outcome shift for the dataset. Table 4 shows the difference in AUCs for the revised models based on top features from the proposed method versus RandomForestAcc method.

NeurIPS Paper Checklist

1. Claims

Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?

Answer: [Yes]

Justification: Main contributions are stated in Section 1. Theoretical and experimental results are presented in Sections 3 and 5 respectively.

Guidelines:

The answer NA means that the abstract and introduction do not include the claims made in the paper.

Detailed:Conditional outcome

Totalperson'sincome

Married

Nevermarriedorunder15years

Employment status employed

Divorced

Citizenshipstatusnotacitizen

Citizenship statusborninUS

Educationalattainment

Disability

Employment status not in labor force

Nativity

Employmentstatusofparents

Mobility status

Vision difficulty

Variable

Citizenshipstatusbornabroad

Gavebirthtochild inpast12months

X

Cognitivedifficulty

Widowed

School atleastbachelor

Schoolatleasthighschoolor GED

Employmentstatuspartialemployed

School postgrad

Separated

Citizenship statusborninPuertoRico,Guam

Hearing difficulty

Citizenship status naturalized

Militaryservice

HDPD debiased

Employmentstatus unemployed

△x

Employmentstatusarmed

×

■

HDPD plugin

Employmentstatuspartialarmed

Ancestry

文

RandomForestAcc

0.00

0.05

0.10

0.15

0.20

0.25

AttributionValue

The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.

The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.

It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.

2. Limitations

Question: Does the paper discuss the limitations of the work performed by the authors?

Answer: [Yes]

Justification: Limitations are discussed in Section 7.

Guidelines:

The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.

The authors are encouraged to create a separate "Limitations" section in their paper.

The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.

The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.

The authors should reflect on the factors that influence the performance of the approach. For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be

used reliably to provide closed captions for online lectures because it fails to handle technical jargon.

The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.

If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.

While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.

3. Theory Assumptions and Proofs

Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?

Answer: [Yes]

Justification: Assumptions and proofs for the theoretical results are provided in Sections 3, F, and G.

Guidelines:

The answer NA means that the paper does not include theoretical results.

All the theorems, formulas, and proofs in the paper should be numbered and crossreferenced.

All assumptions should be clearly stated or referenced in the statement of any theorems.

The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.

Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.

Theorems and Lemmas that the proof relies upon should be properly referenced.

4. Experimental Result Reproducibility

Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?

Answer: [Yes]

Justification: Pseudocode is provided in Algorithms 1, 2, 3, and 4. Implementation details are described in Section H.

Guidelines:

The answer NA means that the paper does not include experiments.

If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.

If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.

Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.

While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example

(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.

(b) If the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.

(c) If the contribution is a new model (e.g., a large language model), then there should either be a way to access this model for reproducing the results or a way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).

(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.

5. Open access to data and code

Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?

Answer: [Yes]

Justification: Code is available at the link https://github.com/jjfeng/HDPD .

Guidelines:

The answer NA means that paper does not include experiments requiring code.

Please see the NeurIPS code and data submission guidelines ( https://nips.cc/ public/guides/CodeSubmissionPolicy ) for more details.

While we encourage the release of code and data, we understand that this might not be possible, so 'No' is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).

The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines ( https: //nips.cc/public/guides/CodeSubmissionPolicy ) for more details.

The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.

The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only a subset of experiments are reproducible, they should state which ones are omitted from the script and why.

At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).

Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.

6. Experimental Setting/Details

Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?

Answer: [Yes]

Justification: Implementation and simulation details are described in Section H and Section I. Full details are in the provided code.

Guidelines:

The answer NA means that the paper does not include experiments.

The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.

The full details can be provided either with the code, in appendix, or as supplemental material.

7. Experiment Statistical Significance

Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?

Answer: [Yes]

Justification: Results are accompanied by confidence intervals.

Guidelines:

The answer NA means that the paper does not include experiments.

The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).

The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)

The assumptions made should be given (e.g., Normally distributed errors).

It should be clear whether the error bar is the standard deviation or the standard error of the mean.

It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified.

For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).

If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.

8. Experiments Compute Resources

Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?

Answer: [Yes]

Justification: Compute resources are described in Section I.

Guidelines:

The answer NA means that the paper does not include experiments.

The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.

The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.

The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).

9. Code Of Ethics

Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines ?

Answer: [Yes]

Justification: We adhere to the NeurIPS Code of Ethics.

Guidelines:

The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.

If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.

The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction).

10. Broader Impacts

Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?

Answer: [Yes]

Justification: Although we do not envision any direct negative impacts of the work, potential positive and negative impacts are discussed in Section B.

Guidelines:

The answer NA means that there is no societal impact of the work performed.

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.

Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.

The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.

The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.

If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).

11. Safeguards

Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?

Answer: [NA]

Justification: We do not release a model or a new dataset.

Guidelines:

The answer NA means that the paper poses no such risks.

Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.

Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.

We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.

12. Licenses for existing assets

Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?

Answer: [Yes]

Justification: Medical dataset used for the case study is accessed and analyzed under the terms of an IRB.

Guidelines:

The answer NA means that the paper does not use existing assets.

The authors should cite the original paper that produced the code package or dataset.

The authors should state which version of the asset is used and, if possible, include a URL.

The name of the license (e.g., CC-BY 4.0) should be included for each asset.

For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset.

For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.

If this information is not available online, the authors are encouraged to reach out to the asset's creators.

13. New Assets

Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?

Answer: [NA]

Justification: We do not introduce new assets. The code is provided only for reproducing the experiments.

Guidelines:

The answer NA means that the paper does not release new assets.

Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.

The paper should discuss whether and how consent was obtained from people whose asset is used.

At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.

14. Crowdsourcing and Research with Human Subjects

Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?

Answer: [NA]

Justification: Study does not involve crowdsourcing or research with human subjects.

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper.

According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.

15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects

Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?

Answer: [Yes]

Justification: Medical dataset was accessed according to the terms of use of an IRB.

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.

For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper proposes a hierarchical decomposition model for binary machine learning classifiers. The nonparametric model allows for a detailed decomposition of distributions shifts at aggregate or partial levels. Additionally, confidence intervals for the proposed estimators are presented.\n\nSTRENGTHS:\n1. Introducing a new general concept to decompose distribution shifts for classifiers has a wide range of applications. Specifically basing the evaluation on $R^2$-measures enhances interpretability.\n\n2. While no causal graph is required, causal knowledge can easily be incorporated into the setting.\n\n3. The paper is well-organized and gradually motivates the complex definitions.\n\nWEAKNESSES:\n1. The proposed method is restricted to (binary) classification problems. This limits the applicability and should already be noted in the abstract and/or title.\n\n2. The framework requires baseline variables W, which can not be used for partial shifts. This is already sufficiently acknowledged and well motivated, especially combined with a causal graph.\n\n3. The conditions should be discussed in more detail. Specifically, the fast convergence rate (8) is a very strong assumption that is not sufficiently discussed.\n\nMinor Comments:\n- Figure 2(a): No dashed lines as mentioned in the caption.\n- L.162: Typo “toderive”\n- L. 170: Typo “psuedocode”\n\nQUESTIONS:\n1. Regarding Weakness 3. The reference [2] only achieves fast convergence under strong and specific conditions. I am not quite sure whether these conditions are even achievable in the presented setting. Specifically, the binned $q_{\\text{bin}}$ is not smooth, which would already violate the Hölder condition. Further, the margin condition should be discussed as this would require comparatively “simple” classification problems based on the bins.\n\nLIMITATIONS:\nSee Weaknesses", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 2, "DATE": "2024-06-21"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors describe a novel method to detect root causes of distribution shift (performance variability) of ML classifiers across domains. The method estimates how much of the shift is due to covariate shift vs. outcome shift, and which input features contribute most to the said shifts.\n\nSTRENGTHS:\nVery significant topic and, as far as I can tell, original approach. Good Real-world case studies. Code is provided.\n\nWEAKNESSES:\nThe paper is not well written because most NeurIPS audience (myself included) may not be able to understand the methods. For example, this\nis almost incomprehensible (lines 76-80):\n\nAn aggregate shift substitutes a factor from the source domain in (1) with that from the target domain, i.e. we swap the factor from p0 to p1. A partial shift with respect to variable subset s (or an s-partial shift) shifts a factor from the source domain in (1) only with respect to variable subset s; we denote this by swapping a factor from p0 to ps.\n\nI am guessing the authors did not have enough space to write elaborate explanation, or maybe reading it requires extensive background in this\nparticular area. Suggestion: write only the final equations and high-level explanation how it would be applied in ML practice; the rest of the derivation should be moved to Technical Supplement, and since there is no length constraint there, you can provide extensive and clear derivation of your result.\n\nBottom line: my best guess is this is a strong paper, but I have low confidence in my assessment due to the above.\n\nQUESTIONS:\nWhat is the connection between equation (2) and VI? Is (2) the\ndefinition of VI? \n\nWhat are E000, E100 etc. between lines 89 and 90?\n\nIn Fig. 3, two types of results are shown: HDPD debiased and HDPD\nplug-in. If someone decides to use your method, which version do you\nrecommend?\n\nLIMITATIONS:\nThe limitations are addressed adequately.", "TITLE": "Official Review", "RECOMMENDATION": 8, "REVIEWER_CONFIDENCE": 2, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-06-28"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe problem of explaining and remedying a dropoff in the accuracy of a supervised learning model when applied to a new environment (i.e. a new joint distribution between inputs and output) is studied. The paper tackles these problems for a non-parametric setting and with the goal of identifying specific individual input variables which most contribute to the dropoff. Confidence intervals are also generated. Previous work in the area does not meet all these criteria. Input variables are segmented into two disjoint sets: a set of \"baseline\" variables W and a set of \"conditional covariates\" Z which are often (but not necessarily) causally downstream of the baseline variables. The total dropoff in performance is decomposed into the dropoff attributable to each of the 3 shifts: a shift in P(W), a shift in P(Z | W) and a shift in the conditional expectation of the target variable given the inputs, i.e., P(Y | Z; W).  The method is able to evaluate the impact of partial shifts in the Z variables, i.e. shifts in a subset of the Z variables. Partial shifts in W cannot be evaluated, though. To rank the impacts of shifts in individual Zs, Shapley values are used. To overcome nonparametric estimation challenges, a binning procedure as well as a Shapley value sampling procedure is used. The method is shown to give intuitive results for simulated data. Two real world problems are studied. The first problem is hospital readmission prediction. The shift in environment is from a general patient population to a patient population restricted to those with heart failure.  The second problem is health insurance coverage prediction where the shift in environment is from Nebraska to Louisiana. The method is evaluated in terms of the quality of the selection of the top-k variables most responsible for performance dropoff. The quality of the selection is evaluated by training and testing a model on the second environment where the inputs include the output of the original-environment variable, the baseline variables and only those top-k of the Z, rather than all the Z.  Superior results as compared to competing techniques are obtained.\n\nSTRENGTHS:\nThe paper is original to the best of my knowledge.\n\nIf the flaws in the paper could be fixed, I certainly agree that the contribution of the work could be significant. Understanding and adapting to changing environments for tabular models is definitely an important problem and it's valuable to do this for general ML models rather than just linear stuff as in Oaxaca-Blinder. \n\nAlthough I didn't have time to check every technical detail super-carefully, the paper does seem to overcome major technical challenges in a clever way. In particular, I quite liked the use of the random subsampling Shapley value approach to overcome the combinatorial explosion of exact Shapley value computation. I've wondered vaguely in the past whether a technique like that could be valid and I feel I benefitted from learning about that reference and seeing its application here. \n\nSome parts of the paper are well written, although I also think there are serious clarity problems, as I'll explain in the next section.\n\nWEAKNESSES:\nI'm sure it's unpleasant for the authors to see my score of 3. I probably have never liked a paper I've given a 3 nearly as much as I liked this paper. I think the potential is here to achieve (eventually) a quite significant contribution to the ML literature. However, I feel strongly that the paper is not ready for submission as is.  I'm somewhat open to raising my score if my concerns can be addressed in rebuttal, but it woh't be easy to address my concerns given the short rebuttal time window.  \n\nThe biggest flaw I see in the paper is that the distinction between baseline variables W and conditional covariates Z is very problematic. The distinction is not well explained and it's poorly illustrated both by the real-world problems and the synthetic experiments. The authors say that Z can often be thought of as causally downstream of W but this not required. Ok, if it's not required, then how else would you segment the variables into W and Z ? I don't think this is ever explained.  For the real world problems (unless I missed something), there doesn't seem to be any explanation why some variables are W and others are Z. For hospital readmission, why would demographic variables be W and diagnosis codes be Z ? I get that demographics could cause the diagnoses to some extent (although the diagnoses are surely a long way from being fully caused by a small set of demographic variables) but I still needed a lot more explanation here. For ACS insurance, why are the 3 demographic variables (age, race etc) the baseline Ws and the others (marital status, etc) the conditional covariates?  For the synthetic stuff, on line 243, why are W, Z1, Z2, Z3 independent normal if we are usually supposed to think of Z as causally downstream of W ? \n\nI also consider only 2 real world problems to be a bit thin in terms of real-world results for NeurIPs acceptance.\n\nAnother problem I have with the experiments is that more naive  baselines are needed to double-check that the partial retraining beats the baseline. I'll take the authors' word that this procedure of taking the old-environment model output as a new-environment input and combining that with top-k most impactful Zs is some sort of standard in the literature. Nonetheless, I think you should compare it against:\n\n1) Throw away all the old-environment data and fully retrain a new-environment-training-data-only model with all available W and Z inputs. \n\n2) Do some sort of weighted-data retraiing using all inputs, where old environment and new environment training data are both used by old environment is weighted less (with the weight being a CV-optimized hyperparameter).\n\nThere are also some clarity problems.\n\nOn line 147, it's wrong to use \"risk\" as the probability of class 1. \"Risk\" has an established meaning in the ML community as \"loss\" or \"error\". E.g. \"empirical risk minimization\" means loss minimization, not minimization of the odds of class 1.\n\nThe paper also makes various variable/terminology choices which aren't a big problem but could be improved so that the reader finds it easier to follow.\n\nI would suggest using \"environment\" where \"domain\" is used because \"domain\" can also mean the input space, i.e. a function mapping from domain to range, so that could be confusing.\n\nI would not call the 2nd domain/environment \"target\", since that can be confused with Y. Maybe the \"shifted\" environment or the \"new\" environment ?\n\nI wouldn't denote the two domains/environments as 0 and 1. I would use letters, e.g. O and S for original and shifted.\n\n\nTypos:\n\nL94 which have the benefits-> which has the benefits\nL162 toderive\n\nQUESTIONS:\nHow does one segment between W and Z if not by casual knowledge (since this is not required) ?\n\nYou need at least one W for your method. Can you just arbitrarily pick 1 variable for your W ? Is there some sort of bias/variance or other tradeoff if you picked 1 variable as W and e.g. the other 99 as Z or would 20/80 or 50/50 be more reasonable for some reason?\n\nWhy do you think the W you chose cause the Z you chose in hospital readmission and insurance coverage?\n\n** update after rebuttal **\n\nIn light of the additional explanations for how to distinguish between W and Z and in light of the additional experiments evaluating the baselines I suggested, I raised my score to a 5.\n\nLIMITATIONS:\nAdequately addressed.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-11"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces a hierarchical decomposition framework aimed at explaining performance discrepancies of machine learning models across different domains. It proposes both aggregate and detailed decompositions to quantify the impact of shifts in feature distributions (marginal and conditional) on model performance, without necessitating detailed causal graphs. The framework leverages nonparametric methods and Shapley-based variable attributions, enhancing understanding and facilitating targeted interventions to mitigate performance gaps. Coverage rates and further experiment analysis are provided.\n\nSTRENGTHS:\nThe scope of the paper significantly extends that of the most relevant prior work, which provides the aggregate decomposition that this work builds on.\n\n This work further extends several other related works by characterizing shifts that are more complex and in a setting where no causal graph encoding prior knowledge of the data generating process or shift structure is assumed.\n\nThe estimation and inference procedures are provided, with convergence rates.\n\nWEAKNESSES:\nI find the paper, at times, is difficult to follow. I would suggest a formal definition on the framework, along on definition of performance gap, and each level of explanation. Some causal interpretation in appendix could be useful in Section 2 to understand the framework.\n\nExperiment results bring some questions.\n\nQUESTIONS:\n- L85 aggregate: the first level mentioned here represents the aggregated level?\n- linear decomposition: although it looks like just a log operation from equation, it would be good to give intuition on such a linear decomposition. What assumptions are made exactly here?\n- More importantly, the performance gap may be from  a combination of variables, instead of one variable at a time. How does the proposed framework address such cases?\n- How scalable is the proposed method? Would the proposed method handle discrete variables?\n- Fig 3b: some variables seems to be highly correlated or even negation of each other ( e.g. Married, Divorced, Never married, and e.g. citizenship status). Yet they do not seems to be consistent to each other (e.g., married and divorced). What is the reason for such a difference?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 2, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-12"}