A hierarchical decomposition for explaining ML performance discrepancies
Authors: Harvineet Singh
Fan Xia
Adarsh Subbaswamy
Alexej Gossmann
Jean Feng
Venue: NeurIPS 2024 poster (2024)
Keywords: explainability
distribution shift
double machine learning
Primary area: interpretability_and_explainability
TL;DR: we propose a method to explain performance difference of a model between populations by highlighting influential variables

Abstract: Machine learning (ML) algorithms can often differ in performance across domains. Understanding why their performance differs is crucial for determining what types of interventions (e.g., algorithmic or operational) are most effective at closing the performance gaps. Aggregate decompositions express the total performance gap as the gap due to a shift in the feature distribution $p(X)$ plus the gap due to a shift in the outcome's conditional  distribution $p(Y|X)$. While this coarse explanation is helpful for guiding root cause analyses, it provides limited details and can only suggest coarse fixes involving all variables in an ML system. Detailed decompositions quantify the importance of each variable to each term in the aggregate decomposition, which can provide a deeper understanding and suggest more targeted interventions. Although parametric methods exist for conducting a full hierarchical decomposition of an algorithm's performance gap at the aggregate and detailed levels, current nonparametric methods only cover parts of the hierarchy; many also require knowledge of the entire causal graph. We introduce a nonparametric hierarchical framework for explaining why the performance of an ML algorithm differs across domains, without requiring causal knowledge. Furthermore, we derive debiased, computationally-efficient estimators and statistical inference procedures to construct confidence intervals for the explanations.

A hierarchical decomposition for explaining ML performance discrepancies
Harvineet Singh'
Fan Xia
Adarsh Subbaswamy?
Alexej Gossmann2
Jean Feng
University of California, San Francisco
U.S. Food and Drug Administration, Center for Devices and Radiological Health
Abstract
Machine learning (ML) algorithms can often differ in performance across domains. Understanding why their performance differs is crucial for determining what types interventions (c.g., algorithmic or operational) are most effective at closing the performance gaps. Aggregate decompositions express the total performance gap as the gap due to a shift in the feature distribution p(X) plus the gap due shift in the outcome's conditional distribution (X). While this coarse explanation is helpful for guiding root cause analyses, it provides limited details and can only suggest coarse fixes involving all variables in an ML system. De- tailed decompositions quantify the importance of each variable to each term in the aggregate decomposition, which can provide deeper understanding and suggest more targeted interventions. Although parametric methods exist for conducting a full hicrarchical decomposition of an algorithm's 's performance gap at the aggregate and detailed levels, current nonparametric methods only cover parts of the hier- archy; many also require knowledge of the entire causal graph. We introduce a nonparametric hierarchical framework for explaining why the performance of an ML algorithm differs across domains, without requiring causal knowledge. Fur- thermore, we derive debiased, computationally-effic.. estimators and statistical inference procedures to construct confidence intervals for the explanations.
1 Introduction
The performance an MI. algorithm can differ across domains due to shifts the data distribution. Understanding what contributed this performance gap help teams choose the most effective corrective action(s), ranging from algorithmic modifications (e.g. model retraining) to operational fixes (e.g. updating data pipelines). Prior works have focused primarily which decompose the performance gap into that due to a shift in the marginal distribution of the input features p(X) (covariate shift (37]) and that due shift in the conditional distribution of the outcome pYY (X) (concept shift or conditional outcome shift) 5 50 30, 36 [6] However, coarse decompositions can only suggest coarse corrective actions, such as investigating data pipelines for all features. The goal of this work is to provide hicrarchical nonparametric framework that first decomposes performance gap into aggregate terms and then each aggregate term into detailed terms. This helps narrow down the features to investigate and understand how they affect the gap.
aggregate decompositions,
If one is willing to make the strong assumption that the expected loss of model is linear function of some feature set X, the problem of obtaining aggregate and detailed decompositions drastically simplifies. This is the key assumption underlying the Daxaca-Blinder (OB) decomposition, one of the most widely used frameworks in the (income and health) disparities literature (32 3 Given an ML
Corresponding author: jean. fenggfcceueeee. edu
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

AGGREGATE
DETAILED Covariate
AGGREGATE
DETAILED
2
0.96
Covariate shift
Covarlate shi Duo to 42 dist
5%
CA 0.1%)
2g
DETAILED
(t: 0.8%)
Outcome shift
0.5
Accuracy drop
Figure/ 1: (left) Proposed framework called Hicrarchical Decomposition of Performance Differences (namely, HDPD) helps to understand performance gaps of an ML algorithm between two domains. It decomposes the overall gap (say, in classification accuracy) into gaps due to shifts in the covariate versus outcome distribution (Aggregate). Then, it quantifies the importance of each feature the two components (Detailed). (right) In terms of directed acyclic graphs, aggregate decompositions describe the effect of shift interventions, for instance, on the outcome Y distribution while keeping all else fixed between domains. Detailed decompositions quantify how well can we explain those shift interventions by more targeted shifts with respect feature subsets Za alone.
model with average loss [p[{] in domains = 0 and 1 and assuming the linear loss relationship [ = JDX,, the OB framework decomposes the performance gap at the aggregate level into that due to a covariate shift (8T [X] - Eol (x)x and that due to conditional outcome shift (B1 - 30) E, [x]. That is, the former is due shift in the feature means and the latter is due to shift in the coefficients. At the detailed level, the aggregate terms corresponding to covariate and conditional outcome shifts are further broken down into the contributions from each feature, i.e. (E, [X]] - [X,)) and (B1, 3 E [X,]], respectively. Although the highly intuitive nature of the OB framework has led its widespread popularity, the terms are difficult to interpret under model misspecification. As such, this work aims to define similar hierarchical decomposition framework for explaining ML performance disparities, without making strong parametric assumptions.
There is currently no unified, nonparametric framework that obtains aggregate and detailed decompo- sitions. Instead, solutions have been proposed for parts of the hierarchy (see Table T]: nonparametric and, assuming the causal graph is known,
methods exist for the aggregate decomposition 5 30
48
4 23 However, the causal graph is unlikely to be known in high-dimensional settings and, more importantly there are no methods for simultaneously obtaining detailed decomposition of the conditional outcome shift. There are also methods that do not decompose the performance gap and instead describe distribution shifts in the variables [29] or model explanations 12 31 However, such approaches do not quantify how such shifts ultimately contribute to an ML performance gap. We make the following contributions.
detailed decompositions of the covariate shift 40
44 50 38
We introduce unified hierarchical nonparametric framework for decomposing the performance gap of an MI. algorithm (Fig leftt. Using the concept of partial distribution shifts, we generalize shifts with respect to variable subgroups to encompass not only covariate shifts but also conditional outcome shifts. We then introduce a unified scoring rule for (candidate) partial shifts, which can be used even when the causal graph is not known.
We derive novel debiased and asymptotically normal estimators for terms in the decomposition, which allow us to construct confidence intervals (CIs) with asymptotically valid coverage rates. We demonstrate the utility of our framework in real-world examples of prediction models for hospital readmission and insurance coverage. Code for reproducing experiments available at https: gigghu...b.... wtw.ttt...oooo...eeeeeeetttttttt jjfeng/HDPD
2 A unifying framework for explaining performance gaps
Notation. Consider prediction algorithm f: X' C R - [0. for binary outcomes Y across source and target domains, denoted by D = 0 and D = 1, respectively. Let the performance of be quantified terms loss function loss ( * Y). Suppose variables X can be partitioned into disjoint sets W E mm and Z X W E T miz where m = my + m2. Although our framework does not require knowing the causal ordering between variables, the interpretation is more intuitive when W is causally upstream
X x {0, 1] + RR, such as the 0-1 misclassification

---

Table 1: Comparison of HDPD to prior works that decompose ML performance gaps. The distin- guishing contribution of this work is that it unifies aggregate and detailed decompositions under nonparametric framework with uncertainty quantification.
Papers
Asgregate
Detuiled decomp. for
Does not require cousal gruph
Confidence intervals
Nonparametric
decomp.
p( J-shin
PLY]X
Zhung ct
Cai
Quintas-Martinee Mantiner
Water al
Liu at al
X0
Deddlad
Chanaca
Blinder
HDPD (this paper)
/
of Z and Y (Fig I right). Variables Z can be chosen to be mediators modifiers of the effect of the domain shift D on Y. For instance, if Z are treatment variables and W are baseline variables, one can interpret covariate shift a change in the treatment policy and an outcome shift change in the treatment effect across the two environments. In absence of any causal knowledge, another option is choose W the variables for which one would like the expected loss given W to be invariant across the two environments; this can be useful to promote fairness of ML algorithms across environments. When this invariance does not hold, the framework explains how variables Z contribute to these differences. We refer to W as baseline variables and Z as conditional covariates. Please refer Appendix C for more discussion on choosing W and Z as well a summary of notation (Table 2
Our proposed hierarchical decomposition of an ML performance gap is based on a stratification of distribution shifts into aggregate and partial shifts. At the aggregate level, the joint distribution of (W, Z, Y) can be factorized with respect the aggregate variable groups W; Z, and Y, i.e.
PD. ( )P,, (Z (Y Z),
(1)
where subscripts Du, Dz and Dy indicate the domain of that factor. An aggregate shift substitutes a factor from the source domain in ① with that from the target domain, i.e. we swap the factor from Po to P1- partial shift with respect variable subset s (or an s-partial shift) shifts a factor from the
source domain in (1] only with respect to variable subset we denote this by swapping factor from Pn to ps- (We keep the precise definition s-partial shifts purposely vague until Section 2.2 We denote expectations with respect to the joint distribution ① as Br..DD
The overall performance gap between domains, A E11 [((W, Z, Y)] Eooo [&(W, Z, Y)] can be decomposed hierarchically as follows. In the Appendix D we also discuss how this decomposition can be interpreted causally under certain conditions.
Aggregate. At the first level of the hierarchy, the framework quantifies how aggregate shifts contribute to the performance gap individually. This leads to the decomposition A Ay Az + Ay, where Ag quantifies the impact of a shift in the baseline distribution p(W). ). Az quantifies the impact of a shift in the conditional covariate distribution p( [ ()), and Ay quantifies the impact of shift in the outcome distribution pY Z). More concretely,
Au 1100 [{] Eo0o [C]
A2 E1o 1100 (f] = ET.
E.10 [e] W]
E. [E W
A 10(W
Ay Enn 1 E11- E..
[
W. Z
E.. [E W, 2
A. (W,2)
The same (or similar) aggregate decompositions have also appeared in prior works 5
30
15
50
38
Detailed. At the detailed level, each aggregate term is further broken down into variable-level attributions. The effect of each variable can be isolated using partial shifts. However, because variables can interact induce complex partial distribution shifts, we define variable importance (VI) using the Shaplcy attribution framework 41 6, [7]. which has the benefits of satisfying axiomatic properties such as fairness, monotonicity, and full attribution. Thus, given a real-valued value function that quantifies the contribution of s-partial shift to an aggregate shift for all
3

---

S f1.... m), the attribution to variable j is the average gain in value when additionally shifting with respect to J, i..
|s| m m-1) (((sU)))=(s))
*C(1." mm
Interpretation. Such VI values can help ML teams identify the underlying cause(s) for a performance gap and design targeted operational and/or algorithmic interventions. For instance. a variable with high importance to the conditional covariate shift term Az may indicate differences in the variable's missingness rates, prevalence, or selection bias across domains. If instead the variable is highly important to the conditional outcome shift term Ay, it may indicate inherent differences in the conditional distribution (i.e. effect modification), differences in measurement error or the way outcome is defined between domains, or omission of variables predictive of the outcome. Finally, note that variable importances should be viewed as relative the variables included in the framework rather than absolute importances, as one cannot include all possible explanatory variables.
To define VI values, the key question is how to define value function e that is applicable to different types of s-partial shifts, even when the causal graph is not known. It turns out that the answer is far from straightforward. The next section discusses how the value function and candidate s-partial shifts must be defined with care
2.1 Value of partial distribution shifts
When the true causal graph is known, prior works define s-partial covariate shift as the substitution of nodes s with mechanisms from the target domain and its value u(s) as the difference in the average loss, e-g. [o] 10 [ entire causal graph is often impractical, (ii) in the absence of such a graph, this value function is not a proper scoring rule and can assign high values to partial shifts that contradict the true causal graph (see Example E. for details), and (iii) oss) can be high even if the shift does not induce similar shifts in the loss the aggregate shift.
[50 38 However, this has number of limitations: (i) knowing the
Instead, we propose to evaluate candidate s-partial shifts by how closely they approximate aggregate shifts, using nonparametric extension of the traditional R² measure. In the case of conditional covariate shifts, an aggregate shift induces a performance difference of A (WW) in strata W while candidate s-partial shift induces performance difference of . (W)) E En [ W] oo The value of this s-partial shift is then the percent variation of A. explained by s0, i.e.
exp
E1. 5o () - (w)))
(8))) =1
E
(W")
Likewise, for conditional outcome shifts, an aggregate shift induces a performance difference of (W, Z) in strata (W, Z) while a candidate s- s-partial shift induces performance difference of (W, Z) [e Z] [. 2). The value of this s-partial conditional outcome shift is then defined as the percent variation of A. explained by A.... i.e.
En. (A... .A..... (W, 2) A (W, 2))2
By (3) = 1
(4)
E11- A (W, 2)]
This formulation of the value function in terms of R22 provides a unified way to score partial conditional covariate and outcome shifts, does not require knowledge of the true causal graph, and is expect the
strictly proper scoring rule under certain conditions (see Appendix E). In general, highest scoring candidate s s-partial shifts to be those that are close to the true causal graph and induce large shifts in the MI algorithm's 's loss. Finally, because some variables must be held out to define the R² measure, we cannot score partial shifts in the baseline variables W. We hope close this gap in future work.
acknowledge one caveat with this framework:
2.2 Candidate partial distribution shifts
We now present the set of candidate partial shifts considered in this work. High-level illustrations for the candidate partial shifts are given in Fig I righht top; more detailed illustrations are given in

---

of the Appendix. We emphasize that these are candidates, as the true causal graph is not known. While there are certainly other partial shifts that one may consider, many have various disadvantages. As such, we leave the investigation of other partial shifts to future work.
s-partial conditional covariate shift: Suppose 7-s is downstream of Zz- Then ps(z[w)
Pi (zalw))o (2))p(/=/22,.1)) (2) Wu et al. [88] considered a similar proposal.
s-partial conditional outcome shift: Shifting the conditional distribution of Y only with respect to variable subset Z, but not Z-5 requires care. We cannot simply define p. (Y[W, 2) as a function of effect, even in
only W and Z,- Such definition would imply that s-partial shift has settings with no shift in the conditional outcome distribution (i.e. P1 WY Z) = po (Y|W; Z))
Instead, we define an s-partial outcome shift based models commonly used in model recalibra- tion/revision [42 where the modified risk (conditional probability of Y) is function of the risk in the source domain Q == (W, Z) = PY 1 2), W, and Zg. That is, we define the shift as
Ps ( w) == pi (,. w) / s; Rs, w))p ( s/2,, q(w, 2s, k_s) - w)dē-. (5)
By defining the shifted outcome distribution solely function of Q, and Zg, any direct effect from -_, to Y is eliminated and has the desired behavior in the setting where there is no conditional outcome shift.
3 Estimation and statistical inference
Here we discuss estimation and statistical inference for the aggregate terms (Aq, A2, and Ay), the value functions (s) and by (y(s), and the Shapley- -based detailed terms Φz.j and Ør.5 for j E (0.... m2). One approach is to rely plug-in estimators, which plug in estimates of conditional means (also called outcome models) or density ratios [43,, which we collectively refer to as nuisance parameters.
For instance, one can estimate the conditional means 10 (w) [[] and (1- oo [e] using MI. and take the empirical mean of PO110 f0.00 with respect to the target domain to get a plug-in estimator for Az Ei. p. However, because estimation of the true nuisance parameters using ML typically converge a rate slower than 1-12
1/2 plug-in estimators generally fail to be consistent rate of 1/2 and cannot be used to construct CIs (25
To this end, we use the method of one-step correction from semiparametric inference to derive debiased MI. estimators [45 [7] The core idea is to subtract the first-order bias of a plug-in estimator, which requires characterizing the canonical gradient (or efficient influence function) of the estimand (25). The primary technical contribution in this section is the derivation of debiased estimators for the detailed decompositions. (Estimation and inference for the aggregate decomposition is well-studied, as the aggregate terms can be formulated average causal effects.) Due to space limitations, this section only presents estimators for the detailed decomposition of the conditional outcome shift. This estimand is particularly interesting, as its unique structure is not amenable to standard techniques for debiasing ML estimators. We refer the reader to the Appendix for derivations, pseudocode, and proofs for all the estimators.
Let PD denote the expectation with respect to domain D. For ease of exposition, suppose the number of IID observations from each domain is the same, denoted by n. We present split-sample estimators, though the results can be readily extended using cross-fitting 7 25). Let the data be randomly split into "training" and "evaluation" partitions. Let P denote the empirical average in the evaluation partition for domain D. All estimated quantities are denoted using hat notation.
Notation.
3.1 Value of s-partial conditional outcome shifts
Here we describe the high-level steps for deriving a debiased estimator for (r)s,, the value of candidate s- s-partial conditional outcome shift. The following section describes a computationally efficient procedure for combining such estimates obtain Shapley values.
Standard recipes for deriving asymptotically normal, nonparametric-efffciee estimators rely on pathwise differentiability of the estimand and analyzing its efficient influence function 25) However, by (s) is not pathwise differentiable because it is function (5). which conditions on the source risk g(w, z) equalling some value r. Taking the pathwise derivative of vy(s) requires taking derivative
5

---

of the indicator function 1 [q((w, 2) = r), which generally does not exist. Given the difficulties in deriving an asymptotically normal estimator for (s). we propose estimating a close alternative that is pathwise differentiable.
The idea is replace q in with its binned variant Jean (w, Dann
Lg((w,=)) + 2 for some B E zt, which discretizes outputs from q into B disjoint bins. As long as B is sufficiently high, the binned version of the estimand, denoted dy (s),,, is a close approximation to Uy(s), (We use B = 20 in the empirical analyses, which we believe be sufficient in practice.) The benefit of this binned variant is that the derivative of the indicator function 1 [90in(us 2)=r} is zero almost everywhere as long as observations with source risks exactly equal a bin edge have measure zero. More formally. we require the following:
Condition 3.1. Let E be the set of (W, Z) such that qW, falls precisely on some bin edge and not equal to zero or one. The set E is measure zero.
Under this condition, ty by (s) is pathwise differentiable and, using one-step correction, we derive debiased ML estimator that has the unique form of a V-statistic (this follows from the integration over "phantom" 2_, in (5)) We represent V-statistics using the operator P1,a II.. which takes the average over all pairs of observations O with replacement, i.e. 15 Σ i=1 Σ j-1 g(O,, O₃) for some function g. Calculation of this estimator and its theoretical properties are as follows.
Estimation. Using the training partition. estimate the outcome models p.. (W, Z) = E... (, Z for D = 0, 1. the shifted outcome model p... ( (W.Z) 2) = [., 21: and the density ratio mod- els 1110 ( 2)
(W, (W, 2) and π x(W, Z. Z- Quin) P1 (Z_ Ze Idin (W, 2) Qban)/Di (Z The outcome and density ratio models can be fit using ML-based regression models and probabilistic classifiers [43), respectively (see Section for details). The estimator for By aa (+) is the ratio by,bin(()
Y,n
idde where the numerator and denominator are estimated using the (
evaluation partition
& 2)² + P. & () (..0ee----1..... (W,
P1, & Z,
(,,)),,, [W,) Z,, 7,,, (W, 7, 7, Quin)
2 P1, &, Z, (..) (W, 7,, / 7,, 7.. Qusa)
(6)
60m
PI,n (a... (W, 2) -,, 21, (W, (W, 2), (E- P.. (W, 2)) 2 Pa.n (A.. ( Z) (W, Z)) (f. a (,, 2)) 2),
(7)
where (.. 2) 1 (W.)) 2)- fi.... 2). Note that the first terms in (6) and are the plug-in estimates, followed by additional terms that correct its bias.
Inference. This estimator is asymptotically normal assuming the estimators for the nuisance parame- ters converge a fast enough rate, per the following theorem.
Theorem 3.2. Suppose Condition 3.7 holds. For variable subset 8, suppose the density ratios a (W, Z,, Z=s: Quan) and (W, Z) are bounded; denominator in case of no shift ug)(()) > 0: estimator is consistent; estimators pt..o., p... and ji... converge at an 01/41 rate, and
Pilâosn qbaa)² cy(nn
(8)
Pi - (i....) - #) 0p 11/2,, Polu-- - (1..)) T110) = Op On(n-1/2)
Then the estimator (5)) ) is asymptotically normal centered at the estimand (s).
Note that the product terms in mean that the estimator converges to normal at n r/te even 1/22 one of the nuisance parameters is estimated rate slower than -1/2, Hence, it is multiply-robust to nuisance model misspecification. A convergence rate of op (/211) can be achieved by estimators in wide variety of conditions, and such assumptions are commonly used to construct debiased MI. estimators. The additional requirement in (8) that Jaan converges (n-) rate is new, but fast or even super-fast convergence rates of binned risks is achievable under suitable margin conditions 2 such Condition G.7 in the Appendix.
3.2 Shapley values
Calculating the exact Shapley value computationally intractable as it involves an exponential number of terms. However, Williamson and Feng [47 showed that calculating the exact Shapley

---

Coverage
1.0
Subset (21)
Coverage
1.0
Subset (21)
(i) Conditional covariate
0.5
8
MounChurge CS
0.0
0.0
Subset (22)
Subset (22)
Coverage
Coverage
0.0
0.5
(ii) Conditional outcome
1.0
Subset
(23)
1.0
Subset (23)
Coverage
Coverage
HaU
0.5-
0.0
een0
18000
0.0
4000
8000
18000
busion
(b)
Figure 2: (a) Coverage rates of 90% CIs for value of s-partial shifts for the conditional covariate (first column) and outcome shifts (second column) across dataset sizes n. Dashed horizontal line indicates 90% coverage rate. (b) Comparison of variable importance reported by proposed method HDPD (debiased) versus existing methods for conditional covariate and outcome shift terms.
value is unnecessary for the purposes of statistical inference. Because there is inherent uncertainty in estimates of the value functions v(s), one only needs to sample and estimate the values for enough variable subsets such that the uncertainty due estimation dominates that due to subset sampling. This leads to a drastic reduction in computation time: Williamson and Feng [47] proves that the number of subsets one needs sample only needs to be linear or super-lines -ineer in the total number of observations n. Using this result, Algorithm I Fotttieeeootiesss outlines a computationally efficient procedure for estimation and inference of the detailed decomposition.
Simulation
We now present simulations show that the proposed procedure achieves the desired coverage rates (Section 1 and illustrate how the HDPD framework provides more intuitive explanations of performance gaps (Section 4.2).. all empirical analyses, performance of the ML algorithm is quantified in terms of 001 accuracy. Below, we briefly describe the simulation settings; full details are provided in Section[[in the Appendix.
4.1 Verifying theoretical properties
We first verify that the inference procedures for the decomposition terms have CIs with coverage close to their nominal rate. We check the coverage of the aggregate decomposition as well as the value of s-partial conditional covariate and partial conditional outcome shifts for (71). (Z2), {Za). (W, Z1, Z2, Zz) are sampled from independent normal distributions with different means in source and target, while Y is simulated from logistic regression models with different coefficients. CIs for the debiased ML estimator converge to the nominal 90% coverage rate with increasing sample size, whereas those for the naïve plug-in estimator do not (Fig 2a and Fig 6).
4.2 Comparing explanations
We now compare the proposed definitions for the detailed decomposition with existing methods. For the detailed decomposition due to conditional covariate shift, the comparators are:
MeanChange Tests for difference in means for each feature. Defines importance as 1- p-value. Daxaca-Blinden Fits a linear model of the aagit-transformed expected loss with respect to Z in the source domain. Defines importance of Z, as coefficient multiplied by the difference the
means of Z; 12 3
WuShift 48] Defines importance of subset s as change in overall performance due to s-partial conditional covariate shifts. Applies Shapley framework to obtain VIs.
7

---

(a) Readmission risk (General-Heene Failure)
(b) Insurance coverage (NE-
Aggregate
Aggregate
HDPD deblased
HDPD
HDPD plug-in
Digpness coder
HDFD plug-in
Herith, work eh
-0115
8:00
odt
Decompootion oike
Decompast lake
Detailed: Conditional covariate
Detailed: Conditional outcome
Drog indueeditee ne and coedition Moroll is suletance use dlander ione
"Toal personts income
Resginatrry signs s-mpptors
Wever monmed or under 15 yeart
HDPD deolased
Orteeniiig
Aedorend
Uneilooe the rerolomi inotrrent
Mavca sihestid part rot Excotare iil. dieass
nobetecrralliie whiee complicatinn
Sapmpteru
HDPD plug-in
MeanChange
WUShiit
Drppossent
Eroloyenn ptorpleeed
Employmee stava arred
coucabornl
saato lorein
Fandomoorrsttcc
HDFD plug
HDPD debiesed
0.2
0.2
0.4
08 Value
04
0.0
0.23
210
G15 Kala
3.00
C20
Figure 3: Aggregate and detailed decompositions for performance gaps of (a) model predicting readmission risk across patient populations and (b) model predicting insurance coverage across US states. subset of VI estimates is shown; see full list in Section D in the Appendix.
For the detailed decomposition due to conditional outcome shifts, we compare against:
ParamerricChange Fits a logistic model for Y with interaction terms between domain and Z. Defines importance of Z; as the coefficient of its interaction term. ParametricAce Same as Parametri cChange but models the 0-1 loss rather than Y. RandomForestAce Compares VI of random forest models trained on data from both domains with input features D, Z, and W to predict the 0-1 loss.
Oaxaca-Blindn inder:: Fits linear models for the logit-transfommed expected loss in each domain. Defines importance of Z; as its mean in the target domain multiplied by the difference in its coefficients across domains.
Although the proposed method may agree with these other methods on the top features in certain data settings, we highlight important situations where the methods differ.
Conditional covariate. (Fig 2b i)) We simulate (W, 21) from a standard normal distribution,
Zz from mixture of two Gaussians whose means depend on the value of Z1 (i.e. Z - Z2). and Y from a logistic regression model depending on (W, Z1, Zz). We induce a shift from the source domain to the target domain by shifting only the distribution of Z,, so that (Z))) = Po(Za|Z, W) )Pi (Z, WW). Only the proposed estimator correctly recovers that Z, more important than Zz, as the -pparill conditional covariate shift explains all the variation in performance gaps across strata W (i.e. the corresponding R2- value function uz((1)) is equal to 1). The other methods incorrectly assign higher importance to Zz. MeanChange only measures shifts but not loss due to shifts, Daxaca-Blindes Blinder uses a misspecified linear model, and WuShift estimates the performance change due to hypothesized s- s-partial shifts but does not check if the partial shifts are good explanations in the first place.
Conditional outcome. (Fig 2b(ii) W and Z E R are simulated from the same distribution in both domains. Y is generated from logistic regression model with coefficients for (W, ZW Zi.... Z,) as (0.5, 0.5, 1, 0.3, 0.3) in the source and (0.5, 0.3, 1, 1.3, -0.1) in the target. Interestingly, none of the methods have the same ranking of the features. Parametri cChange identifies Zι as having the largest shift on the logit scale, but this does not mean that it is the most important explanation for changes in the loss. According to our decomposition framework. Z3 is actually the most important for explaining changes in model performance due to outcome shifts. Laxaca-Blinder ParametricAce and RandomFor ooeeroorettcoo have odd behavior. Oaxaca- Seeraca-Blindee Blinder assigns Z3 second assigns Zz the highest importance), likely because
to the lowest importance and Parametricha they misspecify the outcome models. RandomForestae likely ranks Zz highly because its VI values quantify which variables are good predictors of performance, not performance shift.
more objective evaluation is to compare the performance of fixes based on the different explanations. To this end, we re-fit the MI. algorithm in the target domain with respect to input features Q, W. and the top variables Z, from each explanation. We find that model revisions based on the proposed method achieve the highest performance gain (Table Bln in Appendix).
8

---

5 Real-world data case studies
We now demonstrate applicability of the framework on two datasets with naturally-occurrin shifts.
Hospital readmission. Using electronic health record data from the Zuckerberg San Francisco General Hospital, we analyzed performance of a Gradient Boosted Tree (GBT) trained on the general patient population (source) to predict 30-day readmission risk but applied to patients diagnosed with heart failure (HF, target). Features include 4 demographic variables (W) and 16 diagnosis codes (2). Each domain supplied n 3750 observations from which we keep 20% in the evaluation partition.
Model accuracy drops from 70% to 53% in HF population. From the aggregate decompositions (Fig we observe that the drop is mainly due to covariate shift. If one performed the standard check to see which variables significantly changed in their mean value (MeanChange), then one would find significant shift in nearly every variable. Little support is offered to identify main drivers of the performance drop. In contrast, the detailed decomposition from the proposed framework estimates diagnoses "Drug-induced toxic-related condition" and "Mental & substance use disorder in remission" as having the highest estimated contributions to the conditional covariate shift, and most other variables having little no contribution. Upon discussion with clinicians from this hospital, differences in the top two diagnoses may be explained by (i) substance use being a major cause of HF this hospital, with over eighty percent of its HF patients reporting current or prior substance use, and (ii) substance use and mental health disorders often occurring simultaneously in this HF patient population. Based on these findings, closing the performance gap may require a mixture of both operational (e.g. care programs centered around substance use) and algorithmic interventions (e.g. reweighting data with respect to the top two features). Finally, CIs from the debiased ML procedure provide valuable information on the uncertainty of the estimates and highlight, for instance, that more data is necessary to determine the true ordering between the top two features. In contrast, existing methods do not provide (asymptotically valid) Cls.
ACS Public Coverage. We analyze a neural network trained to predict whether person has public health insurance using data from Nebraska in the American Community Survey (source, n 3000), applied to data from Louisiana (target, n 6000). Baseline variables include 3 demographics (sex, age, race), and covariates Z include 31 variables related to health conditions, employment, marital status, citizenship status, and education.
Model accuracy drops from 84% to 66% across the two states. The main driver is the shift in the outcome distribution per the aggregate decomposition (Fig(3p) and the most important contributor to the outcome shift is annual income, perhaps due to differences in cost of living across the two states. Income is significantly more important than all the other variables; the ranking between the remaining variables is unclear. In comparing the performance of targeted model revisions, we find that revising the model based on top variables identified by the proposed procedure leads AUCs that are better or as good as those based on RandomFond ForeatAceeeeooooo (Table 4 in the Appendix).
6 Prior work
Describing distribution shifts. This line of work focuses on detecting and localizing which distribu- tions shift between datasets [29] 39]. Budhathoki et al. [4] identify the main variables contributing to distribution shift via a Shapley framework, Kulinski and Inouye (28) fits interpretable optimal transport maps, and Liu et al. 30 finds the region with the largest shift in the conditional out- come distribution. However, these works do not quantify how these shifts contribute changes in performance, the metric of practical importance.
Explaining loss differences across subpopulations. Understanding differences in model perfor- mance across subpopulations in a single dataset is similar to understanding differences in model performance across datasets, but the focus is typically to find subpopulations with poor performance rather than to explain how distribution shifts contributed to the performance change. Existing ap- and structured representations of the
proaches include slice discovery methods B5 22 10 subpopulation using e.g. Euclidean balls [
[3]
Attributing performance changes. Prior works have described similar aggregate decompositions of the performance change into covariate and conditional outcome shift components (6] To provide more granular explanations of performance shifts, existing works on causal attribution 50 38 and

---

mediation analysis 44 quantify the importance of shifts in each variable assuming the causal graph is correctly specified; covariate shifts restricted to variable subsets assuming that the partial shifts follow particular structure [8]; and conditional shifts in each variable assuming a parametric model [.]] However, the strong assumptions made by these methods make them difficult to apply practice, and model misspecification can lead to unintuitive interpretations. Furthermore, such methods do not provide hierarchical decompositions, i.c. VIs for each type of shift. Decomposition methods such as
similarly make strong parametric assumptions [32 3 16 14 49 [S]. which is inappropriate for the complex data settings ML. In addition, there no unifying nonparametric framework for decomposing both covariate and outcome shifts, and many methods do not output Cls, which is important when the amount of labeled data from a given domain is limited. A summary of how the proposed framework compares against prior works is shown in Table
7 Discussion
ML algorithms regularly encounter distribution shifts in practice, leading to drops in performance. We present novel framework that helps ML developers and deployment teams build more nuanced understanding of the shifts. Compared to past work, the approach provides nonparametric hierarchical framework for decomposing both conditional covariate and outcome shifts, does not require fine-grained knowledge of the causal relationship between variables, and quantifies the uncertainty of the estimates by constructing confidence intervals. We present real-world case studies to demonstrate how this framework can help diagnose performance drops and guide corrective actions. This framework requires overlapping support the covariates, which may not always be applicable in practice. In such cases, one solution is to restrict to the common support [5].
Important extensions of this work include decompositions of more complex measures of model performance such as AUC and analyzing other factorizations of the data distribution (c.g. label/prior shifts (27)) For unstructured data (e.g. image and text), the current framework can be applied to low-dimensional embeddings or by extracting interpretable concepts [26]; more work is needed to directly analyze unstructured data. Finally, while the focus of this work is to interpret performance gaps, future work may take this work one step further to design optimal interventions for closing the performance gap.
Acknowledgments and Disclosure of Funding
We would like to thank Lucas Zier for supplying the dataset from the Zuckerberg San Francisco General Hospital and providing their clinical expertise to interpret results. We are grateful to Nicholas Petrick, Berkman Sahiner, Gene Pennello, Mi-Ok Kim, Romain Pirracchio, Julian Hong, Avni Kothari, and the anonymous reviewers for helpful feedback. This work was funded through a Patient-Centered Outcomes Research Institute@ (PCORI@) Award (ME-2022C1-2561) presented in this work are solely the responsibility of the author(s) and do not necessarily represent the views of the PCORI@, its Board of Governors or Methodology Committee. The contents are those of the author(s) and do not necessarily represent the official views of, nor an endorsement, by FDA/HHS, or the U.S. Government.
The views
References
[1] Alnur Ali, Maxime Cauchois, and John C. Duchi. The lifecycle of statistical model: Model failure detection, identification, and refitting, 2022. URL htt.:tts///axxi.ogg//aaaa 04166
[2] lean-Yves, Audibert and Alexandre B Tsybakov. Fast learning rates for plug-in classifiers. Ann. Stat., 35(2):608-61) -633, April 2007.
[3] Alan S. Blinder. Wage discrimination: Reduced form and structural estimates. The Journal of Human Resources, 8(4):4364455 1973. ISSN 0022166X. URL http:://www.jjstttt org/
stable/144855
[4] Kailash Budhathoki, Dominik Janzing, Patrick Bloebaum, and Hoiyi Ng. Why did the dis- tribution change? In Arindam Banerjee and Kenji Fukumizu, editors, Proceedings of The
10

---

24th International Conference on Artificial Intelligence and Statistics, volume 130 of Pro- ceedings of Machine Learning Research, pages 1674. PMLR, 13-15 Apr 2021. URL htttps://proceediin httppp/cccii......ppppppeaaaaaaa http./ppeeee//1///uuaaaaattonooa
mlr
html
[5] Tiffany Tianhui Cai, Hongseok Namkoong, and Steve Yadlowsky. Diagmosing model perfor- mance under distribution shift. March 2023, URL http::ttttt.ii.../////// http//bs//3...... 02011
abs
[6] A Charnes, Golany, M Keane, and J Rousscau. Extremal principle solutions of games in characteristic function form: Core, chebychev and shapley value generalizations. In Jati K Sengupta and Gopal K Kadekodi, editors, Econometrics of Planning and Efficiency, pages 123 133. Springer Netherlands, Dordrecht, 1988.
[7] Victor Chernozhukov, Denis Chetverikov, Mert Demirer, Esther Duflo, Christian Hansen, Whitney Newey, and James Robins. Double/debiased machine learning for treatment and structural parameters. Econom. J. 21(1):C1-C68 February 2018.
[8] Juan Correa and Elias Bareinboim. A calculus for stochastic Interventions:Causal effect identification and surrogate experiments. AAAI, 34(06):10093 April 2020. URL
https://ojs.. http///sssaaaa..... http:///wiw...ppp///aaaaaaaaaaaa
[9] P Dawid. Influence diagrams for causal modelling and inference. Int. Stat. Rev, 70 (2):161-189, August 2002. URL
www.w. wwwp:wwc.weooooooo..ooooooooooo. www.oooooo.ooooooooooooo.ooi
j 7.1751-5822 2002. tb00354.x
[10] Greg d'Eon, Jason Eon, James R. Wright, and Kevin Leyton-Brown. The spotlight: A general method for discovering systematic errors in deep learning models. In 2022 ACM Conference on Fairness, Accountability, and Transparency, FAceT '22, page 1962-1981, New York, NY, USA, 2022. Association for Computing Machinery. ISBN 4701453335 doi: 10.1 10.1145/3531 146.35332444 URL /ttttp/://bol..org//0001///////
3533240
[11] Lori E Dodd and Margaret Sullivan Pepe. Semiparametric regression for the area under the receiver operating characteristic curve. J. Am. Stat. Assoc., 98(46)):409 417, 2003.
[12] Christopher Duckworth, Francis P Chmiel, Dan K Burns. Zlatko D Zlatev, Neil M White, Thomas W V Daniels, Michael Kiuber, and Michael Boniface, Using explainable machine learning characterise data drift and detect emergent health risks for emergency department admissions during COVID-19, Sci. Rep., 1 111)2330 November 2021.
[13] Sabri Eyuboglu, Maya Varma, Khaled Kamal Saab, Jean-Benoir Delbrouck, Christopher Lee- Messer, Jared Dunnmon, James Zou, and Christopher Re. Domino: Discovering systematic errors with gross-modall embeddings. In International Conference on Learning Representations, 2022. URL htttts://oppeeeeeeeeeeeee httpe/vwwwwwnwtffeeeeeee
JO
[14] Robert W Fairlie. An extension of the blinder-oaxaca decomposition technique logit and probit models. Journal of economic and social measurement, 30(4):305 -16, 2005.
[15] Sergio P. Firpo, Nicole M. Fortin, and Thomas Lemieux. Decomposing wage distributions using recentered influence function regressions. Econometrics, 6(2), 2018. ISSN 2225-1146. doi: (0.330/ccommmmetris6000022 URL https::tttwtwtttett mdpi com httpp/ppp....p.p111111////////// 200///22222111/////222
[16] Nicole Fortin, Thomas Lemieux, and Sergio Firpo. Chapter decomposition methods in economics. volume 4 of Handbook of Labor Economics, pages 1-102. Elsevier, 2011. doi:
https://o..orr//...00///0000o 7218(1 2000477-2. URL /ttp:///00../..tat////////rrrro htttps://ww..siieneedieeee com/
[17] Amirata Ghorbani and James Zou. Data shapley: Equitable valuation of data for machine learning. Kamalika Chaudhuri and Ruslan Salakhutdinov, editors, Proceedings of the 36th International Conference Machine Learning, volume 97 of Proceedings of Machine Learning Research, pages 2242-2251. PMLR, 09-15 Jun 2019. URL httpt:: htttps:/prroceedingg mlr
press/v97/en htt.///ghobaaiiii 19w.ni199..... html
[18] Tilmann Gneiting and Adrian E Raftery. Strictly proper scoring rules, prediction, and estimation. Journal of the American Statistical Association, 102(477):35 2007. doi: 10.1 198/ 01621450600000
UtL/nttss: /octps::/doi.ooo 0111:111111111000111110000000000
11

---

[19] Chuan Guo, Geoff Pleiss, Yu Sun, and Kilian Q Weinberger. On calibration of modern neural networks. International Conference on Machine Learning, 70:1321-1330, 2017.
[20] Oliver Hines, Karla Diaz-Ordaz, and Stijn Vansteelandt. Variable importance measures for heterogeneous causal effects. April 2022.
[21] John W Jackson. Meaningful causal decompositions in health equity research: definition, identification, and estimation through a weighting framework. Epidemiology, 32(2):282-290, 2021.
[22] Saachi Jain, Hannah Lawrence, Ankur Moitra, and Aleksander Madry. failures as directions in latent space. In The Eleventh International Conference on Learning Representations, 2023. URL httpp:://opeeeeviiitie wwwriiewwweefffrrui www./eruuumr 11-99mmBVPd-9
Distilling model
[23] Yonghan Jung, Shiva Kasiviswanathan, Jin Tian, Dominik Janzing, Patrick Bloebaum, and Elias Bareinboim. On measuring causal contributions via do-interventions. In Kamalika Chaudhuri, Stefanie Jegelka, Le Song, Csaba Szepesvari, Gang Niu, and Sivan Sabato, editors, Proceedings of the 39th International Conference on Machine Learning, volume 162 of Proceedings of Machine Learning Research, pages 10476-10501. PMLR, 17-23 Jul 2022. URL https //proceedings.nut mlr wwwpppeesssepp..p..e prres/vv622/ w162/jung2222 html
[24] Joseph D. Y. Kang and Joseph L. Schafer. Demystifying Double Robustness: A Comparison of Alternative Strategies for Estimating a Population Mean from Incomplete Data. Statistical Science, 22(4):523 539, 2007. doi: 10. 10.1214/07-SSS22 PRL.htts:://do. wrw.ooggr/.0 1rgg/10.5 1214/
07-STS227
[25] Edward H Kennedy. Semiparametric doubly robust targeted double machine learning: review. March 2022.
[26] Been Kim, Martin Wattenberg, Justin Gilmer, Carrie Cai, James Wexler, Fernanda Viegas, and Rory sayres. Interpretability beyond feature attribution: Quantitative testing with concept activation vectors (TCAV). In Jennifer Dy and Andreas Krause, editors, Proceedings of the 35th International Conference Machine Learning, volume 80 of Proceedings of Machine Learning Research, pages 2668-2677. PMLR, 10-15 Jul 2018. URL httt::ts:////oceedding html
mlr. pwwps//v0/kiiia
[27] Wouter M. Kouw and Marco Loog. An introduction domain adaptation and transfer learning, 2019.
(28) Sean Kulinski and David I. Inouye. Towards explaining distribution shifts. In Andreas Krause, Emma Brunskill, Kyunghyun Cho, Barbara Engelhardt, Sivan Sabato, and Jonathan Scarlett, editors, Proceedings of the 40th International Conference on Machine Learning, volume 202 of Proceedings of Machine Learning Research, pages 17931-17952. PMLR, 23-29 Jul 2023. mlr. wtw.eesss..rrrr.sseeeeeeee w202/kulinskkaaa
URL httppsss//pccceeeiiigg
html
[29] Sean Kulinski, Saurabh Bagchi, and David Inouye. Feature shift detection: Localizing which features have shifted via conditional distribution tests. In H. Larochelle, M. Ranzato, R. Had- sell, M.F. Balcan, and H. Lin, editors, Advances in Neural Information Processing Systems, volume 33, pages 19533, Curran Associates, Inc., 2020. URL httpttts///pooceedinng Paper. pdf
[30] Jiashuo Liu, Tianyu Wang, Peng Cui, and Hongseok Namkoong. On the need for language describing distribution shifts: Illustrations on tabular datasets. July 2023.
[31] Scott M Lundberg and Su-In Lee. A unified approach to interpreting model predictions. In
I. Guyon, U. Von Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett, editors, Advances in Neural Information Processing Systems, volume 30. Curran Associates, Inc., 2017. URL http:::tt//////.t. URL/ http:p/pweeeaaaae...eeoooo.ppppp http//pper..lleepppp//////////
pdf
[32] Ronald Oaxaca. Male-female wage differentials in urban labor markets. International Economic Review, 14(3):693-09,, 1973. ISSN 00206598, 14682354. URL, http:: h/tb:://www //www. jstor org/ stable/ 22259811
12

---

[33] F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel, Prettenhofer, R. Weiss, V. Dubourg, J. Vanderplas, A. Passos, D. Cournapeau, M. Brucher, M. Perrot, and E. Duchesnay. Scikit-learn: Machine learning in Python. Journal of Machine Learning Research, 12:2825-2830, 2011.
[34] John Platt. Probabilisticc outputs for support vector machines and comparisons regularized likelihood methods. Advances in large margin classifiers, 10(3):61-74, 1999.
[35] Gregory Plumb, Nari Johnson, Angel Cabrera, and Ameet Talwalkar. Towards a more rigorous science of blindspot discovery in image classification models. Transactions on Machine Learning Research, 2023. ISSN 2835-8856. URL MaDvbLaBiF Expert Certification.
[36] Hongxiang Qiu, Eric Tchetgen Tchetgen, and Edgar Dobriban. Efficient and multiply robust risk estimation under general forms of dataset shift. June 2023.
[37] Joaquin Sarthonero-Candell Masashi Sugiyama, Anton Schwaighofer, and Neil Lawrence. Dataset shift in machine learning. Mit Press, 2022.
[38] Victor Quintas-Martinge Mohammad Taha Bahadori, Eduardo Santiago, Jeff Mu, Dominik Janzing, and David Heckerman. Multiply-robust causal change attribution, 2024.
[39] Stephan Rabanser, Stephan Günnemann, and Zachary Lipton. Failing loudly: An empirical study of methods for detecting dataset shift. H. Wallach, H. Larochelle. A. Beygelzimer, F. d'Alché-Buc, E. Fox, and R. Garnett, editors, Advances in Neural Information Processing Systems, volume 32. Curran Associates, Inc., 2019. Utttttts::// htttt::://prrceeddigng ing..uriing. cc/paper/2019) 20tp//201/////// wttpo///te/ee///e//eeeeeeeeeeeee B488e66006655e000508880 fad5 wdw.17051166..... Paper . pdf
[40] James M Robins, Thomas S Richardson, and Ilya Shpitser. An interventionist approach to mediation analysis. In Probabilistic and Causal Inference: The Works of Judea Pearl. Association for Computing Machinery, August 2020. URL http:://aaa/i org/ hbsg//bs//// 2008
06019
[41] L Shapley, 17. a value for n-person games. Harold William Kuhn and Albert William Tucker, editors, Contributions to the Theory of Games (AM-28), Volume II, pages 307-318. Princeton University Press, Princeton, December 1953.
[42] Ewout W Steyerberg. Clinical Prediction Models: A Practical Approach Development, Validation, and Updating. Springer, New York, NY, 2009.
[43] Masash Sugiyama, Shinichi Nakajima, Hisashi Kashima, Paul Buenau, and Motoaki Kawan- abe. Direct importance estimation with model selection and its application covariate shift adaptation. Adv. Neural Inf. Process. Syst. 2007.
[44] Eric J. Tehetgen Tchetgen and llya Shpitser. Semiparametric theory for causal mediation analysis: Efficiency bounds, multiple robustness and sensitivity analysis. The Annals of Statistics, 40(3):1816 1845, 2012. doi: 10.1214/12-AOS900 URL httthhtt////do. 1214/12-A0S99
[45] Anastasios A Tsiatis. Semiparametric Theory and Missing Data. Springer New York, 2006.
[46] A W van der Vaart. Asymptotic Statistics. Cambridge University Press, October 1998.
[47] Brian D Williamson and Jean Feng. Efficient nonparametric statistical inference on population feature importance using shapley values. International ContreeeeeenncCCn Machine Learning, 2020.
URL
pdf
[48] Eric Wu, Kevin Wu, and James Zou. Explaining medical AI performance disparities across sites with confounder shapley value analysis. November 2021. URL httpp:/arxivv.or
abs/2111.0811
[49] Myeong-Su Yun. Decomposing differences in the first moment. Economics Letters, 82(2): 275-280, 2004. ISSN 0765- doi: http:://oo.aay////oooooocceeeeee
URL.
13

---

[50] Haoran Zhang, Harvineet Singh, Marzyeh Ghassemi, and Shalmali Joshi. "Why did the model fail?": Attributing model performance changes to distribution shifts. In Andreas Krause, Emma Brunskill, Kyunghyun Cho, Barbara Engelhardt, Sivan Sabato, and Jonathan Scarlett, editors, Proceedings of the 40th International Conference on Machine Learning, volume 202 PMLR, 23-29 Jul 2023.
of Proceedings of Machine Learning Research, pages mlr, wtt.m..pp.....22// 2atpa///222222ahg2222222 zhang html
URL
14

---

A
Appendix
Contents of the Appendix are as follows.
Table collects all the notation used for reference.
Section B discusses broader impacts of the work.
Algorithms and provide the steps required for computing the aggregate and detailed decomposi- tion respectively. Detailed decompositions require computing the value of s-partial conditional outcome and conditional covariate shifts which is described in Agsorithmsaannd
Section C discusses considerations for choosing baseline variables W and conditional covariates Z.
Section D describes a causal interpretation of the aggregate and detailed decompositions effects of stochastic (or shift) interventions on structural causal model.
Section B explain why value functions in prior work give unintuitive attribution and that the R2-based value functions are proper scoring rules.
Section describes the estimation and inference for aggregate decomposition and detailed decom- position of conditional covariate shift.
Section provides the derivations of the results.
Sections and describe the implementation and simulation details.
Section J provides additional details on the two real world datasets and results.
Broader impacts
This work presents a method for understanding failures of ML algorithms when they are deployed in settings populations different from the ones in development datasets. Therefore, the work can be used to suggest ways of improving the algorithms or mitigating their harms. The method is generally applicable tabular data settings for any classification algorithm, hence, it can potentially be applied across multiple domains where ML is used including medicine, finance, and online commerce.
Care must be taken while interpreting the results. As usual, assumptions underlying the decompo- sitions such as the coarse causal ordering between the variables W, Z, and should be validated through domain knowledge.
Algorithm 1 Aggregate decompositions into baseline, conditional covariate, and conditional outcome shifts
Input: Source and target data {(W!(d) -
) for d E f0, 1], loss function E(W, Z, Y; f). Output: Performance change due to baseline, conditional covariate, and conditional outcome shifts Ag. Az, Ay-
Split source and target data into training Tr and evaluation Ev partitions. Let Fx be the total number data points in the Ev partition.
2 Fit nuisance parameters V/u, the defined in Section G.1 on the Tr partition as outlined in Section H.I
Estimate Au, Az, Ay using fitted nuisance parameters the Ev partitions following the equations in Section
Estimate variance of influence functions (v, w, y: fm)) vz(d, w, 9: fm)), and shy(d, w, y; fm) as defined in
and 25). respectively.
Compute a-leeel confidence intervals ÂØ =1 v (ar w, 2, v: Br for E Y.
where is the inverse CDF of the standard normal distribution. return Av. Az. Ay and confidence intervals
15

---

Table 2: Notation
Symbol
Meaning
W,Z,Y
Variables: Bascline, Conditional covariates, Outcome Prediction model being analyzed
f
E(W, Z, Y) or
Loss function c.g. 0-1 loss
D= and
Indicators for source and target domain
Po.P1
Probability density (or mass) function for the two domains D = 0, Probability density (or mass) function when only variable subset shifts from source to target
Ps
Expectation over the distribution pn, (W)pn) Jpp, (2,),,
Q (W, 7)
Source domain risk at W, Z, i.e. Po = =1 Z)
and Ev
Training dataset used fit models and evaluation dataset used to compute decompositions
Φr, and Φr
Shapley values for variable in the detailed decomposition of conditional covariate and outcome shifts Value of subset for s-partial conditional covariate shift and s- s-partial outcome shift
Uz (s) and (s())
u (8) and (s)
Numerator and denominator of the ratio defined in the value a subset Outcome models for the conditional expectation of the loss across differ- ent settings
Models f..
Models P
Density ratio models for feature densities across datasets Notation for expectation sample average over source and target data the evaluation dataset Influence function defined in the linear approximation of an estimand, see e.g.
Po,n and Pi, (d, w. 2,
Algorithm 2 VALUUECONDITONNA.OOCTCCMEE Value for s-partial conditional outcome shift for subset
Input: Training Tr and evaluation Ev partitions of source and target data, subset of variables Output: Value for s-partial conditional outcome shift for subset s.
Fit nuisance parameters 1y,,", oo defined in Sections G.3 on the Tr partitions outlined in H.2
Estimate (s) by (que( (s) where (s)) estimated using (6) and édea is estimated using on the Ev partition.
Estimate variance of influence function Vry,ii,. z, yú oy.. (den) as defined in 74
Compute a-level confidence interval as (x()) + 21 21-0/2// var www.WWy,ssa.... (d, W, R, yi 17 19 ( F
return (y(s) and confidence interval
Algorithm VALUEECONDIIONALLOOWARRE.EE Value for s-partial conditional covariate shift for subset s
Input: Training Tr and evaluation Ev partitions of source and target data, subset of variables Output: Value for s-partial conditional covariate shift for subset s. Fit nuisance parameters nun defined in Sections G.2 on the Tr partition, outlined in H.3
Estimate Uz(s) by (gnn(s) (s) (s)) using 21 the Ev partition.
Estimate variance of influence function 1/,,sd,, (d, w, 2, yj ( defined in 41
Compute a-level confidence interval as (2(5) + Z1-a/2 var (wz, (d, w, 2, y 12.6
n/
$ return 0z(s) and confidence interval
16

---

Algorithm 4 Detailed decomposition for conditional outcome and covariate shift
y for d E {0, 1}, loss function <(W, Z, f)) Output: Detailed decomposition for conditional outcome or covariate shift, {Ør : = 0, ... m2 or {ozzá j = 1, ... m2).
Input: Source and target data {(W/d) Z y((1)
Y E IR.
Split source and target data into training Tr and evaluation Ev partitions. Let n be the total number of data points in the Ev partition.
Subsample Lym subsets from = (1,'* , with respect to Shapley weights, including 0 and 2, denoted 81."11
3 Estimate (8) <- VALUECOONDITOXAAIOTTCCOMEEE and 12(()) <- VALJUCCONDTIONA.CCO VARIATE(S) for s 81,"
8k.
Get estimated Shapley values {dyj} } and { by solving constrained linear regression problems in (7) in Williamson and Feng [47] with value functions (y(s) and Uz(s), respectively.
5 Compute confidence intervals based on the influence functions defined in Theorem in Williamson and Feng 47).
return Shapley values (ór3 j 0,... , m2) and {0z. :j ..... m2) and confidence intervals
C Considerations for the choice of W and Z variables
We suppose that variables X are partitioned into baseline variables W and conditional covariates Z. We suggest selecting Z to be the variables that may act as mediators of the environment shift and/or variables whose associations with Y are likely to be modified by the environment shift (i.c. effect modification). This selection can be chosen based on high-level causal graph, where W are variables known to be upstream of Z. For instance, if Z are treatment variables and W are baseline variables, one can interpret a covariate shifts as change in the treatment policy and an outcome shift as change in the treatment effect across the two environments.
In the absence of any prior knowledge, another option is choose W the variables for which one would like the expected loss given W to be invariant across the two environments; this can be useful promote fairness of ML algorithms across environments. When this invariance does hold, the proposed framework explains how variables Z contribute to these differences, which can inform efforts to eliminate performance gaps. This last option is similar how variables are typically chosen in disparity analyses (21] For instance, to understand why income differs between males and females controlling for age, one would domain D) gender. W age, and Z as variables that may explain this disparity (e.g. marital status, employment status).
In general, including more variables in W and Z to explain the performance difference is preferable. Nevertheless, there are tradeoffs. For instance, including more variables in W leads to higher variance of A 10 (W), Z for explaining its variability. On the other hand, when more variables are assigned to W, the performance gap with respect to W (A (A.))(W)) is more complex function. Thus we may have more uncertainty in our estimate of (W), which may lead to wider confidence intervals for the variable importance values.
allows one better distinguish the relative importance of variables in
D Causal interpretation of aggregate and detailed decompositions
Partial distribution shifts that we define in the framework can be equivalently described as stochastic (or shift) interventions in structural causal model (SCM) respecting a causal directed acyelic graph (DAG) [8. To represent an intervention on variable X, we use regime indicator oxx which means that the conditional probability distribution for X in the SCM has been updated to new one (9.
Suppose that the source and target data (W, Y) are generated by SCMs respecting the same DAG G in Figure 4 with no unmeasured confounders. Intervening variable in source SCM sets its
17

---

AGGREGATE
DETALED
Figure 4: Decomposition framework for explaining the performance gap from source to target domain, visualized through causal directed acyclic graphs. Aggregate decompositions describe the incremental effect of stochastic interventions on each aggregate variable's ' distribution at the source with that in the target, indicated by regime indicators dy, oz. and Øy. Detailed decompositions quantify how well candidate partial distribution shifts explain the variability of performance gaps across strata. The candidate partial shifts considered in this work are shown in the DAGs on the right. An s- s-partial conditional covariate shift is a stochastic intervention on variable subset Z,. An s-partial conditional outcome shift a stochastic intervention on variable Y, which the conditional outcome distribution fine-tunes the risk in the source domain (indicated by the additional node Q -Po(Y 1|W, Z)) with respect to Z..
conditional distribution to the corresponding one in the target SCM. Under the assumption of no unmeasured confounders, we have
Po (o II PV))lpag(VV::0x
II (Vilpag(VV)) Vie
(10)
VeVi
for any variable set V where pag denotes parents in g. Therefore, the target data is obtained by intervening on all the variables Po (W, Z, Y; dy. oz σy) = pi (W, Z, Y). Expectation E is taken over the source distribution or its shifted version based on the specified regime indicator.
Aggregate decompositions can then be written as causal effect of intervening on W, Z, Y incremen- tally as follows
Ag = E[E]av] E[e]
Az E[Elau, Øz] - B[(lov]
Ay = E[flov, oz, σy] E[ ow, Øz]
Due the factorization (10), the above are equivalent aggregate decompositions presented in Section 2
For detailed decomposition of the conditional covariate shift, assume the causal DAG in Figure
4 with Zg + Z_, for variable subset s. The s-partial conditional covariate shift is represented and, under the factorization (TO), is equivalent to the considered in Section 2.2
by (oV Thus, the performance difference A (W)) can be equivalently written as E[ (2 R[& W) and A 10 (W) E[e [[([W]. The value function is then
E (A s0 (W) Aooo :oy
oz(s)
(W))
The s-partial conditional outcome shift corresponds to stochastic intervention that changes the
distribution Y to (5). Denoting it as (V;V; oy.,, the performance difference A... (W, Z) can be
equivalently written as B[qWW 7; oy2, B 2] and A., (W, 7) E W. 2: E Z]. The value function is then
W. 7) A. (W, 2)) ou, oz E A
(y(3) =
(11)
2 (W, Z);
18

---

E Proper scoring rules for partial distribution shift
Prior works have considered scoring rules for partial distribution shifts in terms of their change in the average loss [8]. For instance, for conditional covariate shifts, prior works have considered scoring rules of the form
E - E100 = / e(Y, (P<())))) - Po(z|w))
P1 (1 (w)dzdw.
(12)
However, in the absence of detailed causal graph, show that this can lead to unintuitive attributions. Consider the following counterexample. We drop W from the data example for simplicity of exposition.
Example E.I. Consider the following data- data-generating process with random variables Z1 and Zz, real-valued outcome Y, and loss function as squared error ( = (f(Z,, Z2) - r)
Z1 N (D +111)
(13)
Zz = Z1
(14)
( 1)
(15)
== + Z2 + ( - 0}
(16)
where D = and D correspond to the source and target domains, respectively. Suppose the ML model is the optimal model for minimizing the expected loss in the source domain, i.e.
f(,,, Z2) = Zi + Zz. Consider the following candidate partial distribution shifts for explaining the performance change, where Option hypothesizes Z causes Zz and Option 2 hypothesizes Za
causes Z1.
(Option 1) For 5 =
(Option 2) For = (2), (2 Z2)
Per the given data-generating process, the partial shift in Option exactly corresponds to the true (aggregate) dataset shift; thus would expect it to have higher value than Option 2. Nevertheless, the MSE resulting from Option 2's shift is 0.5 (the marginal distribution of Z, under Option is symmetric around 0). In contrast, the MSE resulting from Option I's shift is the probability that standard normal variable is less than -1, which is 0.159.
In Example E.1 scoring rule (12 assigns a higher value to a partial shift that contradicts the true causal graph than even the true aggregate shift, because the rule assumes the hypothesized dataset shift is true. Consequently, (12) is not a proper scoring rule. contrast, the proposed value functions described in Section 2. are.
More formally, we extend the traditional definition for proper scoring rule 18] to the context of explaining performance changes as follows. Let a scoring function X Ou x 0₁ x Q be defined as a function of source observation Og- (Wo: Zo, Yo), target observation O (W1. Z,, Yi). and candidate shift probability model Q. A scoring rule x is proper with respect to the set of distribution shift models F if the following holds: for any true model of the post-shift data distribution p" (W, Z, Y) E F, the expectation of the scoring function xOo, O1,p²) with respect to Oo po and O₁ ~ p" is always no smaller than the expectation of x(Op, O1.p) for any p E F. i.e.
Epo.r (X,,, Or,p") 2 Epo.p" [x(Oo, O1,p)] Vp EF Moreover, we say that x is strictly proper if (17 holds with equality iff = pp
(17)
Let Fz be the set of candidate s-partial conditional covariate shifts p.20 and Fy be the set of candidate s-partial conditional outcome shifts s. i.e.
Fz = (p-00 (v, z w/w)- = po(ylz, w) (2|w) is S Fr = (v. w) s S (E.....,,a..
Then, uz defined in (3) is proper scoring rule with respect to Fz, as Uz attains its largest value when A. 1. which holds when p.s0 matches the true covariate-only distribution shift model p-10- Similarly, defined in 44) is a proper scoring rule with respect to Fy. Moreover, these are strictly proper scoring rules as long as the conditional expectation of the losses of candidate distribution shifts are not adversarially aligned. That is, under the assumptions that
E.so [e] [) = Eo [$W)
+
p.so = P-10
Vp Fz p... P.1 Vp.., E Fr
(A.1)
E.., [, Z = E.., [[], Z]
+
(A.2)
19

---

2z and ty are strictly proper scoring rules with respect to Fz and Fy, respectively. As the candidate distribution shifts considered this work are functionals of the observed distribution shift and observed distribution shifts are unlikely to be adversarial in many real-world situations, Assumptions (A.1) and ( are likely reasonable in practice.
F Estimation and Inference
Let ((W(") - y : 1, na) denote n.a independent and identically distributed (IID) observations from the source and target domains d and 1, respectively. Let fixed fraction of the data be partitioned towards "training" (Tr) and the remaining to "evaluation" (Ev); let ngyv be the number observations in the evaluation partition. Let a denote the expectation with respect to domain d and Pd,, denote the empirical average over observations in partition Ev from domain {0, 1}.
F.1 Aggregate decomposition
The aggregate decomposition terms can formulated as an average treatment effect, a well-studied estimand in causal inference, where domain D corresponds to treatment. As such, one can use augmented inverse probability weighting (AIPW) define debiased ML estimators of the aggregate decomposition terms (e.g. Kang and Schafer (2 We review estimation and inference for these terms below.
Using the training data, estimate outcome models (4-00 (W) E. [e = -w] and WW, Z) E[{ W, Z] and density ratio models 1o00 (W) = Pi (W))) (W() and T1o Z) (WW, 2) Po (W, 2). The debiased ML estimators for Ay, Az, Ay are
Estimation.
Ay Po,, (e - 00 (W)) 100 ( (W) + P1. - Fo
Σ₂ =a (f - o.. 7)) #110 (W, 7) + f1. Z) - Po,n (E - oo W)) (1o)(W) - n0
Ày =P Po,n (E - po..o (W: Z)) #(W, 2) 1, (W, W)
Inference. Assuming the estimators for the outcome and density ratio models converge fast enough rate, the AIPW estimators for the aggregate decomposition terms are asymptotically linear and, thus, facilitate the construction of Cls.
Theorem F.1. Suppose T100 and T110 are bounded: estimators P..0.. a. 1000 and 1 are consistent; and
Po (ñ-00 (4-00) (+ TI00) = 0n 1/2) 11/2)
Po (f..0 p...o) (f110 T110) op(n" 1/2).
Then Åg, Åz, and & are asymptotically linear estimators of their respective estimands.
F.2 Value of s-partial conditional covariate shifts
Using the training partition, estimate the density ratio #1e0(2s, w) Pr (1 (p), w) and the outcome models
Estimation.
/ epo =))o(==alzz w w)dddz
p.o p-o ,0 w) EE [E.0|/|2...0 w|
(18)
(m0(w))= = [o[/w]]
(19)
(w)) E0 [[[w] / (po(y)w, =))o( w)p (xd)
(20)
20

---

in addition to the other nuisance models previously mentioned. We propose the estimator 0z(s)
(s) 02 og where
(8) :- PI 1. (W) - (o))
2
o. (A s0 (W) (o)) _i.._..oo ,. 2 (W, Zo)
p.., u, (1 (W) [W)1( 1/c(E--..0001 (W, 2) 110 ( 2)
(21)
+ 2PP 1, (0. (W) (L - p. (W))
2P (f- (W) (==0 -i-1o(W))
and 02 ; 222m((0)
Inference. The estimator is asymptotically normal as long as the outcome and density ratio models estimated fast enough rate defined formally follows.
Condition F.2. For variable subset s, suppose the following holds
Po (W) - Moi 2 is bounded
Po(M-o_,.024 ,, W) - P.... (1) W) - -misol (Z, W)) = =0p(n
Polu..(W,2)) - ( 2)
(W Z)) = op
Pi()oo((W) - (.))(2)22 on(n 1/2)
Pi(iio((W) (W))2 (n -1/2)
(Positivity)) Pa(2s; w) 0 and Do(w, 2) > O almost everywhere, such that the density ratios Tiso(w, 2.) and T110 (w, 2) are well-defined and between (0, 1).
Theorem E3. For variable subset s, suppose (s) )) > O and Condition F2 hold. Then the estimator (s) is asymptotically normal.
G Proofs
Notation. For all proofs, we will write IP to mean expectation on the evaluation partition (and likewise for the empirical version) for notational simplicity.
Overview of derivation strategy. We first present the general strategy for proving asymptotic
normality of the estimators for the decompositions. Details on nonparametric debiased estimation can be found in texts such as Tsiatis [45] and Kennedy 225).
Let (P) be pathwise differentiable quantity that is function of the true regular (differentiable in quadratic mean) probability distribution P over random variable O. For instance, v in the case of mean is defined as uPP)= == (o)) (o). Let fe denote an arbitrary regular estimator of TP, such as the maximum likelihood estimator. The plug-in estimator is then defined as vfP)..
The von-Mises expansion of the functional v (which linearizes in analogy the first-order Taylor expansion), given it is pathwise differentiable, gives
e(P) v(P) = P ()o +(P,))
(22)
Here, the function vb is called an influence function (or functional gradient of u at P).. R(P,, P) is second-order remainder term. The one-step corrected estimators consider have the form of v(P) P. (o; where P. denotes a sample average. Following the expansion above, the one-step corrected estimator can be analyzed as follows,
(v(t)) + (P.((o:P)) v(IP)
=(Pn - -P)((o P) + R(P,P)
=(P, -P)(( (P,, - ())((; ((; + R(P.P)
21

---

Our goal will be to analyze each of the three terms and to show that they are asympiotically negligible vn-rate, such that the one-step corrected estimator satisfies
(T(P)) P Pntb(oo P)) v(P) P, (b(;; + on(n=113)
where we used the property of influence functions that they have zero mean. Thus the one-step corrected estimator is asymptotically normal with mean v(IP) and variance var(v; P)) which allows for the construction of Cls. In the following proofs, we present the influence functions without derivations; see Kennedy [25] and Hines al. [20] for strategies for deriving influence functions.
G.1 Aggregate decompositions
Let the nuisance parameters in the one-step estimators Av, Az, Ay be denoted by T/ (p-00, T100), flz (Je..0 pl-00, T110), "r (J..o. T110) respectively. Denote the estimated nuisances by fla, iz. tr. The canonical gradients for the three estimands are
ldd= 0} p(d= 0)
(d, w, w, 2, = [[E (1, 2, v) (w)) (w) f(w, 2, v)]
+ (2)0(w) (w) 1dd 1}
p(d Aw
(23)
1[d = 0}
+ v.. 2) 1 11
wa (d, w, 2, 12) = [[((w, 2, 1w 2)) x110(1, =)]
p(d 0)
p(d 1)
[(d=0) = 0}
1(d=1 1}
[[((w, 2, - (.oo(w)) [Iol(W)]
+ (oo0(w)
Az
p(d 0)
p(d 1)
(24)
(y w, 2, = (f(w, 2, (w. 2)) 1fd 1fd=1} = p(d = 1)
-[(((, - n.o z)) TI1O(W, 2)1 1id 0}
Ay.
(25)
p(d 0)
Theorem G.1 (Theorem F.I. Under conditions outlined in Theorem F.1 the one-step corrected
estimators for the aggregate decomposition terms, baseline, conditional covariate, and conditional
outcome Ag Az. and Ky are asymptotically linear, i.e.
As- As = Pn + op 1/2) VE { 2, r].
(26)
Proof. The estimands Ag. Ag. Ay have similarities to the standard average treatment effect (ATE) in the causal inference literature (see (25] Example 2]. Hence, the estimators and their asymptotic
properties directly follow. For treatment T, outcome O, and confounders C, the mean outcome under
= 1 among the population with T = 0 is identified as
op(olc,t 1)p(clt = O)dode
(27)
and its one-step corrected estimator can be derived from the canonical gradient of o, which takes the following form after plugging in the estimates of the nuisance models:
1[T 1[T= 1) # (0 - (c)) + 1(T = 0} 1))
Sn = P.
Pr(T = 0) in(0))
satisfies
In - = P
Prr(T = (c) (0 (i()) + 1(T 0}
1(T - 1}
-0-10)))))
Pr(T = in (c) (c)
of + 1pp(n11//
where (c(c)= = -E/ooe t = 1] and (c) = p(ct = 0)/p(clt = 1) as long as the following conditions hold:
p(clt = =1) > 0 almost everywhere such that the density ratios T (c) are well-defined and
bounded,
22

---

Fi - f1)(f x) = Op(n 1/2).
We establish the estimators and their influence functions by showing that they all be viewed as mean outcomes of the form (27).
Baseline term Ag. The first term E100 [[(w, 2, y)] is mean outcome with respect to wr d = O)p(wle w = 1), which is the same as that in (27) but with E(w, z, y) as the p(E(w, 2, w) outcome, the confounder, and d as the (flipped) treatment. The second term Eg00 [8(w, 2, y)] is a simple average over = 0 population whose influence function the E(w, 2, y) itself.
Conditional covariate ferm Az. First term E110 [f(w, 2, 2.y) is the mean outcome with respect to p([(w, 2, y) [w, 2, d ()p(w, z/d = 1), where the chief difference is (w, z) is confounder. Second term E100 [e(w, v)] is also mean outcome, as discussed above.
Conditional outcome term Ay- First term B111 (f(w,,, 2, v)] is simple average the D population.
G.2 Value of s-partial conditional covariate shifts
72.a (p- 10,10, ,4-0_,0: Poo1, o4.00 T1s0, T110) and the set of estimated nuisances by f2,5 The canonical gradient of o2. (s) is
Let nuisance
parameters in the one-step estimator "2.s
nun be denoted
mun
D, W. Z, ( = (mssO(W) - )k)) 1(D =
mun 2,
Pr(D =)
+ sWW - #1o(WW) p1o /oo_ (V) (W 1[D=0)
Pr(D 0)
1(D =
2(u - (lo(W)) - u.. 100 2)
Pr(D = 0)
1(D 1}
+ (W)) (=vo(W))(Ho (W, 2.) 1. (W))
Pr(D = 1)
11D = 1}
(W)) (ww.(W))(...WWWW.. -W)W)) Pr(D = 1)
-2 (s).
(28)
Lemma G.2. Under Condition
F22
(2 satisfies
02
PP (D, W, 2 + (n
(29)
02
den
den
PROZA
quc
(D, W, Z,Y; 772m) + 0p((1-1//
(30)
Proof. Consider the following decomposition
2 5) 2z
(s)
(Pn P) (D. W. Z, 02,5
(31)
+ (P,, (U (D, W, Z, Y 12 2/2 bun (W, 2, (T
(32)
PP (D, W, Z, Y; (2
1 (, W. Z, Y; m
(33)
We note that 32
converges to a normal distribution per CLT assuming the variance of v is finite.
The empirical process term (32) is asymptotically negligible, as the nuisance parameters ng.5
estimated using a separate training data split from the evaluation data and 25 Lemma I] states that
(P, (V (D, W. Z. Y, fz.
2
cun (D, W. Z.Y; T2
on
23

---

as long as estimators for all nuisance parameters are consistent. We now establish that the remainder term 333 is also asymptotically negligible. Integrating with respect to Y, we have that
1 0} p 0)
33
P 2() (o1)(W)) (mO. ,o,,, W) - puo 1o.oooo0200.. (2,,,
2.)
+ (p.0_.0Zzs W) 20 (W)) 1(D =
p(D = 1)
(m.o(W, 2) p.. 1)) 2) 1 = 0} p(D 0)
- (A. 2) )-1((W)) 11
p(D -0=1] =11
(34)
+ P(W) - ())(())) - (w) -110)--11000 11D= =}
(35)
p(D= =)
1(D 0} p(D = 0)
=P 2o) (.10(W))) (HO. W) p.00 00 W)) (Wi 2.)
+ (m-m ( (z W) uo (Z,, Wo Z.) 1(D 0} p(D 0)
+ (A._,00 00 (Z,, W) (W) 1(D 1(D=1) =
p(D = =)
1(D = 0} p(D 0)
(m.oWW, 2) (., 2)) (#110 W, 2) - 110 (W 2))
(m.o(W,)) - 11o 2) 1(D = =0 p(D 0)
- (i..0 2) - 110 = 13 p(D =)
(36)
+ (W) - 2= (W) (w))) (oo)())))))(()))))) 1(D 1} p(D 1)
(37)
From convergence conditions in Condition
F22 this simplifies to
(W, IP ((0 (o))) (no .0 (Zs, W) iu. ,00,,, W)) (W, 2.) 1DD = 0} p(D 0)
33
+ (/000,00 (Z,, (0,,, W) (o 1(D =1
p(D =
(m.o(W, - (O, 2) 1 = 0} p(D 0)
110=1}
- -R.o 2) - iw))
p(D =1
(38)
+ (W) (o)) (W) - -(o)))))-o(W)))) 1(D 1) p(D = 1)
(39)
+ on (012),
(40)
Given the true density ratios, we can further simplify the expectations over D = 0 weighted by the density ratios in the expression above to expectations over D = 1. By definition of p.o..o(Z, (Z,, W) in
24

---

18
and (W) in 20 and the definition of p.. 2) and p.-10 (W) in Section F.1
33 simplifies
33
(W) (O (W))
P2 so - i- (110 (u.10
+ Pi oo fo))( - (H.s0(W) (.1o(
which is on as long as the convergence conditions in Condition F.2 hold.
As the denominator ugen is equal to the numerator o2 (0), it follows that the one-step estimator for the denominator i asymptotically linear with influence function 2/.00
Proof for Theorem
A.3
Combining Lemma G.2 and the Delta method 466 Theorem 3.1], the estima-
(s) mmm
2
is asymptotically linear
Gmum
2 1/
don) P. Uz (D, W, 1/, 112
+ op(n (1/2).
den
with influence function
2 mun
2 (D, W, W, 1225s 12,
22 (D, W, (
2 (D, W, Z, 1/2 deu)
1z
(ugen
(41)
where
Luu (D, W, Z, Y, 12,8
is defined
in and
videa (D, (D, Z, Y; 1 doa
jum (D, W, Z, Y; 72 v
2,0
Accordingly, the estimator asymptotically follows the normal distribution,
(éz(s) uz(s)) PP (0, var(Vz,. W, Z, Y; 72,5 12,8
den
(42)
L
G.3
Value of s-partial conditional outcome shifts
Let the nuisance parameters in Uybii be denoted ry (Quin pI.11 p.... T) We represent the one-step corrected estimator for u,,bin as the V-statistic
and its estimate Amun.
Amum YYbii
(s)
PP.n IP I.n (f... (W, 2) - i.. ()2
(43)
1II., PI,n (f... (W, Z) po... (W, Z)) (6 (e-1..11 pp.
(44)
i.... 2PP 12P 1,m (p.. (W,, 2o, - (W, /Z, -=-))
(e(W, Z.. 7_.. Y) u... Z., 2 ) (Z Zs W, Join (W, 2))
(45)
IP W Y, W, 1,,,,,,3 2,Y: kmmn
(46)
In more detail, the conditions in Theorem 3.2
follows.
Condition G.3. For variable subset 8, suppose the following hold
x (W, Z., Z-.. @0sa) is bounded
â is consistent
PP (fi.. - (1.00) 2- = 0n (n -1/2)
P, (fr..1 1M.1) 2- 0 0p(11 (n 1/2)
P (f... 21..)) -1/22)
25

---

P1 (4bsn [bin)² (p(n
P1 (fi... - (...s) (f - x) = 1/2).
sun Lemma G.4. Assuming Condition G.3 holds, 7,bbcn is an asymptotically linear estimator for v Uy,bs i.e.
yyb (5)
nuve U. bon (8) Pi F.s (D, W, Z, 2.YY mT.55) op(n
(47)
with influence function
47. (d, w; 2, yt 1T.
= (1..1 (w, 2) - p...s =))2
+ 2 2) -.... 2)) [e(w, 2, pr. (w. 2)]
2F, ( 2æ, 2)) - p ( (w, ns, 23)
[e(w, Re. - y)) - u.. (w, 2g. Z=,)] x (Z
2o, w. (osn(w, z))
B T.n (s).
nam
(48)
Proof. Defining the symmetrized version of h in (46) as
haym (W, 2, W, 2,
k.W.Z.YWW +. 2,Y W.Y.Y. we rewrite the estimator as
nun
(s) = PI, Po bym
W, 2, W, 2, Y; 17.5
Y. "in
Per Theorem 12.3 in [46], the Hájek projection of 14 (s) is
Pi Pi,m Pi,
(W, W, W. 2, y; 11,
mun
5gum (s) Xi, Yi
YYbiin
Pi /hsym (xi, Vi, (2); mmn - (s) Xz, V33
1sym,1 (Xi, Yi; 1y,
where @gam (s) P P1 haym
W. W, W, 2, Y: 1
Consider the decomposition
Yybbin (s) UY,
(s) =P P. h.nhsym
(W,Z.Y, Y. W, Y,Y: mmun
P.I h (W, 2, W, -. Y: 1y,
=Pp 1, P., ham
W, Y, W, Y amun
P PI, [haym..1
(X س (s)
(49)
+ i. Pi) (haym,1 1 (X, mmm
+ 5qun (s) hsym, (X,Y)
pun (s)
(50)
in Pi) (hsym,1 (X,
o mmn (s))
(51)
PI (hagm,1 (X, Y cuum
(s)
haym,1 (x, Y bty (s)
(52)
We analyze each term in turn.
Term 49 Suppose P 22 (W. 7, W. V,V; ( < co. Via a straightforward extension of the proof Theorem 12.3 in van der Vaart [46] one can show that
var (Pr,, P1.n
( J, W. 2, V;
17.8
7p 1.
var (P hsym. (W, Z, Y; (
26

---

Then by Theorem 11.2 in van der Vaart 46 and Slutsky's 's lemma, we have
Pi,n-i Pnnhem Z,, W, Y.Y; 17,5 -i. [haym.1 (W, Z, Y,; (V,5) + 0y (s)] = =Op (n-11/2)
Term 50))
We perform sample splitting to estimate the nuisance parameters and calculate the estimator for (s) Then by Lemma 1 Kennedy 25 we have that
11,, - PP)) (heym,1 (W, 2, Y;; (2.5) + ōque (s) - heym.1 ZW -2t = 0p(11-112
as long as the estimators for the nuisance parameters are consistent.
Term (TI):
This term
(Fi, F1) (haym,1 1 (W, Z, Y; :/,,)) + uqt (s))
LI, P1) 1aym.1 (W, Z, Y; (is) follows an asymptotic normal distribution per CLT.
Term (52g: We will show that this bias term is asymptotically negligible. For notational simplicity,
let EWW Zsy 7 a) fi..1 (W, Z) (W, 7)-
P (haym (.. (W, D. W. Y: f/y
+ Soum
h haym
W. Z, Y, W. Z, Y; n7.5
by mum (5)
PP PI (l<W.. (W. W, 2, 8; 1/8,0 - h W, Z, Y, W, 2, Y; 1/t,=
=P, (p.. () - s....W;; (), - IP, (W, 2)- (W, ())
2P, (i... (W, (W, k)) [1..1 (W, (W,2) (W.
- f { (W, Z., 2- - (W, Ze, (.,) [e(W, 2z, 2-, JW, 2-x)
( Zo W. Gada (W, 21) }
-P, (A.. 2) - (W, 2))² PI (H.. (W, Z) - (W, (W.2)) 2))² 2P (p... (W, pa... (W, k)) [M.. IW, 2)- (W, Z)1
- Z., (_.) (W, (W, 2_.) (W, Z., Zz 2-.)] $ (Z Ze, W.o.oa 2))
-2P, {(W, Z,, 2_.) [(W. Zz, 2,, - pr.. (W, (W,s,,2 2-x)) x (2_., Zn,
2,, W. Goin (W, (W;2)) (2_,, Zo W, Jonn (W.2)]}
=Pi (W, Z)- - (W, Z)) ( (W,2) -f...... (W, D) + (W, 2) H... (W, W. Z))
(53)
+ Pi (fr.. (W: Z) - (W, 2)) (,,,11,222
(..,(W: Z) - ft.. (W, + (W, Z))
(54)
- (W, Zo, . (... (W, Z,, 2_,) P... Z,; 2_.)] (ž_., Zo, W. 9oin (,, Z))
(55)
Pr ((W, Zz. 2-) (e(W, Zo, -_,, - - Zo, 2_:) (W,2)) 2)) - 7 (7_5, 7s, W, Gosn W, 2))]3
x (2 7s, W, Josn (W,
(56)
Note that (56) is (1/2) under the assumed convergence rates for Guin. In addition, On 1122. under the assumed convergence rates for pa.. and p.....
54 is
27

---

Analyzing the remaining summands 53 + 55 we note that it simplifies as follows:
Pi (X) - (X)) (f.. i.... + nh 1 - p... (X))
-P {E(W, 7s 3- (..., (W, ?w, 2-,) - it... (W, 7, 7-x)]
ft (2=ss Zo, W. 9oin (W:
(Z Zs W, qbinl (W, 2))))}
-2P,PIE(W: Ze --:) (p.... (W. Zs: Z_,) i... (W, Z,, 2=-)
(Z_ Zoo W, Toin (W; 2)))
-Pi (u..s()) jt..s( (X)) (H.. (X) f... ....(()) ))+ + fr.. (X))
PP ((W,, Zs. 2- [ (,, 2,, 2-)) - - (W, Ez, 2-)
a (Z_..2 Za. W, Pbin (,, (,in( ())) (2_.. Ze W, Poin (,, Z))
which is 0p(n=1/2) under the assumed convergence rates for pt.... f and it
Condition G.5 (Convergence conditions for 04 Suppose the following holds
Pi(p..1 - p4.00 (fi.. - -a..0)) = op(n
Po(p..o - ( - #110)= = 0p(n2 1/2)
Po(u..1 - -m.0)² is bounded
(Positivity) p(w, z|d = 0) > 0 almost everywhere, such that the density ratios T110( 2) are well-defined and bounded.
Let the nuisance parameters in the one-step estimator uden be denoted by r (J...o, pl.. T110) and the set estimated nuisances by aden.
Lemma G.6. Assuming Condition G.5 holds, then 6f an asymptotically linear estimator for uy i.e.
64th vgt (D, Z.W. m + (n(n21/2)
with influence function
e (D, W, Z, Y; f = (n..1 (W, ps. (W, 2)) 11D 1} p(D 1)
(57)
(2.. u. (W, Z)) (e- 11-...1 (.., Z)) 11D
(58)
1)
1(D
0}
- 2 (1.11W, um 2)) (e (W)
p(D
(59)
oeen
(60)
Proof. Consider the following decomposition of bias in the one-step corrected estimate
jden u
= (P, P) D, W, Z, Y dea)
(61)
+ (P, - (1) dan [D, W. 7, Y; den
- 1/4 den [D, W, 7, den)
(62)
+ P(uy den (D, W, Z, Y: ( 1 den (D, W, Z, Y; Y; dan
(63)
We observe that 61 converges to normal distribution per CLT assuming that the variance of v/dde finite. The empirical process term 62) is asymptotically negligible since the nuisance parameters
ded are evaluated on an separate evaluation data split from the training data used for estimation.
28

---

In addition assuming that the estimators for the nuisance parameters are consistent, Kennedy 25 Lemma states that
(I ( WYY
v Z, m) 1 (1(221/2)
We now show that the remainder term (63) is also asymptotically negligible. Substituting the influence function and integrating with respect to Y (63) becomes
P (f.. (i, (W, 2) (μ... (W, 7) 1 (w (W, 2))) 1(D =
63
(64)
p(D =)
L(D = 1)
+ 2P (u..1 (W, - O., (n... (W, ()) -W.
p 1)
(65)
+ 2PP (f..1 (W. Z) (W, Z)) (4 (W, Z) - P., 2)) 11D = p(D =
(66)
-2P (p.. Z) -i.o 2)) (H.. Z) - 2)) 2) 1 0}
(67)
p(D 0)
-P(A.1(W, 2) - ( 2) - (n,1 , 2) - -. 2))) 1(D=1)
(68)
p(D =1)
+ 2P(u (W, 7) (., (1.. (W, 2) - p -w Z)) 1(D =)
(69)
p(D =
1 0}
21P (f.. (W, 2) ii..o (W, k)) (w. 2) p. () -110
1110
(W, (W, 2))
p(D = 1)
(70)
2IP (p.. Z)= (.. Z)) (W.. Z) - (W, 1(D 0}
(71)
p(D 0)
-f. o (W, Z) (u..1 (W, 2) -W 2))) =(D=1)
(72)
1 = 0}
- 2P ff....1... Z) ao (W, 2)) (fr.o (W, Z) - ia.. (. (
T110 (W, 2))
p(D 0)
(73)
Thus the remainder term is op(n=1/2) if Condition G.5 holds.
Proof for Theorem
3.2
Combining Lemmas G.4 G.6 and the Delta method 46
Theorem 3.1], the
estimator Dy,bia( (s)
D.bin (s)/ is asymptotically linear
Y.bin
(8)
TTbin
mun
(+)
P. n dhy bin, (D, W, 2, Y; Ty,s The
pun dan
+ op(n (1/2).
oden
den
with influence function
"Ybbi (8) den
a (D, Z,
den
LUM (D, W, Z, Y: 17, uden
17 (udon) 3
(D, W. Y,Y; ( )
(74)
where v/pim and vienn are defined in 48 and 60
Accordingly, the estimator follows normal distribution asymptotically, (ōr(s) - vy(s)) -a N (0, var var( (L/r,iin, (D, W, 2, Y
nun m m)m)
(75)
L
Remark on super-fast convergence of Gose- One of the conditions in Condition G.3 states that the Obin converges at (11) rate. While this seems restrictive, binned risk converges exponentially under suitable margin conditions presented in Audibert and Tsybakov (2] In particular, consider the
following margin condition which is relaxation of Condition 3.1
29

---

Condition G.7. For all bin edges b except b E [0,1), the set E. {z (g(x) b) S e) is measure zero for some >
Suppose the function & belongs to the Hölder class E(3, L, Rd) for positive integer 3 and L > (see definition in Audibert and Tsybakov (2), the marginal law of X satisfies the strong density assumption, and the margin condition G.7 satisfied. Then the rate of convergence of Íuin is exponential, i.c.
(Abin(X) (X) - Obin (X))² s Ci exp(-C2n)
(76)
for constants
C1 > O that do not depend
Remark on value of s-partial conditional outcome shift. Unlike the case of s-partial conditional covariate shift, note that oy(0) may be non-zero when the risk in the target domain is recalibration (i.e. temperature-caaing of the risk the source domain [34 19]. For instance, there may be general environmental factors such that readmission risks in the target domain are uniformly lower.
H
Implementation details
Here describe how the nuisance parameters be estimated in each of the decompositions. In general, density ratio models can be estimated via a standard reduction to a classification problem where probabilistic classifier trained discriminate between source and target domains (43).
Note on computation time. Shapley value computation can parallelized over the subsets. For high-dimensional tabular data, grouping together variables can further reduce computation time (and increase interpretability).
H.1 Aggregate decompositions
Density ratio models. Using direct importance estimation [43 density ratio models т 1000 (W))(W) and ( Z) can be estimated by fitting classifiers on the combined source and target data predict D= 0 or 1 from features W and (W, 2), respectively.
Outcome models. The outcome models 1t-00 (l-oo(W) and J..o Z) can be fit in number of ways. One option is to estimate the conditional distribution of the outcome (i.e. po (Y or PY W. 2)) using binary classifiers, from which one can obtain estimate of the conditional expectation of the loss. Alternatively, one can estimate the conditional expectations of the loss directly by fitting regression models.
H.2 Detailed decomposition for s-partial outcome shift
Density ratio models. The density ratio (WW Zg Z). (2 Ze, QW, Z) Q)/P1 (Z in (6) can be estimated as follows. Create second ("phantom") dataset of the target domain which Z_, is independent of Z, by permuting the original Z_, in the target domain. Compute Ybtn for all observations in the original dataset and the permuted dataset. Concatenate the original dataset from the target domain with the permuted dataset. Train classifier to predict an observation is from the original versus the permuted dataset.
Outcome models. The outcome models p..1 and J.... Z) can be similarly fit by estimating the conditional distributions PY Z) and (Y W, Z, quan (W, Z)) on the target domain, and then laking expectation of the loss.
Computing U-statistics. Calculating the double average Pi,, n in the estimator requires evaluating all n² pairs of data points in target domain. This can be computationally expensive, so a good approximation to subsample the inner average. We take 2000 subsamples. We did not see large changes in the bias of the estimates compared to calculating the exact U-statistics.
H.3 Detailed decomposition for s-partial conditional outcome shift
Density ratio models. The ratio F1 (W, Z,) Pı (W, (x), Z.) can be similarly fit using direct importance estimation.
Outcome models. We require the following models.
30

---

_.o_.0 w) = [4.0 w/, defined in (18 can be estimated by regressing loss against w, on the source domain.
(-1o(w) = 10 [f]w], defined (99). can estimated by regressing (t- =) against w in the target domain.
(w) 20 [2 w]. defined in 20 can be estimated by regressing y.o_,, (Ze, w) against in the target domain.
For all models, we use cross-validation select among model types and hyperparameters. Model selection is important so that the convergence rate conditions for the asymptotic normality results are met.
I Simulation details
Data generation: We generate synthetic data under two settings. For the coverage checks in Section 4.1, all features are sampled independently from multivariate normal distribution. The mean of the (W, Z) in the source and target domains are (0, 2, 0.7. 3) and (0, O, O, 0), respectively. The outcome in the source and target domains are simulated from logistic regression model with coefficients (0.3, 1, 0.5, 1) and (0.3, 0.1, 0.5, 1.4).
In the second setting for baseline comparisons in Figure i)) each feature in W E R and Z E R sampled independently from the rest from a uniform distribution over [-1, 1). The binary outcome Y is sampled from a logistic regression model with coefficients (0.2, 0.4, 2, 0.25, 0.1, 0.1) 1) in source and (0.2, 0.4, 0.8, 0.1, 0.1, 0.1) ) in target.
In both the settings, we analyze performance gap of logistic regression models fit on held-out source dataset.
Sample-spiittng: We all models on 80% of the data points from both source and target datasets which is the Tr partition, and keep the remaining 20% for computing the estimators which is the Ev partition.
We use scikit-learn implementations for all models [33]. We use 3-fold cross validation to select models. For density models, we fit random forest classifiers and logistic regression models with polynomial features of degree 3. We clip the predicted probabilities from the density model for ı at 10 avoid very large density weights. Depending on whether the target outcome in outcome models is binary real-valued, we fit random forest classifiers or regressors, and logistic regression or linear regression models with ridge penalty. Specific hyperparameter ranges for the grid search are provided in the code.
Model types:
Computing time and resources: Computation for the VI estimates can be quite fast, as Shapley value computation can be parallelized over the subsets and the number of unique variable subsets sampled in the Shapley value approximation is often quite small. For instance, for the ACS Public Coverage case study with 34 features, the unique subsets is 131 even when the number of sampled subsets 3000, and it takes around 160 seconds to estimate the value of single variable subset. All experiments are run on 2.60 GHz processor with 8 CPU cores.
Data analysis details
Synthetic. We describe accuracy of the MI. algorithm after it is retrained with the top k features and predictions from the original model (Table 3). Proposed method results in the revised model with
highest gain in accuracy and AUC. Figure shows the aggregate and detailed decompositions for the simulation setup Section 4.1 decompositions.
Figure 6 shows the coverage rates 90% CIs for the aggregate
Hospital readmission. Using data from the electronic health records a large safety-ner hospital in the US, we analyzed the transferability of performance measures of Gradient Boosted Tree (GBT) trained to predict 30-day readmission risk for the general patient population (source) but applied to patients diagnosed with heart failure (target). Each of the source and target datasets have 3750 observations for analyzing the performance gap. The GBT is trained on held-out sample of 18,873 points from the general population. Features include 4 demographic variables (W) and 16 diagnosis
31

---

Aggregate
Proposed
Plugin
-0.55
-0.005
8.90
odes
Decampestie
Dotalled: Conditional covariate
Proposed
Sfugin
0827
HB0
Cracie
(93)
Delailed:
0272
Plugin
Cracie
Aatibuttor
Figure 5: Sample estimates and Cls for simulation from Section 4.1 Proposed is debiased ML estimator for HDPD.
Subset W
Coverage
1.0
0.5
0.0
Subset
Coverage
1.0
0.5
Subset
1.0
Coverage
0.0
8000
16000
Figure 6: Coverage rates of 90% CIs for the aggregate decomposition terms for the simulation setup in Section
Table 3: Accuracy and AUC for the revised model with respect to the top k: = (1,2,3) variables identified by different methods. Proposed method leads model with highest improvement in performance, thus, reducing the performance gap. We compare against two additional baselines that outperformed by the proposed method: retraining model on the target data with all features (AUC 0.89, Acc 0,84) and retraining on a weighted source-target data where loss for each point in source and target is weighted by tuned parameter and a respectively (AUC 0.89,
0 Acc 0.85).
Method
AUC-1
Acc-1
AUC-2
Acc-2
AUC-3
Acc-3
ParanetricChange
0.87
0.82
0.87
0.82
0.91
0.86
ParametricAcc
0.87
0.82
0.87
0.82
0.91
0.86
RandomForestAce
0.87
0.82
0.87
0.82
0.91
0.86
OaxacaBlinder
0.87
0.82
0.87
0.82
0.87
0.82
HDPD (proposed)
0.91
0.86
0.91
0.86
0.91
0.86
32

---

Table 4: Difference in AUCs between the revised insurance prediction model with respect the top variables identified by the proposed versus HandomForestAco orestAcc procedures (Diff AUC-k Proposed RandomForestAAc)) 95% Cls are shown.
k
Diff AUC-k
Lower CI
Upper CI
0.000
0.000
0.000
2
0.006
0.001
0.010
-0.002
-0.007
0.002
4 0.004
-0.002
0.008
-0.001
-0.006
0.003
6 -0.002
-0.008
0.003
0.007
0.002
0.011
8
0.006
0.001
0.010
Detailed: Conditional covariate
Drug induced/toxic related condition Mental & substance use disorders in remission Respiratory signs symptoms
X
X
X
Diabetes mellitus with complication
X
electrolyte disorders
Diabetes mellitus without complication
HDPD debiased
X
Variable
Abdominal pain, digestive signs & symptoms
Conditions due neoplasm its treatment
HDPD plug-in
X
Musculoskeletal pain, not low back pain Exposure infectious disease
MeanChange
X
X
Tobacco-related disorders
WuShift
X
Nervous system signs symptoms Personal/family history of disease
X
Injury. sequela
Essential hypertension
X
Abnormal findings without diagnosis
0.0
0.2
0.4
0.6
0.8
1.0
Attribution Value
Figure Detailed decompositions for the performance gap a model predicting hospital readmission in HF population. Plot shows values for the full set of 16 variables.
codes (Z). While training, we reweigh samples by class weights to address class imbalance. Figure shows the detailed decomposition of conditional covariate shift for the dataset.
ACS Public Coverage. We extract data from the American Community Survey (ACS) to predict whether person has public health insurance. The data only contains persons of age less than 65 and having an income of less than $30,000. We analyze a neural network (MLP) trained on data from Nebraska (source) data from Louisiana (target) given 3000 and 6000 observations from the source and target domains, respectively. Another 3300 from source for training the model. Figure S shows the detailed decomposition of conditional outcome shift for the dataset. Table 4 shows the difference in AUCs for the revised models based on top features from the proposed method versus RandomFord FaelomForestAcc method.
NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Theoretical and experimental results are presented in Sections and respectively.
Justification: Main contributions stated in Section 1
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
33

---

Detailed: Conditional outcome
Total person's income
Married
married under years
Employment status employed
Divorced
Citizenship status citizen Citizenship status born US Educational attainment
Disability
Employment status not labor force
Nativity
Employment status parents
Mobility status
Vision difficulty
Variable
Citizenship Gave birth to child in past 12 months
abroad
Cognitive difficulty
Widowed
School least bachelor
School at least high school GED
Employment status partial employed
School postgrad
Separated
Citizenship status born in Puerto Rico, Guam
Hearing difficulty
Citizenship status naturalized
Military service
HDPD debiased
Employment status unemployed Emplayment status armed
HDPD plugin
Employment status partial amed Ancestry
x RandomForestAcc
0.00
0.05
0.10
0,20
0.25
Attribution Value
Figure 8: Detailed decompositions for the performance gap of model predicting insurance coverage prediction across two US states (NE + LA). Plot shows values for the full set of 31 covariates.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: Limitations are discussed in Section
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested few datasets with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be
34

---

used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community, Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?
Answer: [Yes]
Justification: Assumptions and proof' for the theoretical results are provided Sections FF and
G
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should properly referenced.
4. Experimental Result Reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless whether the code and data are provided or not)?
Answer: [Yes]
Justification: Pscudocode is provided in Algorithms 2 and Implementation details are described in Section
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
35

---

While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily new algorithm, the paper should make it clear how to reproduce that algorithm. (b) the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is new model (e.g.. a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. the case of closed-source models, may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification:
Code is available at the link sttpps//githuub.com//jtttt//
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips.. cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).
The instructions should contain the exact command and environment needed to to reproduce the results. See the NeurlPS code and data submission guidelines (https
//wpps.ccppuliiii muublic/guides Soaeee/CCSSuubmsiiii.. stiobbmiiisioooooocccioc for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing much information possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental Setting/eetails
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: Implementation and simulation details are described in Section and Section Full details are in the provided code.
Guidelines:
The answer NA means that the paper does include experiments.
The experimental setting should be presented the core the paper to level of detail
that is necessary to appreciate the results and make sense of them.
36

---

The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment Statistical Significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
Justification: Results are accompanied by confidence intervals.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the enror bars capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall
with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula,
call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
should be clear whether the error bar is the standard deviation the standard error of the mean.
It is OK to report 1-sigma error bars, but should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information the puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: Compute resources are described in Section I
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't 't make it into the paper).
9. Code Of Ethics
Question: Does the research conducted the paper conform, in every respect, with the NeurlPS Code of Ethics http://///eaaaepp.cccccpppt///tt
Answer: [Yes]
Justification:
We adhere to the NeurlPS Code of Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
37

---

If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g.. there a special consid- eration due to laws or regulations in their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: Although we do not envision any direct negative impacts of the work, potential
positive and negative impacts are discussed in Section B
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative socictal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deploymeett of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied partieular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point that improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
Safeguards
Question: Does the paper deseribe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, scraped datasets)?
Answer: [NA]
Justification:
We do not release a model new dataset.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
38

---

Answer:
[Yes]
Justification: Medical dataset used for the case study is accessed and analyzed under the terms of an IRB.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version asset used and, possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g.. website). the copyright and terms of service of that source should be provided.
If assets released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com/ datasets has curated licenses for some datasets. Their licensing guide can help determine the license dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if has changed) should be provided.
this information is not available online, the authors are encouraged to reach out to the asset's creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer:
[NA]
We do not introduce new assets. The code is provided only for reproducing the experiments.
Justification:
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details the dataset/code/mooel as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include anonymized zip file.
14. Crowdsourcing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification:
Study does not involve crowdsourcing or research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included the main paper.
According to the NeurIPS Code Ethics, workers involved in data collection, curation, other labor should be paid at least the minimum wage in the country of the data collector.
Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
39

---

Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [Yes]
Justification: Medical dataset was accessed according to the terms of use of an IRB.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with
human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent)
may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such the institution conducting the review.
40


Reviews:
[{"review_id": "6y6KLWnIdf", "reviewer": "Reviewer_tcwU", "summary": "This paper introduces a hierarchical decomposition framework aimed at explaining performance discrepancies of machine learning models across different domains. It proposes both aggregate and detailed decompositions to quantify the impact of shifts in feature distributions (marginal and conditional) on model performance, without necessitating detailed causal graphs. The framework leverages nonparametric methods and Shapley-based variable attributions, enhancing understanding and facilitating targeted interventions to mitigate performance gaps. Coverage rates and further experiment analysis are provided.", "questions": "- L85 aggregate: the first level mentioned here represents the aggregated level?\n- linear decomposition: although it looks like just a log operation from equation, it would be good to give intuition on such a linear decomposition. What assumptions are made exactly here?\n- More importantly, the performance gap may be from  a combination of variables, instead of one variable at a time. How does the proposed framework address such cases?\n- How scalable is the proposed method? Would the proposed method handle discrete variables?\n- Fig 3b: some variables seems to be highly correlated or even negation of each other ( e.g. Married, Divorced, Never married, and e.g. citizenship status). Yet they do not seems to be consistent to each other (e.g., married and divorced). What is the reason for such a difference?", "limitations": "yes", "rating": 5, "confidence": 2, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "The scope of the paper significantly extends that of the most relevant prior work, which provides the aggregate decomposition that this work builds on.\n\n This work further extends several other related works by characterizing shifts that are more complex and in a setting where no causal graph encoding prior knowledge of the data generating process or shift structure is assumed.\n\nThe estimation and inference procedures are provided, with convergence rates.", "weaknesses": "I find the paper, at times, is difficult to follow. I would suggest a formal definition on the framework, along on definition of performance gap, and each level of explanation. Some causal interpretation in appendix could be useful in Section 2 to understand the framework.\n\nExperiment results bring some questions.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for helpful comments and are glad that they appreciated the novel aspects of the work. We summarize the framework in the global response for more clarity and respond to the reviewer's questions below.\n\n* **Aggregate decomposition**: Each term in the aggregate decomposition $\\Delta = \\Lambda_W + \\Lambda_Z + \\Lambda_Y$ quantifies the performance change when we vary only one factor of the joint distribution of $(W,Z,Y)$.\nThey are defined as:\n$$\\Lambda_W = \\mathbb{E}\\_{100}[\\ell] - \\mathbb{E}\\_{000}[\\ell] = \\int \\ell(f(w,z), y) p_0(y|w,z) p_0(z|w) \\left(p_1(w) - p_0(w)\\right) dy dz dw$$\n$$\\Lambda_Z = \\mathbb{E}\\_{110}[\\ell] - \\mathbb{E}\\_{100}[\\ell] = \\int \\ell(f(w,z), y) p_0(y|w,z) \\left(p_1(z|w) - p_0(z|w)\\right) p_1(w) dy dz dw$$\n$$\\Lambda_Y = \\mathbb{E}\\_{111}[\\ell] - \\mathbb{E}\\_{110}[\\ell] = \\int \\ell(f(w,z), y) \\left(p_1(y|w,z) - p_0(y|w,z)\\right) p_1(z|w) p_1(w) dy dz dw$$\nwhere $\\Lambda_W$ corresponds to the effect of varying $p(W)$, $\\Lambda_Z$ corresponds to the additional effect of varying $p(Z|W)$, and $\\Lambda_Y$ corresponds to the additional effect of varying $p(Y|Z,W)$.\nFrom the equations, one can see that the terms sum up to $\\Delta$ by definition; no assumptions are needed.\n\n* **Multiple variables**: As the reviewer rightly noted, multiple variables can shift to induce a performance gap. This, in fact, is one of the major contributions of this work, as it provides a framework that (i) formally defines an $s$-partial shift for variable subsets $s$ and a value function $v(s)$ that quantifies how well a partial shift with respect to subset $s$ explains the performance gap and (ii) a procedure for estimation and statistical inference of the value function. Shapley values should be viewed as an abstraction that lies on top. Its primary use is to provide a simple digestible summary at the level of individual variables. Shapley values also account for interactions between variables, as it defines the importance of variable $j$ as the average increase in the value function when a variable $j$ is added to a subset $s$. Our framework allows one to directly analyze the importance of variable subsets *or* Shapley values of individual variables.\n\n* **Scability**: The method is highly scalable. Scalability is achieved in two ways.\nFirst, we use a subset sampling procedure to estimate Shapley values that maintains statistical validity. This allows us to compute hundreds of Shapley values.\nSecond, one can group together variables and use the framework to quantify the importance of variable groups (rather than individual variables).\nFor instance, variables may be grouped based on prior knowledge (e.g. genes in the same pathway) and a clustering procedure (e.g. grouping together variables that are highly correlated).\n\n* **Discrete**: The method handles discrete variables, as illustrated in the real-world experiments section. In the submission, we obtain Shapley values for binary variables as well as categorical variables.\nFor categorical variables, we can either obtain importance for each category (e.g. report importance of different marital statuses) or group together all the categories to report a single importance score (e.g. report importance of marital status as a whole). We opted for the former but our procedure can also do the latter.\n\n* **Fig 3b question**: For binary variables, the framework reports a single importance; there is no consistency problem. For categorical variables, the importance of different values may not necessarily be similar, which should be expected. A shift with respect to a specific category is not directly connected to a shift with respect to a different category. This is why the different categories of marital status have different values; marital status has five categories in the ACS dataset.\n\n* We appreciate the suggestion to surface the causal interpretation in the main paper rather than the appendix. Indeed, our explanation framework can be interpreted from the view of stochastic interventions, assuming causal sufficiency, positivity, and SUTVA hold.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "n4uKiWS78A", "reviewer": "Reviewer_tcta", "summary": "The problem of explaining and remedying a dropoff in the accuracy of a supervised learning model when applied to a new environment (i.e. a new joint distribution between inputs and output) is studied. The paper tackles these problems for a non-parametric setting and with the goal of identifying specific individual input variables which most contribute to the dropoff. Confidence intervals are also generated. Previous work in the area does not meet all these criteria. Input variables are segmented into two disjoint sets: a set of \"baseline\" variables W and a set of \"conditional covariates\" Z which are often (but not necessarily) causally downstream of the baseline variables. The total dropoff in performance is decomposed into the dropoff attributable to each of the 3 shifts: a shift in P(W), a shift in P(Z | W) and a shift in the conditional expectation of the target variable given the inputs, i.e., P(Y | Z; W).  The method is able to evaluate the impact of partial shifts in the Z variables, i.e. shifts in a subset of the Z variables. Partial shifts in W cannot be evaluated, though. To rank the impacts of shifts in individual Zs, Shapley values are used. To overcome nonparametric estimation challenges, a binning procedure as well as a Shapley value sampling procedure is used. The method is shown to give intuitive results for simulated data. Two real world problems are studied. The first problem is hospital readmission prediction. The shift in environment is from a general patient population to a patient population restricted to those with heart failure.  The second problem is health insurance coverage prediction where the shift in environment is from Nebraska to Louisiana. The method is evaluated in terms of the quality of the selection of the top-k variables most responsible for performance dropoff. The quality of the selection is evaluated by training and testing a model on the second environment where the inputs include the output of the original-environment variable, the baseline variables and only those top-k of the Z, rather than all the Z.  Superior results as compared to competing techniques are obtained.", "questions": "How does one segment between W and Z if not by casual knowledge (since this is not required) ?\n\nYou need at least one W for your method. Can you just arbitrarily pick 1 variable for your W ? Is there some sort of bias/variance or other tradeoff if you picked 1 variable as W and e.g. the other 99 as Z or would 20/80 or 50/50 be more reasonable for some reason?\n\nWhy do you think the W you chose cause the Z you chose in hospital readmission and insurance coverage?\n\n** update after rebuttal **\n\nIn light of the additional explanations for how to distinguish between W and Z and in light of the additional experiments evaluating the baselines I suggested, I raised my score to a 5.", "limitations": "Adequately addressed.", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "The paper is original to the best of my knowledge.\n\nIf the flaws in the paper could be fixed, I certainly agree that the contribution of the work could be significant. Understanding and adapting to changing environments for tabular models is definitely an important problem and it's valuable to do this for general ML models rather than just linear stuff as in Oaxaca-Blinder. \n\nAlthough I didn't have time to check every technical detail super-carefully, the paper does seem to overcome major technical challenges in a clever way. In particular, I quite liked the use of the random subsampling Shapley value approach to overcome the combinatorial explosion of exact Shapley value computation. I've wondered vaguely in the past whether a technique like that could be valid and I feel I benefitted from learning about that reference and seeing its application here. \n\nSome parts of the paper are well written, although I also think there are serious clarity problems, as I'll explain in the next section.", "weaknesses": "I'm sure it's unpleasant for the authors to see my score of 3. I probably have never liked a paper I've given a 3 nearly as much as I liked this paper. I think the potential is here to achieve (eventually) a quite significant contribution to the ML literature. However, I feel strongly that the paper is not ready for submission as is.  I'm somewhat open to raising my score if my concerns can be addressed in rebuttal, but it woh't be easy to address my concerns given the short rebuttal time window.  \n\nThe biggest flaw I see in the paper is that the distinction between baseline variables W and conditional covariates Z is very problematic. The distinction is not well explained and it's poorly illustrated both by the real-world problems and the synthetic experiments. The authors say that Z can often be thought of as causally downstream of W but this not required. Ok, if it's not required, then how else would you segment the variables into W and Z ? I don't think this is ever explained.  For the real world problems (unless I missed something), there doesn't seem to be any explanation why some variables are W and others are Z. For hospital readmission, why would demographic variables be W and diagnosis codes be Z ? I get that demographics could cause the diagnoses to some extent (although the diagnoses are surely a long way from being fully caused by a small set of demographic variables) but I still needed a lot more explanation here. For ACS insurance, why are the 3 demographic variables (age, race etc) the baseline Ws and the others (marital status, etc) the conditional covariates?  For the synthetic stuff, on line 243, why are W, Z1, Z2, Z3 independent normal if we are usually supposed to think of Z as causally downstream of W ? \n\nI also consider only 2 real world problems to be a bit thin in terms of real-world results for NeurIPs acceptance.\n\nAnother problem I have with the experiments is that more naive  baselines are needed to double-check that the partial retraining beats the baseline. I'll take the authors' word that this procedure of taking the old-environment model output as a new-environment input and combining that with top-k most impactful Zs is some sort of standard in the literature. Nonetheless, I think you should compare it against:\n\n1) Throw away all the old-environment data and fully retrain a new-environment-training-data-only model with all available W and Z inputs. \n\n2) Do some sort of weighted-data retraiing using all inputs, where old environment and new environment training data are both used by old environment is weighted less (with the weight being a CV-optimized hyperparameter).\n\nThere are also some clarity problems.\n\nOn line 147, it's wrong to use \"risk\" as the probability of class 1. \"Risk\" has an established meaning in the ML community as \"loss\" or \"error\". E.g. \"empirical risk minimization\" means loss minimization, not minimization of the odds of class 1.\n\nThe paper also makes various variable/terminology choices which aren't a big problem but could be improved so that the reader finds it easier to follow.\n\nI would suggest using \"environment\" where \"domain\" is used because \"domain\" can also mean the input space, i.e. a function mapping from domain to range, so that could be confusing.\n\nI would not call the 2nd domain/environment \"target\", since that can be confused with Y. Maybe the \"shifted\" environment or the \"new\" environment ?\n\nI wouldn't denote the two domains/environments as 0 and 1. I would use letters, e.g. O and S for original and shifted.\n\n\nTypos:\n\nL94 which have the benefits-> which has the benefits\nL162 toderive", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for a thorough reading of the work and the constructive feedback. We are excited to hear that you liked the work and hope that our response will convince you to raise the score.\n\n* **Selecting $W$ and $Z$**: Thank you for asking this question on how to partition variables into $W$ and $Z$! Your question has prompted us to think more deeply about the selection procedure, and we sincerely thank the reviewer for the opportunity to clarify our thinking in this area. We have responded in the global response with both causal and non-causal approaches for choosing $W$ versus $Z$. We believe these options provide users many practical approaches to applying our method.\n\n* **Choice of $W$ and $Z$ in experiments**: Our reasoning was as follows:\n   + **Hospital readmission**: The primary reason for setting demographic variables to $W$ and diagnoses to $Z$ is that demographics are causally upstream of diagnoses, and diagnoses are causally upstream of readmission status $Y$. As the reviewer rightly noted, other variables can also cause diagnosis codes. However, this is not an issue as long as we correctly interpret the results. In particular, Figure 3(a) reports how well shifts in disease diagnoses explain the variation in the performance gaps (between the two environments) across demographic groups $W$. Omitting variables from $W$ simply changes the performance gaps we are trying to explain (see Subbaswamy et al. 2021 for a formal discussion of why there is a causal interpretation of distribution shifts with respect to $W$, even if it does not contain all possible confounders). At the same time, because one is unlikely to include all possible variables that cause $Y=$readmission in $Z$, the estimated variable importances should be viewed as *relative* to variables included in $Z$ and not as *absolute* importances with respect to all possible variables. We will discuss this nuance in more detail in the manuscript.\n   + **ACS insurance**: Most of the variables available in the ACS dataset could be considered as demographics. Nevertheless, the selected $W$ (sex, age, race) are still causally upstream of the remaining variables we assigned to $Z$ (health condition, marital status, education, etc). One can then interpret the explanations similar to the aforementioned example. Alternatively, one can think of this from an ML fairness perspective, where our goal is to explain the performance gap overall as well as performance gaps within groups defined by the protected attributes of sex, age, and race. Again, as the causal graph may be missing variables, the results should be interpreted per the discussion above.\n   + **Simulation setup described on line 243**: This simulation tests our explanation method for conditional outcome shifts. Because the outcome distribution $Y|W, Z$ is (variationally) independent of the conditional distribution of $Z|W$, both the estimand and the estimation procedure are not affected by the dependence structure between $Z$ and $W$. As such, we considered a simple simulation where $Z$ and $W$ are completely independent. Nevertheless, we see how this can be confusing to the reader and will revise the manuscript so that $Z$ depends on $W$.\n\n* **Bias-variance tradeoff between $W$ and $Z$**: This is an interesting question! When more variables are assigned to $W$, the performance gap with respect to $W$ (i.e. $\\Delta_{\\cdot 10}(W)$) is a more complex function. Thus we may have more uncertainty in our estimate of $\\Delta_{\\cdot 10}(W)$, which may lead to wider confidence intervals for the variable importance values. On the other hand, more variables in $W$ lead to higher variance of $\\Delta_{\\cdot 10}(W)$, so it allows one to better distinguish the relative importance of variables in $Z$ for explaining its variability. We will include this nuance in the manuscript as an additional consideration when choosing $W$ and $Z$.\n   \n* **Partial retraining experiments**: We have included the two suggested baselines from the reviewer in the table below. Partial retraining based on the proposed framework is better than the two baselines: it results in models with a 2\\% improvement in AUC. This is because targeted retraining is more statistically efficient (i.e. requires less training data), so it can quickly adapt to the new environment.\n\n* We thank the reviewer for the suggested terminology changes such as risk to conditional outcome, domain to environment, and target to shifted environment. We will incorporate these into our revision. \n\n**Table**: AUC and accuracy of retrained models are reported. Target-only model is retrained with all features ($W,Z$) but only on the target data. Weighted source-target model is retrained on pooled source and target data with all features, where source data are assigned weight $r$ to optimize AUC in the target environment. Retrained models based on the proposed method achieve the best performance.\n| Method                               | AUC      | Accuracy |\n|--------------------------------------|----------|----------|\n| ParametricChange                     | 0.87     | 0.82     |\n| ParametricAcc                        | 0.87     | 0.82     |\n| RandomForestAcc                      | 0.87     | 0.82     |\n| OaxacaBlinder                        | 0.87     | 0.82     |\n| HDPD (proposed)                      | **0.91** | **0.86** |\n| *(new) Target-only model*            | 0.89     | 0.84     |\n| *(new) Weighted source-target model* | 0.89     | 0.85     |\n\nReferences:\n\nSubbaswamy, Adarsh, Roy Adams, and Suchi Saria. “Evaluating Model Robustness and Stability to Dataset Shift.” International Conference on Artificial Intelligence and Statistics 2021 https://proceedings.mlr.press/v130/subbaswamy21a.html.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "nj3uRnRXUr", "reviewer": "Reviewer_fcBN", "summary": "The authors describe a novel method to detect root causes of distribution shift (performance variability) of ML classifiers across domains. The method estimates how much of the shift is due to covariate shift vs. outcome shift, and which input features contribute most to the said shifts.", "questions": "What is the connection between equation (2) and VI? Is (2) the\ndefinition of VI? \n\nWhat are E000, E100 etc. between lines 89 and 90?\n\nIn Fig. 3, two types of results are shown: HDPD debiased and HDPD\nplug-in. If someone decides to use your method, which version do you\nrecommend?", "limitations": "The limitations are addressed adequately.", "rating": 8, "confidence": 2, "soundness": 3, "presentation": 1, "contribution": 4, "strengths": "Very significant topic and, as far as I can tell, original approach. Good Real-world case studies. Code is provided.", "weaknesses": "The paper is not well written because most NeurIPS audience (myself included) may not be able to understand the methods. For example, this\nis almost incomprehensible (lines 76-80):\n\nAn aggregate shift substitutes a factor from the source domain in (1) with that from the target domain, i.e. we swap the factor from p0 to p1. A partial shift with respect to variable subset s (or an s-partial shift) shifts a factor from the source domain in (1) only with respect to variable subset s; we denote this by swapping a factor from p0 to ps.\n\nI am guessing the authors did not have enough space to write elaborate explanation, or maybe reading it requires extensive background in this\nparticular area. Suggestion: write only the final equations and high-level explanation how it would be applied in ML practice; the rest of the derivation should be moved to Technical Supplement, and since there is no length constraint there, you can provide extensive and clear derivation of your result.\n\nBottom line: my best guess is this is a strong paper, but I have low confidence in my assessment due to the above.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their time and positive feedback. We appreciate the constructive criticism to improve the clarity of the paper, and have included a high-level description of the procedure in the global response. We will also revise the manuscript to focus more on the high-level explanation.\n\n* **Definition of VI**: Yes, the Shapley value formula in equation (2) defines the variable importance.\n\n* **Notation E000**: $\\mathbb{E}\\_{D\\_W, D\\_Y, D\\_Y}$ denotes an expectation with respect to the joint distribution $p\\_{D\\_W}(W) p\\_{D\\_Z}(Z|W) p\\_{D\\_Y}(Y|W,Z)$. Thus $\\mathbb{E}\\_{000}$ and $\\mathbb{E}\\_{111}$  denote the expectations with respect to the source and target environments, respectively.\n\n* **Which version of method to use**: We generally recommend using the HDPD debiased method because it provides valid uncertainty quantification. In particular, its confidence intervals have valid coverage, asymptotically.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Hrbvj0h6pN", "reviewer": "Reviewer_7cwp", "summary": "The paper proposes a hierarchical decomposition model for binary machine learning classifiers. The nonparametric model allows for a detailed decomposition of distributions shifts at aggregate or partial levels. Additionally, confidence intervals for the proposed estimators are presented.", "questions": "1. Regarding Weakness 3. The reference [2] only achieves fast convergence under strong and specific conditions. I am not quite sure whether these conditions are even achievable in the presented setting. Specifically, the binned $q_{\\text{bin}}$ is not smooth, which would already violate the Hölder condition. Further, the margin condition should be discussed as this would require comparatively “simple” classification problems based on the bins.", "limitations": "See Weaknesses", "rating": 6, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "1. Introducing a new general concept to decompose distribution shifts for classifiers has a wide range of applications. Specifically basing the evaluation on $R^2$-measures enhances interpretability.\n\n2. While no causal graph is required, causal knowledge can easily be incorporated into the setting.\n\n3. The paper is well-organized and gradually motivates the complex definitions.", "weaknesses": "1. The proposed method is restricted to (binary) classification problems. This limits the applicability and should already be noted in the abstract and/or title.\n\n2. The framework requires baseline variables W, which can not be used for partial shifts. This is already sufficiently acknowledged and well motivated, especially combined with a causal graph.\n\n3. The conditions should be discussed in more detail. Specifically, the fast convergence rate (8) is a very strong assumption that is not sufficiently discussed.\n\nMinor Comments:\n- Figure 2(a): No dashed lines as mentioned in the caption.\n- L.162: Typo “toderive”\n- L. 170: Typo “psuedocode”", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for a careful reading of the work. We are encouraged to hear that the reviewer found the manuscript well-organized and the method widely applicable.\n\n1. **Binary classification**: While the manuscript focuses on binary classification, the framework can be readily extended to multi-class classification problems as well as to regression problems. The only section of this work that is specific to binary classification is importance estimation of $s$-partial outcome shifts with respect to the binned probabilities $\\Pr(Y=1|Z,W) = E[Y|Z,W]$. However, one can just as easily define $s$-partial outcome shifts with respect to the binned expectations, which would make the framework applicable to regression and multi-class settings. We will add a section to the appendix to describe how such an extension can be achieved. Thank you for this suggestion!\n\n2. **Detailed decomposition of $W$**: As the reviewer noted, the current framework does not provide a detailed decomposition of $W$ variables. We had previously considered simply rotating the roles of $W$ and $Z$, which would give us detailed decompositions for all variables. However, it was unclear how to interpret the resulting values. As such, we have left this open question for future work.\n\n3. **Fast convergence rate assumption**: Thank you for the question on the fast convergence rate assumption! While we agree that fast convergence rates are not achievable in all settings, it does hold, for instance, when the set of $x=(w,z)$ with probabilities near the bin edges $b$ is measure zero. That is, super fast (exponential) convergence rates hold if the set $\\Xi\\_{\\epsilon} = \\left\\\\{x: \\left|Q(x) - b\\right| \\le \\epsilon \\right\\\\}$ is zero for some $\\epsilon > 0$, which is a relaxation of Condition 3.1 in the manuscript. We do not need $q_{\\text{bin}}$ to satisfy the Hölder condition; instead, we just need $q$, the function prior to binning, to satisfy the Hölder condition. In fact, our convergence rates follow from reference [2] quite directly, as they similarly threshold the estimated probability function and demonstrate fast convergence of the binned function.\n\nWe believe that this margin condition is quite reasonable in practice, as there may very well be some small $\\epsilon$ for which the margin condition holds. More importantly, we have found that the method performs quite well, even under violations of this assumption. As shown in the empirical results, our method is able to achieve the nominal Type I error rates, even though the generated data does not satisfy the margin condition. We will revise the manuscript with this discussion of the fast convergence rate assumption.\n\nReferences:\n\n2. Jean-Yves Audibert and Alexandre B Tsybakov. Fast learning rates for plug-in classifiers. Ann. Stat., 35(2):608–633, April 2007.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
We sincerely thank the reviewers for their constructive feedback and the positive response. 

To recap, the aim of the paper is to address a major **methodological gap**: there are currently no nonparametric methods that provide a detailed explanation for why the performance of an ML algorithm differs between two environments, without assuming the detailed causal graph is known. Due to this vacuum, applied ML papers often contain a long list of factors that differ between two environments as the set of potential causes; however, such lists contain many false positives, as variables with large shifts may not necessarily contribute to the performance drop, and miss true positives, as variables with small shifts can still have a large impact on algorithm performance. 

There is a **significant need for explanation methods**, as the lack of generalizability of ML algorithms presents major problems in the **safety** of ML algorithms (particularly in clinical settings) and is a major barrier to widespread adoption. Having explanations is crucial to ML consumers for taking steps to mitigate performance drops, regulators for outlining environments where the algorithm can be safely deployed in, and ML developers for designing methods that are more robust and generalizable.

Our proposed framework aims to address this methodological gap. We are heartened by the reviewers agreeing that this is **"a very significant topic" [fcBN]** and the method has the potential to make **"a quite significant contribution to the ML literature" [tcta]**. Moreover, all reviewers acknowledge that the method **"significantly extends prior work" [tcwU]** and **"overcomes major technical challenges" [tcta]**.

Reviewers asked for more clarity on the framework at a high-level, which we add here. The proposed hierarchical framework decomposes the performance gap (i.e. the difference in the expected loss $\Delta = \mathbb{E}_1[\ell] - \mathbb{E}_0[\ell]$) of a model between two environments. We suppose variables $X$ are partitioned into baseline variables $W$ and conditional covariates $Z$. 

We suggest selecting $Z$ to be the variables that may act as mediators of the environment shift and/or variables whose associations with $Y$ are likely to be modified by the environment shift (i.e. effect modification). This selection can be chosen based on a high-level causal graph, where $W$ are variables known to be upstream of $Z$. For instance, if $Z$ are treatment variables and $W$ are baseline variables, one has the nice interpretation that a covariate shifts is a change in the treatment policy and an outcome shift is a change in the treatment effect across the two environments. 

In the absence of *any* prior knowledge, another option is to choose $W$ as the variables for which one would like the expected loss given $W$ to be invariant across the two environments; this can be useful to promote fairness of ML algorithms across environments.
When this invariance does not hold, the proposed framework explains how variables $Z$ contribute to these differences, which can inform efforts to eliminate performance gaps. This last option is similar to how variables are typically chosen in disparity analyses [Jackson 2021]. For instance, to understand why income differs between males and females controlling for age, one would set domain to $D=$ gender, $W =$ age, and $Z$ as variables that may explain this disparity (e.g. marital status, employment status).

The **aggregate level** decomposes the performance gap $\Delta$ into $\Lambda_W + \Lambda_Z + \Lambda_Y$, which are the performance gaps when only one factor of the joint distribution of the aggregate variables $(W, Z, Y)$ is varied one at a time.
The three terms are (1) the shift in the expected loss due to a shift in the distribution of $W$ from source to target ($\Lambda_{W}$, baseline covariate shift), (2) the shift due to a shift in the distribution of $Z|W$ from source to target ($\Lambda_{Z}$, conditional covariate shift), and (3) the shift due to a shift in the distribution of $Y|Z,W$ from source to target ($\Lambda_{Y}$, outcome shift). For large terms in the aggregate decomposition, we can drill down deeper to get a more detailed understanding of which variables are most important.

A **detailed decomposition** of $\Lambda_Z$ quantifies the importance of variable $Z_j$ for explaining the variation in expected loss differences (performance gaps) due to aggregate conditional covariate shifts given $W$.
We do so by first defining an $R^2$ measure that quantifies the percent variability explained by *partial* shifts in the conditional covariate distribution with respect to variable subsets $s$.
For interpretability, we then summarize the importance of variable $Z_j$ using Shapley values, which quantify how the addition of variable $Z_j$ to variable subsets $s$ increases the $R^2$ value, on average.
The **detailed decomposition** of $\Lambda_Y$ is defined similarly, where the importance of variable $Z_j$ is how well it explains the variation in the expected loss differences due to aggregate outcome shifts given $W,Z$.

This framework is the first of its kind that **satisfies all of the following desiderata** crucial to being useful in real-world scenarios: (i) it does *not* require knowledge of the detailed causal graph, (ii) is non-parametric, and (iii) quantifies uncertainty in explanations through confidence intervals.

References:

Jackson John W. Meaningful Causal Decompositions in Health Equity Research: Definition, Identification, and Estimation Through a Weighting Framework. Epidemiology 2021.

Meta-review: The paper solves an interesting and important problem: namely understanding how the performance of a trained model may diminish when it is deployed in a new environment. As pointed out by reviewers 7cwp and tcta, I strongly recommend that the authors add some clear, practical guidance about selecting W & Z and to include some details about the rate assumptions (as discussed in their back-and-forth with 7cwp) when revising this work.
Decision: Accept (poster)