Explanation Shift: How Did the Distribution Shift Impact the Model?
Authors: Carlos Mougan
Klaus Broelemann
Gjergji Kasneci
Thanassis Tiropanis
Steffen Staab
Venue: Accepted by TMLR ()

Abstract: The performance of machine learning models on new data is critical for their success in real-world applications. Current methods to detect shifts in the input or output data distributions have limitations in identifying model behaviour changes when no labelled data is available. In this paper, we define \emph{explanation shift} as the statistical comparison between how predictions from training data are explained and how predictions on new data are explained. We propose explanation shift as a key indicator to investigate the interaction between distribution shifts and learned models.  We introduce an Explanation Shift Detector that operates on the explanation distributions, providing more sensitive and explainable changes in interactions between distribution shifts and learned models. We compare explanation shifts with other methods that are based on distribution shifts, showing that monitoring for explanation shifts results in more sensitive indicators for varying model behavior. We provide theoretical and experimental evidence and demonstrate the effectiveness of our approach on synthetic and real data. Additionally, we release an open-source Python package, \texttt{skshift}, which implements our method and provides usage tutorials for further reproducibility.

Published in Transactions on Machine Learning Research (01/2025)
Explanation Shift:
How Did the Distribution Shift Impact the Model?
Carlos Mougan Al Office European Commission University of Southampton.
Klaus Broelemann Schufa Holding AG, Germany
Gjergji Kasneci Schufa Holding AG & Technical University of Munich
Thanassis Tiropanis University of Southampton
Steffen Staab University of Stuttgart & University of Southampton
Reviewed on OpenReview: http:s//weerriieeeee////mmmotsss
Abstract
The performance machine learning models on new data is critical for their success real- world applications. Current mothods to detoct shifts in the input or output data distributions have limitations in identifying model behavior changes when labeled data is available. In this paper, we define explanation shift the statistical comparison between how predictions from training data are explained and how predictions new data are explained. We propose explanation shift as key indicator to investigate the interaction between distribution shifts and learned models. We introduce an Explanation Shift Detector that operates on the explanation distributions, providing more sensitive and explainable changes interactions between distribution shifts and learned models. We compare explanation ion shifts with other methods that are based on distribution shifts, showing that monitoring for explanation shifts results in more sensitive indicators for varying model behavior. We provide theoretical and experimental evidence and demonstrate the effectiveness of our approach on synthetic and real data. Additionally, we release an open-source Python package, skshift, which implements our method and provides usage tutorials for further reproducibility.
1 Introduction
Machine Learning (ML) theory provides means to forecast the quality of ML models unscen data, provided that this data is sampled from the same distribution the data used to train and evaluate the model. If unseen data is sampled from a different distribution than the training data, model quality may deteriorate, making monitoring how the model's behaviour changes crucial.
Recent research has highlighted the impossibility of reliably estimating the performance of machine learning models on unseen data sampled from a different distribution in the absence of further assumptions about the nature of the shift (Ben-David et al., 2010; Lipton al., 2018; Garg et al., 2021). State-of-the-art techniques attempt to model statistical distances between the distributions of the training and unseen data the distributions of the model predictions. However, these approaches only partially capture how distribution shifts affect the interaction between new data and trained models. Additionally, they often require causal graphs or specific shift assumptions, limiting their general applicability, mainly since such assumptions can depend on the data modality. In particular, tabular data- an economically critical domain presents unique challenges. Thus, it often necessary to go beyond detecting changes input data distributions and understanding how they impact and relate to changes the model given that performance degradation can not be accurately estimated.

---

Published in Transactions on Machine Learning Research (01/2025)
The field of explainable Al has emerged way to understand model decisions and interpret the inner workings of MT. models. The core idea of this paper is to go beyond the modeling of distribution shifts and monitor for explanation shifts to signal a change of interactions between learned models and dataset features in tabular data. We newly define explanation shift as the statistical comparison between how predictions from training data are explained and how predictions on new data are explained. In summary, our contributions are:
We propose measures of explanation shifts as a key indicator for investigating the interaction between distribution shifts and learned models.
We define an Explonation Shift Detector that operates on distributions model explanations, allowing for more sensitive and explainable changes of interactions between distribution shifts and learned models in Section 3.
We compare monitoring method that is based on explanation shifts with methods that are based on other kinds of distribution shifts. We find that monitoring for explanation shifts results in more sensitive indicators for varying model behavior.
We release an open-source Python package skshift, which implements our " "Eaplanation Shift Detector", along usage tutorials for reproducibility (cf. Statement 6).
2
Foundations and Related Work
2.1 Basic Notions
Supervised machine learning induces function fo dX - dom(Y), from training data Der {(=8", (xtt; )- Where dom(X) refers to the domain of X, which is the set of all possible values that the variable can take, and dom(Y) denotes the domain target of the target variable Y. Thereby, Sa is from family of functions So E F and Der is sampled from the joint distribution P(X, Y) with predictor variables X and target variable Y. fe is expected generalize well on new, previously unseen data DY {=ze..... ........._...................... ....,, =bew) S dom(X). We write DY to refer to { :.... " and D to refer to D {1g .... v.T) For formalizations and to define evaluation metrics, it is often convenient to assune that an oracle provides such that D = {< yo " (zze" "er"","""" v."")} C dom(X) x dom(Y'.
values D = typew... ,?..., y.ew)
The core machine learning assumption is that training data D and novel data Deew are sampled from the same underlying distribution P(X, Y). Where P()) are densily functions for continuous variables or probability mass functions if the variables are discrete. The twin problems of model monétoring and recognizing that new data is out-of-distribution can now be described as predicting an absolute or relative performance drop between perf(D"") and perf(D"ew), where perf(D) (x.))D leval (fe(), v), leval is a metric like 0-1-loss (accuracy), but Drew unknown and cannot be used for such judgment in an operating system.
Therefore, related work analyses distribution shifts between training and newly occurring data. Let two datasets D, D' define two empirical distributions P(D), P(D)., then write P(D) P(D') to express that P(D) is sampled from different underlying distribution than P(D').
Definition 2.1 (Input Data Shift). We say that data shift occurs from D to Die if ((D)) ~ P(Dx"").
Specific kinds of data shift are: Definition 2.2 (Univariate data shift). There is a univariate data shift between P(D)) = P(D . and P (DD P(D).". ...... D,,"") ), if E {1...pp P(D))) na P(Dx.").
D
Definition 2.3 (Covariate data shift). There is a covariate data shift between P(D)) = P(DY, D2,,) and (DD P(Dee ....., DK."") if (DK)) ~ P(Dx<"") which cannot only be caused by univariate shift.
The next two types of shift. involve the interaction of data with the model Sa, which approximates the conditional P(D (.D)) Abusing notation, we write fe()) to refer to the multiset {.f(()) D).
P(D))
2

---

Published in Transactions on Machine Learning Research (01/2025)
Definition 2.4 (Predictions Shift). There is a predictions shift between distributions P(D and ((D related to model fo if (()( ~ "))
Definition 2.5 (Concept Shift). There is a concept shift between P(D P(D D.) and P(Dnew) = P DD if conditional distributions change, i.e. PDDF,D%) P(D .D
P(D)) P(D
In practice multiple types of shifts co-occur together, and their disentangling may constitute a significant challenge that we do not address here.
2.2 Related Work on Tabular Data
Classifier two-sample test (C2ST): Evaluating how two distributions differ has been widely studied topic in the statistics and statistical learning literature (Hastie et al., 2001; Quiñonero-Candee Candela et al., 2009; Liu et al., 2020a) and has advanced in recent years (Lee et al., 2018; Zhang et al., 2013). Using supervised learning classifiers to measure statistical tests has been explored by Lopez-Paz & Oquab (2017) proposing a blassifier-based approach that returns test statistics to interpret differences between two distributions. We adopt their power test analysis and interpretability approach but apply it to the explanation distributions instead of input data distributions. Another noteworthy recent contribution comes from Barrabés et al. (2023), who leverages a tree-based classifier as C2ST. Their approach, augmented with iterative hcuristics, aims at localizing and rectifying feature shifts within input data. In contrast, our work is distinctive in investigating the impact of distribution shifts on the model behaviour. We achieve this by applying C2ST distributions of fcature attribution explanations, providing insights into how alterations in distribution impact the model's predictive behavior.
Other methods to detect if new data is distrf-distribution (OOD) have relied on neural networks based the prediction distributions Fort et al. (2021); Garg et al. (2020). They use the maximum softmax probabilities bobaablittes/likelhoood /ikelihood as confidence score Hendrycks & Gimpel (2017), temperature or energy-based scores Ren et al. (2019); Tiu et al. (2020b); Wang et al. (2021), they extract information from the gradient space Huang et al. (2021), relying on the latent space Crabbé et al. (2021), they a Gaussian distribution to the embedding, or they use the Mahalanobis distance for out-of-distribution detection Lee al. (2018);
Park et al. (2021).
Many of these methods are explicitly developed for neural networks that operate image and text data, and often, they can not be directly applied to traditional ML techniques. For example, in deep neural networks, one can define invariances in the latent space, which do not apply to tabular data, In this work, we focus on tabular data where techniques such as gradient-boosted decision trees achieve state-of-tho-art model performance.
Detecting distribution shift and its impact on model behaviour: at detecting that data is from out-of-distribution. To this end, they have created several benchmarks that measure whether data comes from in-distribution or not (Malinin et al., 2021; Barrabés et al., 2023). In contrast, our main aim is to evaluate the impact of the distribution shift on the use of model. A typical example is two-sample testing on the latent space, as described by Rabanser et al. (2019). However, many methods developed for detecting out-of-distribution data are specific to neural networks processing image and text data and can not be applied to traditional machine learning techniques. These methods often assume that the relationships between predictor and response variables remain unchanged, i.c., no concept shift occurs. Our work is applied to tabular data where techniques such as gradient-boosted decision trees achieve state-of-the-ar model performance (Grinsztajn et al., 2022; Elsayed et al., 2021; Borisov et al., 2021).
related work has aimed
Impossibility of model monitoring: Recent rescarch findings have formalized the limitations of monitoring machine learning models in the absence of labelled data. Specifically (Garg et al., 2021; Chen el al., 2022) prove the impossibility of predicting model degradation or detecting out-of-distribution data with certainty (Fang et al., 2022; Zhang et al., 2021; Guerin et al., 2022). Although our approach does not overcome these limitations, provides valuable insights for machine learning engineers to better understand changes in interactions between learned models and shifting data distributions.
3

---

Published in Transactions on Machine Learning Research (01/2025)
Model monitoring and distribution shift under specific assumptions: Under specific types of assump- tions, model monitoring and distribution shift become feasible tasks. One type of assumption often found in the literature is to leverage causal knowledge to identify the drivers of distribution changes (Budhathoki et al., 2021; Zhang et al., 2022; Schrouff et al., 2022). For example, Budhathoki et al. (2021) use graphical causal models and feature attributions based on Shapley values to detect changes in the distribution. Similarly, other works aim to detect specific distribution shifts, such as covariate or concept shifts. Our approach does not rely on additional information, such a causal graph, labelled test data, or specific types of distribution shift. Still, by the nature of pure concept shifts, the model behaviour remains unaffected and new data need to come with labelled responses to be detected.
Explainability and distribution shift:
Lundberg et al. (2020) applied Shapley values to identify possible bugs in the pipeline by visualizing univariate SHAP contributions. Following this line of work, Nigenda et al. (2022) compare the order of the feature importance using the Normalizod Discounted Cumulative Gain (NDCG) between training and unseen data. We go beyond their work and formalize the multivariate explanation distributions on which we perform two-sample classifier test to detect how distribution shift. impacts interaction with the model. Furthermore, contribute to detecting distribution shift. In Appendix A we provide a formal comparison against Nigenda et al. (2022). Recent work by Kulinski & Inouye (2023) introduced framework for explaining distribution shifts using a transport map between a source and target distribution; our work does not only change on the distribution shift but on how these changes impact the model.
provide mathematical analysis of how the SHAP values
2.3 Explainable Al: Local Feature Attributions
Attribution by Shapley values explains machine learning models by determining the relevance of features used by the model (Lundberg et al., 2020; Lundberg & Lee, 2017). The Shapley value is a concept from coalition game thcory that aims to allocate the surplus generated by the grand coalition in a game to cach of its players (Shapley, 1953). The Shapley value Sj for the j'th player is defined via a value function valf, NN + R of players in T:
T' [2] - 1)!
5y(val5,)) =
( (TU - val),
(1)
TENVO p!
where val),=(1) = Exix -xX)))
(2)
In machine learning, N (1,...,.pp)) is the set of features occurring in the training data. Given that x is the leature vector of the instance to be explained, and the term val represents the prediction for the feature values in T that are marginalized over features that are not included in T. The Shapley value framework satisfies several theoretical properties (Molnar, 2019; Shapley, 1953; Winter, 2002; Aumann & Dreze, 1974). Our approach is based on the efficiency and uninformative properties:
Efficiency Property. Feature contributions add up to the difference of prediction from xf and the expected Valu a)(f = f(x*)= - (x)).
Uninformativeness Property. A feature j that does not change the predicted value has a Shapley value of vero.Va, f((aN(() 2,,) 3(()))))=+))) = Vr a)(f,))= 0.
Our approach works with explanation techniques that fulfill efficiency and non-informative properties, and we use Shapley values example. It is essential to distinguish between the theoretical Shapley values and the different implementations that approximate them; in Appendix D, we provide an experimental comparison of different approaches.
LIME is another explanation method candidate for our approach (Ribeiro et al., 2016b;a) that can potentially satisfy efficiency and uninformative properties, even though several rescarch has highlighted instability and difficulties with the definition of neighbourhoods. In Appendix C, we analyze LIME's relationship with Shapley values to describe explanation shifts.
4

---

Published in Transactions on Machine Learning Research (01/2025)
3 A Model for Explanation Shift Detection
Our model for explanation shift detection is sketched in Fig. 1. We define it follows:
Definition 3.1 (Explanation distribution). An explanation function S F x dom(X) + Re maps a model fo E F and data R² to a vector of attributions S(fa. x) Rr. We call S(fa, an explanation. We write S(Jo, D) to refer to the empirical erplanation distribution generated by {S(fa, x) D)
We use local feature attribution methods SHAP and LIME explanation functions S.
Definition 3.2 (Explanation shift). Given a model So learned from D explanation shift with respect to the model fo occurs if S(fa, "x ~ S(fe, D)F)
Definition 3.3 (Explanation shift metries). Given a measure of statistical distances d, explanation shift is measured as the distance between two explanations of the model fo by d(S(Ja, S(J, D)
We follow Lopez et al. (Lopez-Paz & Oquab, 2017) to define explanation shift metrics based on two-sample test classifier. We proceed depicted in Figure 1. To counter overfitting, given the model So trained on Dtr, we compute explanations {S 2x E Dxal) on an in-distribution validation data set D Given a dataset Dxm, for which the status of in- or out-of diti----iistibuttonn is unknown, we compute explanations {S(fa, x))æ x Then, we construct a two-samples dataset E {(S(fa x), az) e Daal a₂ = 0} (() E "x",, aa = 1] and we train discrimination model ge R - {0, 1) on E, to predict if explanation should be classified as in-distribution (ID) out-of-distributiio (OOD):
V arg min
Σ
x))
(3)
DE DED
where is classification loss function (e.g. cross-entropy). Do is our two-sample Lest classifier, based which AUC yields test statistic that measures the distance between the D explanations and the explanations of new data D
Explanation shift detection allows us detect that a novel dataset new changes the model's behavior. Beyond recognizing explanation shift, using feature attributions for the model gv, we can interpret how the features of the novel dataset Dxe interact differently with model Se than the features of the validation dataset D These features to be considered for model monitoring and for classifying new data out-of-distribution.
Train Classifier
Explanations
Dx.
fo: D
D
S(fo, D
Train Classifler for Two-Sample Test
9s 12
{0,1)
Explanations
D
S(fe, D
Explain Explanation Shift Detector
Reject
Ho
Not
Reject
Explanation Shift
Explanation Shift
S(gv, S(Dted)
AUC 0.5
Figure 1: Our model for explanation shift detection. The model fo is trained on Dt implying explanations for distributions Deat Drew. The AUC of the two-sample test classifier gy decides for or against explanation shift. If an explanation shift occurred, it could be explained which features of the Die"" deviated in fo compared to Diat
5

---

Published in Transactions on Machine Learning Research (01/2025)
4 Relationships between Common Distribution Shifts and Explanation Shifts
This section analyses and compares data shifts and prediction shifts with explanation shifts. Section 4.4 draws from these analyses to derive experiments with synthetic data.
4.1 Explanation Shift vs Data Shift
One type of distribution shift that is challenging detect comprises cases where the univariate distributions for each feature are equal between the source D' and the unseen dataset D but where interdependencies among different features change. Multi-covariance statistical testing is hard task with high sensitivity that can lead to false positives. The following example demonstrates that Shapley values can indicate co-variate interaction changes while a univariate statistical test will provide false negatives.
) x XY. Y. We fit a linear model then P(D))) and P(D))) are identicully distribuled with P(D and P(Dx,"), respectively, while this does not hold for the corresponding Sf(fo. D)x) and Sf Dx for j E (1.2).
M1
xx
Example 4.1. (Multivariate Shift) Let Der N
P2
X 0x2 0x
fo(21, 12) = + a. 21 + b. T2. If D N
[ml]
pax,0xx
[22] x Ox2
Si(o, = a. (x) - m1),
1 S Si new) = val( (1.2}) val({2))] [val([1}) val(
where the Shapley value function val is defined as follows:
val([1,2]) E[fe| X1 11 X2 xz] a x1 + bb222
val(0) E[fel 11 b. 12:
val({1}) E[fel X = =1] + b 12:
14.11 + + ON₂ (x) - (11) + b 11:
σx,
val({2)) = E.foo = 22 + 11,
b p22 p σX1
(I2 112 + a 111
0x2
Substituting these values, we conclude:
S S1 new) + a. 601 m1).
False positives frequently occur in out-of-distribution data detection when statistical test recognizes differences wetween source distribution and new distribution, though the differences do not affect the model behavior (Grinsztajn et al., 2022; Huyen, 2022). Shapley values satisfy the Uninformativeesss property, where a feature j that does not change the predicted value has a Shapley value of 0 (equation 2.3).
Example 4.2. Shifts on Uninformative Features. Let the random variables X1, be normally distributed with (O; 1). Let dataset D X, X2 Y with ytr X,, Thus Y LX" Let D X1 x X and Xnea normally distributed with (w) and u.o E When fo trained optimally on Drr then fo(x) F1. P(D)).) is different from P P(D but S2(Jo, DtF) = =S2(Jo, Dxe").

---

Published in Transactions on Machine Learning Research (01/2025)
Dx, N(33, 03), Dx N( og)-
If 1ig + u3
03 x c3 - P(Xx) + P(X3 aa (X1 m1) a u2)
ai (X1 10---111 m1)
X(fo,X)=
a2 (X2 - M2)
as
(X3 ax(x3
13)
Sa(fo, Dx) Ss D
4.2 Explanation Shift vs Prediction Shift
A Analyses of the explanations detect distribution shifts that interact with the model. In particular, if prediction shift occurs, the explanations produced are also shifted.
Proposition 1. Given model fo Dx - Dy. If fo(z') + fe(2), then S(fe. + S(fo.z).
This follows from the efficiency property of Shapley values (Aas et al., 2021). For example, given:
Given fo(x) + fo(z') Σ x) = fo(x) Ex(fo(Dx)] then S(fe,z) + S(fo,
(4)
diffcrence in predictions implics at least one component of the explanation vectors must differ.
However, the reverse is not necessarily true: explanation shift does not always imply a prediction shift. The following example illustrates this asymmetry.
Example 4.3. (Eæplanation shift not affecting prediction distribution) Given D is generated from (X, x X2 x X,, ~ (0, 1), X2 2), Y = X1 + X2 + and thas the optimal model is f(x) = + T2- If Dnew is generated from X ~ (1, Xy ~ U), Yaow = X + X2 + the prediction distributions are identical So (o( Jo ~ (1,3), but explanation distributions are different S(Jo, (()o *
S(e,, ) because S,(fo, x) = 04 Z4. for i (1,
We [1,2} S,(fo,z) as II:: Si(fo, Dx)) # Si(fo. Dx
fx(xx) = (D(
4.3 Explanation Shift vs Concept Shift
Concept shift comprises cases where the covariates retain a given distribution, but their relationship with the target variable changes (cf. Section 2.1). This example shows the negative result that the detection of explanation shift cannot indicate concept shift.
Example 4.4. Concept Shift Let D ~ X1 X X2 YY, and create a synthetic target y!" = + Da,1 + 02 Fi,2 e. As new dala we have Drew ~Xnew x Xnew x Y, with yeew = by + by 3i,1 + b2 2222 + e whoss coefficients are unknown at prediction stage. With coefficients ag + by, a1 + b a2 # bz. We train a linear regression fg D'g Dir. Then explanations have the same distribution. P(S DK)) PSSfee Dxe")) input data distribution P(D))) P(Dae") and predictions P(fo P(fo(Dr())) But there is no guarantee on the performance of fo Dnew (Garg et al., 2021).

---

Published in Transactions on Machine Learning Research (01/2025)
X X), N(
P((xx) = P(Dx
Y~at voataN(u,o)) + (N(H, +O,,0')
ynew ~at (N, 0²) aN o²) N(0,0"""
P(Dy) P(D
S(fo,Dx) (a(X1 mm)
((X2 :3))
Nu a²
(N2,, (P?))
(3(X, m1)
N B202)
S(h, Dx) (a(X2 M22)
(222
If a a B S(Jo, Dx) + S(hs: Dx).
In general, concept shift cannot be detected because Dr is unknown (Garg et al., 2021). Some research studies have made specific assumptions about the conditional P(Dgew), ),De in order to monitor models and P(D
detect distribution shift. (Lu et al., 2023; Alvarez al., 2023). Appendix 4.3 sketches a situation where explanation distributions are used in the context of labelled data indicate concept shift which is not the paper's main target.
4.4 Experiments on Synthetic Data
This experimental section explores the detection of distribution shifts via explanation shifts in the previous analytical examples. In this case, the model is non-linear, gradient-boosted decision tree from the XGBoost Python implementation.
4.4.1
Detecting Multivariate Shift
Given two bivariate normal distributions Dx (x), N N(., (o.b [ 9) and D (Xpem Xgem)
0.2
1 ) then, for each feature j the underlying distribution is equally distributed between Dx Vj E [1,2}: P(Dx,)==PP)) P(D))") and what is different are the interaction terms between them. We now create a synthetic target. Y X1 X2 + with < N(0, 0.1) and fit gradient-boosted decision tree fe(xx). Then we compute the SHAP explanation values for S(fa, Dx) and (()e,Dxe))
N
0,
0.2
and D
Table 1: Displayed results are the one-tailed p-values of the Kolmogorov-Smimng Smirno (KS) test comparison between two underlying distributions. Small p-values indicate that compared distributions are unlikely to be equally distributed. SHAP values correctly indicate the interaction changes that individual distribution comparisons cannot detect.
Comparison
v-value
Conclusions
P(Dx.), P(DRew) P(xx,), P(Drew)
0.33
Not Distinct
0.60
Not Distinct
Si Dx), (Jo, DR
3.9e-155 153
Distinct.
So Dx), (2 Drew)
2.9e 148
Distinct
Having drawn 50, 000 samples from both Dx and Di in Table 1, we evaluate whether changes in the input data distribution or the explanations can detect changes in covariate distribution. For this, we compare the one-tailed p-values of the Kolmogorov-Smirnoo test between the input data distribution and the explanations distribution. The "Distinct/Na Distinct" conclusion is based on the one-tailed p-value of the Kolmogorov- Smirnov test drawn out of 50, 000 samples for both distributions, this comparison methodology is used similarly for the rest of the experiments on synthetic data section 5.2. Explanation shift correctly detects the multivariate distribution change that univariate statistical testing can not detect.
8

---

Published in Transactions on Machine Learning Research (01/2025)
4.4.2 Detecting Concept Shift
As mentioned before, concept shifts camnot be detected if new data comes without target labels. However, if new data is labelled, the explanation shift can still be useful for detecting concept shifts.
Given a bivariate normal distribution Dx = (X1, ~ N(1,)) where is an identity matrix of order two. We now create two synthetic targets X? X2 + and Ynew X1 X2 + and fit two machine learning models fo : Dx + Dy and hy Dx - D Now we compute the SHAP values for S(fe, Dx) and S(rr, Dx).
Table 2: Comparison of distribution shifts in syuthetic concept shift analysis, highlighting the distinctiveness of SHAP value distributions in detecting relational changes between features and targets, even when other distributions appear equivalent.
Comparison
Conclusions
P(Dx), "(Dx P(Dy), P(D
Not Distinct
Not Distinct
P(v(Dx)), P(hr (Dx P(S(fo, Dx)), P(S(hr, Dx))
Not Distinet
Distinct
In Table 2, we see how the distribution shifts cannot capture the change the model behavior while the SHAP values are different. As in the synthetic example, in table 2, SHAP values can detect relational change between Dx and Dr, even if both distributions remain equivalent.
4.4.3 Uninformative Features on Synthetic Data
To have an applied use case the synthetic example from the methodology section, we create a three-variate normal distribution Dx = X2. X3) ~ N(0, 13), where 13 is an identity matrix of order three. The target variable is generated Y X2 X2 + being independent of X3. For both training and test data, 50, 000 samples drawn. Then out-of-distribution data is created by shifting Xa, which is independent of the target, test data D D 1.
Table 3: Distribution comparison when modifying a random noise variable on test data. The input data shifts while explanations and predictions do not.
Comparison
Conclusions
P(D ), P(D fx( fo (D
Distinct Not Distinct
S(fe, D))) S(fo, D
Not Distinct
In Table 3, see how an unused feature has changed the input distribution, but the explanation distributions and performance evaluation metrics remain the same.
4.4.4 Explanation Shift that does not affect the Prediction
In this case, we provide a situation where we have changes the input data distributions that affect the model explanations but do not affect the model predictions because positive and negative associations between the model predictions and the distributions cancel out, producing vanishing correlation in the mixture of the distribution (Yule effect 4.2).
We create train and test data by drawing 50, 000 samples from a bi-uniform distribution X1 ~ U(, 1), X₂ (1, 2) the target variable is generated by Y = X, + X2 where we train our model fo- Then if out-of- distribution data is sampled from Xnew ~ (11 X.ew ~ (O, 1).
In Table 4, we see how an unused feature has changed the input distribution, but the explanation distributions and performance evaluation metrics remain the same.
9

---

Published in Transactions on Machine Learning Research (01/2025)
Table 4: Distribution comparison over how the change on the contributions of each feature can cancel out produce equal prediction (cf. Section 1.2), while explanation shift will detect this behaviour changes on the predictions will not.
Comparison
Conclusions Not Distinct
f(D')) f(Dx S(fe, Dx,,, S(f,, Dx, Sife, X,)) S(fo, (
Distinct Distinct
4.5 Summary Comparison on Synthetic data
To assess the effectiveness of different detection methods in identifying and accounting for synthetic shifts, we present a conceptual comparison in Table 5. We evaluate these methods based on their capacity to capture synthetic shifts. We illustrate this comparison by considering two scenarios: a multicovariate shift (cf. Example 4.1) and a shift involving uninformative features (cf. Example 4.2). The complementary evaluation with related work is in the Appendix in Section A
This comparison focuses their ability to detect synthetic distribution shifts using the examples of covariate shifts and uninformative shifts. It provides valuable insights while ensuring accountability.
Table Conceptual comparison of different detection methods over the examples discussed the mathematical analysis of the main body of the paper(cf. Section 4): multicovariate shift(cf. Example 4.1 Jand an uninformative features shift(cf. Example 4.2) Learning Classifier Two-Sample test g over the explanation distributions the only method that achieves the desired results (v) and is accountabl.. We evaluate accountability by checking if the feature attributions of the detection method correspond to the synthetic shift generated in both scenarios
Detection Method
Covariate Uninformative
Accountability
Imput distribution(g.)
Prediction distribution(gr)
Input KS
X
Classifier Drift
/
X
Oulput KS
/
Output Wasserstein
/
Uncertainty
NDCG
X
X
Explanation distribution (9+)
Explanation Shift Detector
5 Empirical Evaluation
We evaluate the effectiveness of explanation shift detection on tabular data by comparing it against mothods from the literature, which are all based on discovering distribution shifts. For this comparison, we systematically vary models f, model parameterizations 0, and input data distributions Dx. The experimental results include:
Details on experiments with synthotic data (cf. Section 5.2).
Experiments on natural datasets (cf. Sections 5.3, 5.4, and further extended on Appendix B).
broad range of modeling choices for both the model f and the detector y (ef. Section 5.5). Analysis of the effects of hyperparameter variations on explanation shifts for the Explanation Shift. Detector (Appendix B.5) and the estimator (cf. Section 5.6).
A comparison of SHAP-based method against LIME, alternative explanation approach (Appendix C).
Core observations made in this section are further confirmed and refined in the Appendix, without being contradicted.
10

---

Published in Transactions on Machine Learning Research (01/2025)
5.1 Baseline Methods and Datasets
Baseline Methods. We compare method of explanation shift detection (Section 3) with several methods that aim to detect that imput data is on imput data (Rabanscr et al., 2019), (B2) prodiction shift detection by Wasserstein distance (Lu et al., 2023), (B3) NDCG-bascd test of feature importance between the two distributions (Nigenda et al., 2022), test. (Diethe et al., 2019), and (B5) model agnostic uncertainty estimation (Mougan & Nielsen, 2023; Kim al., 2020). All Distribution Shift Metries are scaled between and 1. We also compare against Classifier Two-Sample Test (Lopez- Paz & Oquab, 2017) on different distributions as discussed in Section 4, viz. (B6) classifier two-sample test on input distributions (go) following Barrabés et al. (2023) and (B7) classifier two-sample test on the predictions distributions (gr):
(B1) statistical Kolmogorov Smirnov (KS) test
(B4) prediction shift detection by Kolmogorov-Sminnn
Φ arg min
E DEDYE
Elgs (2)(
(5)
Y = arg min
E
((o+(f((z)).
(6)
DE DED
Datasets, In the main body of the paper we base our comparisons on the UCI Adult Income dataset Dua & Graff (2017) and on synthetic data. In the Appendix, we extend experiments to several other datasets, which confirm our findings: ACS Travel Time, ACS Employment, Stackoverflow dataset (Stackoverflow, 2019).
5.2 Synthetic Data
Our first experiment on synthetic data showcases the two main contributions of our method: (i) being more sensitive to changes in the model than prediction shift and input shift and (ii) accounting for its drivers. We
first generate synthetic dataset Do, with parametrized multivariate shift between (X1, X2), where p is the correlation coefficient, and extra variable X3 (0, 1) and generate our target Y = X1 X2 + X3. D using gradient-boosted decision tree, while for Bv S((s, Dral. + {0, 1),
We train the train different datasests with different values of p. For gy we logistic regression. In Section 5.5, benchmnrkk other models fa and detectors 9ø-
Sensitivity to Covariate Shift of Classifier Two Sample Test
Sensitivy to Covariate Shift Distribution Methods
0.80
Explanation Distribution 92 (ours)
1.0
Input Distribution (86)
0.75
Predidioon Distribution 9y
0.8
Distibution Shifl Metricetrics
Explanation Shift (ours)
0.6
Inpul (B1)
0.65
Pred, KS (B4)
Unisttinty (BS)
0.4
Preds. Wasserstein (B2)
NDOG (B3)
0.55
0.50
0.0
0.45
0.0
0.2
0.6 Correlation confiiclent p
0.4
0.8
1.0
0.0
0.2
0.4
0.6
0.8
1.0
Correlation confriciont
Figure 2: In the left figure, we compare the Classifier Two-Sample Test on explanation distribution (ours) versus input distribution (B6) and prediction distribution (B7). Explanation distribution shows the highest sensitivity. The right figure, related work comparison of distribution shift (ithods(B1-B5) as the experimental setup has gradual distribution shift, good indicators should follow a progressive steady positive slope, following the correlation coefficient, as method does. In Table 6 we provide quantitative evaluation.
The left image Figure 2 compares our approach against C2ST on input data distribution (B6) and on the predictions distribution (B7) different data distributions, for detecting multi-covariate shifts on different distributions. In covariate experiment, observed that using the explanation shift led to higher
11

---

Published in Transactions on Machine Learning Research (01/2025)
sensitivity towards detecting distribution shift. We interpret the results with the efficiency property of the Shapley values, which decomposes the vector (D)) into the matrix S(Jo: Dx). Moreover, we can identify the features that cause the drift by extracting the coefficients of go, providing global and local explainability,
The right image features the same setup compared to the other out- ofttoo-distribution detection methods (B1-B5). Table 6 quantitatively evaluates how the baselines correlate with the covariate correlation coefficient (p). One can see how our method behaves favourably compared to the others.
Table 6: Pearson Correlation of the correlation coefficient and baseline methods, extending Figure 2. Explanation Shift achieves better covariate shift delection on synthetic data.
Baseline
Pearson Correlation with
B1 Input KS
0.01
B2 Prediction Wasserstein
0.97
B3 Explanation NDCG
0.52
B4 Prediction KS
0.70
B5 Uncertainty
0.26
B6 C2ST Input
0.18
B7 C2ST Output
0.96
(Ours) Explanation Shift
0.99
5.3 Novel Group Shift
The distribution shift in this experimental setup constituted by the appearance of a hitherto unseen group at prediction time (the group information is not present in the training features). We vary the ratio of presence of this unscen group Deew data. The experiment is done with two fo models: a gradient- tree and a logistic regression; for Get we use a logistic regression. Results are presented in Figure 3 and Table 7. Confidence intervals extracted out of 10 bootstraps. Furthermore, compare the performance of different algorithms for so and P6 in Section 5.5, and varying hyperparameters in Scction 5.6.
decision
Exlerstion SOCa Lag
Alt Indian
Adinn
9+
Prodktios 6o
91"
fod.f minin
4+
3GB, Exp. NDCC
Frestto dars borm preri urowen grauz
Fradisn data from prevocddy urowwet grouup
data from previcualy unisse group
Novel group shift experiment on the US Income dataset. Sensitivity (AUC) increases with the growing fraction of previously unseen social groups. Left figure: The explanation shift indicates that different social groups exhibit varying deviations from the distribution on which the model was trained (White). Middle Figure: We vary the model fo by training it using both XGBoost (solid lines) and Logistic Regression (dots). The novel ethnicity group is Black. We compare Explanation Shift against C2ST on input (B6) and output (B7). Right figure: Comparison of Explanation Shift against Exp. NDCG (B4). We see how monitoring method (B4) is more unstable with a linear model, and with XGBoost it erroneously finds a horizontal asymptote. We don't compare against methods relying purely on input data such as (B1) as we are changing the model, which they don't 't take into consideration.
Figure 3:
5.4 Geopolitical and Temporal Shift
In this section, we tackle a geopolitical and temporal distribution shift; for this, the training data D for the model fo is composed of data. from California. in 2014 and Dnew for each of the states in 2018. The objective of the prediction task of this dataset is to predict whether individual's income is above $50,000. !
For information consult https://tttttaa..oooeccoeaeeetes the associated publication Ding al.
(2021b)
12

---

Published in Transactions on Machine Learning Research (01/2025)
Table Pearson correlation between baselines and the ratio presence of the unseen group. The Accountable column indicates accountability, measured as providing both theoretical underpinnings and empirical validation of the sources driving model changes, as discussed in Section
Pearson Correlation
Baseline
Accountable
Sa Log
ʃe XGB
B1 Input KS
0.99 + 0.01
0.99 0.01
B32 Pred. Wass.
0.95 = 0.02
0.98 0.01
X
B3 NDCG
0.37 = 0.25
0.81 0.100
B4 Pred. KS
0.97 0.02
0.96
X
B3 Uncertainty
0.73 + 0.10
0.11
B6 C2ST Input
0.95 0.03
0.00
X
B7 C2ST Output
0.67 = 0.13
0.96 0
X
Explanation Shift
0.98 + 0.01
0.98 0.01
Feature importance of the Explanation Shift detector (Wasserstein)
AUC performance the Explanation Shift Detector
0.72
0.88
0.11
In-ikstriudiin (CA14)
Ruce
0.072
0.26
0.29
0.33
0.18
0.18
0.015
NY15
AUC Home (wwssrttettyyy Coeniyy Divitatton
TX18
Ocsozetor
0.31
0.15
0.27
0.11
0.028
0.002
0.015
H118
0:18
0.2
0.15
0.12
0.19
0.017
0.058
750
MN18
PR18
0.11
0.16
0.28
0.14
0.081
0.016
0.099
0.
100
0.085
0.038
0.095
0.056
0.
0.049
0.019
0.00 0.045
0.022
0.13
0.058
10
Sec
0.086
0.03
0.0039
0.018
0.035
0.012
0.015
3.82
0.50
0.70
AUD
PRTS
K316
H12
0811
TXT8
NYT6
GATS
Figure 4: In the left figure, comparison of the performance of Explanation Shift Detector in different states. In the right figure, the strength analysis of features driving the change in the model, on the y-axis are the features, and the x-axis are the different states. Explanation shifts allow identify how the distribution shift of different. features impacted the model.
The model gv is trained each time on each state using only the Die" in the absence of the label, and 50/50 random train-test split evaluates its performance. As models, we use XGBoost as fa and logistic regression for the Explanation Shift Detector (9w). The model's performance was evaluated using the AUC metric in different stales, except PR18, where the model showed an explanation shift.
We hypothesize that the AUC of the Explanation Shift Detector on new data will be distinct from that in-distribution data, primarily owing to the distinctive nature of out-of-distribution model explanations. Figure 4 illustrates the performance of our method on different data dist ributtios, where the baseline is ID hold-out set of CA14. The AUC for CA18, where there is only a temporal shift, is the closest to the baseline, and the OOD detection performance is better in the rest of the states. The most disparate state is Puerto Rico (PR18).
Our next objective to identify the fcatures where the explanations differ between D'S and Drew data. To achieve this, we compare the distribution of linear coefficients of the detector between both distributions. We use the Wasserstein distance sampling fraction from California-14 and 1000 bootstraps from other states in 2018. In the right image of Figure 4, we observe that for PR18, the most crucial feature is the Place of Birth.
distance measure, generating 1000 in-distribution bootstraps using a 63.2%
Furthermore, we conduct an across-task evaluation by comparing the performance the "Explanation Shift Detoctor" on another prediction task in the Appendix B. Although some features are present in both prediction tasks, the weights and importance order assigned by the 'Explanation Shift Detector' differ. One of this method's advantages is that it identifies differences in distributions and how they relate to the model.
13

---

Published in Transactions on Machine Learning Research (01/2025)
5.5 Varying Models and Explanation Shift Detectors
OOD data detection methods based on input data distributions only depend on the detector type, independent of the model fo- OOD Explanation methods rely on both the model and the data. Using explanation shifts as indicators for measuring distribution shifts' impact on the model enables factors of the explanation shift. Therefore, in this section, we compare the performance of different algorithms for explanation shift detection using the same experimental setup. The results of our experiments show that using Explanation Shift enables us to see differences in the choice of the original model fo and the Explanation Shift Detector
account for the influencing
Estimator fe
Detector 9o
XGB
Log.Reg
Lasso
Ridge
Rand.Foress
Dec.Tree
MLP
XGB
0.583
0.619
0.596
0.586
0.558
0.522
0.597
LogisticReg.
0.605
0.609
0.583
0.625
0.578
0.551
0.605
Lasso 0.599
0.572
0.551
0.595
0.557
0.541
0.596
Ridge 0.606
0.61
0.588
0.624
0.564
0.549
0.616
RandomForest
0.586
0.607
0.574
0.612
0.566
0.537
0.611
DecisionTree 0.546
0.56
0.559
0.569
0.543
0.52
0.569
Table 8: Comparison of explanation shift detection performance, measured by AUC, for different combinations of
explanation shift detectors and estimators on the UCI Adult Income dataset using the Novel Covariate Group Shift experimental setup (cf. Section 5.3). The table shows that the algorithmic choice for Sa and ge can impact the OOD explanation performance. We can see how, for the same detector, different fo models flag different OOD explanations performance. On the other side, for the same fo model, different detectors achieve different results.
5.6 Hyperparameters Sensitivity Evaluation
This section presents an extension to our experimental setup where we vary the model complexity by varying the model hyperparamcters X). We use the UCI Adult Income dataset with the Novel Covariate Group Shift experimental setup (ef. Section 5.3). For the Stackoverflow as training data, we use the United States of America and a novel covariate group, France.
In this experiment, we changed the hyperparametors of the original model: for the decision tree, we variod the depth of the tree, while for the gradient-boosted decision trees, we changed the number of estimators, and for the random forest, both hyperparameters. We calculated the Shapley values using TreeExplainer (Lundberg et al., 2020). For the Detector choice of model, we compare Logistic Regression and XGBoost models.
The results presented in Figure show the AUC of the Explanation Shift Delector for the ACS Income dataset under novel group shift. We observe that the distribution shift does not affect very simplistic models, such as decision trees with depths or 2. However, as we increase the model complexity, the impact of ont-of-distribution data on the model becomes more pronouneed. Furthermore, when we compare the performance of the Explanation Shift Detector across different models, such Logistic Regression and gradient-boosted decision trees, observe distinct differences that the y-axis takes different values). Furthermore, in Appendix B.5, we study the effects of varying the complexity hyperparameters for the Explanation Shift Detector.
conclusion, the explanation distributions serve as a projection of the data and model sensitive to what the model has learned. The results demonstrate the importance of considering model complexity under distribution shifts.
5.7 Discussion
The Shapley value, a key component in our method, describes how a model's prediction for specific data point deviates from the mean. These theoretical considerations, which we laid out in Section have been confirmed by experimental sections.
14

---

Published in Transactions on Machine Learning Research (01/2025)
Log. Rug. Exolanation Sinft Oeteeer
XGB Explaration Br Delector
AGB
ABB Oeeven Too
0.00
0.64
0.04
Exclestton
0.00
0.50
0.48
typerpamereer
Hee. Explamaiao Sh Deactor
Delector
0.10
Duterate Erg AUC
Eapleral S1611
0.00
0:08
046
Max Deptoitypeecrrrreeeee
Max Degrhitipeecaaaaa
Figure 5: Images represent the AUC of the Explonation Shift Delector; on Iwo dalasel: Top ACS Income and Bottom Slackoverllow under novel group shifl. In the images on the left, the detector is logistic regression, and in the images the right, it is a gradient-boosted decision tree classifier. By changing the model, we can see that vanilla models (decision tree with depth or 2) are unaffected by the distribution shift, while when increasing the model complexity, the -iitr-iiiitiitttion impact of the data the model starts to be tangible
In Section 3.3, have studied input distribution shift. Our experiment shows that explanation shift detects input distribution shifts better than the best baseline methods. Table 7 showcases the two lop-performing methods comparing input distributions with Dolmogoro--Smiroot (B1) and our method -with statistically insiguificant differences.
In Section 5.2, we have studied co-variate shift. Considering, Table 6, the best method for detecting input distribution shifts, (B1), fails completely on this task. The second best method is (B2) comparison of prediction distributions using the Wasserstein distance, which also did quite well w.r.t. predicting input distribution shifts and came rather close behind our approach in both experiments.
Morcover, in our geopolitical and temporal shift experiment (cf. Scction 5.4), we demonstrate the ability to account for the drivers of model changes under such input data shifts. Cross-lask comparisons in experiments (Figure or Figure 8) highlight how explanation shift feature importance varies even when input distribution shifts remain constant during cross-task. These capabilities are not offered by any of the competing basclines. These observations are further supported by additional experiments in Scction 5.6, where we solely vary model complexity, showcasing the adaptability of explanation shifts to changes in model characteristics.
15

---

Published in Transactions on Machine Learning Research (01/2025)
6 Conclusions
Commonly, the problem of detecting the impact of the distribution shift on the model has relied on measurements for detecting shifts in the input or output data distributions or relied on assumptions either on the type of distribution shift or causal graphs availability. In this paper, we proposed explanation shifts as an indicator for detecting and identifying the impact of distribution shifts on machine learning models. We provide software, mathematical analysis examples, synthetic data, and real-data experimental evaluation. We found that measures of explanation shift can provide more insights than input distribution and prediction shift measures when monitoring machine learning models.
The potential utility of explanation shifts as distribution shift indicators that affect the model in computer vision or natural language processing tasks remains an open question. We have used feature attribution explanations to derive indications of explanation shifts, but other AI explanation techniques may be applicable and come with their advantages. Also, our approach cannot detect concept shifts, as concept shift requires understanding the interaction between imput data and response variables. By the nature of pure concept shifts, such changes do not affect the model. We work under the assumption that such labels not available for new data, nor do we make other assumptions; therefore, our method is not able to prediet the degradation of prediction performance under distribution shifts.
Limitations:
Furthermore, our use of the shap Python package for Shapley values approximation can introduce known drawbacks, as highlighted in recent literature (Bilodeau et al., 2022; Slack et al., 2020a). Additionally, our current. implementation relies on linear Shapley value interaction approximations, which can be extended following the work of Fumagalli et al. (2023); Bordt & von Luxburg (2023).
Reproducibility Statement
To ensure reproducibility, we make the data, code repositories, and experiments publicly available https://github. com/ http/////ooggaa//poaaaattttttttt Also, an open-source Python package skshift, avail- io/For our experiments, we used default scikit-learn parameters Pedregosa. et al. Experinttts were 4 vCPU server with 32 GB RAM.
able at: https://skshifft reatt//ttsshttttrrrdtdddooodo rewdthedocs..
Acknowledgements
This work has received funding from the European Union's Horizon 2020 research and innovation program under Marie Sklodowska- Curie Actions (Grant Agreement Number 860630) for the project
Intelligence without Bias". The views expressed in this article are purely those of the authors and may not, under any cireumstances, be regarded as an official position or policy of the Europcan Commission. We acknowledge the support of the Stuttgart Research Focus Interchange Forum for Rellection on Intelligent. Systems (IRIS).
References
Kjersti Aas, Martin Jullum, and Anders Løland. Explaining individual predictions when features are dopendent: More accurate approximations to shapley values. Artif. Intell., 298:103502, 2021. doi: URL http::////oo..o/11011111111 10.t//.artint....... artint. 103502.
10.1016/.arttm..2221..00
Jose M. Alvarez, Kristen M. Scott, Salvatore Ruggieri, and Bettina Berendt. Domain adaplive decision trees: Implications for accuracy and fairness. In Proceedings of the 2023 ACM Conference on Fairness, Accountability, and Transparency. Association for Computing Machinery, 2023.
Robert J Aumann and Jacques H Dreze. Cooperative games with coalition structures. International Journal of game theory, 3(4):217 237, 1974.
Míriam Barrabés, Daniel Mas Montserrat, Margarita Geleta, Xavier Giró Nieto, and Alexander G Ioannidis. Adversarial learning for feature shift detection and correction. In Thirty-seventh Conference on Neural Information Processing Systems, 2023.
16

---

Published in Transactions on Machine Learning Research (01/2025)
Shai Ben-David, Tyler Lu, Teresa Luu, and Dávid Pál. Impossibility theorems for domain adaptation. In Yee Whye Teh and D. Mike Titterington (eds.), Proceedings of the Thirleenth International Conference Artificial Intelligence and Statistics, AISTATS 2010, Chia Laguna Resort, Sardinia, Italy, May 13-15, 2010, volume 9 of JMLR Proceedings, pp. 136. JMLR.org, 2010. URL http://prooceddnng mlr. www.mmll..r...... press/ v9/david10a.hrr html.
Blair L. Bilodeau, Natasha Jaques, Pang Wei Koh, and Been Kim. Impossibility theorems for feature attribution. CoRR, abs/2212.116 11870, 2022.
Schastian Bordt and Ulrke von Luxburg. From shapley values to generalized additive models and back. In AISTATS, volume 206 of Proceedings of Machine Learning Research, pp. 709-745. PMLR, 2023.
Vadim Borisov, Tobias Leemann, Kathrin SeBler, Johannes Haug, Martin Pawelezyk, and Gjergji Kasneci. Deep neural networks and tabular data: A survey, 2021. URL. atttps///arae http://vwwrr/b//////e/e/e a21///0..//111.../../ //1///1100.0000001 01889.
Kailash Budhathoki, Dominik Janzing, Patrick Blöbaum, and Hoiyi Ng. Why did the distribution change? In Arindam Banerjee and Kenji Fukumizu (cds.), The 24th International Conference Artificial Intel- ligence and Slatislics, AISTATS 2021, April 13-15, 2021, Virtual Evenl, volume 130 of Proceedings of Machine Learning Research, pp. 1666-1677 PMLR, 2021. URL http://proceeding mlr haa..ppress/////// budhathoki21a. html.
Hugh Chen, Joseph D. Janizek, Scott M. Lundberg, and Su-In Lee. True to the model or true to the data? CoRR, (bss/2006.6233/ 2020. URL https://arxiv oooooo////a.. /a/aabb/2000 2006. 16234.
Jianbo Chen, Le Song, Martin J. Wainwright, and Michael I. Jordan. L-shapley and c-shapley: Efficient model interpretation for structured data. In ICLR 2019, 2019.
Lingjiao Chen, Matei Zaharia, and James Y. Zou. Estimating and explaining model performance when both covariates and labels shift. In Alice H. Oh, Alckh Agarwal, Daniclle Belgrave, and Kyunghyun Cho (eds.), NewrlPS, 2022.
Tianqi Chen and Carlos Guestrin. XGBoost: A scalable tree boosting system. In Proceedings of the 22nd ACM SIGKDD International Conference Knowledge Discovery and Data Mining, KDD '16, pp. 785- 794, New York, NY, USA, 2016. ACM. ISBN 978-1-4503-4232-2. doi: 22144//299967222337755, URL http://doi... acm. org/10. 1etg/0..1444 1145/29396777 2933978 2939785
Jonathan Crabbé, Zhaozhi Qian, Fergus Imrie, and Mihaela van der Schaar. Explaining latent representations with a corpus examples. In NewrIPS, pp. 12154 12166, 2021.
Tom Diethe, Tom Borchert, Eno Thereska, Borja Balle, and Neil Lawrence. Continual learning in practice. ArXiv preprint, 2019.
Frances Ding, Moritz Hardt, John Miller, and Ludwig Schmidt, Retiring adult: New datasets for fair machine learning. In NewrIPS, pp. 6478-6490 2021a.
Frances Ding, Moritz Hardt, John Miller, and Ludwig Schmidt. Retiring adult: New datasets for fair machine learning. In Marc' Aurelio Ranzato, Alina Beygelzimer, Yann N. Dauphin, Percy Liang, and Jennifer Wortman Vaughan (eds.), NewrIPS 2021, pp. 6478 6498, 2021b.
Dheeru Dua and Casey Graff. UCI machine learning repository, 2017. URL http://arrchve..ccs.u.... ml.
Shereen Elsayed, Daniela Thyssens, Ahmed Rashed, Lars Schmidt-Thiee and Hadi Samer Jomaa. Do we really need deep learning models for time series forccasting? CoRR, abs/2101.02111 2021. URL https://arxivy wtt.aaawwww..r/ab///.....00.
Zhen Fang, Yixuan Li, Jie Lu, Jiahna Dong, Bo Han, and Feng Liu. Is disstiibbiiintoo detection learnable? In Alice H. Oh, Alekh Agarwal, Danielle Belgrave, and Kyunghyun Cho (eds.), NeurIPS, 2022.
17

---

Published in Transactions on Machine Learning Research (01/2025)
Stanislav Fort, Jie Ren, and Balaji Lakshminarayanan. Exploring the limits of outtof-distribution detection. NewrIPS, 34, 2021. URL https:///rxiiv wtt..rr/////////////////////// httpess//arxiv.or///////0... 030000000///000000404 03004.
Christopher Frye, Colin Rowat, and Lya Feige. Asymmetric shapley values: incorporating causal knowledge into model-agnostic explainability. In Hugo Larochelle, Marc' Aurelio Ranzato, Raia Hadsell, Maria-Florina Balcan, and Hsuan-Tien Lin (ods.), NeurIPS 33: Annual Conference Neural Information Processing Systems 2020, NeurIPS 2020, December 6-12, 2020, virtual, 2020. URL cc/paper /aaaperr2220//aah/oo//7000000
ptt.//rrceedings..ur.p.ppp. htw.://cccccceeettoo----tattataa Abstract..tt.ll
Fabian Fumagalli, Maximilian Muschalik, Patrick Kolpaczki, Eyke Hüllermeier, and Barbara Eva Hammer. SHAP-IQ: Unified approximation of any-order shapley interactions. In Thirty-seventh Conference Neural Information Processing Systems, 2023.
Saurabh Garg, Yifan Wu, Sivaraman Balakrishnan, and Zachary Lipton. A unified view of label shift estimation. In H. Larochelle, M. Ranzato, R. Hadsell, M. F. Balcan, and H. Lin (eds.), NewrIPS, volume 33, Curran Associates, Inc., 2020. URL https://proceedings. metpeeeeddnnnn.nnnppp.psp. neurips. cttp:/ss..cppppp///000///// pdf..
pp. htee:222802222444000001888226621
Saurabh Garg, Sivaraman Balakrishnan, Zachary Chase Lipton, Behnam Neyshabur, and Hanie Sedghi. Loveraging unlabeled data to prodict out-of aut-of-distributinn performance. In NeurIPS 2021 Workshop on Distribulion Shifls: Connecting Melhods and Applications, 2021.
Amirata Ghorbani and James Y. Zou. Data shapley: Equitable valuation of data for machine learning. In Kamalika Chaudhuri and Ruslan Salakhutdinov (eds.), Proceedings of the 36th International Conference on Machine Learning, ICML 2019, 9-15 June 2019, Long Beach, California, USA, volume 97 of Proceedings of Machine Learning Research, pp. 2242 2251. PMLR, 2019. URL http://proceedings..o htt.oeeeddnnn......resss////// prress/97// ghorbani html.
Leo Grinsztajn, Edouard Oyallon, and Gael Varoquaux. Why do tree-based models still outperform deep learning on typical tabular data? Tn NewrIPS, 2022.
Joris Guerin, Kevin Delmas, Raul Sena Ferreira, and Jérémie Guiochet. Out-of-distribation detection is not all you need. In NeurIPS ML Safety Workshop, 2022.
Trevor Hastie, Robert Tibshirani, and Jerome Friedman. The Flements of Statistical Learning. Springer Series in Statistics. Springer New York Inc., New York, NY, USA, 2001.
Dan Hendrycks and Kevin Gimpel. A baseline for detecting misclassified and out-of-distribution examples neural networks. In ICLR 2017, 2017.
Rui Huang, Andrew Geng, and Yixuan Li. On the importance of gradients for detecting distributional shifts in the wild. NewrIPS 2021, abs/2110.0021s. 2021. URL https: http:///awww..o/a/////////// wtgg 0t122.101222211
Chip Huyen. Designing Machine Learning Systems: An Iterative Process for Production-Readyy Applications. O'Reilly, 2022. URI ISBN-13 17---133777-1-11077700
Byol Kim, Chen Xu, and Rina Foygel Barber. Predictive inference is free with the jacklmife- affter- bootstrap. In Hugo Larochelle, Mare' Aurelio Ranzato, Raia Hadsell, Maria-Florina Balcan, and Hsuan- Tien Lin (eds.), NewrIPS 2020, 2020. URL httpp://pproceedings segroceedingss.nurppp..... /ttprr/pss.cpppppe//2//////// hashh/ 233460aa775007666633444411111662 Abstrrcc.btract......m html.
Scan Kulinski and David I. Inouye. Towards explaining distribution shifts. In ICML, volume 202 of Proceedings of Machine Learning Research, pp. 17931-17952, PMLR, 2023.
Yongchan Kwon, Manuel A. Rivas, and James Zou. Efficient computation and analysis of distributional shapley values. In Arindam Banerjee and Kenji Fukumizu (eds.), The 24th International Conference on Artificial Intelligence and Statistics, AISTATS 2021, April 19-15, 2021, Virtual Event, volume 130 of Proceedings of Machine Learning Research, pp. 701- PMLR, 2021. URL http://prooeeding..... httss/v330/kwon22/ a.. html.
18

---

Published in Transactions on Machine Learning Research (01/2025)
Kimin Lee, Kibok Lee, Honglak Lee, and Jinwoo Shin. A simple unified framework for detecting out-of-distribution samples and adversarial attacks. In Samy Bengio, Hanna M. Wallach, Hugo Larochelle, Kristen Grauman, Nicolò Cesa-Bianchi, and Roman Garnett (eds.), NeurIPS 31: An- nual Conference on Neural Information Processing Systems 2018, NewrIPS 2018, December 8-8, 2018, Montréal, Canada, pp. (bbeadf775accc6677777777a86G Abstract html.
2018. URL https:/ /trppe//proccedings www.sseneerrrrreooeeoeo cc/pppper/011//aaaeeeeeae
Zachary C. Lipton, Yu-Xiang Wang, and Alexander J. Smola. Detecting and correcting for label shift with black box predictors. In Jennifer G. Dy and Andreas Krause (eds.), Proceedings of the 35th International Conference on Machine Learning, ICML 2018, Stockholmsmässan, Stockholm, Sweden, July 10-15, 2018, volume 80 of Proceedings of Machine Learning Research, pp. 3128 3136. PMLR, 2018. URL http: html.
/ //proceedings. mlr,
Feng Liu, Wenkai Xu, Jie Lu, Guangquan Zhang, Arthur Gretton, and Danica Sutherland. Learning deep kernels for non-parametric two-sample tests. In Proceedings of the 37th International Conference on Machine Learning, 1CML 2020, 13-18 July 2020, Virtual Event, volume 119 of Proceedings of Machine Learning Research, pp. 0316-6326. PMLR, 2020a. URL http: sttp://proceeddigg... mlr wttppp/eee//i//////// htt.mmmmm.mmmm......mmmmmmmm11 html. /ttpoodddnn.....pppr////////////
Weitang Liu, Xiaoyun Wang, John D. Owens, and Yixuan Li. Energy-based out-of-distribution detec- tion. In Hugo Larochelle, Marc' Aurelio Ranzato, Raia Hadsell, Maria-Florina Balcan, and Hsuan- Tien Lin (eds.), NewrIPS 2020, 2020b, URL Abstract. html.
neurips. cc/paper/2020/ hash//
David Lopez-Paz and Maxime Oquab. Revisiting classifier two-sample tests. In ICLR 2017, 2017.
Yuzhe Lu, Zhenlin Wang, Runtian Zhai, Soheil Kolouri, Joseph Campbell, and Katia P. Sycara. Predicting out-of-distribution error with confidence optimal transport. In ICLR 2023 Workshop on Pitfalls of limited data and computation for Trustworthy ML, 2023.
Scott M. Lundberg and Su-In Lec, A unified approach to interpreting model predictions. In Isabelle Guyon, Ulrike von Luxburg, Samy Bengio, Hanna M. Wallach, Rob Pergus, S. V. N. Vishwanathan, and Roman Garnett (eds.), NewrIPS 2017, pp. 4763-4774, 2017. URL
courips.ccpappeee
Scott M. Lundberg, Gabriel Frion, Hugh Chen, Alex DeGrave, Jordan M. Prutkin, Bala Nair, Ronit Katz, Jonathan Himmelfarb, Nisha Bansal, and Su-In Lee. From local explanations to global understanding with 2020.
explainable ai for trees. Nature Machine Intelligence,
Andrey Malinin, Neil Band, Yarin Gal, Mark J. F. Gales, Alexander Ganshin, German Chesnokov, Alexcy Noskov, Andrey Ploskonosov, Liudmila Prokhorenkova, Ivan Provilkov, Vatsal Raina, Vyas Raina, Denis Roginskiy, Mariya Shmatova, Panagiotis Tigas, and Boris Yangel. Shifts: dataset of real distributional shift across multiple large-scale tasks. In Joaquin Vanschoren and Sai-Kit Yeung (eds.), Proceedings of the Neural Information Processing Systems Track on Datasets and Benchmarks 1, NeurIPS Datasets and Benchmarks 2021, December 2021, virtual, 2021. URL httpe:///actaeetsseccccaassssss proceeddioooceccc....o.o html.
ctariips.cc/papr//2222 httpr//222/hhahaeeeeee httaaaaa.cc..122eee0000000333110
Christoph Molnar. Interpretable Machine Learning. Interpretable-m1-bott
2019. https://christon hww.:opph...thuu...o. github.thhu... 10/
Carlos Mougan and Dan Saattrup Nielsen, Monitoring model deterioration with explainable uncertainty estimation via non-parametric bootstrap. In AAAI, pp. 15037 15045. AAAI Press, 2023.
David Nigenda, Zohar Karnin, Muhammad Bilal Zafar, Raghu Ramesha, Alan Tan, Michele Donīni, and Krishnaram Kenthapadi. Amazon sagemaker model monitor: A system for real-time insights into deployod machine learning models. In KDD, pp. 3671 3681. ACM, 2022.
Chunjong Park, Anas Awadalla, Tadayoshi Kohno, and Shwetak N. Patel. Reliable and trustworthy machine learning for health using dataset shift detection. NeurIPS 2021, abs/2110.14019, 2021. URL https: orgg/abs/1110.10000
19

---

Published in Transactions on Machine Learning Research (01/2025)
Fabian Pedregosa, Gaäl Varoquaux, Alexandre Gramfort, Vincent Michel, Bertrand Thirion, Olivier Grisel, Mathieu Blondel, Peter Prettenhofer, Ron Weiss, Vincent Dubourg, et al. Scikit-learn: Machine learning in python. the Journal of machine Learning research, 12:2825 2830, 2011.
Liudmila Ostroumova. Prokhorenkova, Gleb Gusev, Aleksandr Vorobev, Anna Veronika Dorogush, and Andrey Gulin. Catboost: unbiased boosting with categorical features. In Samy Bengio, Hanna. M. Wallach, Hugo Larochelle, Kristen Grauman, Nicolò Cesa-Bianchi, and Roman Garnett. (eds.), NewrIPS 2018, pp. 6639 6649, 2018. URL https://procceeding/ hewpeedddiinnnnuuiii..e.e..eeeee www.riis.ccppppe///eeea B/hash/ 449117666aa11aaac444444622888833 Absrrrat.ttttttttaa html.
Joaquin Quidonero-Candela, Masashi Sugiyama, Neil D Lawrence, and Anton Schwaighofer. Dataset shift in
machine learning. Mit Press, 2009.
Stephan Rabanser, Stephan Günnemann, and Zachary C. Lipton. Failing loudly: An empirical study of methods for detecting dataset shift. In Hanna M. Wallach, Hugo Larochelle, Alina Beygelz- imer, Florence d' d'Alché-Buu, Emily B. Fox, and Roman Garnett (eds.), NeurIPS 92: Annual Confer- ence Neural Information Processing Systems 2019, NewrIPS 2019, December 8-14, 2019, Vancou- ver, BC, Canada, pp. 1394-1406, 2019. URL Abstract. html.
wwwpings..nuiiipi neurips. cet.ipps.ccpappe/22019 htth////////1/haahh/ /
Jie Ren, Peter J. Liu, Emily Fertig, Jasper Snoek, Ryan Poplin, Mark A. DePristo, Joshua V. Dillon, and Balaji Lakshminarayanan. Likelihood ratios for oft-t--distiibution detection. In Hanna M. Wallach, Hugo Larochelle, Alina Beygelzimer, Florence d' d'Alché-Bue Buc, Emily B. Fox, and Roman Garnett (ods.), NewrIPS 2019, pp. 14680-14691, 2019. URL https: sttps://procceedings neurips. wwtppapppppp//////eeeeeeee 1t7886888:3300a225770266663 Abssraat html.
Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. Model-agnostic interpretability of machine learning, 2016a.
Marco Túlio Ribeiro, Sameer Singh, and Carlos Guestrin. 'why should T trust. you?": Explaining the predictions of any classifier. In Balaji Krishnapuram, Mohak Shah, Alexander J. Smola, Charu C. Aggarwal, Dou Shen, and Rajeev Rastogi (eds.), Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Dala Mining, San Francisco, CA, USA, August 13-17, 2016, pp. 1135-1144, http://9999972.22977..29
ACM, 2016b. doi: 10. 01055//222667229977777 URL https://doi. http::://dooooo///////
Jessica Schrouff, Natalie Harris, Oluwasanmi O Koyejo, Ibrahim Alabdulmohsin, Eva Schnider, Krista Opsahl-Ong, Alexander Brown, Subhrajit Roy, Diana Mincu, Chrsitina Chen, Awa Dieng, Yuan Liu, Vivek Natarajan, Alan Karthikesalingam, Katherine A Heller, Silvia Chiappa, and Alexander D' Ammur. Diagnosing failures of fairness transfer across distribution shift in real-world medical settings. In Alice H. Oh, Alekh Agarwal, Danielle Belgrave, and Kyunghyun Cho (eds.), NewrIPS, 2022.
L. S. Shapley. A Value for n- o-Person Games, pp. 307-318. Princeton University Press, 1953. doi: doi: 10.1515/078140008197770
URL https://doi. http:///ooo.rg////// org/10 10. 151111///1100000190000113
Dylan Slack, Sophie Hilgard, Emily Jia, Sameer Singh, and Himabindu Lakkaraju. Fooling lime and shap: Adversarial attacks on post hoc explanation methods. In Proceedings of the AAAI/ACM Conference on AI, Ethies, and Society, AIES '20, pp. 180 186, New York, NY, USA, 2020a. Association for Computing Machinery. ISBN 9781450371100. doi: 10. 11144//33756677337555883 URL https://doi... aotoo..org/1... 1145/ 3375627. 3375830.
Dylan Slack, Sophie Hilgard, Emily Jia, Samoer Singh, and Himabindu Lakkaraju. Fooling LIME and SHAP: adversarial attacks on post hoc explanation methods. In Annette N. Markham, Julia Powles, Toby Walsh, and Anne I.. Washington (eds.), AIES '20: AAAT/ACM Conference on AT, Ethics, and Society, New York, NY, USA, February 7-8, 2020, pp. 180 186. ACM, 2020b. doi: 10.1155/375607..375500 URL attps://doi.ow 10. 1145/3375622 7377777677733788000
3375830.
Stackoverflov. Developer survey results 2019, 2019. URL 2019/.
20

---

Published in Transactions on Machine Learning Research (01/2025)
Mukund Sundararajan and Amir Najmi. The many shapley values for model explanation. In ICML, volume PMTR, 2020.
119 of Proceedings of Machine Learning Research, pp.
Haoran Wang, Weitang Liu, Alex Bocchieri, and Yixuan Li. Can multi-label classification networks know what they don't know? In Marc' Aurelio Ranzato, Alina Beygekimer, Yann N. Dauphin, Percy Liang, and Jennifer Wortman Vaughan (eds.), NeurIPS 2021, pp. 29074 29087, 2021. URL https: tttps://proceedingg -tt.rbbooooo777--bttttttt.c bbsstact.html.
Eyal Winter. Chapter 53 the shapley value. In volume 3 of Handbook of Game Theory with Economic Applications, pp. 2025-2054. Elsevier, 2002. doi: https://bbi.orr//00000////// 00000000011011114440000000000000 URL htt.://w....................aaaa
Artjom Zern, Klaus Broelemann, and Gjergji Kasneci. Interventional shap values and interaction values for piecewise linear regression trees. In Proceedings of the AAAT Conference on Artificial Intelligence, 2023.
Haoran Zhang, Harvineet Singh, and Shalmali Joshi. "why did the model fail?"; Attributing model performance changes to distribution shifts. In ICML 2022: Workshop on Spurious Correlations, Incoriance and Stability, 2022.
Kun Zhang, Bernhard Schölkopf, Krikamol Muandet, and Zhikun Wang. Domain adaptation under target and conditional shift. In Proceedings of the 30th International Conference on Machine Learning, ICML 2013, Atlanta, GA, USA, 16-21 June 2013, volume 28 of JMLR Workshop and Conference Proceedings, pp-
819 827. JMLR.org, 2013. URL http:// httt:://proceedings.. mlr wtt.gismmmr..ppss///eee wwtppppsss///epppeppppppppp //tp///////1....aaaa html.
Lily H. Zhang, Mark Goldstein, and Rajesh Ranganath. Understanding failures in out-of-distribution detection with deep generative models. In Marina Meila and Tong Zhang (eds.), Proceeulings of the 38th International Conference on Machine Learning, ICML 2021, 18-24 July 2021, Virtual Event, volume 139 of Proceedings of Machine Learning Research, pp. 12427-12436. PMLR, 2021. URL #139//haagg2 html. 2mm.l
mariinnssmlrrrrrrr mlr. prrss/
A Experimental Comparison against Specific Related Work
A.1 Comparison Against Changes on Feature Attribution Relevance
In this section, we present comparative analysis against the work of (Nigenda et al., 2022),
which involves assessing the disparity in feature importance orders between training data and out-of- distribution data. To quantify this disparity, we employ the normalized discount cunulative gain (NDCG) metric. This method is versatile, accommodating both individual sample analysis and distribution-level assessments. In cases involving distributions, we aggregate the average feature importance.
A.1.1 Novel Group Shift
experiment extends the core experiment detailed in Section 5, where distribution shifts arise due to the emergence of previously unseen groups during the prediction phase.
Experimental Set-Up:Thiss
Datasets: We use ACS Income, ASC Employment, ACS Mobility and ACS Travel time (Ding et al., 2021b). The group that not present on the features is the black ethnicity.
Baseline: We compare against the method proposed by Nigenda et al. (2022), (B6) of the experimental comparison of the main body, that compares the order of the feature importance using the NDCG between train and unseen data. We vary fe to be a XGBoost and a Logistic regression. For the "Explanation Shift Detector", F we use a logistic regression in both.
Metrics:To To facilitate a direct comparison with the Area Under the Curve (AUC) metric, we adapt the NDCG metric, to have the same interval range as follows: NI-N (DG) + 0.5, ensuring a consistent metric range.
This extended experiment aims to further validate the effectiveness of the "Explanation Shift Detoctor" under novel group shifts in real-world datasets. It demonstrates how the approach performs consistently across
21

---

Published in Transactions on Machine Learning Research (01/202))
New Category Benchmark ACSEmployment
0.60
Explanation f =
Explanation Shift f, XGB
Explanation Shift f Log
Explanation Shift fç =Log
0.76
Exp:NDGG f XGB
0.75
Exp. NDCG fo XGB
Exp. NDCG f= Log
Exp. NDOG fo Log
0.85
0.05
0.60
0.55
0.50
Fraction of data from previously unseen group
0.0
0.2
Fradion of data from previuusly unseen
0.4
0.6
1.0
New Category Benchmark -aaaaarkkAASMAAA ACSMobilit
New Category Benchmark ACaTravelTime
Explanation Shift f XGB
Explenation Sa Log
Exp. NDCG f. XGB
Exp. NDCG Log
0.70
0.66
AUG
AUC
0.00
0.60
0.50
Explanaticn Shitt -KGBB Explanation Shift Log XGB fy
0.50
Fraction data from
Fradion previcusly
Figure 6: Novel group shift experiment conducted on the 4 Datasets. Sensitivity (AUC) increases as the proportion of previously unseen social groups grows. As the experimental setup has gradual distribution shift, ideal indicators should exhibit steadily increasing slope. However, in all figures, NDCG exhibits saturation and instability. These observations align with the analysis presented in the synthetic experiment section, discussed in Section of the main paper.
multiple datasets and provides insights into the scnsitivity of model behavior as previously unseen social groups become a larger part of the prediction data. The results are presented in Figure 6, where our proposed method is compared against. Exp. NDCG (B6) across the four datasets. We can see how Exp. NDCG (B6) is more unstable and finds often an horizontal asymptot, in all the situations, this is due to changes on the feature importance order do not have information about the value, where our approach of performing a Classifier Two Sample Test the distributions of explanations do.
A.1.2 Synthetic Data Comparison
In this section, we evaluate changes in the distribution explanations and the order of feature importance when faced with a synthetic data shift scenario. We begin with bivariate normal distribution D (X,, X2) N(1,,), where I represents the identity matrix of order two. We create a synthetic target variable Y X X2+ e, and develop a machine learning model fo Dx + Dy using a non-linear model, specifically XGBoost model. Subsequently,
generate new data from D X2), (2, which constitutes shift of Drew = Dd x + 1. We then compute SHAP values for S(ff, Dx) and compare the average contributions orders.
Having sampled 50, 000 instances from both D and D we analyze whether alterations in explanation distributions and explanation importance orders can detect these changes. To achieve this, we compare one-tailed p-values from the Kolmogorov-Sminng test for explanation shifts and the order of average SHAP values between the distributions.
22

---

Published in Transactions on Machine Learning Research (01/2025)
Table 9: Comparison between distribution shifts explanations and shifts in feature attribution importance orders(previons work of (Nigenda et al., 2022)). Explanation distributions exhibit differences, while the importance order remains consistent.
Comparison
Conclusions
p P
Distinct
((S D)), P(S )) (f Dz)) Sa(fo, D)),, (fS Dx > S2(fo.Dxx))
Distinet
Not Distinct
A.1.3 Analytical Comparison under Monotonous Uniform Shift
In this section, we conduct an analytical comparison between changes in explanation distributions and changes in the order of feature importance.
Example A.1. Comparison against NDCG Let DJ (D), DX,) H1),, T) and D (D " N[u2, p2), I) where the relationship between A1 and #2 is monotonous uniform shift characterized by μ22 = N where N is real number. We fit linear model fo(X1, X2) 7 + aj X1 + X2. where >> 02. Then even if the distribution of SHAP values are distinct between S(fe. Dx) and S(fo Dx the order of importance between the distributions is not distinct. If Sioo, (f)) S2(fe. DX) then Si D S2(Jo. D But the distributions are distinct Si D2) * Si ") and S2(fo, D # D.
Dx(,, aj (Dx, - S,), ( aj (DX," u2),
11 + N.
Then Sy Dx) + (So, Dx
But Sifo, Dx) S2(fo.
Si D > S2lfo, D
Conclusion of the comparison to Nigenda et al. (2022) In the context of natural data, when confronted with novel covariate shift, our findings indicate that NDCG demonstrates limited sensitivity and fails to detect shifts when the fraction of data from previously unseen groups exceeds ratios 0.2 to 0.4 threshold.
Furthermore, in our analyses both synthetic and natural data, we observe that NDCG struggles to provide accurate and consistent estimates when faced with multicovariate shifts.
Both analytically and in our experiments with synthetic data, becomes evident that NDCG lacks robustness and sensitivity when confronted with even basic, uniform, and monotonous shift.
B Further Experiments on Real Data
In this section, we extend the prediction task of the main body of the paper. The methodology used follows the same structure. We start by creating a distribution shift by training the model fo in California in 2014 and evaluating it in the rest of the states in 2018, creating geopolitical and temporal shift. The model go is trained each time on each state using only the New in the absence of the label, and a 50/50 random train-test split evaluates its performance. As models, we use gradient-boosted decision tree (Chen & Guestrin, 2016; Prokhorenkova et al., 2018) for fe, approximating the Shapley values by TreeExplainer (Lundberg al., 2020), and using logistic regression for the Explanation Shift Detector.
B.1 ACS Employment
The objective of this task is to determine whether an individual aged between 16 and 90 years is employed or not. The model's performance was evaluated using the AUC metric different states, except PR18, where the model showed an explanation shift. The explanation shift was observed to be influenced by features such Citizenship and Military Service. The performance of the model found to be consistent. across most.
23

---

Published in Transactions on Machine Learning Research (01/2025)
of the states, with an AUC below 0.60. The impact of features such as difficulties in hearing seeing negligible in the distribution shift impact on the model. The left. figure in Figure 7 compares the performance of the Explanation Shift Detector in different states for the ACS Employment dataset.
Feature importance of the Explanation Shift detector (Wasserstein)
AUC performane of the Explanation Shift Detector
0.332
0.041
0.024
0.0093
0.015
0.041
Mi-istriouton (CA14)
Reldtonhip
0.23
0.34
9.965
0.043
0.45
0.011
0.28
0.42
0.014
TX18
Age
0.053
0.18
0.0377
0.25
0.086
0.2
0.064
230
Hi18
KS18
0.42
0.341
0.023
0.059
0.0069
0.022
0.0077
MN16
Edsorknn
0.057
AUC Komel Deeciy
130
0.2
0.02
0.012
0.037
0.027
CA18
Awcrddy
0.14
0.323
0.364
0:087
0.0092
0.018
0.012
0.315
100
Marial
0.21
0.032
0.037
0.0097
0.022
0.0078
0.013
0,054
VilenD11
0.1
0.316
0.022
0.013
0.012
0.007
0.0085
DREM
0.02
0.016
0.037
0.013
0.016
0.023
0.057
10"
0
0.0094
NATTVTTY
AUG
P1818
NY18
H118
CA18
KS18
Figure 7: The left figure compares the performance of the Explanation Shift Detector different states for the ACS Employment dataset. The right figure shows the feature importance analysis for the same dataset.
Additionally, the feature importance analysis for the same dataset is presented in the right figure in Figure 7.
B.2 ACS Travel Time
The goal of this task is to predict whether an individual has a commute to work that is longer than +20 minutes. For this prediction task, the results differ from the previous two cases; the state with the highest OOD score is K518, with the "Explanation Shift Detector" highlighting features such as Place of Birth, Race or Working Hours Per Week. The closest state to ID CA18, where there is only a temporal shift. without any geospatial distribution shift.
Feature importance of the Explanation Shift detector (Wasserttiin)
AUC performance of the Explonation Shift Delector
Rava
0.14
0.48
0.45
0.38
0.11
0.15
0.031
In:Dietnbutoon (CA14)
Citzcrohip
0.89
0.15
0.26
0.23
0.039
0.13
0.0086
NY18
Poyntyiscome
0.42
0.13
0.099
0.323
0.074
0.027
0.04
AJX Seeree Dww.yynera...oo .en
H118
0.054
100
KS18
Ago
0.008
0.05
0.027
0.073
0.035
0.033
0.045
125
PR18
Ratetonohp
Gsouaatinn
0.067
0.067
0.084
0.11
0.3039
0.052
0.025
0.02
0.023
0.064
0.012
0.034
0.0097
0.011
Explyyment
0.031
3.016
0.094
0.344
0.031
0.037
0.015
MonliySterr
0.072
0.024
0.031
0.912
0.021
0.0083
0.011
0.023
0.02
0.008
Education
0.083
3.058
0.017
0.0033
0.0288
0.0068
0.016
0.04
0.0092
0.015
0.932
0.019
0.014
0.0086
0.50
0.00
0.53
1.6
3.70
0.75 0.50
Maral
0.053
0.0041
0.028
0.0061
0.012
0.011
0. 0.0098
AUC
Figure 8: In the left figure, comparison of the performance of Explonation Shift Detector, in different states for the ACS TravelTime prediction task. In the left figure, we can see how the state with the highest OOD AUC detection is KS18 and not PR18 as in other prediction lasks; this difference concerning the other prediction task can be allributed to "Place of Birth", whose feature altributions the model finds to be more different than CA14.
B.3 ACS Mobility
The objective of this task to predict whether an individual between the ages of 18 and 35 had the same residential address as a year ago. This filtering is intended increase the difficulty of the prediction task, as the base rate for staying at the same address is above 90% for the population (Ding al., 2021b).
24

---

Published in Transactions on Machine Learning Research (01/2025)
The experiment shows similar pattern to the ACS Income prediction task (cf. Section 4), where the inland US states have an AUC range of 0.55 - 0.70, while the state of PR18 achieves higher AUC. For PR18, the model has shifted due to features such as Citizenship, while for the other states, it is Ancestry (Census record of your ancestors' lives with details like where they lived, who they lived with, and what they did for a living) that drives the change in the model.
As depicted in Figure 9, all states, except for PRI8, fall below an AUC of explanation shift detection of 0.70. Protected social attributes, such as Race Marital status, play an essential role for these states, whereas for PR18, Citizenship is key feature driving the impact of distribution shift in the model.
Feature importance of the Explanation Shift
detector (Wasserstein)
AUC performance of the Explanation Shift Detector
1.2
UE15
UT1
UR1:
UMS4
0.5
0.35
0:21
0.2
0:17
0.033
0.069
InDKianibttio (CA14)
Razi
0.16
0.24
0.30
0.24
0.17
0.19
0.011
0.32
0:28
0.083
0.31
0.245
0.11
0.02
AUC Korm Daceety Deriheon
TX18
H118
Education
0.067
0:16
0/50
0.028
0.053
0.28
0.079
0.063
0.02
0. 3.037
0 0.046
0.13
0.031
0.059
0:17
0 0.06
0.046
0.023
0.062
0.018
0.099
120
KS18
0.14
0.064
0,082
0.05a
0:11
0.02
0.251
MN11
cow
0.12
0.048
0.05
0.079
0.039
0.044
0.015
-00
PNNG
0.000
0,084
0:17
0.0095
0.043
0.053
0.055
CA18
Maneal
3REM
0.058
0.082
0.09
2.044
0.087
0,024
0.0072
0.036
0.035
0.021
0.039
0.061
0.029
0.011
0:1
0.01
0.013
0.083
0.018
0.05
3.047
0.011
0.00e6
0.024
0.0071
0.057
0:077
0.018
0.0065
0.0057
2.023
0.01
0.021
0.01
0.0062
0.0095
0.032
0.036
NATIVITY
Endloymett
AUIC
PRTS
NNN1
H11
K818
NF18
TX11
GA16
Figure 9: Left figure shows a comparison of the Explanation Shift Delector's performance in different states for the ACS Mobility dataset. Except for PR18, all other states fall below AUC of explanation shift detection of 0.70. The features driving this difference are Citizenship and Ancestry relationships. For the other states, protected social attributes, such Race Marital status, play an important role.
B.4 StackOverflow Survey Data: Novel Covariate Group
This experimental section evaluates the proposed Explanation Shift Detector approach on real-world data under novel group distribution shifts. In this scenario, new unseen group appears at the prediction stage, and the ratio of the presence of this unseen group in the new data is varied. As training data country, use the United States. The model fe used is gradient-boosted decision tree logistic regression, and logistic regression is used for the detector. The results show that the AUC of the Explanation Shift Detector varies depending on the quantification of OOD explanations, and it shows more sensitivity concerning model variations than other state-of-the-art techniques.
The dataset used is the StackOverflow anmual developer survey, with over 70,000 responses from over 180 countries examining aspects of the developer experience (Stackoverflow, 2019). The data has high dimensionality, leaving it with +100 features after data eleansing and feature engineering. The goal of this task is to predict the total annual compensation.
B.5 Sensitivity Analysis of Explanation Shift Detector Hyperparameters
This scction extends our experimental setup by evaluating the impact of varying the complexity of the Explanation Shift Detector through its hyperparameters, represented as gV(S(JO, X)). We focus on the geopolitical and temporal shifts in the US Income dataset, using CA14 for training and PR18 the out-of-distribution (OOD) dataset.
In this analysis, we adjusted the hyperparameters for the Explanation Shift Detector across different model types. For decision trees, we varied tree depth; for gradient gradient-boosted decision trees, we changed the number of estimators; and for random forests, both hyperparameters were adjusted. To evaluate estimator choices, we compared Logistic Regression and XGBoost models. While this setup resembles the one in Section 5.6, here we focus altering the complexity of the Explanation Shift Detector rather than the original estimator.
25

---

Published in Transactions on Machine Learning Research (01/2025)
AUC of Exolanation Shift Detector
Gemaany
of Explanatiion Shift Detector
0.003
United Kingdom of Great Britain and Nottharn Ireland
India
0.9
0.90
Canada
France
Unffled Kingdom of Great Britain and Northern Ireland India
Germiny
0.65
0.7
Canada
0.50
France
0.6
0.00
02
04 65 0. 1.0
02
0.4
0.6 08 1.0
Fraction of data from previously unseen group
Fraction of data from previously unseen group
Figure 10: Both images represent the AUC of the Explanation Shift Detector for different countries the StackOverflow survey dataset under novel group shift. In the left image, the estimator, So, is a gradient-boosted decision tree; in the right image, for both cases the detector, ge, is logistic regression. By changing the type of estimator model, we can see how different types of models are affected differently for the same distribution shift.
Varying the Complexity of the Explanstion Shift Detector
Varying the Com plexity of the Explanation Shift Detector
f=XXBB e
0.75
(r*XSB, Dydttstioon
f=X0 q Randdm Fores!
0.70
0.890
0.629
0.55
5.000
0.00
6.570
0.66
ignine 9 9Ga
Linina Dudssion
Lnnar g Random Forest
centring
Mac www.Seeeellypppppaan Explametion St1
Mac www.ohttiigpppesmeeeee Expleratoo Snf
Figure 11: The AUC of the Explanation Shift Detector under geopolitical and temporal shifts in the ACS Income datasel. Left: results for a linear model as the estimator. Right: results for a gradient-boosted decision tree regressor. Increasing the complexity of the Explanation Shift Detector highlights the impact of distribution shifts, particularly for the XGB model. For linear models, the AUC plateaus due to their limited complexity.
As shown in Figure 11, the Explanation Shift Detector requires a minimum level of complexity to detect the effects of distribution shifts. Beyond this threshold, the AUC values plateau, indicating diminishing returns from increasing the Explanation Shift Detector's complexity. For the linear model as estimators (fa), the AUC of the Explanation Shift Detector plateaus with less complex parameter, likely due to the simplicity of how the distribution shift impact simpler model. In contrast, when the XGB model increases the detector's complexity, it continues to improve performance, reflecting the model's greater capacity to learn complex patterns.
In conclusion, explanation distributions act as projections of the data and model behavior, capturing the impact of learned features. While Explanation Shift Detector complexity is important under distribution shifts, its influence appears secondary to that of the original estimator model complexity, as explored in Section 5.6.
C LIME as an Alternative Explanation Method
Another feature attribution technique that satisfies the properties above (efficiency and uninformative features Section 2) and can be used to create the explanation distributions is LIME (Local Interpretable Model-
26

---

Published in Transactions on Machine Learning Research (01/2025)
Agnostic Explanations). The intuition behind LIME is to create local interpretable model that approximates the behavior of the original model in small neighbourhood of the desired data to explain (Ribeiro al., 2016b;a) whose mathematical intuition is very similar to the Taylor series. In this work, we have proposed explanation shifts as a key indicator for investigating the impact of distribution shifts on ML models. this section, we compare the explanation distributions composed by SHAP and LIME methods. LIME can potentially suffer several drawbacks:
Computationally Expensive: Its current implementation is computationally expensive than current SHAP implementations such TreeSHAP (Lundberg et al., 2020), Data SHAP (Kwon et al., 2021; Ghorbani & Zou, 2019) or Local and Connected SHAP (Chen et al., 2019), the problem increases when we produce explanations of distributions. Even though implementations might be improved, LIME requires sampling data and fitting a linear model, which is a computationally more expensive approach than the aforementioned model-specific approaches SHAP.
The definition of local "neighborhood", which can lead to instability of the explanations. Slight variations of this explanation hyperparameter lead to different local explanations. In Slack et al. (2020b) the authors showed that the explanations of two very close points can vary greatly.
Local Neighborhood:
Dimensionality: LIME requires as hyperparameter the number of features to use for the local linear approximation. This creates a dimensionality problem as for our method to work, the explanation distributions must be from the exact same dimensions as the input data, Reducing the number of features to be explained might improve the computational burden.
Sensitivity Multicovariate Shift
AUC Explanation Shift Detector
0.8
Explanation Shift SHAP
0.000
ShapAsian LimeAsian
Explanation Shift Lime
0.629
0.7
0.600
Shift
0.575
0.6
AUG of
0.5
0.500
0.0
0.2
0.6
0.8
0.75
Correlation coefficient
Fracion 0OD data
Figure 12: Comparison of the explanation distribution generated by LIME and SHAP. The left plot shows the sensitivity of the predicted probabilities to multicovariate changes using the synthetic data experimental setup on the main body of the paper. The right plot shows the distribution of explanation shifts for a New Covariate Category shift (Asian) the ASC Income dataset.
Figure 12 compares the explanation distributions generated by LIME and SHAP. The left plot shows the sensitivity of the predicted probabilities to multicovariate changes using the synthetic data experimental sctup from Figure 2 in the main body of the paper. The right plot shows the distribution of explanation shifts for New Covariate Category shift (Asian) in the ASC Income dataset. The performance of OOD explanations detection is similar between the two methods. Still, LIME suffers from two drawbacks: its theoretical properties rely on the definition of a local neighborhood, which can lead to unstable explanations (false positives or false negatives on explanation shift detection), and its computational runtime required is much higher than that of SHAP (see experiments below).
C.1 Runtime
We analyzed the runtimes generating the explanation distributions using the two proposed methods. The experiments were run on a server with four vCPUs and 32 GB of RAM. We used shap version 0.41.0 and 1ime version 0.2.0.1 as software packages. In order define the local neighborhood for both methods, we use all the data provided background data in this example. As fo model,
XGBoost and compare
27

---

Published in Transactions on Machine Learning Research (01/2025)
the results of TreeShap against LIME. When varying the number of samples use five features and while varying the number of features we use 1000 samples.
method
me
Tree Shap
Shap
Lime
Line
10
Time Tine
187
Figure 13: Wall time for generating explanation distributions using SHAP and LIME with different numbers of samples (left) and different numbers of columns (right). Note that the y-scale is logarithmic. The experiments were run on server with four vCPUs and 32 GB RAM. The runtime required create an explanation distribution with LIME is far greater than SHAP for gradient-boosted decision tree
Figure 13, shows the wall time required to generate explanation distributions using SHAP and LIME with varying samples and columns. The runtime required of generating an explanation distributions using LIME is much higher than using SHAP, mainly when producing explanations for distributions. This is because LIME requires training local model for each instance of the input data to be explained, which can be computationally expensive. In contrast, SHAP relics on houristic approximations to estimate the feature attribution without training a model for each instance. The results illustrate that this difference in computational runtime becomes more pronounced as the number of samples and columns increases.
We note that limiting the number of features be explained can further reduce the computational burden of generating the explanation distributions, this reduces the dimensionality of the explanation distributions. However, this will inhibit the quality of the explanation shift detection, it won't be able to detect changes in the distribution shift that impact the model on those features.
Given the current state-of-the software packages, we have used SHAP values due to the lower runtime required and the theoretical guarantees that hold with the implementations. In the experiments performed in this paper, we are dealing with medium-scaled dataset with around 1, 000, 000 samples and 20 25 features. Further work be envisioned developing novel mathematical analysis and software that study under which conditions which method is more suitable.
D True to the Model or True to the Data?
The "Explanation Shift Detector" proposed in this work relies on the explanation distributions that satisfy efficiency and uninformative theoretical properties. We have used the Shapley values explainable AI method that satisfies these properties. However, the correct way to connect model to coalitional game, which is the central concept of Shapley values, is a source of controversy, with two main approaches (i) an interventional (Aas et al., 2021; Frye et al., 2020; Zern et al., 2023) or (ii) an observational formulation of the conditional expectation @xpuatationSSuudaaaajaan
Najmi, 2020).
In the following experiment, we compare what are the differences between estimating the Shapley values using one the other approach. We benchmark this experiment the four prediction tasks based the US census data (Ding et al., 2021a) and using the "Explanation Shift Detector", where both the model fe(X) and (S X)) are lincar models. We will calculate the Shapley values using the SHAP lincar explainer. 2
The comparison depends on a feature perturbation hyperparameter: whether the approach to compute the SHAP values is either interventional or correlation dependent. The interventional SHAP values break the dependence structure between features the model uncover how the model would behave the inputs
explainers..........eeeeaee..eee Linear html
28

---

Published in Transactions on Machine Learning Research (01/2025)
were changed (as it was intervention). This option is said to stay "true to the model", meaning it will only give allocation credit to the features that the model actually uses (Aas et al., 2021).
On the other hand, the full conditional approximation of the SHAP values respects the correlations of the input features. the model depends on one input that is correlated with another input, then both get some credit for the model's behaviour. This option said say "true the data", meaning that only considers how the model would behave when respecting the correlations in the imput data (Chen al., 2020).In In our case, will measure the difference between the two approaches by looking at the linear coefficients of the model Ie and comparing the performance using the geo geo-political and temporal experiment of the previous section 5, for this case between CA14 and PR18.
Table 10: AUC comparison of the "Explanation Shift Detector" between estimating the Shapley values between the interventional and correlation-dependent approaches for the four prediction tasks based on census dataset (Ding et al., 2021a). The % character represents the relative difference. The performance differences are negligible.
Interventional
Observational
%
Income
0.736438
0.736439
1.1e-06
Employment
0.747923
0.747923
4.44e-07
Mobility
0.690734
0.690735
8.2e-07
Travel Time
0.790512
0.790512
3.0e-07
Table 11: Linear regression coefficients comparison of the "Explanation Shift Detector" between estimating the Shapley values between interventional and the correlation-depedden approaches for one of census-based prediction tasks (ACS Income). The % character represents the relative difference. The coefficients show negligible differences between the calculation methods
Interventional
Observational
%
Marital
0.348170
0.348190
2.0e-05
Worked Hours
0.103258
-0.108254
3.5e-06
Class of worker
0.579126
0.579119
6.6e-06
Sex
0.003494
0.003497
3.4e-06
Occupation
0.195736
0.195744
8.2e-06
Age
-0.018958
-0.018954
4.2e-06
Education
-0.006840
-0.006840
5.9e-07
Relationship
0.03/1009
0.034212
2.5e-06
In Table 10 and Table 11, can see the comparison of the effects of using the aforementioned approaches
to learn our proposed method, the "Explanation Shift Detector". Even though the two approaches differ
theoretically, the differences become negligible when explaining the protected characteristic, i.e. when providing the linear regression coefficients.
29


Reviews:
[{"review_id": "rxa7fH5l4x", "reviewer": "Reviewer_6Tsi", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths\n1. The paper looks into an interesting area of analyzing model performance based on explanation shifts.\n2. The authors have made considerable effort in including the different distribution shifts including their mathematical notations.  \n3. The authors have also attempted to theoretically analyze the sensitivity of different shifts towards model performance on new data.\n\nWeaknesses:\n1. From Eq. 1, we see that use of Shapeley values restricts the number of features to p. How does the explanation shift results vary with different p values?  Specially in complex datasets, the number of features determining the model behavior is large. The evaluation datasets have comparatively smaller feature dependencies such as UCI Adult Income dataset, which has only 14 features. This does not quite bring out the efficacy of the explanation shift. The approach should be evaluated over more complex datasets.\n2. The novelty of the approach seems to be weak. It only deals with tabular data, which greatly restricts application. Also, how does the proposed approach perform over other data-types besides the tabular data. Can this approach be generalized? How does it work for non-tabular data such in case of images or language?\n3. Also how does the proposed approach perform in low-data regime? The authors should evaluate the dependency of explanation shift on the scale of data samples required. Often the out-of-distribution data may have a comparatively smaller sample size, in such a case, can the approach still evaluate the distribution shift and if so to what confidence?\n4. The authors claim that using the explanation shift led to higher sensitivity towards detecting distribution shift. This claim is brought forth using synthetic data only. What about in cases of natural datasets? Also, does that mean it is more prone towards error? What are the chances of false negatives and how does the sensitivity vary with different features?", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper proposes an approach for detecting the impact of the distribution shift on the model performance. The authors focus on detecting shifts that affect the behaviour of a particular model trained on labelled source data and evaluated over an unlabelled target domain. The approach relies on the computation of Shapley values using a pre-fixed set of features in the training data and predicted features on the target data. The authors evaluate their approach over a variety of baselines using both synthetic and real data, revealing the greater sensitivity of explanation shifts towards model performance compared to data shifts.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "The changes requested are related to the weaknesses stated above and have been told elaborately. Here they are reiterated briefly for completeness. But refer to the points above in the 'weaknesses' section above for detailed explanation.\n - p value is an important component of the approach. There should be a proper sensitivity analysis of the p value in the experiemnts along with evaluation on complex datasets.\n - Using only tabular data to show the performance greatly reduces the applicability of the approach. Showing it for visual or text data is necessary.\n - Showing the applicability of the approach with out-of-distribution data having comparatively smaller sample size.\n - claim that using the explanation shift leading to higher sensitivity towards detecting distribution shift needs to be validated on natural datasets beyond synthetic data only.", "broader_impact_concerns": "There isn't any immediate concerns in this aspect.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "ZPhjsiEFoR", "reviewer": "Reviewer_Crub", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "**Strengths**\n- The concept of explanation shift—measuring changes in feature attributions across domains to detect distribution shifts—is both novel and intriguing.\n- The paper provides several case studies in Section 4, aiding in the understanding of the proposed method and its ability to detect various distribution shifts. In Section 5, the authors offer experimental evidence demonstrating improved performance in detecting distribution shifts.\n- Additionally, the source code for the experiments is made publicly available. The authors also contribute an open-source Python package, **skshift**, which implements the \"Explanation Shift Detector\" and includes tutorials to facilitate reproducibility.\n\n**Weaknesses**\n1. **Lack of Detailed Behavioral Analysis**: \n   While the method can detect whether a distribution shift has occurred and the model’s behavior has changed, it does not explain *how* the model's behavior has changed or how feature attributions have shifted. This lack of interpretability limits the method's applicability. The authors should address these concerns, specifically how feature attribution changes and what these changes mean in practice.\n\n2. **Concept Shift Detection**: \n   While it is promising that the method can detect concept shift when test labels are available, this scenario is often impractical. The authors should explore whether it can detect concept shift with only a few available test labels.\n\n3. **Modality Scope**: \n   The proposed explanation shift method appears modality-agnostic, and while the authors acknowledge this in the limitations section, it raises the question of why the scope of the application was limited to the tabular domain. It would be beneficial to explore whether promising results could also be achieved in vision or NLP domains.\n\n4. **Use of SHAP for Feature Attribution**: \n   The method uses SHAP, but the authors mention known drawbacks of SHAP in the limitations section. Could better performance be achieved using more advanced feature attribution methods? The authors should address these concerns and explain whether those drawbacks could be mitigated with other techniques.\n   \n5. **Classifier Choice**: \n   Why didn’t the authors explore using more complex classifiers, like deep neural networks, instead of logistic regression? Is this decision driven by the characteristics of the tabular domain?\n\n6. **Synthetic Data in Experiments**: \n   A significant portion of the analysis and experiments was conducted on synthetic data. In particular, the claim in Table 5 that the explanation shift detector can handle *all* covariate shifts and uninformative shifts seems overgeneralized. The authors should provide more robust guarantees.\n   \n7. **Limited Experimental Scope**: \n   Modern work in the tabular domain often involves testing on dozens of datasets to prove performance improvements [1]. The experimental scope feels too limited, especially since the main experiments were conducted on the ACS Income dataset, with only additional tests on ACS Travel Time, ACS Employment, and StackOverflow. This raises concerns about potential cherry-picking of datasets.\n   \n8. **Limited Performance Improvement**: \n   In Tables 6 and 7, the performance improvement is marginal when compared to methods like B2 (Prediction Wasserstein), B7 (C2ST Output), and B1 (Input KS).\n\n9. **Computational Cost**: \n   The computational cost of the proposed method is likely very high, especially since SHAP must be calculated for every dataset. This raises concerns about scalability, particularly for larger datasets. An analysis of computational cost, comparing it with other baselines, is necessary.\n\n[1] Yan et al. \"Making Pre-trained Language Models Great on Tabular Prediction.\" ICLR 2024.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper introduces the concept of **explanation shift**, defined as a statistical comparison between how model predictions are explained on training data versus target data. The approach involves an **explanation function** $ \\mathcal{S}(f_{\\theta}, x) $, which takes the model $ f_{\\theta} $ and a $ p $-dimensional data point $ x \\in \\mathbb{R}^{p} $, and returns a vector of attributions $ \\mathcal{S}(f_{\\theta}, x) \\in \\mathbb{R}^{p} $.\n\nInitially, this function is fit on the validation set of the source domain. It is then re-applied to the test set of the target domain to detect differences in feature attributions between the two domains. A classifier is trained to distinguish between source and target domain attributions. The classifier’s AUROC is subsequently calculated, and a two-sample classifier test (utilizing one-tailed p-values from the Kolmogorov-Smirnov test) is performed to determine whether the AUROC significantly deviates from 0.5, indicating the presence of a distribution shift.\n\nThe authors compare their explanation shift-based monitoring method with other approaches focused on different types of distribution shifts. The findings suggest that monitoring explanation shifts provides a more sensitive indicator for detecting changes in model behavior.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "It would be nice to answer the points pointed out in the above Weaknesses, reflect the answers to correct the parts that need to be modified in the text of the text, and if there are additional experiments that are conducted, it would be good to include them in the text.\n\n**Writing Suggestions**\n\n- The paper feels like it hasn't been thoroughly proofread. The current quality falls short of TMLR's publication standards, and substantial corrections are required throughout the manuscript.\n- **Example 4.1–4.3**: The equations are understandable but need better alignment and formatting. They are currently ill-posed and visually unclear.\n- **Example 4.2**: The formulas below Example 4.2 appear strange—are they directly related to Example 4.2?\n- **Avoid Redundancy**: Repeating information multiple times should be avoided. For instance, the table caption text like “Displayed results are the one-tailed p-values of the Kolmogorov-Smirnov test comparison between two underlying distributions” or “Novel Covariate Group Shift for the 'Asian' group with a fraction ratio of 0.5 as described in Section 5” only needs to be mentioned once in the main text.\n- **Gradient boosting decision tree** / **gradient-boosting decision tree** ⇒ Should be consistently referred to as **gradient-boosted decision tree**.\n\n**Typos and Formatting Errors**\n\n- (p. 2) `{(xtr_0 , ytr_0) . . . , (xtr_n , ytr_n)}` ⇒ `{(xtr_0 , ytr_0), . . . , (xtr_n , ytr_n)}`\n- (p. 5) `$R^p \\rightarrow {0,1}$` ⇒ `$\\mathbb{R}^{p} \\rightarrow {0, 1}$`\n- (p. 10) `the Appendix in sectionA` ⇒ `the Appendix in Section A`\n- (p. 10) `paper(cf. Section 4):` ⇒ `paper (cf. Section 4):`\n- (p. 10) `shift(cf. Example 4.1 )a` ⇒ `shift (cf. Example 4.1 ) a`\n- (p. 10) `shift(cf. Example 4.2) .` ⇒ `shift (cf. Example 4.2).`\n- (p. 11) `datasests` ⇒ `datasets`\n- (p. 15) `Section› 5.3,` ⇒ `Section 5.3`\n- (p. 15) `wrt.` ⇒ `w.r.t`", "broader_impact_concerns": "There are no broader impact concerns if the requested changes are appropriately addressed.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "X6GyAo7KTD", "reviewer": "Reviewer_NeeN", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Overall, I am very happy with the work, and I do not have many criticisms toward it. \n\nStrength: \n- The demonstration of the pipeline in terms of theories (multivariate shifts and uninformative shift) and empirical results are comprehensive enough \n\n- The effectiveness of the methodology is good in terms of sensitivities \n\nWeakness: \n\n- The pipeline relies on a trained classifier to predict if an explanation should be classified as ID or OOD. This would rely on a well trained classifier as a prerequisite to ensure the pipeline works properly. Though it is followed by roc and two sample test, which could lower the risk of having a weakly trained classifier, the risk would be concerning if applied to more complicated data such as images or texts. \n\n- Perhaps adding some discussions or ablations about the on how the choices and the training quality (well trained or not) of the classifier would impact the result would be good. (Since as discussed in the discussion section that the extension to cv and nlp is still under developing, I agree that it would not be the ultimate focus of this paper. )", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The authors introduce a new definition of explanation shift as the statistical comparison between how train data and new data contribute/ related to predictions measured by shapely values. The author further introduces the pipeline along with metrics to measure distribution and explanation shift. Experiments were done on synthetic and real tabular data to demonstrate the pipeline.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "- When I was reading definition 3.2, I got upset with the implicit assumption that there is no overfitting/underfitting for the model f$\\theta$. Though this is discussed in the later paragraph, I think that all of the assumptions should be clearly stated.", "broader_impact_concerns": "This is well discussed in the discussion section.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: All the reviewers recommend accepting this paper to TMLR. Here are some highlighted comments on the strengths of the paper mentioned by the reviewers:

- The concept of measuring changes in an explanation function is interesting.
- The case studies in Section 4 are useful.
- A good amount of empirical studies in Section 5 show the effectiveness.
- The authors share the work as an open-source Python package and tutorials.
- Several different distribution shifts are considered.

I also recommend acceptance with minor revision. This work makes good contributions to the methodology and its understanding to the domain of distribution shift detection, although some weaknesses are pointed out by the reviewers (see also Explain your recommendation to the ICLR Journal-to-Conference Track).
Decision: Accept (poster)