Data-faithful Feature Attribution: Mitigating Unobservable Confounders via Instrumental Variables
Authors: Qiheng Sun
Haocheng Xia
Jinfei Liu
Venue: NeurIPS 2024 poster (2024)
Keywords: feature attribution
Shapley value
integrated gradients
Primary area: interpretability_and_explainability

Abstract: The state-of-the-art feature attribution methods often neglect the influence of unobservable confounders, posing a risk of misinterpretation, especially when it is crucial for the interpretation to remain faithful to the data. To counteract this, we propose a new approach, data-faithful feature attribution, which trains a confounder-free model using instrumental variables. The cluttered effects of unobservable confounders in a model trained as such are decoupled from input features, thereby aligning the output of the model with the contribution of input features to the target feature in the data generation. Furthermore, feature attribution results produced by our method are more robust when focusing on attributions from the perspective of data generation. Our experiments on both synthetic and real-world datasets demonstrate the effectiveness of our approaches.

Data-Faithful Feature Attribution: Mitigating Unobservable Confounders via Instrumental Variables
Qiheng Sun 1:2, Haocheng Xia Jinfei Liu Li
17hejiang University
Hangzhou High-Tees Tech Zone (Binjiang) Institute of Blockchain and Data Security Siebel School of Computing and Data Science
University of Illinois edu. en, hxia7@11linois..or
(qiheng_sun,
Abstract
The state-of-the-art feature attribution methods often neglect the influence of unobservable confounders, posing a risk of misinterpretation, especially when it is crucial for the interpretation remain faithful to the data. To counteract this, we propose new approach, data-faithful feature attribution, which trains confounder-free model using instrumental variables. The cluttered effects of unobservable confounders in a model trained as such are decoupled from input features, thereby aligning the output of the model with the contribution of input features to the target feature in the data generation. Furthermore, feature attribution results produced by our method are more robust when focusing on attributions from the perspective of data generation. Our experiments on both synthetic and real-world datasets demonstrate the effectiveness of our approaches.
1
Introduction
The increasing complexity and opacity of machine learning (ML) models real-world applications boost the demand for feature attribution 8 The feature attribution methods have been developed to help users understand why a model produces certain outputs from specific inputs. For instance, loan applicant rejected by bank's 's decisioonmmking model might seek reasons behind the denial and what changes could potentially reverse the model's 's decision. Some recent studies [19 9 have shifted the focus of feature attribution from traditional model-centric perspective to new perspective that data-centric. Specifically, users may wish to assign importance values to features according to the data generation process, referring to the causal relationships through which features influence the target feature. For example, consider a medical application setting where a patient seeks to understand which personal features aggravated the illness and the cooperative impact of all features on the illness. What the patient really wants to know is how all features collaboratively contribute to the illness, rather than diagnostic model's prediction output. These aspects of feature attribution align with the concepts of model fidelity and data fidelity, respectively. Here, model fidelity refers to the attribution being consistent with the output of the explained model, while data fidelity pertains the attribution being consistent with the data generation process.
SHapley Additive exPlanations (SHAP) [27] and Integrated Gradients (IG) 38 are prevalent rep- resentatives of two distinct series of feature attribution methods, each uniquely satisfying critical axioms including sensitivity, implementation invariance, completeness, and symmetry (28). These properties are essential for ensuring reasonability and fairness in feature attribution. The SHAP-basee methods, grounded in game theory and particularly the Shapley value (32. offer interpretations by evaluating all possible combinations feature contributions a discrete feature space. In contrast,
*Jinfei Liu the corresponding author.
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

Unobservable Confounder
Other Features
Ability
Instrumental Variable
Parental
Education
Income
Education
Figure I: Arrows indicate direct effects. Since Ability s not directly observed and is correlated with education, the influence that should be attributed to Ability (arrow 1) is erroneously attributed to Education (arrow 2) in feature attribution. To fix this issue, Parental Education can be used as an instrumental variable for investigating the true impact of Education on Income.
the IG-based methods, which are an analog of the Aumann-Shaple method 39 from cost-sharing, focus on continuous feature spaces by using path integration over gradients. It is worth noting that even when we aim for interpretations that are faithful to the data, we still rely on a model to predict the target feature values when certain features are selected or excluded in the attribution process. However, when unobservable confounders exist, both SHAP-based and IG-based methods may lead to misunderstandings if applied directly to the widely used predictive model. This is because unobservable confounders, although impacting the output, are entirely overlooked from the process of attribution. Consequently, their influence is instead attributed to other correlated features.
Motivation example. As shown in Figure T suppose model predicting personal income includes Education as an input feature, and Ability serves as a confounder if the model does not incorporate it input feature. Because Ability has indirect impact Income through its influence on Education and a direct impact on Income simultaneously, the existing feature attribution methods tend to incorrectly attach the direct impact of Ability on Income to the impact of Education on Income due to their correlation. This concealed correlation may lead to incorrect attribution on the role of educational level in personal income, resulting in overestimation of the education returns, as demonstrated in our experiments using real datasets ($ 5.
In this paper, develop method to eliminate unobservable confounder effects in feature attribution in order to achieve a deeper understanding from the perspective of data fidelity. The instrumental variable method is widely used for causal analysis (4 It lies in identifying features that directly affect those influenced by confounders, while not having a direct impact on the outcome themselves. By using the instrumental variable to control confounders that influence specific features, any resulting changes in the outcome variable are driven solely by how the instrumental variable alters that feature. For example, in Figure I when examining the impact of Education on Income, suitable instrumental variable could be the variable Purental Education as analyzed in appendix Section F.2 By observing the changes in Education resulting from variations in Parental Education, we can then discern the true effect of Education on Income, effectively isolating it from other confounders. Intuitively. the instrumental variable approach can help mitigate the impact of unobservable confounders in feature attribution. However, the instrumental variable approach mainly focuses on evaluating the isolated impact of individual features influenced by unobservable confounders on the outcome variable, while feature attribution lies in considering the cooperative attribution of features. This means evaluating the combined contribution of Education and Other Features on Income.
To bridge this gap, we propose using a two-stage model training with the instrumental variable that disrupts the association between confounders and other features. Specifically, the model is trained using features re-estimated through instrumental variables and collaborative variables. This ensures that the influence of confounders remains consistent despite variations in feature coalitions. Therefore, the marginal contribution of each feature, which is determined by assessing the impact on the model's output with and without the feature, is not affected by any confounders. Furthermore, the attribution value of each feature, calculated as the average of its marginal contributions across different feature coalitions, remains influenced by confounders. This alignment allows the contribution of input features to the model output to mirror their contribution in data generation to the target feature, thereby facilitating the attribution to be faithful to the data. Feature attribution involves explaining a model output by assigning attribution scores to the input instance. However, our focus is on data-faithful feature attribution, i.e., we are not trying to explain the output of a specific model but trying to explain the target feature through model.

---

Contributions. To the best of our knowledge, we are the first to identify crucial issue: unobservable confounders compromise feature attribution, especially when data fidelity essential. To tackle this challenge, we propose training models free of confounders using instrumental variables, ensuring the feature attribution will remain faithful to the data. We validate the effectiveness our proposed methods using both real and synthetic datasets, observing that our method achieves up to a 67% relative improvement over the baseline methods in terms the error of attribution ratio metric the real dataset.
Preliminaries
2.1 Problem Setup
We aim to quantitatively assess the influence of each input feature the target feature. This assessment can be viewed as a contribution assignment problem in the context cooperative game theory [41] Formally, given explained input vector of d features x" = {æi..... 22), a baseline input x', and a model f g + I which approximates the data generation equation for the target feature, our objective is to explain the difference target feature, i.c., y y/' conducting data-faithful attribution for the input features. We assume x' and x' are of the same dimensionality d, and each entry can be either discrete or continuous. We denote by X the set of input features and the target feature, partitioning X into two subsets: X, which is influenced by unobserved confounders (denoted as E), and X, the set of other observable features. For clarity and convenience, we use æ, y. a T. e to denote possible values within feature sets Y, X, X, &, respectively. For given subset of features S, we denote the subset of the original vector of values by using S as a subscript, e.g.. as = (2: Jives.
2.2 SHAP-bases Attribution
Shapley Value. Consider set of players N
coalition is subset of / that cooperates to complete a task. A utility function (IS) (S S .)) is the utility of coalition S for task. The marginal contribution player with respect to coalition is (i)) -U((). Shapley value is the unique metric that satisfies the properties of fair reward allocation, including balance, symmetry, additivity, and zero element 41. It measures the expectation of marginal contribution by i in all possible coalitions. That is,
u(su (1))-u(S) (W---)
SV.
Σ
W
SCA(HO)
Computing the exact Shapley value requires enumerating all utilities for all player subsets, Therefore, the computational complexity of exactly calculating the Shapley value is exponential [46].
SHAP 27 utilizes the concept of Shapley values attribute the contribution of each feature in
model. In the existing SHAP-based methods [2] the definition of utility functions for interpreting
an input æ can be divided into categories ( condition expectation Shapley and intervention
Shapley. In condition expectation Shapley, following the assumption that the features are generated according distribution D), the utility function defined by 21 UC((S) = E' (x) = x33 (37 based on the condition expectation of the model prediction under feature set S. In intervention Shapley, the utility function is defined by (z2(S) [ do
ED fxr
= a3)] 44 where the
operation do(æs x²)) means we intervene on the features S in variable x to be the same as the
features x while the features outside of S in are influenced following the causal relationships
of the features [30]. For conciseness, we omit the subscript D in the expectation term in the rest of the paper.
2.3 IG-based Attribution
IG is pivotal method for model attribution 388 particularly well-suited for deep neural networks due to its prerequisite that the model be continuously differentiable. This approach calculates the cumulative gradients along a straight-line path extending from a baseline input x' to the explained input x". Mathematically. the attribution IG, assigned to particular feature x; for given input x"
3

---

and baseline x' is defined as:
Ig. (a", f) (a) a!d) [" Of (æ' + (a x')) do.
(I)
dx;
Remarkably, IG shares similarities with the Aumann-Shaples approach [39 and satisfies several essential properties, including linearity, dummy attribution, Affine Scale Invariance (ASI), propor-
tionality, and symmetry [37]. Recent research has enhanced IG through its application to complex models and the refinement of the integrated path 28).
3 Misattribution with Unobservable Confounders
In this section, we carry out a theoretical analysis to demonstrate how unobservable confounders mislead the feature attribution of both SHAP-based and IG-based methods. To show the influence of unobservable confounders in feature attribution, we first employ simple structural equation to characterize the data-gent gett-generating process, expressed as
y g (z. σ) +.,
where g(t T) can represent any linear or nonlinear continuous relationship involving both क and at. The equation allows us to clearly recognize the individual contributions of ã and to y, while also considering the unobserved effects encapsulated in the error term Given that X is the set of features influenced by unobservable confounders E, generally follows that given two data instances a and 22, E|e [ *f EE [Ø2] when # A 2 since X influenced by & while |E|e [z]]] = IR(e [T2) is valid as the feature set X not affected by E
3.1 Example of Errors for Data-Faithful Feature Attribution
We discuss how unobservable confounders introduce errors in data-faithful feature attribution with toy example of Figure in condition expectation Shapley. We assume as ability, & as the measurement of education Tevel, æ the work time in a week (i.e., the other variable), and y the weekly income. ă. T. and each represents single numerical variable. The data generation equations are defined as follows:
Uniform(O, 1), z Uniform((. 1) +,,
Uniform(O, 1), E-
In this case, ability influences education, and the three features all have a direct influence on income. Consider specific data instance x = [Ø* æ=] [1.5, 1] we are curious about the contributions of the individual's s education level and work hours to their income compared to the features distribution. In this example, means assigning value to the individual's education level ã* = 1.5 and work time क" = 1 to evaluate their contribution to weekly income in comparison to the distribution of education levels and work hours of the population.
First, we conduct feature attribution with a model f that is trained to fit E[ylœ] = Efyl ã. x] g( Φ) - EE (ē, ø which simulates the widely used supervised machine learning model train- ing paradigm in reality. The utility derived from the model is (S) = E( æs 23. B[g(@, Φ) æts 25s] E[e|æs az). The condition expectation Shapley value of x' is SVC. ([((((()))
4c (Ø)) + [2/ x²)) - u ((x*(()))) })}} Replacing the according util- [E]g(#", x) + Bie|ă", x - E[[g( x) Elelä, x) + Elg(ā", x²)]
ities, we have sve. a.
B . Blg(Ø, )") E(e|ū, T'}
Then, we conduct feature attribution for H" with the term which it really contribute to y, i.c., g x). T²(5) (S) Eīg(ū l xz). According to the definition of condition expectation Shapley, S 17 ((w*)) - 77 (0) C7 (1 Th) tf (==1)) Replacing (
The utility derived
the accrrording utilities, we have SVC
+[gg .े' -
Errors in Feature Attribution Values. The values of each expectation term computed according to the data generation equations are shown in Table []] Since きे is independent from we have Ele/ Ø"] = [[e[@, क and Ble|ä" Ø²) =Eee[* .... By substituting the values for each expected
term, we can obtain that SVC. 0.875, SV 0.625. SVC. 0.325, and SV 0.325. It's

---

Table 1: Value each expectation term in SV
E1
g σ
g T
g C. T
9 2
T
E c 2. T
E
0.5
0.75
0.5
1.5
0.5
0.75
clear that attribution based on the model trained to fit Ely[a]] which is a common training paradigm in supervised learning, tends to give wrong attribution value of ø", which is the excessive attribution value education level in this example. The intuitive reason is that the model associates the direct effect of ability income with the education level, i.e., the influence of is attached to at. However, Ele|@", ; æ: - Ele at, a: is not actually in the effect of x" on y because ă does not influence during the data generation process.
3.2 Errors in Feature Attribution with Unobservable Confounders
We analyze the attribution errors when the feature attribution conducted on a model f trained to fit E[ylæ] = Blylë, u. æ] in supervised learning for SHAP-based method (Propositions) and and IG
(Proposition 3 respectively.
Proposition 1. The expected error for marginal contribution of in condition expectation
Shapley with model trained to fit Ely|z] is E[c) = z'uunt - -Ele = z'gl. resulting
an expected deviation of attribution value by ASV; x EscAr( (M57 -1 s EEe/zs = z's}..
=
Proof. Due to the limited space, please see the appendix for detailed proof. The same to the following propositions.
Proposition 2. The expected error for marginal contribution of feuture in intervention Shapley with mode J trained to fit Elyja] is Eppeldo((sssu = =FSu())) Ele Eee]do(xrs .r's),, resulting
an expected deviation of attribution value by ASV, = EscM(( ( [Bleldo(xxs ='su(a) - TE|[dd((xs- = a3))]]-
Proposition 3. The expected error for attribution value of feature i using IG with model ff trained to fit E[yla] is AIG; (x) x²() f 0f(x'
((e'+a(("=-- +a(="=='))) 3g( +c ca(="= "e'))))
do.
dm:
On,
4 Mitigating Unobservable Confounders via Instrumental Variables
As demonstrated in Section 3 when it pertains to unobservable confounders, the prevalent feature attribution methods including SHAP and IG inevitably lead to misunderstandings that are not faithful to the data, since they rely on predictive model f trained to fit Ely|æ=] This fundamental because the trained predictive model has already associated the unobservable confounders with the input features. Therefore, it is tempting to ask: how can we decouple the confounders from their correlations with other features in the used model f?
4.1 Motivation of Using Confounder-free Model
One may think a straightforward solution is directly training a model f to fit g( .:). Unfortunately, it is nearly impossible since we cannot remove the influence of unobservable confounders in the target feature y. To bridge the gap, we provide an alternative solution to train model that gives the same attribution results for the input features as it is trained to fit g g æ). Denote by á = g(Ø Ø) + E(e]. and f is trained to Ely ã.. æ7. The influence of ã and 7 on ÿ is identical to their impact on y, as both are encompassed within ()).
Example. For the toy example Section 3 the utility calculated with f trained to fit E ã
g(ā T Ele] is ú (S) B[g(@, x las a'sl + E[e]. The condition expectation Shapley value
of ãt SV :. 1/ (((((())) - ((( + (( क }} - (' {}]} By replacing the according
utilities, we have SV = (B(g(("" .b) + [(() [=) - REe] + B[g("", , x²)) + E(c]
Elg(ă, x²)] E(e] [Big(@", x)] E[g(@, m] [ xx)] Eig(ū, x")] SV
5

---

The advantage of attribution based on ÿ lies in the term E[c] being constant, thereby breaking the association between and the imput features.
Proposition 4. From the perspective of data generation, the contributions of features in o and to are equivalent to their contributions to ÿ. When using the condition expectation Shapley, intervention Shapley, and Integrated Gradients (IG) methods, the attribution values of each feature in &. and are identical for both models f = g x.) and f = g x.) E(c)
4.2 Confounder-free Model Building
With Proposition 4 the problem becomes how to train the model f to fit Eljja, क now. To achieve this, we introduce the instrumental variables.
Instrumental Variable. The features that are used as instrumental variables, denoted as V can be effectively utilized in our model if they satisfy the following three key properties. 1) relevance: should correlate with X, ensuring that y can serve as reliable proxy for these features. 2) exogeneity: V/ should be uncorrelated with the latent confounders E, ensuring that it is not influenced by these unobserved factors. 3) exclusion restriction: V/ should influence the outcome Y solely through its effect on X. In other words, apart from its interaction with X, should not have any other direct indirect pathways affecting Y. This ensures that the effect of v/ on Y can be unambiguously attributed to its relationship with X. The effectiveness of IV-SHAP and IV-IG may be reduced when the three assumptions of instrumental variables are violated. However, the extent of this reduction depends how severely the assumptions are violated.
With the help of instrumental variables, we can establish the following equation by taking the expectation of given E and 2b,
Bly M bb} = Elg( T) 2b
B[c] = 9 Elc v/),
where zb is a possible value of V and dM 1b) is the conditional distribution of X. Given the T training data instances, the optimal parameters of model trained to fit g σ)) + E(e] within the function space H. are identified by minimizing the following objective:
(c. - dM (Z)Z,, (M v))
min JEN
yr
(2)
where represents the loss metric we used evaluate model performance and is the index of specific data instance. We provide the training methods for supervised neural network models which are extensively employed in the real world Section 43 Specifically, we discuss their loss functions and gradient computations when the objectives are regression and classification problems. The training steps are inspired by the two-stage training causal effect estimation [2] and counterfactual prediction [18]. The re-estimated unconfounded values are sampled from the first-stage trained model, the sampling has little influence on the implementation and computation complexity of the second-stage model training. Therefore, the two-stage training process does not limit the method's practical usability. Due to the limited space, we provide details of model training with discrete input features and non-gradient model training in appendix Section D
4.3
Confounder-free Model Training
Model Training for Continuous Feature Attribution. In the regression task, where the model a neural network trained for forecasting continuous value, it is also denoted as fe (w, T), where represents the model parameters. As our objective, adopt a l2 loss function. With the unknown conditional distribution of X given X and V, we initially utilize a neural network model, denoted Mg. where Ø is the model parameters, to approximate this distribution. The l2 loss function for determining the optimal model parameters subsequently approximates as per the following equation
C(T:0) = |TI-1 Σ yt (t, d.d. (5 Iz, 2b,)
(3)

---

where the integral term estimates the expected output of fe under the distribution approximated by Ma. By employing the relevant calculations, we ascertain that the gradient of the loss function with respect to the thh training data point is
VeL. - 21E M m.)))) [y fo (τ, xx)] ( Lff (ă, xt)].
(4)
In short, our training process comprises two fundamental steps: 1) an instrumental variable method is applied to re-estimate æ. mitigating the impact of unobservable confounders, and 2) this refined æ is utilized to calculate the gradients for f.
Model Training for Discrete Feature Attribution. In the classification task. where is a discrete variable representing classes, we adapt the loss function the multi-class cross-entropy
(f yvrr In for (æe, Te )dM (Ø Te, Wi)
L(T; - ΣΣ r=1
(5)
In this formulation, Vt.r represents the true label of the tth data point in the 7h category. denotes the total number of distinct classes into which the target variable can be classified. The probability of the model classifying data point into the ,Th category given by fo,r (Ø, Tz). The gradient calculation for the tt training data point, considering this loss function, is then
R Σ
Yt.r So.r (+. x..) Sorr (Ø, Ft)
VoL. -E M
(6)
This adaptation of the loss function for discrete target variables ensures that model can handle classification tasks, effectively optimizing its performance across multiple categories.
Feature Attribution Computation. The exact computation of Shapley value and integrated gradients needs huge cost while approximation methods are widely used. Towards practical applications, we further propose a Shapley value approximation method and integrated gradients approximation method for saving computation costs in Sections E. and E.22 respectively. The correlation of input features may affect the fatthflailsllness of IV-SHAP and IV-IG. We can combine methods which deal with the correlated input features to the two-stage model to better capture these correlations. For example, on-manifold Shapley (6 can used to account for feature correlations, while causal Shapley [19 can be applied if the causal structure of the input features is known.
5 Experiments
In this section, present empirical evaluation in detail. We employ synthetic and real-world datasets to evaluate the faithfulness and robustness against unobservable confounders of feature attributions given by proposed data-faithie feature attribution methods prevalent SHAP-basee and IG methods. Comparisons non-gradient training models, and the approximation methods of SHAP and IG given in appendix Sections F3 F.4, and F.3] respectively. Our code can be found in the repository at https://gi thtp::://githbb.... com ZJJU DIEE-DIVER//A DIVER/IV-SHAA SHAP
feature attribution for classification problems, feature attribution
5.1 Experiments on Synthetic Datasets
We first conducted data simulation experiment to validate our proposed methods' effectiveness. Synthetic datasets offer an advantage studying feature attribution, as we can obtain the ground truth of attribution values according to the data generation equation which is unobtainable in most real datasets.
Dataset Generation Process. We generated two synthetic datasets, each containing four parts: an unobserved confounder e, a variable influenced by the unobserved confounder, collaborative variables æ = { (1 < i < 6), and the target feature y. Notably, dataset A and dataset B share the same æ and tb. 6 is formulated by uniform variable and parameter p which controls the noise level. The generation of these features adhered to specific functional relationships, illustrated in
7

---

P+RA
Figure 2: Evaluation results on synthetic Dataset A.
2+6.3
2-0.4
p=0.6
6.11
Figure 3: Evaluation results synthetic Dataset B.
the equations below.
V ~ Uniform (0, 1),
I4 ~ Uniform (1 sis 6),
& ~ Uniform(O, 1),
p,
2 Ev 1 + + +²²²) /3,
Dataset A
. + I2 + v Iz + t
1 4 +
+ +eA
6
B exp(v p p-1)
5 (VeB vi t + +122) /3,
Dataset
yd B exp(X<1 F32
VT
exp( ) +Tg + V
2
+ 6
6
Compared Methods. We utilized two representative feature attribution algorithms, SHAP 27 and IG (38), as the baseline methods. Specifically, we trained a neural network model on synthetic datasets to fit [E[y|a) ax) as the baseline model. Then we applied the two feature attribution methods to attribute contributions for input features. For our proposed approach, we employed the same neural network architecture but trained the model fit Elgla [=] with instrumental variables. Our attribution approaches, applied to the model trained with the instrumental variable, are referred to as IV-SHAP and IV-IG, corresponding to SHAP and IG, respectively.
worth noting that for the model, the explicitly input data features have no causal relationships among them. Therefore, Causal SHAP [19] Asymmetric SHAP [4], BSHAP [37] and SHAP are equivalent in this context.
Experimental Results. We randomly generated 1000 data points based on the data generation cquations. We then adjusted features स and of each data point by subtracting certain value as baseline input. We conducted experiments with varied subtracted values set at 0. 125, 0.25, 0.375, and 0.5. For each data point, we applied IV-SHAP, IV-IG, SHAP, and IG to attribute the contributions of the features. Subsequently, we compared the attribution values of feature 2 against the ground truth obtained directly from the data generation equations. We observe that the errors in attribution results provided by IV-SHAP and IV-IG are significantly smaller than those of SHAP and IG. The absolute errors in the attribution values of each algorithm for every data point, as compared to the benchmark, illustrated in Figures 2 and
8

---

5.2 Experiments on Real-world Datasets
We conducted experiments on two real-world datasets to demonstrate the efficacy of IV-SHAP and IV-IG in practical scenarios where data generation processes are black-box. Initially, we excluded specifie features to simulate unobservable confounders. Subsequently, we trained model using the complete set of features to establish ground-trut -tuut attribution values, thereby assessing the robustness and reliability of the compared methods under realistic conditions.
consisting of 758 entries with 20 variables each, gathered from the U.S. labour market. This dataset is extensively used in research explore the impact of education on income. In the study examining the relationship between the logarithm of weekly carnings (Iw) and other features such as educational years (edu), years of work experience (expr), tenure at the current organization (tenure), marital status (mr)) residence in the South (ms), and urban residence (smsa), there exists a significant challenge. Ability, as an unobservable confounder, not only directly influences also their income. Using feature attribution methods like SHAP and IG without accounting for the confounder might incorrectly attribute the effect of ability on income to correlated educational levels. To address this issue, we incorporated the educational years of the mother (medu) as an instrumental variable to affect an individual's ss education. Additionally, IQ scores (iq) and knowledge in the world of work test (kww) in the dataset, serving as crucial indicators of ability, offer a unique opportunity to approximate the ground truth of the real contribution of each feature.
Real-world Datasets. The first real dataset we used is the Griliches76 dataset 17 36
individual's education level but
Compared Methods. We assume a decrease in educational years for each individual as baseline inputs and execute a two-phase experiment. Initially, we omit IQ and the world of work test scores, calculating attribution values for IV-SHAP, IV-IG, SHAP, and IG. Note that the process of computing these attribution values using IV-SHAP, IV-IG, SHAP, and IG is consistent with the synthetic dataset experiments. These methods are employed to evaluate the impact of reduced education years. Subsequently, we incorporate IQ and the world of work test scores train a new model and recalculate attribution values using SHAP and IG. Due to the absence of real-world benchmark in the reality dataset, we adopt the attribution results from the model, which includes unobservable confounders in its training process, as our benchmark for comparison.
Evaluation Metric. Denote the average attribution ratio of IV-SHAP by EARIVSHAP 1 -E= IVSHAP= where IVSHAP, refers to the educational attribution value for the 7th data point, bw
n calculated by the IV-SHAP method, IVSHAP, represents the extent to which changes in educational years influence the income in the jth data point. (w; is the income for the ith data point. EARIvSHAP the average of the absolute values of the ratios between the educational attribution values and income across all data points. This measure provides a comprehensive quantification of the impact of educational years on income. The average attribution ratio for reduced educational years, cal- culated by SHAP in the model trained with IQ and the world of work test scores, is denoted as EAR = > BMSHAAP; where BMSHAP; represents the benchmark attribution of educa- tion on income, incorporating IQ and the world of work test. We then compute the absolute relative error between the attributions of IV-SHAP and the benchmark using the formula FARyySSP E.ARgSSKA
EARauSHAP
For the SHAP algorithm, EAR still serve as a benchmark for EARSHAP. For the attribution values calculated by IV-IG and IG, we use the average attribution ratio obtained by the IG algorithm on the model that includes IQ and kww as inputs as the benchmark.
Experimental Results. The experimental results are shown in Table 2 which demonstrates that our methods can significantly reduce the attribution error. The values in the table represent the mean of five independent runs, with the standard deviation following each mean.
Table 2: Relative error of each attribution algorithm.
YEAR
SHAP
0.566 + 0.041
0.569
+ 0.053
0.569 + 0.040
0.548
+ 0.047
0.552 + 0.038
IV-SHAP
0.184
0.052
0.162
0.026
0.172
0.025
0.157
0.019
0.146 + 0.021
IG
0.554
0.044
0.582
# 0.052
0.583
+ 0.044
0.467
0.047
0.538
+ 0.043
IV-IG
0.178
1 0.025
0.152 :: 0.020
0.165
1 0.028
0.149
1: 0.023
0.135 1 0.011

---

Empirical Analysis. The second real dataset we use is the Angrist and Krueger dataset [3, which is an American census dataset consisting of statistical data on people born in specific years, including variables such as age (AGE), an education level (EDUC), weekly wage (LWKLYWGF)) marital status (MARRIED), and race (RACE). We employed this dataset to examine the confounder effects of the ability on the attribution of education for income. In this dataset, we used the quarter of birth as an instrumental variable for years of education. The rationale behind this is the compulsory education laws in various states, which typically mandate schooling until the age of 16. Students born early in the year often start school later, leading to systematic differences in educational duration based on birth quarter. However, this dataset lacks measures of intelligence the factor of ability. so we cannot conduct the experiment like the previous one. Nevertheless, the average attribution ratio of one additional year, calculated by SHAP, IV- IG, and IV-IG, 0.0218, 0.0206, 0.0218, and 0.0207, respectively. This aligns with the observation that people may overestimate educational returns because of neglecting the confounder ability in [6]
work capability to assess
Limitations
Despite the strengths of our approach, there are several limitations to consider which are shown as follows:
Dependence on the Availability of Instrumental Variables: Our approach assumes the presence of suitable instrumental variables for features affected by unobserved confounders. However, in practical scenarios, finding appropriate instrumental variables can be challeng- ing sometimes. For further information on identifying instrumental variables, refer to works such as
10, and 23
Linearity Assumption in Theoretical Derivations: Our theoretical derivations are based on the assumption that the influence of unobserved confounders on the target features is linear. This assumption does not hold in all real-world situations. Nevertheless, in our experiments with real datasets in Section 5.2 our attribution method showed significant improvements over existing methods, even when the influence of unobservable confounders on features was non-linear.
These limitations highlight areas for future research, particularly in developing methods that do not rely on the availability instrumental variables and that can give theoretical analysis of non- linear effects of unobserved confounders. Addressing these aspects enhance the practicality and applicability of our methods.
7 Conclusion
In this paper, we focus on addressing the effects of unobservable confounders in feature attribution, emphasizing feature attribution being faithful to data. The proposed method improves the under- standing of the causal factors driving an outcome variable, going beyond standard attribution scores that simply deseribe predictive models. Our approach of training confounder-free models using instrumental variables effectively isolates the impact of confounders, enhancing the robustness of feature attribution results. Our validations using real and synthetic datasets confirm the effectiveness of the proposed methods. For future work, intend develop methods that do not rely on the availability of instrumental variables and that can provide a theoretical analysis of the non-linear effects of unobserved confounders. For the broader impacts of the paper, please see Section A in the appendix due to the limited space.
Acknowledgment
The authors would like to thank the anonymous reviewers for their helpful comments. This work was supported in part by the National Key RD Program of China (FB311111)))000, NSFC grants (62102352, U23A20306), The Zhejiang Province Pioneer Plan (2024C01074), and NSF grant (CNS-2125530).
10

---

References
[1] N. Akhtar and M. A. Jalwana. Towards credible visual model interpretation with path attribution. In International Conference Machine Learning, pages 439-457. PMLR, 2023.
[2] D. Angrist and G. W. Imbens. Two-stage least squares estimation of average causal effects in models with variable treatment intensity, Journal of the American statistical Association, 90(430):431 442, 1995.
[3] J. D. Angrist and A. B. Krueger. The effect of age at school entry on educational attainment: an application of instrumental variables with moments from two samples. Journal the American statistical Association, 87(418):328-333 1992.
[4] M. Baiocchi, J. Cheng, and D. S. Small. Instrumental variable methods for causal inference. Statistics in medicine, 33((3):2297-2340 2014.
[5] Bordt and U. von Luxburg. From shapley values to generalized additive models and back. In International Conference on Artificial Intelligence and Statistics, pages 709-745. PMLR, 2023.
[6] D. Card. The causal effect of education earnings. Handbboo of labor economics, 3:1801- 1863, 1999.
[7] J. Castro, D. Gómez, and J. Tejada. Polynomial calculation of the shapley value based on sampling. Computers Operations Research, 36(5))1726-1777) 2009.
[8] H. Chen, I. C. Covert, S. M. Lundberg, and S.-L. Lee. Algorithms to estimate shapley value feature attributions. Nature Machine Intelligence, 566):590-601, 2023.
[9] H. Chen, D. Janizek, Lundberg, and S.-I. Lee. True to the model true the data? arXiv preprint arXiv:2006. 16234, 2020.
[10] N. M. Davies, G. D. Smith, F. Windmeijer, and R. M. Martin. Issues in the reporting and conduct of instrumental variable studies: a systematic review. Epidemiology, 24(3):363-36,, 2013.
[11] X. Deng and C. H. Papadimitriou. On the complexity of cooperative solution concepts. Math. Oper: Res., (9(2):257-266 1994.
[12] R. Dwivedi, D. Dave, H. Naik, S. Singhal, R. Omer, P. Patel, B. Qian, Z. Wen, T. Shah, G. Morgan, et Explainable (xai): Core ideas, techniques, and solutions. ACM Computing Surveys, 55(9))1-33, 2023.
[13] Enguchard. Sequential integrated gradients: simple but effective method for explaining language models. In Findings of the Association for Computational Linguistics: ACL 2023, Toronto, Canada, July 9-14, 2023, pages 7555-7565 Association for Computational Linguistics, 2023.
[14] C. Frye, C. Rowat, and I. Feige. Asymmetric shapley values: incorporating causal knowl- edge into model-agnostic explainability. Advances in Neural Information Processing Systems, 33:1229 12212299999999921111111111 2020.
[15] H. Gao, J. Li, W. Qiang, L. Si, B. Xu, C. Zheng, and Sun. Robust causal graph representation learning against confounding effects. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 37, pages 7624-7632. 2023.
[16] A. Ghorbani and J. Zou. Data shapley: Equitable valuation data for machine learning, In International conference machine learning, pages 2242-2251. PMLR, 2019.
[17] 7. Griliches. Wages of very young men. Journal of Political Economy, 84(4, Part 2):869-S85, 1976.
[18] J. Hartford, G. Lewis, K. Seyyoon-roow, and M. Taddy. Deep iv: A flexible approach for counterfactual prediction. In International Conference on Machine Learning, pages 1414-1423. PMLR, 2017.

---

[19] T. Heskes, E. Sijben. I. G. Bucur, and T. Claassen. Causal shapley values: Exploiting causal knowledge to explain individual predictions of complex models. Advances in neural information processing systems, 0144778-4889 2020.
[20] N. Jethani, M. Sudarshan, Covert, S.-I. Lee, and R. Ranganath. Fastshap: Real-time shapley value estimation. ICLR 2022, 2022.
[21] Y. Jung, S. Kasiviswanathan, J. Tian, D. Janzing, P. Blöbaum, and E. Bareinboim. On measuring causal contributions via do-interventions. In International Conference on Muchine Learning, pages 10576-10501. PMLR, 2022.
[22] D. Kaltenpoth and J. Vreeken. Nonlinear causal discovery with latent confounders. In Interna- tional Conference on Machine Learning, ICML 2023, 23-29 July 2023, Honolulu, Hawaii, USA, volume 202 of Proceedings of Machine Learning Research, pages 15639-15654 PMLR, 2023.
[23] Y. Kawakami. Instrumental variable- identification for causal effects using covariate information. In Thirty- AAAI Conference on Artificial Intelligence, AAAI 2021, Thirty- Third Conference on Innovative Applications of Artificial Intelligence, IAAI 2021, The Eleventh Symposium on Educational Advances in Artificial Intelligence, EAAI 2021, Virtual Event, February 2-9, 2021, 1 pages 12131-12138. AAAI Press, 2021.
[24] Y. Kawakami, M. Kuroki, and J. Tian. Instrumental variable estimation of average partial causal effects. In International Conference on Machine Learning, 1C232023, 23-29 July 2023, Honolulu, Hawaii, USA. volume 202 of Proceedings of Machine Learning Research, pages 16097-16130. PMLR, 2023.
[25] R. Liu, C. Yin, and P. Zhang. Estimating individual treatment effects with time-varying confounders. In 2020 IEEE International Conference on Data Mining (ICDM), pages 382-391. IEEE, 2020.
[26] S. M. Lundberg and S.-I. Lee. Consistent feature attribution for tree ensembles. arXiv preprint 170.006660. 2017.
[27] S. M. Lundberg and S.-I. Lee. A unified approach to interpreting model predictions. Advances in nearul information processing systems, 30, 2017.
[28] D. D. Lundstrom, T. Huang, and M. Razaviyayn. A rigorous study of integrated gradients method and extensions internal neuron attributions. In International Conference on Machine Learning, pages 14485 14508. PMLR, 2022.
[29] S. Maleki, L. Tran- Thanh, G. Hines, T. Rahwan, and A. Rogers. Bounding the estimation error of saapling-based shapley value approximation with/without stratifying. CORR, abs/1306.4265. 2013.
[30] Pearl. Causality. Cambridge university press, 2009.
[31] M. T. Ribeiro, S. Singh, and C. Guestrin. " why should i trust you?" explaining the predictions of any classifier. In Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining, pages 1135-1144, 2016.
[32] L. S. Shapley. A value for n-person games. Contributions to the Theory of Games, 2(28):307- 317, 1953.
[33] A. Shrikumar, P. Greenside, and A. Kundaje. Learning important features through propagating activation differences. In International conference on machine learning, pages 3145-3153. PMLR, 2017.
[34] R. Singal, G. Michailidis, and H. Ng. Flow-based attribution in graphical models: A recursive shapley approach. In International Conference on Machine Learning, pages 9733-9743. PMI.R, 2021.
[35] D. Smilkov, N. Thorat, B. Kim, F. Viégas, and M. Wattenberg. Smoothgrad: removing noise by adding noise. arXiv preprint 03827.066.388....0.....0 2017.
12

---

[36] H. Stock and W. Watson. Instructional stata datasets for econometrics. Boston College Department of Economics, 2003.
[37] M. Sundararajan and A. Najmi. The many shapley values for model explanation. International conference machine learning, pages 9269-9278. PMLR, 2020.
[38] M. Sundararajan, A. Taly, and Q. Yan. Axiomatic attribution for deep networks. In International conference on machine learning, pages 3319-3328. PMLR, 2017.
[39] Y. Tauman. The aumann-shapley prices: a survey. The shapley value, page 279, 1988.
[40] J. Wang, J. Wiens, and S. Lundberg, Shapley flow: A graph-based approach to interpreting model predictions. In International Conference on Artificial Intelligence and Statisties, pages 721-729. PMLR, 2021.
[41] E. Winter. The shapley value. Handbook of game theory with economic applications, 3:2025- 2054, 2002.
[42] A. Wu, K. Kuang, B. Li, and F. Wu. Instrumental variable regression with confounder balancing. In International Conference Machine Learning, ICML 2022, 17-23 July 2022, Baltimore, Maryland, USA, volume 162 of Proceedings of Machine Learning Research, pages 24056- 24075. PMLR, 2022.
[43] Y. Xu, Zhu, C. Shi, S. Luo, and R. Song. An instrumental variable approach to confounded off-policy evaluation. In Proceedings of the 40th International Conference on Machine Learning, volume 202 Proceedings of Machine Learning Research, pages Jul 2023.
PMLR, 23-21
[44] A. Zern, K. Broelemann, and G. Kasneci. Interventional shap values and interaction values for piecewise linear regression trees. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 37, pages 11164-11173, 2023.
[45] J. Zhang, D. Kumor, and E. Bareinboim. Causal imitation learning with unobserved confounders. Advances in neural information processing systems, 33:12263: 33:12263-12274, 2020.
[46] J. Zhang, Q. Sun, J. Liu, L. Xiong, J. Pei, and K. Ren. Efficient sampling approaches to shapley value approximation. Proceedings of the ACM Management of Data, 1(1):1-24, 2023.
[47] Zhang, H. Xia, Q. Sun. J. Liu. Xiong, Pei, and K. Ren. Dynamic shapley value computation. In 39th IEEE International Conference on Data Engincering, ICDE 2023, Anaheim, CA, USA, April 3-7, 2023, pages 639 652. IEEE, 2023.
13

---

Appendix
In the appendix of our paper, we provide comprehensive additional content. We discuss the Ibroader impacts of the paper in Section A respectively. Section B reviews the related works. In Section D we delve into the computation of gradients when dealing with discrete features and discuss training methods for non-neural network models. Following this, Section presents the algorithm for computing Shapley values optimized using confidence intervals, along with an error analysis for unbiased sampling in integrated gradients. Subsequently, Section Fofferss supplementary material related to our experimental procedures. This includes a detailed analysis of the characteristics of experimental dataset, justifying experimental design. Additional results from classification experiments and non-neural network models are also provided.
A Broader Impacts
While we believe our paper has many positive social impacts, we think it can particularly affect:
Fairness and Equity in Automated Systems: Reduces biases caused by unobservable confounders in feature attribution, promoting fairness in systems like credit scoring and hiring. This helps to build trust in these systems and supports fair decision-making in various areas.
Improved Mecksign-Making in Healthcare: Our method enables accurate identifica- tion of factors affecting patient outcomes, leading to better diagnosis, treatment plans, and personalized medicine. Hcalthcare professionals can make better decisions, which improves patient care and outcomes.
We do not think our paper has any negative social impacts.
B
Related Work
In this section, we first introduce seminal works that have a significant influence on the feature attribution domain. This is followed by an exploration of works integrating causal knowledge into feature attribution. Additionally, we introduce the widespread presence of confounders in machine learning. Finally, we discuss advancements computational optimization for SHAP-based methods. For more detailed survey of feature attribution, please see [12].
B.1 Classic Feature Attribution Techniques
LIME (Local Interpretable Model-agnostic Explanations) facilitates the understanding of individual predictions of complex models by creating explanatory models (31). It reveals the impact of features on predictions by perturbing the input and observing the resultant changes in output. DecpLIFT (Deep Learning Important FeaTures) offers method for assessing feature importance in deep neural networks by comparing the activation of each feature against a reference activation, proving particu- larly effective in interpreting deep learning models 33) SmoothGrad enhances visual interpretations of gradient-basse baased methods by applying multiple small random perturbations to the input data and aver- aging the gradients of these perturbations (35] Meanwhile, researchers have increasingly recognized that for interpretability methods to be effective and credible, they need to satisfy axiomatic properties. SHAP (SHapley Additive exPlanations) employs Shapley values from cooperative game theory to measure feature contributions, offering model-agnostic approach with broad applicability (27]. adheres desirable allocation properties, ensuring both consistency and equity in attributing feature influence on predictions. Meanwhile, Integrated Gradients (IG) calculates feature importance through the integration of gradients along straight path from bascline to the input, making ideal for scenarios with continuous features and differentiable models [8]
B.2 Causal Feature Attribution Techniques
In the evolving field of feature attribution, the significance of causal relationships for data-faithful interpretations is increasingly recognized (21). Asymmetric Shapley values (ASVs) are developed
14

---

to infuse causal understanding into model explanations [14]. symmetry axiom in the Shapley value framework, allowing for the inclusion of causal relationships. Notably, ASVs can provide insights even without complete causal graph. Causal Shapley values stand out in their capacity to distinguish between direct and indirect feature impacts on model predictions, offering profound understanding of data generation [9. Shapley Flow distinguishes itself by evaluating the entire causal graph, attributing influence across its edges rather than focusing solely on nodes [40]. Recursive Shapley Value (RSV) presents specialized approach for graphical models, quantifying the propagation of changes from source nodes throughout the graph [34] However, despite the advancements made by these methods in considering the causal relationships between model input features, these methods overlook the impact of unobservable confounders on feature attribution.
They achieve this by modifying the
B.3 Confounders in Machine Learning
Researchers have recently begun exploring methods to identify and adjust for confounders in algo- rithmic models to enhance decision-makee masking quality [22] 42). Gao et al. [15] identify that pre-trained graph neural networks perform better on pruned graphs than on full graphs due to confounders and introduce Robust Causal Graph Representation Learning (RCGRL.) to effectively address this issue by eliminating confounders. Zhang et al. 455 introduce a method for causal imitation learning in the presence of unobservable confounders, featuring a graphical criterion to evaluate its feasibility despite partially observed decision variables behind expert actions. Deep Sequential Weighting (DSW) is proposed for estimating individual treatment effects in healthcare, accounting for virevngrying hidden confounders using deep learning 25) In confounded sequential decision-making, Xu et al. [3] study introduces an instrumental variable (IV) method for off- off-policy evaluation (OPE) to estimate policy returns accurately in infinite horizon settings.
B.4 Approximation of SHAP-based Methods
TreeSHAP [26.. for tree-based models, enhances SHAP value calculation efficiency by utilizing tree structures to skip redundant feature combination evaluations. Dynamic Shapley, discussed in the paper by [47]. focuses on dealing with scenarios where the players may change. Kernel SHAP [27), suitable for various models, approximates SHAP values by sampling in the feature space and assessing the impact of different feature combinations. Among the recent advancements in optimizing SHAP computation are TMC (Truncated Monte Carlo) [I6] and FastSHAP (20), each offering unique approaches to enhance efficiency. TMC employs truncation technique for rapid, biased sampling approximations. FastSHAP biased too, employing a pre-trained auxiliary model, speeds up SHAP value prediction. Moreover, unlike methods that approximate Shapley values through sampling, its ability to accurately estimate Shapley values does not improve with more samples, the precision of the auxiliary model is predetermined upon training. Recently, researchers have proposed one Shapley value approximation method based on the complementary contribution which can be adapted to the general class of feature attribution scenarios [46].
C Proofs
C.1 Proof of Proposition 1
Proof. From the perspective of the data generation process for y, the marginal contribution of feature i is exclusively linked to the function g. Thus, the marginal contribution of feature i in condition expectation Shapley for the target feature generation is t (SU ( (S) = E Eig(E x)) = =su()}] - (2 x) a x... For the model trained fit Ely|a, we have E Eylēā Elg(@, T
~. क| Eie ã Φ-- = g(, T) + E[e[@] Therefore, given explained input æ. the marginal contribution of particular feature with S in condition expectation Shapley derived with model f can be represented follows 2/ (S
= ([ = [[ -E] [x)) (r))e las = [s)
E( ær) sss(()) ] E(e|=suç1)
ss)( [[ x>) lats ars] B ags ms)-
Thus, the marginal contribution caleulated by the model f which is trained to fit E[y ãū, उ includes
an error term Ep je| ( = {su(n]] [[ = {. arises from the model's ss reliance on
the correlations within features X and E. By averaging the errors in the marginal contributions of
15

---

feature i with all possible cooperate coalitions, we can get the expected deviation of attribution value -1
by ASV; = N Escri( |s| {Eplekasuts - [stnn Ele æs - ='']}.
C.2 Proof of Proposition 2
Proof. From the data generation perspective, the marginal contribution of feature i intervention
Shapley for the target feature generation should be 77 (S)) (10))(77((5 Elg(2,,))do(( d)ldo(zs.
{su(iy)] - E[g((;, Ido ass .s)] However, for the intervention Shapley, the marginal contribu- tion derived with a model trained to fit Ely] is (S {i}) - -4 [[[
=s + [(e)do( = {(s(()))] (a)(oo)) = ==s]] Ep]ccdo(ss = æg)] Thus, we have the expected error <su(i))] - B[g(a, T |oo{xs = æs)
((g( (o(
for marginal contribution of feature i in intervention Shapley with mode I trained to fit E[y|ær] is Ep(e|do( (u(()) = =[Sugg)]] - En fe (do(æss =5)]]. By averaging the errors in all the marginal con- tributions of feature with all possible cooperative coalitions, we can get the expected deviation of attri- bution value by ASV; x Escrr(0 (11) (Ep]e)do((ssu() =su(a)) (ole/d
C,3 Proof of Proposition 3
Proof. For IG, where is the unobservable confounder correlated with æ. the derivative Øf is likely to incorporate the effect of 6 on æ, as it is trained to fit g(x) + c. Consequently, the following inequality typically holds of + 0g Therefore, the attribution [G; æ', () derived from the 7e.
de predictive model f generally differs from the attribution IG, (z, x',9) that should be obtained based on the actual data generation process. When we accumulate the difference of Øf and dg we can get the error for attribution value of feature i using IG with model f trained to fit E |x]] is af aaaeaaaa"""""" 0g (0' aaa"aaa"" aaaaa=== -xx)
Tw zur in the path,
ATG, = (x: - xx() fazo
da.
Oz:
Ox.
C.4 Proof of Proposition 4
Proof. The marginal contribution of a particular feature with S in condition expectation Shapley de- rived with model f = g(ū, )) + B[ can be represented as follows 2e (S U {i}) (S) =
[[ .su(n]] E f=]()) lag
æs! Bgg ( क [ suf()
T + Ele]
Elg(@, F) æzs æs] E[e] Elg( x) }) (ou(()) @ Elg( (w, x)) |æs æs] Thus, the attribution are identical for models f g( σ) and f 9 कं) + Ble] in condition expectation Shapley.
The marginal contribution derived with model trained to fit f (2 ar) E (F) = [d)
is (SU
zs( ) + E[c] Elg( doo(x xsS(())] [ as)) Elg a. oddo( B[g(ar, 2)) do
ms) E[c] E[g( क) do (
=su(i))
æs)]. Thus, the attribution identical for models f g x)) and
f= ( x) E[e] in intervention Shapley.
as = 8y holds when model f ((x)+((())))) +E(c as E[c] is a constant. Thus, the attribution The derivative de de
identical for models f g( x) and f g x) + E[c] in Integrated Gradients(IG).
D Discrete Confounded features and Gradient-Free Model Training
D.1 Training with Discrete Features
We extend our discussion to scenarios where the targets predicted by Mrg (# It, 1/2) are discrete. In cases where the prediction features P of M, are discrete, the fundamental approach to optimizing the loss function I.(T: 0) remains similar. The primary modification involves substituting the integral over the probability distribution of P with summation across discrete points. Assuming P has K
16

---

categories, and denoting the probability of the tt data point being classified into the h category by Mό. (i* Le; zt), the loss function when is continuous is reformulated as:
K Ma
2
1
Σ
1/2
Ft; th) So (#* .25))
(7)
For the 9th training data point, the gradient of this loss function is:
K Σ
VoL. = 2 1/
Mp
t Te, 1/)s fo (#* Ttt)
Mo (#*
Tee v1) fo (a
(8)
Furthermore, the gradients of mini-batch comprising training data tuples are computed as:
VTL.m
Σ
8+
K Me
(a ke,, (h) fo (** (e*,z,))
)]
Σ Me (a* a... v.). S6 (a", 2x)
(9)
In situations where is a discrete variable, representing categories or classes, the multi-class cross- entropy can be formulated as:
R
((T;0)= [T]
Mo (a |x2, vh) yerr In fer (Æt, xxi)
(10)
1=1
In this formulation, Mt.r represents the true label of the th data point in the T category. denotes the total number of distinet categories classes into which the target variable y can be classified. The model's prediction for this category is given by for (æc, æt). The gradient calculation for the eth training data point, considering this loss function, is then:
R
K
VoL.
Ma
Ft. wh)
ytrr
fé,, (#* xt.)
(11)
forr (Ø Tz.)
k=1
Furthermore, the gradients of mini-batch comprising training data tuples are computed as:
L. =
Mo k De, vi) for (w tz)
yerr So.r (#t Tt)
(12)
This adaptation of the loss function for discrete target variables ensures that model handle classification tasks, effectively optimizing its performance across multiple categories.
D.2 Gradient-Free Model Training
When training models to fit ý in scenarios where gradient-based optimization is not feasible, we introduce an alternative approach that effectively addresses the influence of confounding factors. The essence of this approach lies in the generation of synthetic data, which is derived from the predicted distribution of p. By sampling each original data point times, we create B synthetic data points for every original point. This process results in synthetic dataset that embodies the controlled effeets of the confounders. The creation of this dataset is a vital step towards ensuring that the subsequent model training is less influenced by confounding variables.
key advantage of this method is its independence from any specific model type. The generated synthetic dataset can be utilized to train variety of machine learning models, not limited to those that rely on gradient-base based optimization. This model-agnostic nature significantly widens the applicability of our approach, making it suitable for various scenarios and models. Through this method, we ensure that the training of models occurs in an environment where the impact of confounders is mitigated, thereby enhancing the reliability of the feature attribution.
17

---

E
Supplement to SHAP and IG approximation
SHAP-based methods and IG-hased methods be applied to the proposed confounder-free models. However, the computational complexity poses a significant barrier to real-world applications. The exact computation of the Shapley value is proved to be an #P-hard problem (II), and the exact computation of integrated gradients requires the antiderivative of the gradient, which is infeasible due their complexity, necessitating the use of approximation methods. The approximation cost of the Shapley value is higher than the straightforward sampling in the path of integrated gradients due to extensive feature subset evaluations. To further enhance the applicability. we develop optimizations for the approximation of SHAP- based methods. Research has shown that Shapley values can be represented not only based on marginal contributions but also complementary contributions, which allows for reusing samples in estimations, offering an advantage [46]. We propose an enhanced approach for SHAP-based methods, optimizing complementary contribution-bast sampling using confidence intervals. This optimization is designed to minimize estimation errors in all utility functions within SHAP-based methods.
E.1 Estimation Techniques for SHAP
Recent work [46] suggests that the Shapley value formula can be equivalently transformed into a form expressed based complementary contributions. Here, the complementary contribution refers to the difference in utility between complementary subsets. The Shapley expression is given by
1
u(S U {d}) 1 U(S)
SV.
Σ
(13)
W
(
SCA(())
u(S U (zi)) U(NY (SU
Σ SCAM(=)
(14)
("s))
The formulas based on complementary contributions offer advantages in terms of sample reusability during approximate computation. Building this foundation, we propose adjustment based on the confidence intervals of Shapley value estimates in stratified sampling.
a dynamic sampling
Denote by G (S [S C W { |S] = - 1) (1 | S n) the set of coall)ccollsssons, and by SV i. the expected complementary contributions of (2,, coaliitons. That is,
-(S) - U(N \ S)
SV Σ
(15)
G-)
SEeW
Complementary contributions CC(S) U(S U (z,)) UWN (S (z,))) naturally stratified into strata ....... Gy²² according to the coalition size. We start by deriving the confidence interval of the estimator of SVi, using l-test.
Lemma 5. According the Central Limit Theorem, the sample mean approximates a normal distribution when the sample size is sufficiently large. Assuming confidence level of a, the confidence
interval for SVij based on the 1-test is SVi.j + Aa Moi..
St,, which can also be represented
S.a SV < <VV,j* Ao
S
P(SV Av
=a,
(16)
mma.
mm
where Aq is the t-score corresponding to a, my is the sample size of G and Sa,5 is the sampling variance of SVi.-
Denote by 6 (S|S | }} the set of j-colitions (1 i After drawing a coalition = S from @) we can estimate the complementary contribution CCx(SS, which can be used in SV for z; in S and SV SVinn,, for z; in A S.. In light of the fact that each sample can influence multiple strata, how should we allocate the number of samples to optimize the precision of the estimated values? We denote T3y as the sum of the confidence intervals for strata which can be influenced by random sample of Gyy that is
N
Sij
N
Sinn-
K = 2
An
Aa
(17)
mi
Minnn
i=1
18

---

For each sampling iteration, we select the stratum that maximizes the sum of the corresponding confidence intervals.
Next, we will outline the algorithmic procedure, which is divided into two primary stages. Due to page limitations, the pseudo-code of the algorithm is presented in Algorithm I the appendix. In the first stage, we sample at least Minit samples for SV.,j- We then compute unbiased estimations of o using Bessel's 's correction based on samples collected in the first stage. In the second stage, the stratum for each individual sampling is determined based the sum of confidence intervals (1 < j n/2). Furthermore, this sum is updated upon the completion of each sampling. Specifically, let CCN(S)
U (zi3) be mijj samples for computing SVi.. then of mi:; EKE (SCN(Ss U {zit) m.,, Ei CCx(S& (zi1)) Let Müst be the number of samples used in the first stage, and the number of remaining samples is m - Mfistt We I n/2) according to Equation (17) using the unbiased sample variance oi We randomly sample from the stratum with the largest sum of confidence intervals. Following this sampling, the sum of confidence intervals will be updated. We will continue to repeat this process until all samples have been utilized. The final estimation of Shapley value is the average of all complementary contribution means in each stratum.
calculate 1/ 13-(1
SHAP experiences an exponential increase in computational complexity with the addition of more features. However, for IG, the increase in features does not significantly escalate the complexity of the integral path. Therefore, in terms of efficiency in approximate computations, IG generally outperforms SHAP. This makes IG preferable choice in situations where computational complexity for interpretability is a critical concern. However, it is important to note that SHAP, as a model- agnostic method, is applicable for interpreting models that not based gradient optimization.
Algorithm of SHAP Computation Based on the Confidence Interval. For the algorithm process propose, which utilizes confidence intervals to optimize the calculation of Shapley values, refer to Algorithm|] 1 is important to note that our algorithm is utility function-agnostic [29], meaning can be applied across various SHAP-based algorithm variants. This is achieved by simply substituting the utility function defined by each method into our calculation. Furthermore, we provide an unbiased proof of our method in Theorem 6
Theorem 6. Given set of players N = {21..... }. Algorithm 7 gives an unbiased estimation of Shapley value for every player. that is, E(SV. = SV, (1 i n).
Proof. Denote by CCx(S)), CCN((mm..) sample of GV² (1 S i,j S n) drawn by Algo- rithm The expectation of the sample SV.J ma.j Ekmi (Sx(S)). We can compute the
expectation of SVij with
CCN(SK) 1 mij
E E(SV
E
Σ
E[CCx(SA))
(18)
mi,j
mi.
kml
km
According to Equation E[CCN((A)] SV Thus, E(SV, SVas that means SV., an unbiased estimation of SV i.j.
15
Then, we can compute the expectation of SV; produced by Algorithm
We have
1
E SVV.
E
SV
ESS
SV. SV
(19)
n2
j=1
That is, SV, is an unbiased estimation of SV:
E.2 Unbiased Integrated Gradients Approximation
In existing literature related to Integrated Gradients (IG), interpolation methods are commonly used for approximation [3), which already exhibit high efficiency compared to SHAP-like approximation methods. In contrast, our paper introduces the use of Monte Carlo methods for the integration of gradients. It's important clarify that the aim of proposing an unbiased estimate for IG is not
19

---

Algorithm Shapley value computation based on the confidence interval.
Input: players N {z1,.... .... ) Manit > and 0
Output: approximate Shapley value SV, for each player =; (1 i S n) Mij Mi. + 0 (1 S S n);
f- -1;
while c Σド= M1,5 do
c= -= m1,j;
for i= to to.
if ma,j < Minit then
let 8 be sample drawn from 6%:
u u u UUN SS);
for S do
SV.,si+ = u; mi,ss =; 1;
end for
for 2, E do do
SV,,viis1 u; mi.NNS = 1;
end for end if end for end while
compute 57 (1 S i,j < n);
fifist f- -5=1 M..
for k = 0 to
first do
for j=1 to n do
T/- 2
N
Ag
Em1
Aa
S.n-i (
end for
let S be sample drawn from G where j corresponding to the stratum with the maximum R3:
u (()) U(N \ ();
for z; E S do
SV.,ss w; m+,ss++ 1;
end for for 2; N S do
SV. u; m.
end for update 52 (1 < í, < n); end for for i=1 to do
SV. 1 Σj= SV Vi.g/ /maj;
end for return SV,,..., SVn.
20

---

to optimize sampling efficiency but rather to provide an error analysis for the sampling process. Our proposed approach not only provides an unbiased estimate of the integrated gradients but also includes this crucial enror analysis.
Let be uniformly distributed random variable over the interval [o, 1J. where
(x,-<x)) Of (x' + uu(x x²))
h(u) = (z; --x()
dz.
Then, h(u) is an unbiased estimator of (x, (2, x', due
(z:- - x/)) / (x' + u (I - x')) du (x; (b;)
S
Of (x' + + (I xx))
[[b(w)] (z;
do.
dz.
Ox:
(20)
Next, we can obtain a more accurate unbiased estimate through the following steps: First, we generate my random samples u1.-.. Um, of u. Then, sample h(u) based these random numbers to get an independent and identically distributed sample h(uz), .h(um.) Finally, we use the observed values of
= d Em= h(m4) as the estimate for 1G,( x', f') The proof mh Ei hu(.) is unbiased estimator of (x, x', is straightforward, due the fact that El 1 Ee h(ue)] = 2. Ei F([(us)) = TG,(,,,',,, f)-
that
Lemma 7. The probability that IG,(I, x',x,,',J f) < < n) deviates from IGe( x', f) be equal to or greater than any fixed 2 0 given the sample size m, is bounded by
2m,
(1, -
2 e/m,) S 2exp(
(21)
r
[ax h(u) [in h(u). where ri,
Proof. According to Hoeffding's inequality, we have
PP P( x', f) IG,(,, x', fIl 2 clm,)
(22)
_P(IG x', - EEIG; x', fI] 2 El e/m,;
(23)
h(u)) E m.
hww))] 2 m.c(m,)) 2 exxp( 2m,
TP(
(24)
r
F Supplement to Experiments
F1
Experiments Compute Resources
We conduct experiments on machine with 2 Montage(R) Jintide(R) C6226R @ 2.90GHz and 256GB memory. Our experiments do not require high-end hardware, and our algorithm is not time-consuming. For feature attribution experiments on synthetic datasets, each attribution algorithm takes about 10 seconds to attribute one data point. Thus, an attribution algorithm takes several hours to attribute an entire dataset. The time consumption on our real dataset similar. Also, since our algorithm uses very little memory, it is easy to run multiple processes and algorithms in parallel on machine, so the time cost for reproduction is friendly.
F.2 Analysis Experiment Design
Statistical Characteristics of the Synthetic Datasets. We present the mean and variance of each feature in our synthetic datasets to provide crucial insights into their characteristics, as shown Table
3 The mean offers understanding of the average behaviour of features, while the variance indicates their variability. This information is vital for assessing the data's overall distribution and quality, and it plays key role in interpreting the results of our proposed methods and baseline algorithms.
21

---

Table 3: Mean and Sed.Dev. of Features as a Function of p.
p 0.2
0.4
0.6
0.8
1.0
Ca 0.10
+ uos
0.20 + o.n
0.30 + o.1
0.40
+ ozs
0.50 + 0.23
to
0.21
+ 0.12
0.27 + 0.14
0.32 + 0.177
0.37
+ 0.18
0.42 +: 0.11
Yo 0.68
4 0.18
0.87 4 0.24
1.0 0.3
1.15
+ 0.29
1.30 +: 0.47
Eb 2.03
0.122
1.12 0.13
0.83 0.15
0.69 0.16
0.61
:: 0.18
tb 1.11
1:011
0.72
+: 0.19
0.60 0.18
0.53
+: 0.18
0.50
+: 0.18
ybb 2.83
0:25
1.65 0.23
1.27 0:244
1.09 0:2
0.99 0:27
These statistics help validate the synthetic data's consistency and reliability, which essential for the credibility of our experimental findings.
Statistical Characteristics of the Real Datasets. In the analysis of the Griliches76 dataset [36 we observe various degrees of correlation between the mother's 's years of education (denoted as med) and several key variables. Firstly, the correlation coefficient between the mother's education and marital status (variables mrt and mrt80) is close to O, indicating almost no correlation between the mother's s level of education and her marital status. The correlation coefficients with urban residence status (variables smsa and smsa80) are 0.098 and 0.031, respectively, suggesting a slight positive correlation. This implies that there a weak but positive association between the mother's educational attainment and living in urban area. A moderate positive correlation is observed with IQ, as indicated by a correlation coefficient of 0.226 with the mother's education. This suggests that higher maternal education is somewhat associated with higher IQ scores. Similarly, the correlation between the mother's 's education and scores in the world of work test is 0.195, which also reflects a moderate positive correlation. This indicates that higher maternal education levels might be linked to better performance in job-related knowledge. Most notably, the correlation coefficients with personal education years (variables and s80) 0.340 and 0.341, respectively, indicating relatively strong positive correlation. This suggests that the mother's 's level of education is considerably associated with the individual's own educational attainment. While there is certain degree of correlation between maternal education and both IQ and job knowledge test scores, factors like regression to the mean in intelligence suggest that the correlation between mother's education and her child's IQ is weaker than the correlation between a mother's education and the child's own educational attainment. Therefore, we posit that selecting maternal education as instrumental variable, although not perfectly ideal, still holds validity and can be utilized to verify our methodology.
In the Angrist dataset, child must be six years old within the current year enroll in school under the U.S. Compulsory Education Law. In the U.S., the school year typically starts in August, meaning child turning six in December child born in the fourth quarter, such as December, can start school before reaching six. Conversely, a child born in the first quarter, like January, must wait until the autumn term after their sixth birthday to begin school. U.S. law mandates students must be at least 16 years old to legally drop out of school. Therefore, students dropping out at 16 may have varying years of education based on their birth month. For instance, those born between 1920-1929 have average educational years of 1 11.39, 11 11.44, 11.55, and 11.57 for each quarter, respectively. Parents, when deciding to have children, seldom consider such subtle differences in birth months. Thus, the month of a child's birth, independent of other factors affecting educational levels like intelligence, family background, and environment, can be seen as random assignment. This inadvertently creates variations in education duration based on birth month akin randomized controlled trial where children born in the fourth quarter represent the "experimental group" with longer education, while those in the first quarter are the "control group" with shorter education. Hence, the birth quarter serves as an instrumental variable in this context.
still commence their education in the same year. Consequently,
F.3 Classification Task with DNN Model
In this experiment, we continued to utilize synthetic datasets a and b for a classification study. This time, the labels were processed for binary classification. Specifically, we computed the probabilities for data points being classified into category by applying sigmoid function to the y values in the datasets; otherwise, the labels were assigned to category 0. Due to the inherent randomness in
22

---

generating labels, it was not feasible to directly determine benchmark for each feature's 's contribution
to the data classification.
To validate the efficacy of our proposed method, we designed a comparative experiment based on the symmetric properties of SHAP and IG. In this experiment, we set baseline inputs by reducing the values of feature I and the collaborative variables с in each data point. The reduction followed a specific rule: the decrease in and the decrease in e should result in equivalent Shapley/IG values for the change in y. Under this setup, the SHAP/IG values attributed to the classification into category 1 should be identical for both and c. We assessed the effectiveness of our approach by comparing the difference in SHAP/IG values for and between our method and baseline algorithms. The results indicated that our approach significantly reduced errors compared to baseline algorithms. This outcome suggests that even though and might have similar Shapley values in altering y. the baseline training method may inaccurately estimate changes in latent confounders, leading to different impacts of i and c on the final classification.
Figure 4: Evaluation results on synthetic Dataset A with DNN Classifier.
Pestur 1.23
Figure 5: Evaluation results on synthetic Dataset B with DNN Classifier.
F.4 Regression Task with XGBoost Model
We opted for XGBoost as the representative of non-deep learning models for our experiments. As gradient accumulation not feasible on XGBoost, the Integrated Gradients (IG) algorithm cannot be applied. Hence, our comparisons were primarily focused SHAP-based algorithms. The experimental results indicate that our method outperforms the baseline in most scenarios. When the impact of the unobservable confounder is minimal, the baseline. This is considered reasonable, as there are inherent errors in training the model with features re-estimated using instrumental variables. In such scenarios, the influence of these errors on the model surpasses that of the unobservable confounder.
method is less effective compared to
Pacfure
Postars Borichie
Figure 6: Evaluation results on synthetic dataset a with XGBoost.
F.5 Efficiency of Our Approximation Methods
We utilized widely-used algorithms MC (Monte Carlo, referring to the Monte Carlo sampling method based on marginal contributions) [7], CC (Complementary Contribution, referring to the stratified sampling method based on complementary contribution), and the state-of-the-art CCN
23

---

PARO
PARA
Figure 7: Evaluation results synthetic dataset b with XGBoost.
Table 4: MSE of the SHAP values estimation.
SAMPLES
56*100
56*200
56*300
564400
56%500
MC
3.20e-5
1.74E-5
1.28B-5
9.81B-6
8.37E-6
CC
1.78p-5
9.45g-6
6.89B-6
5.37E-6
4.58E-6
CCN
1.76F-5
9.62F-6
6.71R-6
5.45n-6
4.96F-6
OURS
1.48E-5
8.66R-6
6.15E-6
4.92k-6
4.25k-6
(Complementary Contribution Neyman, referring to the sampling based on Neyman allocation with complementary contribution) [46] as baseline methods for our experiment on the real-world Spambase dataset. We chose the Spambase dataset for its larger number of features (56), compared to the two other real datasets we previously used. This higher feature count offers a better testbed to evaluate our proposed methods. It is worth noting that both our proposed method and these baseline methods are unbiased sampling estimation approaches. We trained a regression neural network model on random selection of 1000 data points from this dataset. The efficiency of each method was assessed by comparing the Mean Squared Error (MSE) of SHAP values against a benchmark for the same number of samples. For this, denoted the SHAP value of the jth feature for the ,th data point as SVi,j in the benchmark, and SVi., in the estimation algorithm, with the MSE
calculated using - > , (SV.., SV..)) where n=1000 and m=56. Given that the computation of exact SHAP values requires exponential time complexity. we used the results obtained from extensive sampling via the CC method as our benehmark, involving 10,000x56 samples. These results served as specific benchmark, separate from the baseline CC method used earlier in the experiment for comparative analysis. We then analyzed errors for sampling algorithms various sample sizes, ranging from 100x56 to 500x56. Our findings revealed decrease in error for all algorithms as sample size increased, with our method exhibiting lowest error as shown Table [ This superior performance is attributed to our method's unique ability to estimate each stratum's 's confidence interval from sample variance during sampling.
NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: The introduction our paper clearly state claims made, including the contributions made in the paper and important assumptions and limitations.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
It is fine to include aspirational goals as motivation as long as is clear that these goals are not attained by the paper.
2. Limitations

---

Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: Please see Section
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g.. independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-ter system might not be used reliably to provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and faimess.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't It acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?
Answer: [Yes]
Justification:
We provide the full set of assumptions and a complete (and correct) proof.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should properly referenced.
4. Experimental Result Reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: The experiments sections (Section and the appendix) provided in the paper disclose all the information needed to reproduce the main experimental results.
25

---

Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurlPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily new algorithm, the paper should make it clear how to reproduce that algorithm. (b) the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is new model (e.g.. a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open- open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. the case of closed-source models, may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes)
Justification: Our code can be found the repository at htttss com/ ZJU- FJU-DIVER/IV -V-RRRIIVIHAPPP
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips.. cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https
/ ctiips.cc/puulic/guui.
for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
26

---

At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code is permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification:
The full details are provided with the code.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment Statistical Significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
Justification: For the results on synthetic datasets, we used bar charts, which effectively demonstrate the stability of the experimental results. For the experimental results presented in tabular format, we included the standard deviation to show statistical significance.
Guidelines:
The answer NA means that the paper does include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, overall
run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
If error bars reported in tables plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: Please see Section the appendix for the information of computer re- sources.
27

---

Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, cloud provider, including relevant memory and storage.
The paper should provide the amount of compule required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).
9. Code Of Ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics
Answer: [Yes]
Justification: We have thoroughly reviewed the NeurIPS Code of Ethics and confirm that our research conforms to it in every respect.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special cireumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g.. there a special consid- eration due to laws or regulations in their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive socictal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: Please see Section in the appendix for societal impacts.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (c.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility ML).
11. Safeguards
Question: Does the paper describe safeguards that have been put in place for responsible release of data models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
28

---

Answer: [NA]
Justification:
The paper does not pose such risks.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model. for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do not require this, but encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: We have properly cited the original papers that produced the real datasets used in the paper.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website). the copyright and terms of service of that source should be provided.
assets are released, the license, copyright information, and terms use in the package should be provided. For popular datasets, paperswi thcode. com/ datasete has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that re-packaged, both the original license and the license of the derived asset (if has changed) should be provided.
If this information is not available online, the authors are encouraged to reach out to the asset's 's creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification: The paper introduces new code assets, which are well documented. The documentation includes details about the usage, the limitations, and the corresponding license.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details the dataset/code/mmddl as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include an anonymized zip file.

---

14. Crowdsourcing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer[[NA]
Justification:
The paper does not involve crowdsourcing nor research with human subjects. We used two real-world datasets that contain information about education and income collected from the market, which are anonymized and not involve direct human sub- jects research. Therefore, the requirements for providing instructions, screenshots, and compensation details not apply.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then much detail as possible should be included in the main paper.
According to the NeurlPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country or institution) were obtained?
Answer: [NA]
The paper uses two real-world datasets that contain information about edu- cation and income collected from the market. These datasets are anonymized and do not involve direct human subjects research. Therefore, IRB approval is not required.
Justification:
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent)
may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions NeurlPS Code of Ethics and
and locations, and we expect authors to adhere guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
30


Reviews:
[{"review_id": "Odb4H9t2ru", "reviewer": "Reviewer_p3Si", "summary": "This paper addresses the problem of estimating causal effects with feature attribution, by applying SHAP and Integrated Gradients (IG) to two-stage models with instrumental variables. On synthetic datasets, the proposed methods IV-SHAP and IV-IG can better recover the ground-truth causal effects, compared to SHAP and IG applied to predictive models. On real-world datasets, IV-SHAP and IV-IG are more aligned with prior knowledge than SHAP and IG.", "questions": "- Besides SHAP and IG, is it appropriate to apply other feature attribution methods to a two-stage model in order to estimate causal effects?\n- For the real-world datasets, what are the baseline values for IV-IG, and how are features removed in IV-SHAP?\n\nMinor comments:\n- There are typos in lines 172 and 178: $\\frac{1}{\\mathcal{N}}$ should be replaced with $\\frac{1}{|\\mathcal{N}|}$.\n- I recommend omitting the $D$ in $\\mathbb{E}_D$ of Proposition 2 for consistency.\n- The use of Hoeffding's inequality in the proof for Lemma 7 require some boundedness condition on $h(u_i)$, which is reasonable to assume but should be mentioned nonetheless.", "limitations": "The limitations of the proposed approach itself seem adequately addressed.", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- This paper addresses an important problem in the field of feature attribution: gaining causal insights from the attribution scores. Although it seems intuitive that a predictive model cannot provide causal insights when paired with feature attribution methods, this paper formally illustrates the need to account for confounding.\n- The writing is clear. The pronounced focus on data fidelity in the introduction makes it clear that model-centric interpretation is not the goal of this paper. The conceptual examples are illuminating.", "weaknesses": "- It is unclear how the expectations with respect to $\\hat{M}_{\\phi}(\\tilde{x} | \\bar{x}_t, \\psi_t)$ in Equations (4) and (6) are empirically computed in the experiments. The (approximate) computation can potentially be expensive.\n- What is the impact on IV-SHAP and IV-IG when the three assumptions of instrumental variables are violated? The paper lacks a discussion on this.\n- What is the impact on IV-SHAP and IV-IG when the input features are correlated? The paper lacks a discussion on this.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your encouraging and insightful comments. Please find our response below. \n\n**Response to Weaknesses**\n\n* W1: For the case of discrete $\\tilde{\\boldsymbol{x}}$, $\\hat{M_\\phi}$ is trained as a DNN classifier with softmax output, where the output element represents the probability of the corresponding category. We can enumerate the probability of each category and compute the weighted sum of all categories when we need to compute the expectation term condition on  $\\hat{M_\\phi}\\left(\\tilde{\\boldsymbol{x}} \\mid \\overline{\\boldsymbol{x_t}}, \\psi_t \\right)$. The computation complexity is $O(c*n)$, where $c$ is the number of categories and $n$ is the number of sampled data tuples in model training. For the case of continuous $\\tilde{\\boldsymbol{x}}$, we implement a mixture density network to fit the distribution of $\\hat{M_\\phi}\\left(\\tilde{\\boldsymbol{x}} \\mid \\overline{\\boldsymbol{x_t}}, \\psi_t\\right)$. We use 20 random samples from the distribution output by the mixture density network to approximate the expectation, which has demonstrated great stability in our experiments. Thus, the computation complexity is $O(n)$. Both cases can be effectively computed given typical computational resources. We will add the details in the paper.\n\n* W2: The effectiveness of IV-SHAP and IV-IG may be reduced when the three assumptions of instrumental variables are violated. However, the extent of this reduction depends on how severely the assumptions are violated. To mitigate the influence of unobservable confounders, the instrumental variables need to have a more direct influence on $\\tilde{X}$ while having less correlation with other features. Additionally, the instrumental variables should have little or no direct influence on $Y$. The instrumental variables are still effective at reducing the impact of unobservable confounders if the assumptions are only mildly violated. For example, in our experiment with the Griliches76 dataset, the chosen instrumental variable, Parental Education, has a weak correlation with the unobservable confounder, Ability, as we analyzed in Appendix G.2. However, the influence of parental education on child education is much stronger, as demonstrated by the statistical characteristics of the real dataset in Appendix G.2. Despite this mild violation, the effectiveness of IV-SHAP and IV-IG remains significant. We will add a discussion in the paper/\n\n* W3: The correlation of input features may affect the data-faithfulness of IV-SHAP and IV-IG. We can combine methods which deal with the correlated input features to the two-stage model to better capture these correlations. For example, on-manifold Shapley [1] can be used to account for feature correlations, while causal Shapley [2] can be applied if the causal structure of the input features is known. We note that our primary focus in this paper is on addressing the influence of unobservable confounders. Therefore, we chose the two most representative feature attribution methods, SHAP and IG, to formulate the problem. We will include this discussion in the paper.\n\n**Response to Questions**\n\n* Q1: Other feature attribution methods can be applied according to the demands. For example, when considering the correlation between input features, we can apply on-manifold Shapley or causal Shapley to our two-stage model if sufficient prior knowledge is available. Our methods focus on addressing unobservable confounders by re-estimating $\\tilde{\\boldsymbol{x}}$. Therefore, we can combine other feature attribution methods to handle the input features after re-estimating $\\tilde{\\boldsymbol{x}}$ with the two-stage model.\n\n* Q2: We experimented with multiple baseline values by subtracting specific values from the features $\\tilde{\\boldsymbol{x}}$ and $\\overline{\\boldsymbol{x}}$ in both synthetic and real datasets. Lines 283-286 and 311-312 in Section 5 provide the details of experimental setting. The removed features are replaced with the average values from the dataset in IV-SHAP, which is consistent with the approach in SHAP.\n\n**Response to Minor comments**\n\n* M1: Fixed. Thanks! \n\n* M2: We have omitted the $D$ following your suggestion.\n\n* M3: $h(u_i)$ is the weighted gradients of a deep neural network, which is continuously differentiable and bounded in our setting. We will clarify it in the paper.\n\n\nReferences\n\n[1]Frye C, de Mijolla D, Begley T, et al. Shapley explainability on the data manifold[C]. International Conference on Learning Representations (ICLR).\n\n[2]Heskes T, Sijben E, Bucur I G, et al. Causal Shapley values: Exploiting causal knowledge to explain individual predictions of complex models[J]. Advances in neural information processing systems (NeurIPS).", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "OusdavJyNV", "reviewer": "Reviewer_Ye8R", "summary": "The paper addresses the challenge of unobservable confounders in feature attribution methods, which can lead to misinterpretations. The authors propose a novel approach called \"data-faithful feature attribution,\" which trains models free of confounders using instrumental variables (IVs) to ensure that feature attributions are faithful to the data generation process.", "questions": "1. How can one reliably identify suitable instrumental variables in practice? What criteria should be used to ensure their effectiveness?\n2. How does the method perform with non-linear effects of unobserved confounders? Have you conducted any experiments or simulations to demonstrate its robustness in such scenarios?\n3. Have you conducted sensitivity analyses to evaluate the robustness of the method under different conditions, such as varying degrees of distributional shifts or the quality of IVs?\n4. How does the proposed method scale with larger datasets and higher-dimensional data? Are there any benchmarks or performance metrics available to demonstrate its scalability?", "limitations": "Please refer to the above questions.", "rating": 5, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. The introduction of a confounder-free model training approach using instrumental variables is a significant advancement. This method addresses a critical gap in feature attribution by ensuring data fidelity.\n2. The paper provides a comprehensive theoretical analysis that clearly explains the impact of unobservable confounders on feature attribution and how the proposed method mitigates this issue.\n3. The authors validate their approach on both synthetic and real-world datasets, showing up to a 67% improvement in attribution accuracy over baseline methods. This demonstrates the practical applicability and effectiveness of the method.\n4. The proposed method significantly reduces errors in feature attribution, ensuring that the contributions of input features to the target feature are accurately represented.", "weaknesses": "1. The two-stage training process and the use of advanced techniques like IVs may be complex to implement and require significant computational resources, potentially limiting the method's practical usability.\n2. The theoretical derivations are based on the assumption that the influence of unobserved confounders on the target features is linear. This assumption may not hold in all real-world scenarios, limiting the method's applicability.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your encouraging and insightful comments. Please find our response below.\n\n**Response to Weaknesses**\n* W1: The implementation and computation complexity of the two-stage training process is similar to the model training in the model-faithful feature attribution. In the first stage, we need to train a DNN classifier \n$\\hat{M_\\phi}$ if $\\tilde{\\boldsymbol{x}}$ is discrete (or mixture density network $\\hat{M_\\phi}$ if $\\tilde{\\boldsymbol{x}}$ is continuous) to approximate the distribution of unconfounded $\\tilde{\\boldsymbol{x}}$. Thus, the implementation is easy and computation complexity does not require significant computation resources in the first stage.  The second stage of model training is the same as the training of the predictive model in model-faithful feature attribution, except that the feature values of $\\tilde{\\boldsymbol{x}}$ are replaced with the re-estimated unconfounded values. The re-estimated unconfounded values are sampled from the first-stage trained model, the sampling has little influence on the implementation and computation complexity of the second-stage model training. Therefore, the two-stage training process does not limit the method's practical usability. We will clarify it in the paper.\n\n* W2: We conduct the theoretical derivations based on the assumption that the influence of unobserved confounders on the target features is linear. We can give an accurate formula for the error in the linear assumption while other types of influence are difficult to decouple in theoretical analysis. However, our method can still work well in the practical scenario when the effect of the confounders is not linear. In the real datasets Griliches76 dataset and the Angrist and Krueger dataset we used in Section 5.2, the confounder of Ability has a non-linear impact on income.  The experimental results still demonstrate significant effectiveness. \n\n**Response to Questions**\n\n* Q1: To identify suitable instrumental variables (IVs), we can first conduct a preliminary screening, such as LASSO regression, to assess the correlation between potential IVs and confounded features. Variables with non-zero coefficients may be correlated with $\\tilde{\\boldsymbol{x}}$ and are considered potential IVs. Next, we can use causal interventions to empirically verify these IVs, ensuring their effectiveness through statistical tests, such as checking if the F-statistic exceeds a threshold or using Hansen tests for validity. For a more in-depth study on IV identification, we recommend the seminal works [1][2][3]. Although identifying IVs is beyond this paper's scope, we will add a discussion to make this paper more self-contained and accessible to readers.\n\n* Q2: Our method can effectively address the non-linear effects of unobserved confounders. In experiments with real datasets, such as Griliches76 and Angrist and Krueger, the confounder Ability has a non-linear impact on income. For example, while factors like IQ positively correlate with income, the relationship is not linear. Although we assume linear effects of confounders for the convenience of theoretical analysis, our method remains effective for non-linear effects. This is because we re-estimate $\\tilde{\\boldsymbol{x}}$ using instrumental variables in the first stage, allowing us to obtain unconfounded $\\tilde{\\boldsymbol{x}}$ for attribution, regardless of how unobserved confounders influence $Y$.\n\n* Q3: In Section 5.1, we examined distribution shifts using synthetic datasets because real datasets do not allow control over external confounders and instrumental variables (IVs). In our synthetic datasets, we use $\\rho$ to control the magnitude of unobserved confounders, $\\tilde{\\boldsymbol{x}}$ and the unobserved confounders are correlated. Consequently, the distribution of $\\tilde{\\boldsymbol{x}}$ also changes. Figures 1 and 2 demonstrate that our method's effectiveness increases with confounder influence. Notably, even at $\\rho=0.2$, our method is effective.  Additionally, experiments show that higher-quality IVs reduce confounder-induced errors more effectively. For example, we test the average error of IV-SHAP on Dataset A when $\\rho=0.2$ with $\\psi \\sim (0,0.5), \\psi \\sim (0,1), \\psi \\sim (0,2)$ where $\\psi$ controls the correlation of IVs with $\\tilde{\\boldsymbol{x}}$. The results are shown in the table below.\n\n     |  Feature Deviation   | 0.125   | 0.250   | 0.375   | 0.500   |\n     |----------|---------|---------|---------|---------|\n     | $\\psi \\sim (0,0.5)$ | 0.034 | 0.174 | 0.268 | 0.397  |\n     | $\\psi \\sim (0,1)$ | 0.018 |  0.057 | 0.166 | 0.280  |\n     | $\\psi \\sim (0,2)$ | 0.012 | 0.039 | 0.0127 | 0.204  |\n\n\n* Q4: Our method scales well with larger datasets and higher-dimensional data. While larger datasets increase the training cost of the two-stage model, they do not complicate implementation. The training method costs twice as much as a typical predictive model, but remains manageable. Besides, the larger datasets increase the robustness and stability of the two-stage model. For higher-dimensional data, the computational demand increases, so we introduced approximation methods for IV-SHAP and IV-IG (see Appendix Section F), which we validated with the Spambase dataset. Extending our experiments to a 100-dimensional synthetic dataset, we found our method remained effective. Since data-faithful feature attribution is used mainly for tabular data to help understand feature relationships, and typically does not involve large dimensions. The average attribution ratio can still be used to evaluate scalability in the experimental section.\n\nReferences\n\n[1]Angrist, J. D., \\& Imbens, G. W. Identification and estimation of local average treatment effects. Econometrica, 1994.\n\n[2]Baiocchi, M., Cheng, J., \\& Small, D. S. Instrumental variable methods for causal inference. Statistics in Medicine, 2014.\n\n[3]Murray, M. P.  Avoiding invalid instruments and coping with weak instruments. Journal of Economic Perspective, 2006.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "sjUD7s6Vnc", "reviewer": "Reviewer_6T6P", "summary": "The paper introduces the problem that unobservable confounders in data can mislead and negatively affect the quality of feature attribution explanations. Feature attribution methods do not consider unobservable confounders which poses the risk of misinterpreting the attribution scores found, which could have consequences, especially in high-risk application domains where users rely on feature attribution to calibrate their trust in the model. To address the issue, the authors propose to decouple input features affected by unobservable confounding factors and those not. The latter are then used to train a confounder-free model, for which they show that feature attribution methods SHAP and Integrated Gradients are faithful to the true data generation process and robust.", "questions": "- I did not fully understand how you identified the unobservable confounders in the real dataset use case. How could practitioners identify unobservable confounders in their models? \n- Feature attribution methods explain a model output $f(x)$ by assigning attribution scores in the input instance $x$. I am unsure if I may have misunderstood your approach, but by fitting a model with a different data (one time with all variables, and once without variables affected by unobserved confounders), are you not training and explaining a different model? If this is the case, I see your contribution rather providing a training procedure on only instrumental variables and without confounding variables, leading to data-faithful attribution. Currently, the paper's contribution often reads as if you are proposing a new attribution method (i.e. referring to your approaches as IV-SHAP and IV-IG). Could you please clarify? \n- By excluding variables affected by unobserved confounders in training, does the model's predictive performance (e.g. measured in accuracy) suffer or does the task become easier for the model? Is there a trade-off between data-faithful feature attribution and model performance?\n- How similar are the resulting attribution scores of IV-SHAP and IV-IG? It has previously been found that local explanation methods like feature attribution often disagree because they only locally approximate the model (Krishna et al., 2022: The Disagreement Problem in Explainable Machine Learning). Does removing the confounding variables lead to less disagreeing feature attribution? \n- In feature attribution, we are often interested in the features with the largest positive or negative attribution scores as we want to understand which features a model relied on most for its output. In this case, the faithful ranking of the scores matters more than the score itself. I'm curious, in your experiments, did the ranking of scores change with the instrumental variable approach? How much does being faithful to the data matter in practice if not? \n- Out of curiosity: What do you think, how would this approach / does this approach transfer to high-dimensional problems like image or text classification?\n\nI will happily adjust the score when I understand the paper better.", "limitations": "Limitations and broader impact are adequately discussed in the appendix.", "rating": 7, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The paper addresses a novel perspective on feature attribution: Unobserved confounders in the data generation process can mislead feature attribution methods to assign inflated or deflated scores to features that are affected by unobserved confounders. I have not come across this perspective of connecting the data generation process to local explanations before. I believe it is an interesting viewpoint, particularly for tabular data use cases where causal relations may be discoverable. \n- The paper uses an example to illustrate the problem it is addressing and the approach to solving it, which helps the reader follow the methodology. \n- The authors provide the source code for their experiments.", "weaknesses": "- The paper is quite inline-math-text heavy, which limits its readability. I believe the readability and therefore clarity can be improved by restructuring math-heavy paragraphs (room for this could be made by e.g. shortening the introduction).\n- Some things were unclear, please see the questions.\n- Minor comments:\n     - Lines 146-154: The long equations are slightly difficult to follow when written inline, and will be clearer if using \\equation or \\align.\n     - Lines 156-159: Thought experiment results could be shown in a table, for more clarity.\n     - Line 191: it’s -> it is\n     - Please increase the font size in Figures 2 and 3 for better readability.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your encouraging and insightful comments. Please find our response below.\n\n**Response to Weaknesses**\n* W1: We will follow your suggestions to enhance the readability.\n\n* Minor comments: Fixed. Thanks!\n\n**Response to Questions**\n\n* Q1: In our paper, when addressing the impact of unobserved confounders, it is not necessary to identify what these confounders are. Instead, we only need to determine their existence and, if present, employ our method to mitigate their effect on feature attribution. For example, we can use potential instrumental variables for detection. When we intervene on an instrumental variable, we can then analyze whether the correlation between variable $\\tilde{\\boldsymbol{x}}$ and the target feature aligns with the change of the target variable caused by the instrumental variable. If the correlation is strong, but the change in the target variable caused by the instrumental variable is small, this suggests the presence of unobserved confounders which increases the correlation between $\\tilde{\\boldsymbol{x}}$ and the target feature [1].\n\n* Q2: Feature attribution indeed involves explaining a model output $f(x)$ by assigning attribution scores to the input instance \n$x$. However, our focus is on data-faithful feature attribution, i.e., we are not trying to explain the output of a specific model but trying to explain the target feature $y$ through a model. For example, consider a medical scenario where a patient is interested in understanding the impact of all features on a disease, rather than just the prediction output of a particular diagnostic model. In data-faithful attribution, the trained model is used to estimate feature subset utilities because the real-world data generation equation is unknown. Thus, the model serves as a tool for data-faithful attribution.  We propose a training method to reduce confounder influence and enhance data-faithful attribution.  We will clarify them in the paper.\n\n* Q3: Our experiments suggest that the model's accuracy might be affected, indicating a trade-off in most scenarios. When we train the model using features that have been re-estimated to remove the influence of confounders, the model may lose the correlation between the predicted output and the confounders, which also affects the target feature $y$. We compared the Mean Squared Error (MSE) of our model with a predictive model in the experiment and found our loss to be slightly higher. We note that our goal is not to maximize the predictive accuracy of the model but rather to ensure that the attribution results are more faithful to data. We will clarify it in the paper. \n\n* Q4: In our experiments, the average change rates between IV-SHAP and IV-IG scores in both synthetic and real-world datasets are below 10\\%. SHAP and IG are inspired by the Shapley and Aumann-Shapley values [2], respectively, and share many common properties [3]. In contrast, methods like LIME and DeepLIFT follow different principles and produce different values. Thus, merely removing confounders is unlikely to resolve inconsistencies between LIME/DeepLIFT and IV-SHAP/IV-IG. Discrepancies among attribution methods also arise from how they handle correlations between input features. For example, on-manifold Shapley [4] uses conditional expectations to supplement unselected features, while causal Shapley [5] uses interventions. These methods can be combined with ours, but addressing confounders alone cannot resolve differences due to varying approaches to feature correlation. Our paper focuses on the impact of unobserved confounders, and we suggest choosing the appropriate method for handling feature correlations based on the specific context.\n\n* Q5: The ranking of scores obtained from our methods and existing methods are different. We statistically analyze the proportion of attribution score ranking changes for IV-SHAP and IV-IG compared to SHAP and IG on Dataset A when $\\rho=0.2$. The results are shown in the table below.\n\n     |     Feature Deviation     | 0.125   | 0.250   | 0.375   | 0.500   |\n     |----------|---------|---------|---------|---------|\n     | IV-SHAP  | 82.7%   | 87.1%   | 92.3%   | 94.2%   |\n     | IV-IG    | 58.4%   | 69.6%   | 77.1%   | 86.3%   |\n\n     In the Griliches76 dataset experiments, 57.1\\% of the attribution rankings changed. Even when the rankings remain the same, \n     changes in attribution scores can offer valuable insights into feature contributions. For example, if the attribution score for \n     education decreases, even without a change in ranking, it could indicate that the return on education for income is lower than \n     initially expected.\n* Q6: The proposed feature attribution approach may be feasible for image and text classification. Images often encounter confounders like lighting. If a model is trained where most cat images are in strong lighting and dog images in weak lighting, the attribution for a cat image taken in weak lighting may be misleading. Identifying suitable instrumental variables to mitigate the confounders can make feature attribution more reliable and intuitive. This is an insightful extension, and we will explore it further.\n\nReferences\n\n[1] Baiocchi M, Cheng J, Small D S. Instrumental variable methods for causal inference[J]. Statistics in medicine, 2014.\n\n[2] Tauman Y. The Aumann-Shapley prices: a survey[J]. The shapley value, 1988.\n\n[3] Sundararajan M, Najmi A. The many Shapley values for model explanation[C]//(ICLR), 2020.\n\n[4] Frye C, de Mijolla D, Begley T, et al. Shapley explainability on the data manifold[C]//(ICLR), 2020.\n\n[5] Heskes T, Sijben E, Bucur I G, et al. Causal shapley values: Exploiting causal knowledge to explain individual predictions of complex models[J]. (NeurIPS), 2020.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
We are very grateful to the reviewers for their encouraging and insightful comments. To address the concerns, we provide detailed point-to-point responses as follows.

Meta-review: The paper makes an important contribution in understanding the causal attribution that drives an outcome variable. Rather than standard attribution scores that explain a predictive model, the authors propose an IV-based method that explains the true outcome variable. The method is motivated through a theoretical analysis of the effect of unobserved confounder on attribution scores. Reviewers agreed that the technical contribution and insight is note-worthy and therefore I'm recommending Accept. However, they also have two suggestions for improving the clarity of the paper:
1) It will be helpful to make it clear in the abstract and introduction that the goal is to explain the true outcome variable (not a given ML model)
2) The limitations in Appendix A and the limitations discussed in the rebuttal period should be included in the main text.
Decision: Accept (poster)