Causal Dependence Plots

Authors: joshua r loftus
lucius e j bynum
sakina  hansen
Venue: NeurIPS_cc_2024_Conference
Published: 2024-05-13 13:19:09

Abstract: To use artificial intelligence and machine learning models wisely we must understand how they interact with the world, including how they depend causally on data inputs. In this work we develop Causal Dependence Plots (CDPs) to visualize how a model's predicted outcome depends on changes in a given predictor *along with consequent causal changes in other predictor variables*. Crucially, this differs from standard methods based on independence or holding other predictors constant, such as regression coefficients or Partial Dependence Plots (PDPs). Our explanatory framework generalizes PDPs, including them as a special case, as well as a variety of other interpretive plots that show, for example, the total, direct, and indirect effects of causal mediation. We demonstrate with simulations and real data experiments how CDPs can be combined in a modular way with methods for causal learning or sensitivity analysis. Since people often think causally about input-output dependence, CDPs can be powerful tools in the xAI or interpretable machine learning toolkit and contribute to applications like scientific machine learning and algorithmic fairness.

Main paper: 
 introduction: 
 This paper develops Causal Dependence Plots (CDPs) to visualize relationships between input variables and a predicted outcome. Motivated by explaining or interpreting AI or machine learning models [8, 16, 17, 36] , for simplicity we consider supervised learning, i.e. regression or classification. We also focus on the model-agnostic or "black-box" setting, where the interpreter can query the model but not access its internal structure. Model-agnostic interpretation methods are functionally limited to observing how the model responds to variation in the inputs. While this initial application forms our practical motivation, we emphasize that CDPs are more general.Simple explanations that focus on one input variable at a time can be powerful tools for human understanding. However, just as with the interpretation of linear regression model coefficients, these simple relationships can be misleading. When varying one input variable, we must make some choice about what values to use for the other inputs. CDPs make this choice using an explicit causal model, and to our knowledge this is the first work that does so. We compare CDPs to other state-of-the-art, non-causal explanation methods like the Partial Dependence Plot (PDP) [12] , Individual Conditional Expectation (ICE) [14] , Accumulated Local Effect (ALE) [3] , and Shapley Additive Explanation (SHAP) feature plot [33] . Explanation methods may respect existing causal dependencies between predictors or break them.Problem statement. If there are causal relationships between predictors but our visualization, interpretation, or explanation method does not respect them the resulting model explanation may be irrelevant or misleading [37, 54] . Such explanations could lead to incorrect decisions for regulating or aligning algorithmic systems, sub-optimal allocations of resources based on model predictions, a breakdown between human feedback and reinforcement learning systems, or other forms of error and harm. In scientific machine learning-where explanations can be used to generate hypotheses for follow-up investigation-a flawed interpretation may support spurious hypotheses. For these reasons, the causal validity of model explanations should be a top priority.High level proposal. We wish to interpret or explain a given supervised machine learning model f (x) with p input features x = (x 1 , . . . , x p ). Specifically, we want to understand how the predictions ŷ = f (x) of this model depend on feature x j for a given j, 1 ≤ j ≤ p. PDPs and ICE plots do this by varying x j and holding the other features x \j constant, where x \j is the (p -1)-tuple containing all features except for x j . This implicitly assumes independence between x j and the other inputs. Our method replaces this independence assumption with an Explanatory Causal Model (ECM) for the input features. This auxiliary ECM is a tool we use to help explain f , it determines how other inputs x \j vary when x j is changed.CDP pseudo-algorithm. To construct a CDP showing how f (x) depends on x j , a user specifies an ECM M containing the predictors x, and an intervention I(x j ) in M. The intervention changes x j , and may change other features if they are caused by x j in M. The type of intervention is chosen based on the type of causal explanation desired, with several example options demonstrated later. An explanatory dataset D = {x i : i = 1, . . . , n} can be given or, if unavailable, generated by M. Note that we use the notational convention where i indexes observations or examples while j indexes features. The horizontal axis of the plot is specified by a grid {x j,k : k = 1, . . . , K} of possible values for x j , with k indexing grid points. For each observation x i in D, and at each grid point xj,k :1. Use the ECM to simulate counterfactual values x * i,k for all features of observation i under the intervention I(x j,k ). 2. Input counterfactual features to the prediction function f , and store the resulting counterfactual prediction ŷ * i,k = f (x * i,k ) in an array indexed by (i, k). For each observation i in D, construct the individual counterfactual prediction curve (x j,k , ŷ * i,k ) by connecting points that are adjacent on the plot grid. Plot the empirical average of these curves, which is the main output of the CDP. The individual curves can be shown or suppressed as desired. The resulting CDP shows how the model's predictions ŷ causally depend on x j when this predictor is varied by the intervention I(x j ) in ECM M.PDP and ICE algorithm. Start with the notation and setup as above but without any ECM or intervention. For each observation x i in D, and at each grid point xj,k : 1. Define x ′ i,k by setting feature x ij to the grid point xj,k and keeping other features x \j fixed at their original values in x i from D, that isEQUATIONDifferent combinations of the predictive setting and choice of ECM generate many uses for CDPs.In general, ECMs can be designed based on a particular desired explanation, make use of prior domain knowledge, or be learned and estimated from data using causal learning methods in a modular fashion. Importantly, the ECM does not need to contain the outcome variable y except in one special case-residual plots-to be discussed later.Causal bridge. In one special case we may wish to understand how f depends on a variable z which is not one of f 's input features but is causally related to them. Other methods cannot do this, but CDPs can provided the ECM also contains z. For simplicity we choose notation in the rest of the paper to reflect the case where the explanatory variable is an input, but this is not a loss of generality since we can simply define fdrop (x, z) := f (x) and apply CDPs to fdrop . Since the ECM may vary x when z is changed, we can see how f depends on z. This could be useful to probe a predictive model for fairness with respect to a sensitive attribute that the model does not use directly.Incomplete causal knowledge. There are various applications where an ECM does not need to be a fully specified or "correct" model for all features. First, CDPs only use the predicted-and not actual-outcome. This is useful for semi-supervised or anticausal learning: given causal structure among predictors only and a supervised learning model, attempt causal inference for the outcome [52, 63] . Second, we may only require explanations or plots for one or a small number of features. In such cases we only need information from the ECM about interventions on the features of interest and their causal descendants, and not other predictors. Finally, predictive models often use features that are known transformations or representations of inputs, and these transformations can be used to construct an ECM. For example, if the features are (x, x 2 , z), an ECM can simply encode the fact that x 2 depends causally on x. Even if we do not know the dependency between x and z, we can make use of our partial knowledge about the features.Multiparty auditing, e.g. for fairness. An owner of a predictive model may not have causal knowledge or incentives to use such knowledge. Predictive accuracy is their only concern. But a separate party, like a regulator, may audit that model. This party may have more causal knowledge due to specializing in auditing, or may be legally obliged to make certain causal assumptions for the purpose of the audit, e.g. to allow disparities only along certain causal pathways and not others. Previous work applied causality to fairness [5, 10, 27-29, 32, 34, 38, 48, 60, 62] , recourse [7, 26, 44, 57] , and other desiderata. Existing methods like PDPs are limited to only showing direct dependence, and this may hide the full extent of unfairness or discrimination [18] . CDPs can be used to probe a black-box for unfairness in the form of total dependence or partially controlled dependence.Explanations under covariate shift. Often a pre-trained model is used for predictions on data from a different data generating process than the training DGP. We can use ECMs and CDPs to visualize how the model will behave out-of-distribution. ECMs could even be chosen adversarially.Scientific theory development. Large and complex models may be fit to data where underlying structure is largely unknown. In such settings, relatively simple ECMs can be used to formulate simple hypotheses relating some predictors and plot causal dependencies to check these hypotheses or generate new ones. This can also be done hypothetically, assuming an ECM for exploration.After defining the CDP framework we demonstrate CDPs on synthetic and real datasets in Section 3 and Appendix B, including in conjunction with structural causal learning in B.2. We compare CDPs with other state of the art competitor visualization methods, for example in Figure 4 . Theorem 2.10 establishes the first universally valid causal interpretation of PDP and ICE plots. Finally, in Section 2.7 we illustrate how to visualize uncertainty about the choice of ECM. 
compute prediction ŷ′: 
 i,k = f (x ′ i,k) and store in an array indexed by (i, k). Plotting (x j,k , ŷ′ i,k ) generates an ICE curve for each i, and the empirical average of these is the PDP.Motivating example. Consider a model for parental income P , school funding F , and graduates' average starting salary S, with ECM shown in the bottom row of Figure 1 . In the top row, the ECM functions are plotted in the left panel, and the remaining panels show visual explanations of supervised models that predict Ŝ = f (P, F ). In this example the training data for black-box models was generated by the ECM, but later we will see real data examples where this is not the case. Blue curves show how Ŝ depends on P when P is causally manipulated without holding F constant, i.e. under the intervention do(P = p). Orange curves show the dependence of Ŝ on P when F is held constant at its observed value, and coincide exactly with standard PDPs. Full definitions of these are given in Section 2. Several key takeaways:• Comparing the direct (or partial) dependence curves and total dependence curves we see there can be qualitative differences depending on the type of explanation desired, for example one can be increasing while the other is decreasing. This is a consequential fact when considering how interventions may change (predicted) outcomes. Increasing P causes larger values of Ŝ, but if the increase in P is done while holding F constant then it could cause a decrease in Ŝ (or a smaller increase). An intervention which does not hold other predictors constant-arguably the canonical causal operation-can be shown by our TDP. • Our framework includes some existing model explanation plots like ICE and PDPs as special cases. In panels (c-d), and later in Theorem 2.10, we see that PDP + ICE = NDDP.P F S Ŝ = f (P, F )            P ∼ U[0, 1.5], F = 2P 3 + N (0, 0.2 2 ), S = F -P 2 + N (0, 0.2 2 ) Ŝ = f (P, F )A practitioner seeing only the PDP in panel (d) may conclude that "dependence" of Ŝ on P is weak, especially if P ≤ 1. The TDP in panel (c) shows a stronger increasing relationship closer to the true total dependence and a more holistic view of how Ŝ depends on P . Our work clarifies that the weaker form of dependence shown by PDPs is natural direct dependence. We also see the same weak dependence empirically for SHAP and ALE plots in Figure 4 .• Explanations of models can be qualitatively different from the underlying causal relationships. For example, the random forest in panel (c) shows a direct dependence of Ŝ on P that is increasing when the true direct dependence of S on P is decreasing. Predictive machine learning models may fail to capture causal dependence, and in this case studying the black-box would not necessarily help us learn about the real world. As another example, panel (b) shows that the total dependence of a linear model on a predictor can be non-linear, in this case because the mediator F depends non-linearly on P .We are given a predictive model f , possibly estimated or learned using empirical risk minimization (ERM) f = arg min h∈H n i=1 ℓ (h(x i ), y i ) with some loss function ℓ, pre-specified function class H, and an independent and identically distributed training sample {(y i , x i ) : i = 1, . . . , n} with feature vectors x T i ∈ R p . In Section 2.6 we focus on simple mediation analysis and partition the predictor variables into subsets so that X and M both notate predictors, M being a mediator.To create an explanation of model dependence on a single feature, like a plot with x j on the horizontal axis and f on the vertical axis, we must decide what to do with the other features when varying x j along the plot axis. Most explanation methods use the same approach as the PDP and ICE plots: they hold other features fixed at values in a (auxiliary, explanatory) dataset. This may be unrealistic if other features depend on x j causally, or even mathematically undefined if features are mutually constitutive, e.g. (x, x 2 ) or the set {GDP, GDP per capita, population}.Our notational conventions and definitions are influenced by [6, 41, 43] . Let U be a set of exogeneous noise variables, V a set of p = |V| observable variables, and G a set of functions such that for each j ∈ 1, . . . , p we have V j = g j (PA j , U j ), where PA j ⊆ V and U j ⊆ U are the observable and exogeneous parents, respectively, of variable V j . Let the directed acyclic graph (DAG) G have vertices given by variables and, for each V j ∈ V and each of the parent variables in PA j and U j , a directed edge oriented from the parents to V j . Definition 2.1 (Structural Causal Model (SCM)). A (probabilistic) SCM M is a tuple ⟨U, V, G, P U ⟩ where P U is the joint distribution of the exogeneous variables. This distribution and the functions G determine the joint distribution P M over all the variables in M. Finally, causality in this model is represented by additional assumptions that M admits the modeling of interventions and/or counterfactuals as defined below. Definition 2.2 (Interventions). For the SCM M, an intervention I produces a modified SCM denoted M do(I) which may have different structural equations G I . Correspondingly, some variables may have different parent sets, so the DAG representation G do(I) may also change. We denote the new, interventional distribution as P M;do(I) . A simple class of interventions involves intervening on one variable, e.g.I = do V j := g( PA j , Ũj ) ,which changes how V j and all variables on directed paths from V j in G are generated. An even simpler sub-class of these are the atomic interventions setting one variable V j to one constant value v, which we denote I j,v := do(V j = v). Note that in this case V j has no parents in the graph G do(I) ; the source of the intervention itself is outside the world of the model.Interventions are useful for modeling changes to a data generating process (DGP), for example, experiments that control a particular variable to see how its value changes other variables, or policy changes aimed at altering or removing existing causal relationships. In addition to generating new observations as a DGP, an SCM can also be used to model counterfactual values for observations that have already been determined. A counterfactual distribution is an interventional distribution defined over a specific dataset with information or constraints given by some of the observed values in that data, as we now describe. Definition 2.3 (Counterfactuals). Let V be the observed variables for observations in a given dataset, PA j = v and U j the observed and exogeneous parents of variable V j , and I and intervention that modifies any of V j 's parents. The intervention I may hold some or all of v fixed and vary U j ← u, passing these through g j (v, u), or through gj (ṽ, u) if the intervention also changes any of v ← ṽ.The counterfactuals V j (ṽ, u) are values V j would have taken if any of its observed and/or exogeneous parents had taken the different values (ṽ, u). To define the counterfactual distribution P M|V=v;do(I) , we use the posterior or conditional (depending on our probability model approach) distribution P U|V=v to model uncertainty about U while computing counterfactual values of variables for an observation in the modified SCM M do(I) . Remark 2.4. Note that if the desired causal explanation uses counterfactuals, then we likely obtain the observed values from an auxiliary explanatory dataset. But since an SCM can generate data, we may also use it to generate the initial observed values and then re-use these when computing counterfactuals for the explanation.Our proposed solution to the fundamental problem highlighted for univariate explanations is to use an auxiliary ECM M j and let this causal model determine how other features vary as functions of x j . We denote these explanations as E j ( f ; M j ) or E j ( f ) if the context is clear. In the deterministic or noiseless case, suppose we know continuous functions g kj such that x k = g kj (x j ), with g jj the identity. In this case the model M j tells us g(x j ) := (g 1j (x j ), . . . , x j , . . . , g pj (x j )) is a curve in R p parameterized by x j , and we generate the explanation E j ( f ) by plotting f (g(x j )) against x j .G X Y E( f ) (a) X M Y E( f ) (b) X M Y E( f ) (c)Figure 2 : An ECM for predictors is used to produce an explanation E( f ) of the predictive model f . Solid arrows represent possible causal relationships in the ECM, and dotted arrows show dependence of the model explanation on predictors. In (a) G X denotes the subgraph of the SCM for predictors. In the mediation example (b), predictor X causes Y directly and also through mediator M , creating an important distinction between direct dependence (orange) and total dependence (blue). The reverse causality example (c) shows variables useful in predicting Y may be caused by Y , and also be causes of prediction Ŷ = f (X, M ) and the explanation of that prediction.To extend this strategy to non-deterministic causal models we use an ECM M X for the predictor variables with G X its associated DAG. We represent this graphically in Figure 2 . The expressive power of SCMs allows us to pose various interpretive questions and compute various kinds of explanations by performing operations in M X .For the following definitions, we assume predictor variables X ∈ Ω X , an outcome of interest Y ∈ Ω Y , and a black-box function f (x) : Ω X → Ω Y with outputs that we may also denote Ŷ . A structural causal model M, either assumed or learned from data, specifies causal relationships for the predictors X, i.e. it need not involve the outcome Y . Note that predictors may only be a subset of the variables in M as in the "causal bridge" application discussed in Section 1.1. Definition 2.5 (Explanatory Causal Model (ECM)). An ECM M ′ augments the SCM containing predictors by including the predicted outcome Ŷ as an additional variable with f as its structural equation.Generating causal explanations for f involves performing abduction, action, and prediction with this ECM. In a large ECM graph we may suppress all arrows into Ŷ except those from the explanatory feature and its descendants. This is to simplify the display, as in Figure 5 .Additional notation and conventions. We use the shorthand f (P M ), where f takes a distribution P M as its argument, to denote using data from that distribution as the input to the black-box function f . For each type of causal explanation with a given Named Effect based on intervention I, we define the Individual Counterfactual Named Effect curves as the set of counterfactual curves f (P M|V=v;do(I) ) for each individual, the Named Effect Function as their (empirical) expectation Ê f (P M|V=v;do(I) ) , and the Named Dependence Plot as a plot displaying all of these curves. EQUATIONThese show the effect of intervention I θ on black-box output for each individual observation in the explanatory dataset as θ varies. The (empirical) average of these (over the explanatory data) is (an estimate of) the Causal Effect Function (CEF), and a plot showing the IC and CEF is a Causal Dependence Plot.We typically apply this to create plots for one explanatory feature X s at a time using interventions like I θ = do(X s = θ). Horizontal axes for plots use a grid over the possible values of X s given by its range in dataset D. Bar graphs can be used when the explanatory feature is categorical.Next is perhaps the most straightforward and important named effect. Definition 2.7 (Total Dependence Plot (TDP)). For an intervention I, the Individual Counterfactual Total Effect (ICTE) curves TE(I) = f (P M|X=x;do(I) )(3) show the total effect of intervention I on black-box output for each individual observation in the explanatory dataset. The (empirical) average of these (over the explanatory data) is (an estimate of) the Total Effect Function (TEF), and a plot showing the ICTE and TEF is a Total Dependence Plot (TDP). We compute the TDP following Algorithm 1.Algorithm 1 Total Dependence Plot (TDP) Inputs: M (ECM), f (black-box predictor), D (explanatory dataset), X s (covariate of interest) Let X be a grid of possible values of X s Set N to the number of observations in D Initialize N × |X| matrix of predictions Ŷ for x in X do Define intervention I = do(X s = x) Sample counterfactual dataset D Xs←x entailed by P M|D;do(I) Set Ŷ [:, x] to f (D Xs←x ) end for Plot N lines (X, Ŷ [i, :]) {(Individual Counterfactuals)} Plot average (X, i Ŷ [i, :]/N ) {(Causal Dependence)}Remark 2.8. In the remaining definitions, we give notation only for the individual counterfactual curves and leave the other objects implicitly defined.We often wish to decompose how much of the total effect of X on Ŷ is attributable to different pathways between the variables. This can be explored via direct dependence below, as well as with other named CDPs described in Appendix A. Definition 2.9 (Natural Direct Dependence Plot (NDDP)). Given intervention I define a corresponding intervention J that intervenes on all descendants of any variables that are changed by I, except for Ŷ , and resets them to their observed values in dataset D. We then define the Individual Counterfactual Natural Direct Effect curves NDE(I) = f (P M|X=x;do(I,J) ).This quantity represents the effect of intervention I on black-box output Ŷ while all variables not directly intervened upon by I are fixed at their 'natural,' i.e., pre-intervention values in D. Algorithm 4 demonstrates how to compute the NDDP.From this construction of NDDP, we see by comparing it to the PDP and ICE algorithm that it is equivalent to these, confirming what we observed in Figure 1(d) .Theorem 2.10 (PDP + ICE = NDDP). When generating plots for the predictive model f using the dataset D and feature X s , the ICE plot curves and Individual Counterfactual Natural Direct Dependence curves are identical. Hence, the NDDP is identical to a PDP that includes ICE curves.Proof. We have implicitly assumed both plots will use the same range for their horizontal axes. This is natural as implementations use the range of the feature in the dataset, and both plots are constructed from the same feature X s in the same dataset D. Since the PDP and NDDP both contain empirical averages of their respective Individual curves it suffices to show these are equal at each point x in the plot grid.Consider individual i in dataset D with features x i = (x i1 , . . . , x ip ). The value of the ICE curve at x for this individual is, from (1), f (x ′ i,k ) where x ′ i,k := (x i1 , . . . , x is ← x, . . . , x ip ), i.e. the original features x i but with entry s set to x. We must show this is the same as the value of the Individual Counterfactual Natural Direct Dependence curve at x for individual i. Applying Definition 2.9, we use interventions The NDDP applies these in the order I followed by J. First, I sets the value of feature X s to x for all individuals, and may modify other features if they are descendants of X s . Then J intervenes on each descendant X j of X s and resets it to its observed values in D, and in particular for individual i these are each reset to x ij . Hence, the value of the Individual Counterfactual Natural Direct Dependence curve at x for individual i is also given by f (x ′ i,k ).I = do(X s = x) and J = do(X j = x j if X s is an ancestor of X j ).Remark 2.11. Note that there is some subtlety in the assumption of using the same dataset: a Bayesian probability modeling approach to SCMs may add more randomness when sampling counterfactuals. In this case, rather than the ICE and ICNDD curves being identical, the ICE will equal the expectation of the ICNDD curves over this additional source of randomness. The additional randomness is specified by priors over exogenous variables, and the expectation can be estimated by more sampling at a computational cost of a constant factor. Since the usage of these plots is visual and somewhat qualitative this subtlety is not an important limitation of the theorem, and it would only apply under particular modeling assumptions. Remark 2.12. To our knowledge this is the first result establishing a universally valid causal interpretation of PDPs. Its most important limitation is that it applies to the model output Ŷ and not necessarily the original outcome Y .Several other types of named CDPs are described in Appendix A.Many applications involve a causal structure we refer to as a mediation triangle, with examples shown in Figure 1 and Figure 2b . In mediation analysis, we often wish to decompose how much of the total effect of X on Y is attributable to the pathway through M and how much of it is direct. CDPs allow us to visualize frequently studied quantities of interest in this setting including other special cases defined in Appendix A.1. Although mediation analysis motivates CDPs and helps build intuition, we emphasize that our definitions can be used with any structural causal model. See Section 3 for other, more complex examples.There are various ways to incorporate uncertainty about the ECM into CDPs. We explore a natural first extension of the CDP that shows a range between possible effect functions induced by a set of auxiliary ECMs. The set of ECMs could be pre-specified or, for example, given by a Markov equivalence class output by a causal structure learning algorithm. Returning to our motivating example from Section 1, we might question whether parental income P impacts school funding F , considering instead an SCM without mediation: P → S ← F . Figure 3 shows a range of possible effect functions interpolating between this ECM without the indirect effect and the original ECM in Section 1, for each of the TDP, NDDP, and NIDP. In this we have assumed the same structural equations for the edges that are common to both models. These plots show a range for how Ŝ may depend on P when we are unsure how F depends on P . Figure 9 in Appendix B.2 shows an example with real data where we use candidate ECMs discovered by the PC algorithm. These examples are not confidence regions, but any method for producing confidence sets in SCMs could also be used with CDPs to display uncertainty regions. Future work can develop additional methods for  
experiments: 
 We demonstrate CDPs in a series of experiments with simulated and real datasets.Comparison with other explanatory plots. Figure 4 shows accumulated local effect (ALE) [3] and Shapley Additive Explanation (SHAP) [33] feature plots for the salary example. These appear similar to the PDP, with a more weakly increasing relationship than that seen in the TDP. TDPs represent a significant and novel contribution to the existing model visualizations.Real data with domain knowledge. An ECM may be constructed using domain expertise. Figure 5 shows an ECM and CDPs for the Sachs et al. [49] dataset of expression levels of proteins and phospholipds in human cells, for which data and a ground-truth DAG1 are publicly available in the Causal Discovery Toolbox [24] . While the actual biology of the problem is not our focus here, there are meaningful takeaways from the figure. For this model, the TDP shows an increasing relationship, while the NDDP/PDP shows a decrease. The overall direction of the trend in predictions based on PKA is reversed if we hold other predictors fixed. This is an important lesson for using model explanations in scientific machine learning.Additional experiments can be found in Appendix B. Notably, in Appendix B.2 we learn an ECM from data with a causal structural learning algorithm and then use it to produce CDPs. The main takeaway of the real data experiments is that CDPs can be useful in practice.In simulation experiments we know the true DGP, so we can compare its functional form to various black-box models and their explanatory plots. Results in Appendix B.1 show that CDPs are sensitive to whether the functional form assumptions of the black-box model fit the DGP. In other words, if a black-box model is a poor fit to the DGP, then CDPs can accurately explain the black-box but will not reflect the true DGP. This limitation is not specific to CDPs but applies to all explanation methods. Figure 6 also shows that different ECMs can produce different CDPs for the same black-box model, and a misspecified ECM can produce misleading CDPs. 
discussion: 
 Related work. Recent work in recourse [25] uses contrastive or counterfactual explanations [56] . Some of this focuses on causal dependence [50] . Blöbaum and Shimizu [4] identify the predictor with the largest total effect, which is most applicable when assuming linearity. Zhao and Hastie [63] investigated causal interpretations of PDP, aiming for causal inference about the underlying DGP, and showed that when the DGP satisfies the backdoor criterion [39] then a PDP visualizes the total effect (TE) of a predictor. Cox Jr [11] observed an association between partial dependence plots and NDE, an equivalence we formally establish in Theorem 2.10, to our knowledge the first such result. Lazzari et al. [30] weight observations when computing PDPs. There has been some recent work creating causal variants of SHAP [1, 13, 19, 23, 58] , and in future work we will explore comparisons of appropriate special cases of CDPs with these. We are not aware of any previous causal explanation work with the generality of CDPs.Limitations. Causal modeling always involves some limitations [15, 47] . For CDPs, full specification of an ECM can be a strong assumption. However, in Sections 1.1 and 2.7 we discussed some ways this can be relaxed. In general, if a causal explanation is desired or necessary, then we cannot avoid making causal assumptions. Model-agnostic explanation methods also always have certain limitations [2, 36] . For example, if the predictive model fails to fit the DGP, then any model explanation will also fail if our interpretive goal is to learn about the DGP [63] . CDPs may be misleading if the true DGP differs in important ways from the ECM, as shown in Figure 6 . However, standard PDPs and similar explanation methods also require auxiliary explanatory data, and that data may also differ from the target DGP. So this is not an additional limitation specific to our method.Causal Dependence Plots use an explanatory causal model to create plots with causal interpretations. This allows us to use the powerful language of structural causal models to pose and answer a variety of meaningful questions. Our framework generalizes Partial Dependence Plots, which Theorem 2.10 shows it includes as a special case, and allows other kinds of causal interpretations we have not seen previously explored in the literature. Future work in this direction could expand on some canonical causal structures for useful applications, or interface with other kinds of models, for example extending to non-tabular data by applying causal representation learning.Relating explanation methods to Pearl's ladder of causation [40] , most previous interpretable machine learning and explainable AI methods-like PDPs-concern associations and hence are confined to the first rung of the ladder. With CDPs we ascend the ladder, creating model interpretations intended to change the world. Interpretability provided the initial motivation for CDPs, but since plots are qualitative CDPs also open the door for future work on causal methodology that relaxes assumptions while maintaining visual validity.Broader Impacts. There are many potential societal consequences of our work: essentially those shared by all tools for model interpretability and explainability. Model explanations can be misleading, either due to error or intentional deception. When a user is convinced by a flawed model explanation to reach misguided conclusions about a model, they may make harmful or sub-optimal decisions about how or whether to use that model. For example, if an explanation tool is used to assess the fairness of a model, a flawed explanation could lead to the conclusion that a discriminatory model is fair or that a fair model is discriminatory. In applications related to science, a flawed model explanation can lead to wasting resources pursuing a dead-end hypothesis or to missing out on an important discovery.Similarly, poor business decisions can be made on the basis of flawed explanations.[ For completeness we include here definitions for several other important, named CDPs. EQUATIONNotice that intervention I is performed before intervention K. This quantity represents the effect of intervention I on black-box output Ŷ that is due only to any indirect pathways to Ŷ . We compute the NIDP following Algorithm 5. The difference between two values of this function can be used to express the natural indirect effect as a special case.This section describes additional algorithms including several special cases of named CDPs. We use this DGP to fit two different black-box models: one model that assumes the correct functional form (i.e., the relationship for Y shown in the DGP above), and an 'incorrect' model that predicts Y via linear regression. We use two different ECMs to construct CDPs, one which is the true DGP and one which incorrectly assumes the structure M 1 → X → M 2 . Figure 6 shows the CDPs for each of these models using the black-box training data as the explanatory data. Similarly, Figure 7 shows SHAP and ALE plots using the same explanatory data.We can glean a couple insights from Figure 6 . First, CDPs are sensitive to whether the functional form assumptions of the black-box model fit the ground truth data generating process. Good explanations for Ŷ may be different from good explanations for Y if the black-box model is poorly specified. The second is that a misspecified ECM can produce bad explanations even when the black-box model correctly fits the true causal relationships in the DGP.In Figure 7 we see that ALE and SHAP produce similar explanations to the PDP/NDDP. We saw this previously for the random forest black-box model in the example from Figure 1 .The Breast Cancer Wisconsin (Original) dataset [35] is a publicly available dataset often used to test algorithms on medical data. The dataset contains 9 ordinal variables, which represent attributes of the cells within a breast mass: Clump Thickness, Uniformity of Cell Size, Uniformity of Cell Shape, For the undirected edge between Cell Size and Cell Shape, we investigate the sensitivity to the different options in Figure 9 . Note: Our intention is not to make conclusive scientific statements, but only to demonstrate how CDPs could be used in conjunction with causal structure learning.Marginal Adhesion, Single Epithelial Cell Size, Bare Nuclei, Bland Chromatin, Normal Nucleoli and Mitoses. The outcome variable is the class of the breast tumor, benign or malignant.We use a causal structural learning algorithm, specifically the PC algorithm [55] implemented in Julia CausalInference [51] , to learn a DAG for this dataset, on a smaller subset of predictor variables for simplicity. Figure 8 shows the resulting DAG and CDPs for a random forest model to classify the Class variable.This shows CDPs can be combined with other causal methods like structural learning algorithms. The PC algorithm output had an undirected edge between Cell Size and Cell Shape. We next explore the graph structures consistent with this uncertain edge.Figure 9 shows the TDP, NDDP, and NIDP for a learned additive noise model with three different structures consistent with G B : (1) with the edge Cell Shape → Cell Size, (2) with the edge Cell Size → Cell Shape, and (3) with no edge between Cell Size and Cell Shape. This figure shows that the takeaway about cell shape impacting tumor class is indeed sensitive to our choice about the uncertain edge, particularly for the TDP.One criticism of PDPs is that they may hide heterogeneity or individual variation, and for this reason it may be good practice to include the ICE curves in any PDP. Just like the individual curves in an ICE plot, the individual counterfactual curves in our CDPs can show important effects that are hidden by averaging as illustrated in Figure 10 . In our implementation the default settings for CDPs-and our recommendation-is to show these individual curves.CDPs are applicable not only to understand model outputs but also to understand model performance. For example, CDPs can be used to probe residuals (or other measures of error) under distribution shift. Figure 11 demonstrates this with the salary example in Figure 1 . Using the same RF model, residuals show no trend with respect to parental income P nor school funding F . However, a TDP over the residuals reveals the random forest model learns a causally incorrect functional form for Ŝ, even if it fits the training data well. By comparison, only the causally correct model shows no trend in residuals with a CDP. This is an empirical verification of the correspondence between robustness to distribution shift and causal learning [46] .Predictive models were fit using scikit-learn [42] . CDP implementations make use of causal modeling functions in dowhy [53] . Figures were generated with matplotlib [20] . For causal structural learning we used the PC algorithm [55] implemented in Julia CausalInference [51] . We used the Python implementation of ALE plots in [22] . In experiments we used the Breast Cancer Wisconsin (Original) dataset [35, 59] Justification: Broader impacts are discussed, and are essentially those common to all model explanation/interpretation tools.Guidelines:• The answer NA means that there is no societal impact of the work performed.• If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact. • Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations. • The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster. • The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology. • If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML). 
safeguards: 
 Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?Answer: [NA] Justification: No such data or models.Guidelines:• The answer NA means that the paper poses no such risks.• Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters. • Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images. • We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.12. Licenses for existing assets Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?Answer: [Yes]Justification: Code packages and datasets are cited and citations include licensing information.Guidelines:• The answer NA means that the paper does not use existing assets.• The authors should cite the original paper that produced the code package or dataset.• The answer NA means that the paper does not involve crowdsourcing nor research with human subjects. • Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper. • We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution. • For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.38th Conference on Neural Information Processing Systems (NeurIPS 2024).Following the discussion in[45] and follow-up ground truth DAG for the Sachs et al.[49] dataset in Figure5of[45], we choose the edge PIP3 → PIP2 in order to eliminate a would-be cycle. 
 
 Appendix:  


Decision: The consensus is full among the four reviewers, they awarded exactly the same rating at the first review.

The authors-reviewers engagement was high with 33 posts published on this manuscript.

The strenghts of the paper overwhelm the weaknesses.

I greatly appreciated the opinion from 6pua who raised several issues on how the paper is presented. In particular, a paper introducing a new visualization and explainability approach should give a clear picture of how the new technique must be applied, which kind of results can be expected and how these results can re interpreted and leveraged on. Another weakness is the limitation implied by the need to have a fully specified ECM. However, the rebuttal and subsequent discussion clarified on this aspect that was finally considered not that a strong limitation.

Amongst the strengths, I feel that the following are of some interest; i) the authors revealed a flaw in explainable ML while proposing a solution, ii)  the manuscript introduces more causal reasoning into the model explainability, while doing so they try to improve the interpretation of graphical representations. I share the same view as two reviewers that this manuscript is a little weak from a causal inference perspective, but quite original and useful in the explainable AI field. An additional strength of the paper is that it presents the novel concepts very well and also develops theoretical arguments (Theorem. 2.9),that are useful.

In conclusion, I think the paper deserves attention even if it could be significantly improved whether the authors can address and solve the issues raised by some of the reviewers. In particular, I think that to improve the quality of the paper the authors can refer to the excellent review from 6pua.

Reviews: review: paper_summary: paper_summary: The authors tackle the problem of evaluating the dependence between the inputs and output of black box machine learning models.  Generally, analysis of this type of dependence is done in a univariate manner, holding constant all but one variable and visualizing how the outcome changes as we vary that one variable.  However, when the variable of interest is causally related to other input variables, these results may be misleading and only show part of the picture.  To help elucidate the bigger picture in these cases, the authors propose a framework that generalizes partial dependence plots, allowing for the visualization of the dependence between input variables of interest and the outcome in terms of metrics like total, direct, and indirect effects.  The authors demonstrate this approach on simulated and empirical data.
strength_weakness: strength_weakness: I really want to like this paper.  I love visualization tools and explainability, so work like this appeals to me.  However, I feel like the way this work is presented does it a disservice.  For a paper introducing a new visualization and explainability approach to be adopted, readers need a clear picture of how to apply these types of visualizations to their problems, what sort of results they can expect, how to interpret those results, and then what to do with them.  The authors do provide an example early on in Figure 1 with some key takeaways and some brief discussion of results in the Experiments section.  However, the discussion is generally both too high-level (such as saying "this plot shows X" without clarifying how to read X of the plot) and stops short of where actual application would need to go (such as saying "we can see from the plots that X and Y differ", without describing why that is significant and how a practitioner should interpret that difference).  This results in a paper that seems to have some interesting visualization ideas and methods but that I feel would struggle to see the methods actually adopted in practice without clearer application guidance.  I struggled a lot with what score to give, and I'm absolutely open to changing it based on other reviewers' comments and the authors' response, but for now, I can't quite vote for accept given these issues.

More specifics:

Figure 1 is close to being very helpful as an early intuition-builder, and, with a little bit more description, could actually be very useful.  I think my main issues with it as-is are:
- The left 3 plots have the orange and blue lines as defined in the figure description, but the description doesn't say what the light grey points are in the plot.
- The caption describes the orange lines as "Natural Direct Dependence", a term that is not actually defined until page 7.  The text does, at least, describe the orange lines as the dependence "when F is held constant at its observed value", but then describe it as coinciding "exactly with a standard PDP", another term which is not defined outside the appendix.
- I actually really like the three points of takeaways from Figure 1.  However, at least for me, they're missing a step: what exactly are you seeing in the plots that leads to these takeaways?  For example, the first takeaway is that "there can be qualitative differences between direct (or partial) dependence and total dependence."  However, given that we're still in the introduction and these terms haven't been clearly defined yet, you should clarify that you're concluding this by comparing the blue and orange within each individual plot and ideally also what this means semantically for the example. (i.e., If we just looked at total dependence, we may conclude X, but this plot shows that partial dependence actually behaves like Y) The third takeaway is also mostly a great description, but I think I'm still missing something.  Does the takeaway that random forest messes up on direct dependence mean that the random forest model is incorrect?  If all I had done was train a random forest model and saw that result, would I have any way of knowing that it's incorrect?

Even as someone who knows what counterfactuals are, I found your definition of counterfactuals in Definition 2.3 confusing.  It sounds like we're reasoning about interventions on $V_j$, and this intervention could be setting it to a constant or defining a new function for its dependence on its parents. (based on Definition 2.2) However, Definition 2.3 also describes the possibility of "also do[ing] an intervention that changes any of the values in $PA_j$".  I'm also not sure what it means that you "may hold some or all of $v$ fixed vary $U_j := u$" - if we're varying $U_j$, does that mean the intervention is touching all of the exogenous parents of $V_j$ but maybe only some of the observable parents? But I don't see how the counterfactual definition requires us to vary the parents - if we're intervening at $V_j$, it shouldn't affect its parents, right?  Just how the values of those parents affect $V_j$.  There must be something I'm missing in this definition. (I also don't think it helps that you use capital $V$ for the variable of interest but lowercase $v$ for its parents)

Section 2.4 has some symbology/variables that don't seem well-defined.  I think line 176 is the only place the symbol $\mapsto$ appears (unless I'm missing something).  What does it mean here? Also, $k$ suddenly appears here as an index to $g$ and $x$, and we see that it ranges from $1$ to $p$.  Given how important this section is, a little more clarity in all of these indices would help.

Algorithm 1 seems strange and unnecessary.  First, on a bit of a side note, line 195 says that Algorithm 1 describes "the construction of an ECM", which seems like an odd title.  Up until now, the ECM has been discussed as the causal model provided by the user, and all algorithm 1 seems to be doing is adding edges from every variable in the ECM to the outcome, so I don't really see how that can be called "constructing the ECM."  More importantly, though, Algorithm 1 seems essentially just like a pre-processing step.  You could just as easily tell the user to provide an ECM describing the causal structure among the input variables that has all of them causing outcome. (which would probably be the default assumption anyway, hence why they're being included as predictors) Given how tight space seems to be, unless there's a nuance I'm missing, Algorithm 1 could just be replaced with the sentence "First, we add edges from all variables in the ECM to outcome."

Line 196 introduces the notation of $\hat{f}(P^M)$.  However, this notation was used for the first time on line 191 - introduce this notation before you use it.

In Algorithm 2, the second line is "Get the possible values of $X_S$ and set it to $X$".  I think you mean the opposite (i.e., "Set $X$ to all the possible values of $X_S$), since I'm not otherwise sure what it would mean to set all the possible values of $X_S$ to $X$...

I appreciate the discussion in 2.7 about uncertainty of the ECM structure, but, as with the discussion around Figure 1, I think it stops just short of providing a strong enough example.  If I were uncertain about the structure and produced the plots in Figure 3, what should be my takeaway besides "the shapes are different for each model"?  I did look at the example in Appendix B.2, hoping that the added space would allow for this type of discussion, but the only written takeaway for Figure 8 is "cell shape impacting tumor class is indeed sensitive to our choice about the uncertain edge, particularly for the TDP."  The difference does appear very large for TDP, so what does that mean?  Can I use these results in any way to figure out which MEC member I should use?  Is the takeaway "when reporting these results, I should report values for all three models?"

In line 273, the authors conclude that, since ALE and SHAP appear similar to the PDP, "our TDPs represent a significant and novel contribution to the existing model visualizations."  I don't necessarily disagree that the authors' TDPs are useful, but I just don't see how that follows from the previous statement.

The authors appear to be very pressed for space, relegating a lot of content to the supplementary material.  However, while some of this is reasonable and understandable, there are places where this is either confusing of feels clunky.
- The authors describe their work as a generalization of PDPs.  However, the authors never actually define PDPs in the body of the paper, leaving that for the appendix.
- Some of the discussion on page 7 seems overly dependent on the supplementary material, to the point where it's not particularly clear or useful without it.  This especially stood out in lines 224 and 225, where we're asked to compare Definition 2.8 to a definition in A.2, which shows that it's equivalent to the PDP (which is defined in A.1), the proof of which is in A.2.  Rather than having this content awkwardly split between the main paper and the appendix, I'd rather have this stuff referenced at a higher level with a pointer to the appendix, freeing up some space in the main paper for more detail on other parts that need it.
questions: questions: I think these are mostly covered by the Weaknesses section.
limitations: limitations: The biggest weakness of this approach is the need to specify the ECM structure ahead of time.  To the authors' credit, they are upfront about this and provide multiple ways to ease this challenge, such as using causal structure learning methods (such as in the example in the appendix) or by allowing the user to supply multiple candidate models.


Based on the authors' responses, I am raising my score from a 4 to a 6.
paper_summary: paper_summary: The authors propose a new approach to visualize the impact of a change in a variable on the outcome of interest. They argue that if some of the other variables in the model are mediators, and that they should not be held constant as is currently done in variable importance measures, as it may lead to bias (post-treatment bias) on the measure of the effect on the outcome. They define estimands derived from the field of causal mediation analysis, and show how to use them in graphical representations that are easy to interpret and can also incorporate uncertainty.
strength_weakness: strength_weakness: - this work is not very theoretical, but it introduces a new way to look at a model results for explainability
- however, it is a small fix on the problem of causally interpret machine learning models instead of doing a complete causal analysis. The PDP approach proposed by the authors require the same assumptions as a causal analysis: definition of the estimand of interest, verification of the identifiability assumptions, possibly through the construction of a causal graph, and estimation. From Figure 1, we can see that the results are sensitive to the choice of mode, and no guidance is provided on how to chose a suitable model. Overall the authors propose a fix for users that want to interpret a machine learning causally. However, if the model does not include a suitable set of adjustment variables, this approach will not work. Maybe a clearer message would be to to causal inference when one wants a causal interpretation of the model, instead of tinkering a non-working approach. However, the provided plots are interesting, and should be used in mediation analysis.
questions: questions: - Can you provide clearer guidance for the user in choosing an adequate model?
- and for users to correctly verify the assumptions are met, especially if they are not familiar with causal inference and come from the field of explainable AI
limitations: limitations: the idea is good, but probably hard to use in practice, especially for users that are not familiar with causal inference. It also encourages users to think that they can obtain a valid causal interpretation from an additional step after fitting a machine learning model, instead of insisting that a valid causal conclusion is prepared by all steps of the analysis, including the study design, which is crucial.
paper_summary: paper_summary: The authors present a novel set of attribution methods called Causal Dependence Plots (CDPs) that extend partial dependence plots (PDP) while respecting possible causal dependencies in the method's inputs. The approach aims to obtain truthful and reliable insights for black box model analyses. In detail, a given Explanatory Causal Model (ECM) is utilized to simulate the propagation of possible input intervention effects onto other causally related input variables. The authors claim that, by propagating causal effects for the inputs, a 'natural' configuration (according to the ECM) is presented to the model. This stands in contrast to existing attribution methods, which only vary features independently -- leading to out-of-distribution configurations that do not truthfully reflect the input domain of the model under consideration. The authors differentiate primarily between direct (or 'partial') dependence plots that only evaluates single input variation (corresponding to previous evaluation methods), and the total dependence, which consider changes in causally related inputs accordingly.

Evaluations are performed on multiple setups of synthetic toy examples and a real-world causal protein data set. In both settings, incorporating causal relations yields a better understanding of the system's behavior than non-causal attribution methods.
strength_weakness: strength_weakness: (A) My main concerns are regarding the significance of this work. The presented contributions concerning causal inference seem rather minor, as the paper is mainly about the visualization of already-known results. In my opinion and the context of this conference, possible considerations and/or analysis about learning or causal inference are lacking.


(B) While the authors discuss possible important applications to fairness, out-of-distribution testing, or theory testing, none of the presented examples feature any of these topics.


(C) The authors primarily compare to associational methods. While not touching on the topic of visualization, causal Shapley values [Heskes et al. 2020] exist and should be compared to.


Minor remarks:
* On page 2, NDDPs are mentioned without being referenced or defined before.
* Sec 1. l.46 typo'containsing'



[Heskes et al. 2020], "Causal Shapley Values: Exploiting Causal Knowledge to Explain Individual Predictions of Complex Models"
questions: questions: My questions mainly concern the mentioned weaknesses. I would, therefore, like the authors to comment on the following questions:

(1) Regarding (A): Could the authors think of possible applications of their method with causal analysis beyond pure visualization? When computing CDPs, how could a practitioner or automated system learn to infer treatment effects and detect deviations from the expected causal effects?

(2) Could the authors discuss possible relations of their work to [Heskes et al. 2020]?

(3) The authors mention the particular analysis of "causal descendants" (l.88). However, inference and predictions can also be performed in the anti-causal direction (consider, for example, [Schölkopf et al., 2012]). How does the presented approach translate to this kind of setting?



[Heskes et al. 2020], "Causal Shapley Values: Exploiting Causal Knowledge to Explain Individual Predictions of Complex Models"

[Schölkopf et al., 2012] "On Causal and Anticausal Learning"
limitations: limitations: By proposing a causal explainability method, various questions regarding the applications and risks of such methods arise for real-world decisions being made based on the resulting visualizations. The authors thoroughly discuss possible societal and ethical impacts due to improper use of their method. Implications of providing incorrect causal knowledge and a sensitivity analyses of the method are provided.
paper_summary: paper_summary: The paper introduces causal dependence plots to visualize how (black-box) model predictions are affected by changes in input variable distributions. The explanations are generated based on an underlying explanatory causal model and can be generated for different types of causal quantities such as direct or indirect effects.
strength_weakness: strength_weakness: 1. The authors mention the limitation of a fully specified ECM. As this not only requires complete knowledge of the DAG but also the functional form of the corresponding functions G, this is a strong assumption. I think the paper would benefit, if the authors discuss the case if these functions are estimated instead. This severely reduces applicability and clarity of the approach. 

Minor Comments:
- L. 45: Define $\hat{f}$ before 
- L. 46: typo in “containsing”
- Please define abbreviations such as TDP (l. 68) and NDDP (l. 71) before mentioning them
- Please clarify the legend of Figure 3. Mention which model corresponds to which ECM
questions: questions: 1. The noisy case is not explained very well. Are the exogenous parents estimated based on an estimated model for the functional relationship? 
The demo.ipynb seems to rely on additive form assumptions, which should be highlighted and discussed in the paper.
limitations: limitations: See Weaknesses

Comments: title: Update after discussion
Urgent need to share your opinion on authors' rebuttals
An invitation to share your opinion on rebuttals
The discussion has opened
comment: Thanks to the Reviewers and AC for a responsive and productive discussion. Here are a few updates about our action items:

- We've found another causal SHAP paper (Counterfactual SHAP, Albini 2022) and will add it to our discussion of related literature. Reviewer M8fQ also clarified that we are only requested to include discussion of causal SHAP, and not necessarily experimental comparisons (since we noted it has been difficult to use some of the existing code implementations)
- Following more discussion with Reviewer M8fQ we will add some discussion about diagnostic plots of residuals, and potentially an experiment that illustrates the basic idea (possibly in an appendix)

Each of the proposed changes is manageably small and together they will improve the existing paper without changing its key contributions. 

We're grateful for all the feedback so far, and will listen if there are any more recommendations accompanying the final decisions.
Thank you for replying and adjusting your rating based on the exchange. No worries about the timing, it would only have been difficult if you had additional questions that took substantial time to answer.
I appreciate the authors' detailed response, and i apologize for the late response.  I think with the proposed changes, this paper will be much more understandable and applicable, so I am happy to raise my score.
Taking into account all reviews, I will maintain my score.
I have updated my review, and the rate.
Dear Authors,
thank you for again addressing my concerns regarding Q1. While residual plotting is a delicate matter on its own, --due to mispredictions possibly not becoming apparent at the root cause of the misprediction, but only later on, due to possible cancellation/reinforcing effects--, I would appreciate the demonstration of such an analysis in the paper or appendix.

Given that the claimed additions towards my remarked points will appear in the final version of the paper, I'm willing to increase my score to a weak accept.

Best,
Reviewer M8fQ
Your review was very thorough and we tried to match that with our response. We hope you found the response satisfactory, perhaps enough to merit increasing your rating. But if you still have follow-up questions or any remaining concerns we kindly request you inform us soon so there is enough time to respond. 

Thanks!
Dear Reviewer M8fQ,

There is still a little time left to discuss our reply to your last point about Q1, that is, about including an example showing the use of diagnostic plots of residuals (which could potentially be used to find problems with the predictive model and/or the ECM). Is this along the lines of what you were asking? Would it help address your Q1 sufficiently to merit any more of an increase in your rating? Do you have any clarifying questions or remaining reservations?

Best,
The Authors
Since the discussion period will end soon we wanted to ask one last time if you have any more follow-up questions or comments.

You previously mentioned you were willing to increase your rating to 6, and we hope our last reply may have improved your opinion more. We discussed a similar point with Reviewer M8fQ:

> Our main goal is not to contribute novel methods for causal learning or inference, but to reformulate (visual) model explanations as a causal problem within a clear conceptual and formal framework. As a comparison, consider the SHAP paper, "A Unified Approach to Interpreting Model Predictions" (Lundberg and Lee, NeurIPS 2017). This paper was not trying to contribute novel results or methods for game theory, but to reformulate model explanations as a game theoretic problem. Can such reformulations be significant? Perhaps- the SHAP paper has now been cited over 24k times. We are not conjecturing that CDPs will have a similar number of citations, but work (like ours) which reformulates an important task (model explanation visualizations) as a causal one (and therefore connects it to much of the existing literature on causality) could end up being significant.

Whatever number you are willing to increase your rating to, if you update the form before the discussion ends then we would be able to see the change, and we would be grateful for that.
Thanks for acknowledging our response. The end of the discussion period is approaching but there is enough time for us to answer any additional questions if you ask them soon. We would also appreciate letting us know if you're willing to increase your rating since we addressed your comments.
Dear reviewers, in particular those of you that did not commented on the authors rebuttals. 
Please share your opinion on what the authors wrote in response to your reviews. 
This is an urgent issue for this paper
All the best
We're grateful that you increased your score and replied to our rebuttal.

Your follow-up on Q1 raises an excellent point. It is actually something we thought about as another potential future application of CDPs: **diagnostic plots of residuals**. We didn't mention it in this paper because we were pressed for space, and also because we thought the topic was less well-known.

We don't assume the explanatory dataset is labeled or that the ECM includes the outcome variable. However, if it does, then we can plot the residuals on the vertical axis instead of the predictions. This could be useful for exactly the type of example you mentioned.

We are willing to add a brief discussion about this and possibly include another plot, for example showing the residuals from the random forest and/or linear models from Figure 1. This might need to go in supplemental material depending on the space remaining. Let us know if you think that would be a valuable addition to the current draft- it would not take much additional work, so it would be another minor change we can add to our action items.

Thanks again for your reply and for raising this point.
Dear authors,
thank you for answering my questions.

Q2: I am not sure whether direct comparisons make sense here, as the related papers put different focus on different aspects. I was mainly concerned with the overall lack of discussion on related methods and believe that this point has been cleared.

Q3: I agree with the authors' view on the topic. Thank you for providing further insights and including a discussion in your paper.

Regarding Q1: While I definitely agree that visualization/explanation methods are relevant contributions to the field, I also believe that the paper is still lacking behind in its possibilities. In particular, if one is already in the position of having access to a causal model, it also enables us to attribute or trace-back model misbehavior to particular factors in the SCM (e.g., identifying a particular variable being ignored by the predictor, thus being the cause of its mispredictions). For now, the paper only utilizes causal models to adjust input data, but does not leverage their ability of inferring explanations using the strong computational/structural implications of an SCM.


I have raised my score to borderline accept, but would still like to encourage the authors to think about whether such an analysis could be applied to any of the existing experiments, or being discussed in the context of some simple artificial example.

Best,  
Reviewer M8fQ
Dear reviewers,
I kindly ask to those of you that did not yet share your opinion on rebuttals to do as such to help tha authors to improve the quality of their work.
All the best
Thank you for reading and responding. We're pleased to hear you are willing to increase your score.

If you have any particular references that you recommend for general guidance on causal modeling let us know so we can consider including them.

Two brief comments follow just so we can be sure we're understanding each other.

> [...] the difficult part of causal inference is establishing a reasonable DAG and ECM [...]

We agree with this, but we also think it is not necessarily a weakness about our paper. Our task and contribution is not a method for causal inference but for model interpretation, that is why the paper is submitted under the "**Primary Area**: Interpretability and explainability." For reasons that model interpretation itself is an important task, we'll refer again to the potential Applications listed in Section 1.1 of the paper, e.g. "multi-party auditing."

There are also some reasons we think the ECM requirement is not too strong, e.g. the paragraph on "incomplete causal knowledge" and Section 2.7, which brings us to the next comment.

> [...] it is not (yet) clear to me how CDPs can overcome those limitations.

It's fair that this is not entirely clear yet, at this stage we have only demonstrated some proofs of concept. What we are thinking about here is something like the example in Section 2.7. We think the problems that arise due to the challenges you mention, like whether some mediator/collider is adjusted for or not, might turn out to be less problematic when we look at their impact *on a plot*. In other words, the assumptions for a certain estimator to have a certain desirable property (e.g. double robustness) might fail to hold, but a plot with an accompanying uncertainty region might show the empirical relationship is qualitatively similar (e.g. increasing but concave) across a large range of conditions. If that qualitative visual conclusion is what's important for the given application (e.g. a paper testing some theory which predicts the relationship should be increasing but concave) then we don't need to worry about the other assumptions that were failed.
I thank you for your answers and clarifications, I agree that you are clear about the distinction of the approach you propose and a causal inference approach, however, I think that the difficult part of causal inference is establishing a reasonable DAG and ECM, which is necessary for the CDP, but does not provide a result as strong as a causal result.

Regarding your final remark, CDPs will definitely provide valuable insights, however, some dependencies can be reversed if some relevant variables (confounders or others) are missing from the model, or if other variables (mediators or colliders) are adjusted for, and it is not (yet) clear to me how CDPs can overcome those limitations.

Providing the rephrasing and explanations and references that you are considering to include in your work I am willing to change my grade to 6 (weak accept).
Thank you for addressing my comments.
Dear reviewers,
thank you so much for the precious work you have done so far. Your comments are fundamental to help the authors of the paper to improve the quality of their manuscript. The authors well recognized the great work you have done so far as well as I strongly agree with them.
The authors too seriously your comments and prepared rebuttals which are well structured and meticolous, thus I kindly ask you all to read them carefully and to discuss directly with author whether their answers to your comments are satisfactory or not.
All the best