Fast Estimation of Partial Dependence Functions using Trees
Authors: Jinyang Liu
Tessa Steensgaard
Marvin N. Wright
Niklas Pfister
Munir Hiabu
Venue: ICML 2025 poster (2025)
Keywords: partial dependence
model explanation
functional decomposition
Primary area: social_aspects->accountability_transparency_and_interpretability
TL;DR: We introduce an efficient algorithm for computing partial dependence functions for tree-based model explanations

Abstract: Many existing interpretation methods are based on Partial Dependence (PD) functions that, for a pre-trained machine learning model, capture how a subset of the features affects the predictions by averaging over the remaining features. Notable methods include Shapley additive explanations (SHAP) which computes feature contributions based on a game theoretical interpretation and PD plots (i.e., 1-dim PD functions) that capture average marginal main effects. Recent work has connected these approaches using a functional decomposition and argues that SHAP values can be misleading since they merge main and interaction effects into a single local effect. However, a major advantage of SHAP compared to other PD-based interpretations has been the availability of fast estimation techniques, such as `TreeSHAP`. In this paper, we propose a new tree-based estimator, `FastPD`, which efficiently estimates arbitrary PD functions. We show that `FastPD` consistently estimates the desired population quantity -- in contrast to path-dependent `TreeSHAP` which is inconsistent when features are correlated. For moderately deep trees, `FastPD` improves the complexity of existing methods from quadratic to linear in the number of observations. By estimating PD functions for arbitrary feature subsets, `FastPD` can be used to extract PD-based interpretations such as SHAP, PD plots and higher-order interaction effects.

Fast Estimation of Partial Dependence Functions using Trees
Jinyang Liu
Tessa Steensgaard Marvin Wright 234 Niklas Pfister 15
Munir Hiabu
Abstract
1. Introduction
As machine learning models become increasingly complex and widely deployed in mission-critee critical applications, in- terpretability has become essential for ensuring fairness and transparency (Adadi & Berrada, 2018). One popular model explanation method is Shapley additive explanations (SHAP) (Lundberg et al., 2020) a post-hoc explanation method that has gained traction for its game-theoretic ap- proach of attributing feature importance based on Shapley values. A value function must be specified for the Shapley value. In this paper we refer SHAP as the Shapley values that use the partial dependence (PD) functions the value function. Others (Chen et al., 2020; Taufiq et al., 2023) have termed it interventional Shapley values.
Many existing interpretation methods are based on Partial Dependence (PD) functions that, for a pre-trained machine learning model, capture how a subset of the features affects the predictions by averaging over the remaining features. Notable methods include Shapley additive explanations (SHAP) which computes feature contributions based on game theoretical interpretation and PD plots (i.e., 1-dim PD functions) that capture average marginal main effects. Recent work has connected these approaches using a functional decomposition and argues that SHAP values can be misleading since they merge main and inter- action effects into single local effect. However, major advantage of SHAP compared to other PD-based interpretations has been the availability of fast estimation techniques, such as TreeSHAP. In this paper, we propose a new tree-based esti- mator, FastPD", which efficiently estimates ar- bitrary PD functions. We show that FastPD con- sistently estimates the desired population quan- tity in contrast to path-dependent TreeSHAP which is inconsistent when features are correlated. For moderately deep trees, FastPD improves the complexity of existing methods from quadratic to linear in the number of observations. By esti- mating PD functions for arbitrary feature subsets, PD can be used extract PD-based interpre- tations such SHAP, PD plots and higher-order interaction effects.
PD plots, i.e. one-dimensional PD functions (Friedman, 2001; Hastie et al., 2009; Molnar et al., 2023) quantify the effect of individual features by averaging predictions while holding a target feature fixed values. However, as noted by Hiabu et al. (2023), both PD plots and SHAP do not provide complete model interpretation. SHAP merges main effects and interactions while PD plots ignore interactions thereby limiting insight into feature contributions.
Alternatively, functional decomposition allows for greater insight by clearly separating main effects and higher-order interactions. This idea has previously been investigated in Stone (1994); Hooker (2007); Chastaing et al. (2012); Lengerich et al. (2020); Herren & Hahn (2022) under the functional ANOVA identification constraint and more gener- ally in Bordt & von Luxburg (2023); Fumagalli et al. (2025). In this paper, we consider an identification constraint based on PD functions for which we develop a fast algorithm.
Computational efficiency improvements for SHAP have been proposed in several contexts: model-agnostic methods such as FastSHAP (Jethani et al.. 2022); deep-network adap- tations (Ancona et al., 2019; Wang et al., 2022); and tree- based approaches via TreeSHAP (Lundberg et al., 2020). The popular path-dependent variant of TreeSHAP-used in XGBoost (Chen & Guestrin, 2016) and LightGBM (Ke al., 2017)-is is derived from approximating PD functions (Fried- man, 2001). Recent work has further optimized these algo- rithms (Yang, 2022; Yu et al., 2022) and extended them to efficient SHAP interaction computations (Muschalik et al., 2024).
'Department of Mathematical Sciences, University of Copen- hagen, Denmark FFaculty of Mathematics and Computer Science, Universität Bremen, Germany 'Leibniz Institute for Prevention Re- search and Epidemiology - BIPS, Germnay DDepartment of Public Health, University of Copenhagen, Denmark 'Lakera, Switzerland. Correspondence Jinyang Liu sill@mtth.u.dkk:
Proceedings of the nd International Conference on Machine Learning, Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).
"The implementation is available as R-package on GitHub: https: / /i thub. ccm/ P1 http///aanttdd///mmmm ex

---

Fast Estimation of Partial Dependence Functions Using Trees
1.1. Contribution
S S [d] and x E X
We propose FastPD, a novel and efficient tree-based al- gorithm for consistently estimating arbitrary PD functions. FastPD can be used to obtain well-known PD-based expla- nations such as SHAP and PD plots. In addition it can be used to extract a functional decomposition that fully char- acterizes the target function with little computational cost. Finally, we show that path-dependen TreeSHAP can be an inconsistent estimate of the population SHAP value and demonstrate in simulations study that there be substan- tial differences between the path-dependent estimates and FastPD estimates. The remainder of this work is structured as follows. Section 2 introduces PD functions and their role in functional decomposition. Section 3 discusses estimation methods, computational complexity, and presents FastPD. Section compares FastPD with existing PD-based expla- nation techniques.
Σ
my (xt) = us(#s),
(2)
UCS
where Us x
R are the PD functions defined as
(s(ss) = Erx [m(rs, Xz)]-
The identification constraint (2) leads to the unique solution
ms(xs) = E(1))150ueo vo
(4)
UCS
which is known as the Möbius inverse (Rota, 1964) in com- binatorics and as the Harsanyi dividend (Harsanyi, 1963) in cooperative game theory.
The function vi can be interpreted as the expected value of the function m the coordinates xa for all k E S are kept fixed while the remaining coordinates &o E 5 vary according to the distribution Pxg. This interpretation of PD functions directly carries over to sums of the components in the func- tional decomposition of the form Lvgs my (œv) via (2). Furthermore, the functional decomposition decomposes the function m into additive contributions that together make up the whole function m. To illustrate this, consider the following two-dimensional example.
1.2. Notation
........i k), and for any subset S S [d]. define 5 = (d) S. For I E X Rdd we use the notation 25 E X to represent the coordinates of corresponding to the indices in S. Random variables are denoted by capital letters. Lastly for d-dimensional function m : X + R, with a slight abuse of notation, we will write m( xg), nevertheless with the interpretation that the coordinates are permuted into the right order before applying m.
For all & e N, we let [k]] = {I.....
Example 2.1 (Functional decomposition). Assume that for 21, 22 E IR the PD functions take values
2. Motivation: PD-Based Explanations
v0 5, ti (x) 10, (2 3, (1,2(11, 12.
Consider a real multivariate function m : 12d 2 X - R and a distribution Px on X' with full support. For example, m could be a black-box machine learning model for esti- mating the credit score of customer and Px a distribution describing the customer base (see also Section 2.1). Our goal is to understand how changes to individual coordinates affect the function value. To this end, we consider a func- tional decomposition {ms] S C [d]] of m such that for all
Then from (4) the functional decomposition can be obtained
as
mg 5, m1 (21) =5, 5, m2(32) = -2, m1 (x1, 4,
with the interpretation that fixing the z1-value adds (x = 5 to the baseline prediction up = mo = 5, leading to un v1(x)) = 10. On the other hand, fixing both x1 and 22 and assuming no interaction, we would expect an output of mg + m=())) ) + mz((2) = 5 5
x E X
8, but
d
since the actual expectation is 21,2 .r2) 12, we have interaction effect of m1,2 = 4.
m(x) = mo +
m& (Tx)
k=1
Σ ma((ZK.) + + (x))
Two popular quantities used to capture feature contribution are PD plots and SHAP values. The latter can be defined for all k E [d] and all x E via a game theoretical motivation (Strumbelj & Kononenko, 2010)
ket
= ms(ss).
(1)
SCM4
k(S,k == USu(n (TSU(N))))))) - (x(s),
Unfortunately, without further assumptions such a decom- position is not unique. We will consider an identification strategy due to Hiabu et al. (2023), and assume that for all
d (d-1)
(x(z) == Σ
k)) (5)
|
2

---

Fast Estimation of Partial Dependence Functions Using Trees
Hiabu et al. (2023) showed that a functional decomposition can represent both the PD plots and the SHAP values as
without drawing conclusions about the underlying data- generating process. (ii) When the emphasis is the re- lationship between input features X and response Y. In this scenario, the machine learning model m is used to ap- proximate a target function m" (e.g., the conditional mean m" x + B|y X = x] or the conditional quantile m x + inf E | S i X = x) 2 ax) which is the actual function we wish to explain.
uk (Zx) = mo mxx)))),
(PD plot)
(x(x)) = (xx) + Σ (15() +...
m
(SHAP value)
These expansions illustrate that PD plots capture the main effects in the functional decomposition nate value Th, while SHAP values aggregate main effects and interaction effects of all orders. As a result, neither PD plots nor SHAP values fully capture the behavior of m, leading to potential misinterpretations as illustrated in Example 2.2 - a similar argument is made by Hiabu et al. (2023).
It has been argued (e.g. Chen et al., 2020) that for the case (ii), when m is the conditional expectation function, it can make sense to consider the functions xτ + Ex [Y Xs = xy] = Epx [m" (I., Xg) X = x²1 instead of PD functions. Janzing et al. (2020) argue from a causal perspective that PD functions are generally preferable to this approach and easier to interpret. We tend to agree with their arguments and only consider PD functions as defined in (3).
specific coordi-
Example 2.2 (PD plots and SHAP values do not fully capture m). Consider the function m : TR2 - IR defined for all x E IR by 42) == I1 + 2x (2,, and let Px be a distribution with mean zero. Then, {mo, M2, m2, m
In this paper, we consider both cases m = nn and m m as possible targets. We formally distinguish them by considering PD functions in two settings:
defined for all x E RR2 by
mo px [1 X2], m1( =1 - px [x1
(i) The model PD function
m2(22) = - p px [x1
Ls (xg) Epp [M Xz]l
(6)
m1,2(21 .22 ) = 20,42 + 2x [X
(ii) The ground truth PD function
is the unique functional decomposition satisfying (1) and (2). For the SHAP value Φ1 we get
US (as) = Exx f Xg)]-
In both cases, PD-explanations are applied to a trained ma- chine learning model m. However, if the target is the ground truth PD function (ii), then additional assumptions are re- quired to ensure valid explanations. An important point is that the ground truth PD function is only identified in settings in which px is a product measure regularity conditions are made that avoid unidentiffability due to ex- trapolation (e.g., assuming a parametric model). In contrast, such assumptions are not necessary in case (i), as it can, in fact, be of interest to understand the behaviour of m outside the training support.
(21, = x1 + 01 Ep, X2)
The PD plot mo-+-01 m- only captures the average dependence on 21, which does not provide any insight on the interac- tion between only captures part of m as it downhweights the interaction contribution between Tj and T'2
and 22. Similarly, the SHAP value Φ1
Instead of focusing on PD plots and SHAP values, we there- fore advocate to estimate the full functional decomposition {ms] S C [d} defined via (1) and (2). However, this en- tails estimation of all PD functions {us | S S [d]] Unlike for SHAP values, where many fast estimation techniques are available (c.g., TreeSHAP), no fast algorithms have been proposed to estimate all PD functions. In Section 3 propose such algorithm that efficiently estimates all PD functions which can then be further used to extract PI)- based explanations such as SHAP values and the functional decomposition with no significant extra cost.
Most algorithms for PD-based explanations rely m being a specific machine learning model. This is also the case for FastPD, which supposes that m is a tree-based model. A possible work-around to this is to train a new (surrogate) tree-based model on (X(),, (())))))..))..... (x(*), (X()),(((())) (m())))) and afterwards ap- ply FastPD to this model. However, the additional approx- imation steps then lead to the same potential difficulties as in case (ii).
2.1. What Is The Target m??
It is worth differentiating two use-cases of PD-based expla- nations: (i) When the function m represents a pre-trained blxck-box model m such as a neural network or tree en- semble, and the sole foeus is to understand the model
3. Estimation of PD Functions
The most direct approach to computing bDsed explana- tions is to directly compute the PD functions us, defined
3

---

Fast Estimation of Partial Dependence Functions Using Trees
Table 1. Comparison of the algorithmic complexity to estimate either the SHAP values or PD functions for all features and ne evaluation samples in a single decision tree using no background samples. Here d denotes the total number of features, D the depth of the tree, / the number features the tree splits on and A the operations required for a single model evaluation (for a single tree this O(D)). The method of Friedman (2001) can estimate SHAP values from the PD functions (5).
Method
Complexity (SHAP)
(x)
Details
VanillaPD
m)) 0(242Pn)) n.)
Yes
Slow if d is large, but applicable to all models Only approximates PD functions, with same incon- sistency as T
(Friedman, 2001)
Yes
TreeSHAP-pate
O(D22Pnd) ne) O(D2
No
Fast but inconsistent if features are correlated Estimates are based on I 100 background sam- ples
TreeSHAP-ing
No
(Zern et al., 2023)
0(2 Th + +3 Dn.) +3DD
No
Fast and consistent with any number of background samples
FastPD
O(2 (ne + mo))
Yes
Fast and consistent for any PD-based explanations
in (3), for all S [d] required to compute the desired PD- based explanation. In most practical examples one does not have access to the data generating mechanism Px directly and therefore need to estimate it. For example, if m is a prediction model for fraud detection and Px is the future distribution of customers' features the algorithm will be applied on, one may use (parts) of the current customer base as background data to approximate Px. Here, we consider the case in which we observe background sam- ple Dr,, = (x ...., X (m)) consisting of n. iid samples from Px based on which we want to estimate the PD-based explanations. An obvious estimator in this setting is the em- pirical PD function, defined for all S S [d] and all x E X by
earliest example we are aware of is an algorithm proposed by Friedman (2001) which proposed to compute PD functions by traversing the tree weighting predictions based on the coverage of each node. Assuming that 2 < Ras. where D denotes the depth of the tree, it has a reduced complexity of ((2)2Pne). compared to VanillaPa algorithm of Friedman (2001) is one of the only implemen- tations we found that attempts to estimate the PD functions, however, by construction it only roughly approximates (8). As with TreeSHAP PrressHAP-path (discussed next) this can lead to inconsistent estimates (see Proposition 3.1). More Re- cent approaches have focused on estimating SHAP directly and hence do not provide estimates of the PD functions. The most notable algorithm TreeSHAP (Lundherg et al., 2020), which comes in two variants: Path-dependent Tree- SHAP (TreeSHAP-pathh and interventional SHPP (IreeSHAP-int) We first examine TreeSHAP By exploiting the tree structure, TreeSHAP-path reduces the factor nd in the complexity of VanillalD to 2 D. While this method is computationally efficient, the SHAP values it computes rely on the same approximation of the PD functions as the algorithm proposed by Friedman (2001). This leads to the undesirable property that for two distinct trees that have the exact same predictions, the estimated SHAP values may differ when the features are not indepen- dent. In particular for post-hoc explanations of black box model, this dependence on the internals of the model is con- cerning. Moreover, even in the limit of infinite (background) data the SHAP values estimated by TreeSHAP-path do not necessarily converge to the model SHAP value. A for- mal statement of this inconsistency is provided in the fol- lowing Proposition 3. A proof is given in the Appendix.
The tree-based
us(5))
1 Σ 116
m(rs, (
(8)
In a model-agnostic setting, using (8) to estimate the PD function Vy for a fixed S at a single evaluation point x has a complexity of O(Rm)) where R is the number of operations needed to evaluate m a single point. We usually refrain from evaluating the PD functions all points in the domain X. and instead only evaluate it at set of The evaluation points that depend on the precise application. Consequently, computing (8) for all ne evaluation points and all sets S C [d] results in a complexity of O n.m)), which in many applications is intractable. We call this baseline approach of estimating PD functions Vani 1 IaPD. In Table we compare its complexity to bree- alternatives which we discuss next.
3.1. Tree-Based Methods
If m is decision tree, the complexity of obtaining various PD-based explanations can be substantially reduced. The
Proposition 3.1 (Inconsistency of SreeSHAP-pathy There exists a distribution Px on X, evaluation point x' E X and distinct trees m A and m B such that
See GitHub issue: https://githhb. com/ shap/ shap/issues/344

---

Fast Estimation of Partial Dependence Functions Using Trees
(i) Va E X : (x) m (x)), but di (ns) #
reduces the complexity of computing (gg)) for evalua- tion point compared with Vani aPD. It operates in two main steps: (1) a tree augmentation step that precomputes partial dependence information using the ns background samples in Dn, and (2) an evaluation step that retrieves partial dependence functions for any desired feature subsets on n. evaluation points.
3m (x', Dio))
(ii) 1om 10 (Dn))= - 0% (2') >0 a.s.;
where Dns- -xx()...., X consists of iid samples from Px, 2K
(/(4)) (x') denotes the population SHAP value com-
puted via the model PD function and 0% (a', Dn.) denotes the TreeSHAP- -paattp-ttttt--- h explanation of feature k at x', where Dng is used to compute the coverage probability f..
The naive estimator VanillaB performs step and step 2 for each evaluation point and background sample together, which leads to complexity of O(n,ns) in the number of the samples. The main observation used by Fast is that the two steps can be separated entirely by exploiting the tree structure, thus resulting in a complexity of O(ne + m²). To motivate the FastPD algorithm, we begin by noting that decision-tree in can be expressed as a weighted sum of indicator functions. More concretely, there exists L N, c1....:: CL E R and AL,...,AA AL C Rd such that for all x E X
TreeSHAP-ing int was proposed as method that consis- tently estimates the SHAP values. However, it has a rather high complexity of (D22 nemss) that scales with the prod- uct Rg Recently, Zern et al. (2023) have reduced the complexity by considering all background samples simul- tancously when traversing the tree. yielding an improved complexity of (22 nu + 3D Dn.). The algorithm by Zem et al. (2023) is an improvement on FreeSHAP-ind which however does not estimate PD functions and instead only estimates the differences (S,, k x) in (5). While this ap- proach reduces the complexity of estimating SHAP values does not allow us to efficiently extract estimates of the PD functions.
L m(x) = Σ Egll,(a)) 1=1
Furthermore, the leaves As are rectangles, such that for each E [L], A, by1l [a bj2l x... x [asa, d determines the bounds of leaf node in the tree. For all S S [d] and j E [4] define the bounds of features S on leaf as (S) = IIceslaja bjal. By substituting m with n in (8) and simplifying, we obtain
In the following section, propose FastPD which esti- mates all PD functions at a similar complexity to Zern et al. (2023). From this, the SHAP values and the complete func- tional decomposition {m.5 S S [d]} can be extracted with practically no additional cost, providing a more complete explanation of m.
0s( = Σ oj (1(5 P P E A,(5)),
(9)
3.2. FastPD Algorithm
Dg - Pap
where I denotes the empirical distribution of the background data Drn, that is, P (Xx E A3 no - 1(x S E AJ))). The factor (i) in (9) iden- tifies the leaves in which x would have landed if the splits corresponding to S were ignored during traversal, while the the factor (ii) in (9) represents the proportion of observa- tions for which the features in 5 fall within the bounds of the leaf.
D
Dj
D3 Pgles of
Dg
Dglos 2
23
R1 2
D2 + Dy
D2
(a1)
P1.2 + Di
D1.2 D
(Pa. D:)
Pg Daleg <
Dg De 22 2
Di Di <
D1 Di 2.0
A naive computation of (ii) in (9) loops over all observations and checks whether the feature in S lie within AS)) The idea of FastPD is to separate the computation of partial dependence information on the background samples from evaluating evaluation points into two phases:
81 <00 20
T (.).2)) P. (Dg. D.. Dz. D1.2)
To (21.52
P2 (Dg. Dj Dg. D12))
Figure 1. Augmentation step in FastPD, Augmentation entails calculating sets D which contain observations that would have reached leaf j if splits on features in S were ignored. Starting from the full background sample Dn, and setting Dp +- Dn,, cach subsequent split creates new sets and updates existing ones. Here,
Augmentation (Algorithm 1): For each leaf j [L], we identify the set of features T, encountered along the path to that leaf. For every subset c T,, we save a corresponding list DG that contains the observations that would have reached leaf if splits features in S were ignored. For any sample i follows that i E DD4) if and only if x() 5
Ds[a, < c] = (i Ds x c).
In this section, we introduce F FastPD, which substantially
5

---

Fast Estimation of Partial Dependence Functions Using Trees
AS).. Consequently, factor (ii) in (9) can be computed efficiently as the ratio |
Algorithm 1 FastPD augmentation step. Nodes are in- dexed by j. where I, and ry represent the indices of the left and right child nodes, respectively. The feature used to split at node j is denoted by dj. and lj is the split threshold and Dj the value.
Evaluation (Algorithm 2, Appendix B): Once the tree is augmented, the PD function (5{ at any fixed evaluation point z and set S can be computed quickly. Due to the augmentation step, every subset 5 _ T has corresponding list D saved in P3 on leaf') j. To compute 0g(25) for a point x and features S, we can first intersect S with the tree's s split features to obtain a subset U of features that is actively used by the tree. Afterwards, we traverse the tree to every leaf j in which zy (U)).
input Tree structure = {vv,,r, r, t, d} and dataset Dns output Augmented data T = {T3 J is leaf), P = (P,
j is leaf)
1: function RECURSE G,T,P)
2: if j is leaf then
T₃ T T
4: Pj P
The procedure guarantees that we arrive the leaves j in which the indicator (i) in (9) is equal to one. Once we are at leaf'> j. we can compute the empirical probability (ii) in (9) by counting the samples in D where U,- = U T.. Note
5:
return
6:
end if
7:
for all Ds) E P do
8:
if dg ES then
that Dij exists in P3 since Uj C T3.
9:
P Pyes f- Pyes U ((S. Ds)}
10:
Pno t Pno U {(S)
Lastly, because different subsets S may lead the same U when intersected with the split features, redundant computa- tions are avoided by saving ier (xt) (see lines 3 and 22 in Algorithm 2). For example, if the tree-depth is less than the number of features, then U = 0 may occur often and the computed PD functions can be saved.
11:
else
12:
Pyes + Pyes U U[SS (s|ra, < sty))
13:
Pmo + Pmo U {(S, Ds [Td,
Ds( 2 4))
14:
end if'
15:
end for
16:
Tnew f- T
17:
if dg & then
The consistency of FastPD follows from the consistency of the empirical estimator (8) which it computes exactly. A proof can be found in the Appendix.
18:
Thew t T {dz}
19:
for all (5,
do
20:
P yes < Pyes U (d,).
21:
Pro < Pno U(SU (dz}. Ds))
3.2. 1. COMPLEXITY OF FASTPD
22:
end for
The main reduction in complexity of FastPD stems from separating the computation of the partial dependence infor- mation on Dr, from the evaluation on n, evaluation points, which breaks a product into a sum.
23:
end
24:
RECURSE((, Tnew: Pyes)
25:
RECURSE(T) T Tnew: Pao Pno)
26: end function
27: Initialize for all leaf nodes j: Tj + 0. P3 + 0
To explicitly bound the algorithmic complexity of FastPD we can proceed as follows. Let F be the number of unique features the tree has split. It will always hold that F < d and usually the inequality is strict. We start at the tree root with a list containing all observations and assign that to the set S = 0, these lists are recursively passed to the child nodes. New lists Dsufk] == Ds are created for all S when new feature, k. is encountered on the path. This means that the total number of lists to keep track of will be at most 2 on each node if every split feature was unique. For every node, every list incurs a maximum of ni operations, yielding a very rough bound of O(2D+Fmp) for the worst- case complexity of augmenting the whole tree. Once the tree has been augmented, the complexity of traversing all nodes is O22P), and doing it for all 2 subsets and evaluation points will result in a complexity of 0(2D+rna)) ne).
RECURSE(0, T = 0. P = {(0, Dm..)))
(Chen & Guestrin, 2016) and LightGBM (Ke et al., 2017) even with large number of features, the complexity of computing the PD function for all subsets S is reduced. This is because the PD functions are calculated separately for every tree and then summed together. Hence for trees with moderate depth D, the main complexity does not stem from the number of subsets S C [d] for which vs needs to be estimated, but in the traversal of all background samples for every new point that needs to be explained. If n. nh ne- then both VanillaPD and TreeSHAP- inn will scale proportionally to n² We hence gain a significant speed-up by reusing the computed empirical probabilities at the leaves whenever new points are explained.
Thus, if D and therefore also F, are not too big which usu- ally is the case for gradient boosted trees as in as XGBoost
Proposition 3.2 (Consistency of FastPD). Let m : X + I be a bounded target function and Px a distribution
6

---

Fast Estimation of Partial Dependence Functions Using Trees
on X. Then, for a sequence of iid background samples ..... Px, it holds for all S E [d] that ' (æz) a.s., where ig is the estimate for Us from FastPD applied with background data Dn,- = "il,,,,,((n,,
TreeSHAP-path
FastPD
10-
1im S,n, (xs)
Moreover, if mmm is a uniformly consistent estimate of m trained on Dwy i.e., limng SuPsEx pmmm (x) - m((r]] = 0, a.s. then Aas ($5) us (2s) a.s.
0-
0-
lim ss 7100000
-3-
4. Experiments
-2
JT1
We consider a supervised learning setup where training data Dirain == (() ) are iid sampled from a distribution P with correlated features X For all experi- ments, an XGBoost estimator, m,, was trained on Dirain and subsequently explained using the same training samples as background data.
Type
Model SHAP
Extimate
Figure 2. Comparison between FastPD (for SHAP) and TreeSHAP-paa on simulated data of two covariates with cor- relation 0.3 and ra, = 500 background samples that are also used as evaluation points. For the two methods, the simulation run that achieved the median MSE was selected. We see that the model SHAP is captured well by the PastPD SHAP estimates (equiva- lent to Vani 1 aa)PD) which is not the case for TreeSHAF -paat h.
Specific simulation settings varied depending on the analy- sis. For the inconsistency and MSE analyses (Figures 2, 3 and 5), we used d 2 covariates sampled from bivariate Gaussian distribution with correlation 0.3 and variance of 1. XGBoost hyperparameters (nrounds E (1...., 1000). eta E [0.01, 0.3), max_depth. E (......)) 6)) were tuned via 5-fold cross- validation with 50 random search evalua- tions. For the runtime comparison (Figure 4), we used = covariates sampled from a multivariate Gaussian distribu- tion (details in Appendix) and a single fixed XGBoost model configuration with 20 trees and a maximum depth of D 5. All other hyperparameters were left as default, except for eta, which was drawn uniformly from [0.01,
does not shrink with increasing number of background sam- ples, whereas the MSE of FastPD decreases substantially, demonstrating that it is consistent towards the model SHAP, Lastly, we observe that accurate estimation might require more than 100 background samples (The Python package shap e.g. does not use more than 100 background samples). In the setting of Figure X1 and X2 have a correlation of 0.3 further correlations of and 0.7 are considered in the Ap- pendix. Notably even when the correlation is zero, Fast turns out to be more accurate then TreeSHAP -patt. This is because T TreeSHAP-pate does not leverage all infor- mation available in the background samples whereby at each inner node, it conditions on the path taken by the evaluation point.
All simulations were conducted on a dedicated compute cluster (2 Intel Xeon Gold 6230 @ 2.1 GHz CPUs, 192 GB RAM). Our implementation of the FastPD algorithm is available as an R package on GitHub?.
Inconsistency of TreeSHAP-path. Figure 2 illustrates the SHAP explanations of m for X2 in 500 observations of () E IR x RR22 We observe that FreeSHAP-path inconsistently estimates the model SHAP obtained via the model PD function. In contrast, FastPD, which estimates the PD function based on the same 500 samples used as the background data, is consistent and lies close to the model SHAP. In the setting of Figure 2, X1 and X2 have cor- relation of 0.3 further correlations of 0, 0.1, and 0.7 are considered in the Appendix.
Comparison of Computational Runtime Figure com- pares the runtime of extracting the PD) functions for all S using FastPD with computing the interventional SHAP values as implemented in the SHAP Python package. An XGBoost model was pre-fitted with 20 trees and max- depth of 5 on a fixed dataset of 8 8000 observations. The number of background samples, Tis was selected to be 000, 2 000..... 800. The same samples were used as evaluation points. VanillaPD and TreeSHAP-ing scale quadratically in comparison to Zern et al. (2023) and Fa FastPD, which has a linear complexity in the number of samples. While Zern et al. (2023) and FastPD have a comparable runtime, F FastPD calculates all PD functions from which not only SHAP values but a complete functional decomposition can be derived.
Non-Decreasing MSE of TreeSHAP-path:
Figure 3 depicts the mean squared errors (MSEs) of the different methods when the model SHAP is taken the target. We observe that the MSE of TreeShAP-path
'R implementation of the FastPD algorithm: https:// github wttp///oo//aa////e
7

---

Fast Estimation of Partial Dependence Functions Using Trees
1e.0-
Method
1000.0
Stape: 2,000GG
Method
Fast2D
MSE
1o-02-
Fast30-500
Time [s]
VanillaFD
Paxt70-100
Skeges 1 1,85
TrecSMMP-aa
10.0
1o403
Past70-50
FastPD
DreesSlAP-1nnt
Slope (Fast3t): 0.88
Zern al.
1e-01-
PreeSSiAP-path
al
ProoSiliP-patte
16-05
Stapee 0.8073
500
1000
3000
5000
Fig, Re
Figure Comparsion of Mean Squared Error (MSE) of
TreeSHAР versus FastFD over B 100 simulations (log- scale). Each boxplot summarizes dependent simulation runs. In each run, an XGBoost model n was trained using n observations, and the same training samples were resampled background data to estimate the partial depen- dence (PD) function. For F FastPD, different numbers of back- ground samples
Figure 4. Runtime in seconds (s) on log-log scale between FastPD and other methods function of the number of back- ground samples and evaluation points which are taken to be the same. Mcasurements where an XGBoost model was fitted on the data with 20 trees and depth each. The mean times for no nR 1000 are (217s, 4.86s, 0.181s, 0.161s, 0.0157s) respectively and (13722s, 295s, 1.26s, 1.26s, 0.0899s) for re, ne 8000. The runtime of TreeSHAP-pon - depends only on the number of evaluation points ne.
results B = 100 in-
the mean times over
100 runs
500 and (50, 100, 500, 5000} for n = 5000. SHAP values were esti- mated using the training observations as evaluation points, and the mean squared error (MSE) was computed as v. Ei= (Ø) (x) (2x(()) )), where Φ1 denotes the model SHAP of feature X1.
used: {50, 100, 500) for
perparameters nrounds E {10, , 200}, max.depth E (1,...,,5), 5], eta E [0, 0.5]. co sample bytree [0.5, and subsample [0.5, 1] were tuned via random search with 5-fold tross-validation over 100 random search evaluations. For each dataset, we computed the following measure of variable importance
Obtaining Functional Components The functional com- ponents can be recovered from the estimated PD functions via (4). Figure compares mi (x1) from Example 2.2 with the functional component computed using FastPD and with the method proposed in Friedman (2001). The esti- mate of FastPD lies close to the component riu (x1) one would have obtained via the model PD function, while the path-dependent (Friedman, 2001) suffers in areas outside the center. We also see that the component estimated by FastPD-100 has slope that is slightly off, which high- lights the need to approximate the PD function with more background samples. In the setting of Figure 5, X1 and X₂ have correlation of 0.3 and setting with correlation of 0.7 is given in the Appendix.
Importance(mss)
Σ ms x
(
(10)
Components ms appearing in the top five of any method (FastPD, FastPD-50, FastpD-100, or Friedman (aadd)an-path) were compared in their importance attributions. Taking FastPD as the reference, the path- dependent algorithm exhibited relative differences exceed- ing +300% in 33 of the 96 datasets. Applying FastPD with only 50 background samples (FastPD- 50) reduced this to 18 datasets, and using 100 samples (F (FastPD-100) further lowered to just 9. The full attribution results for each dataset are provided in the Appendix.
5. Real Data Experiments
5.1. Comparison on Benchmark Datasets OpenML-CTR23 and OpenML-CC18
These experiments show that PD estimation accuracy im- proves with larger background samples. Our results suggest that smaller background sample size may often suffice. Ideally, one would leverage the full dataset to compute PD functions; when this is not feasible, a practical strategy is to subsample (e.g. n, = 100) and then checking whether the resulting PD functions closely match those from a larger sample (e.g. ny 200). Such comparison provides a
We have conducted experiments on significant number of curated datasets in both regression and classification settings. We used the OpenML-CTR23 (Fischer et al., 2023) regression datasets and the OpenML-CC18 (Bis- chl et al., 2021) classification datasets. In these experi- ments, we explained the predictions of a XGBoost model via functional decomposition as done Figure 5. The hy-
8

---

Fast Estimation of Partial Dependence Functions Using Trees
FastPD
FX Friodran-path
0.30
Line
0.05
Line
FaatPD
0.00
Hobband
mi
FastPD-100
-0.05
Wife
Priednan-path
-0.30
Model day(a)
-0.15
Ground Truth mi(rr)
50
Age
Figure 6. The component mage.rei identified by FastED and Fr "riedman-pact" for the adult t datasct. We see that FastPD estimates a positive effect for husbands during the prime working age, while F Fr rriedman-paa h estimates zero effect for both hus- bands and wives.
Figure 5. Comparison of estimated functional component my be- tween FastPD and the path-dependent method of (Friedman, 2001) (Fried) serreddmnn-pah). The components are extracted from an XGBoost model trained on 5000 samples from the data- generating distribution. The model component (mi1, red line) was computed by weighting the leaves using the true probabilities, while the ground truth component (mn", green line) was computed analytically similar Example 2.2.
tional expectation m One limitation of Algorithm 2 its space complexity, which increases with the number of lists at each leaf. However, once the tree augmented, storing only the sample counts instead of the full lists can improve memory usage, but this problem also motivates limiting the tree depth. Fortunately, existing gradient boosting algo- rithms such as XGBoost perform well with shallow trees?, as increasing depth is likely to cause the model to overfit.
simple yet effective check on the adequacy of the reduced background set.
5.2. The Adult Dataset
A particularly illustrative additional example is the adult dataset which contains data on whether an individual's income exceeds $50, 000 per year (Becker & Kohavi, 1996). We ran randomized grid search with 5-fold CV to tune XGBoost hyperparameters (max.depta e {3....,7} 7). eta
It is important to distinguish the model PD function from the ground-truth PD function. If the target the ground- truth PD, our recommendation is use FastPD an exploratory visualization tool, be supplemented by semi- parametric or doubly-robust estimators (Kennedy et al., 2016; Chernozhukov et al.. 2018).
(0.01,0.05, 0.1). nrounds E {100, 200, 300), min child weight subsample E (0.6, 0.8, 1.0), col solmseee..yttee e (0.6, 0.8, 1.0)) over 25 trials. We then visualized the pnt-reeaaiiossiii
{1,3,5),
Although F FastPD can be used to recover functional com- ponents of any order, practice higher-order interaction components may contribute negligibly; therefore one may choose to truncate the intitial black-box model the main effects and pairwise interactions to improve interpretability. Additionally one can report the average discrepancy be- tween the truncated model and the initial black-box model.
interaction component (Figure 6) and noticed that FastPD and Friedman-pate offered conflicting interpretations: With FastFD, we notice that at prime working age there is slight positive effect on income and/or slightly
for husbands, while the effect close negative for wives. In contrast, the path-dependent algo- rithm F r Priedman-pate estimates the effect to be zero for both wives and husbands at working age; suggesting that age has the same effect on husband's and wife's 's income this range.
Finally, if the target is the ground-truth PD, key challenge with PD-based explanations mentioned in Section 2.1 is that they can be distorted by extrapolation outside the support of Px. Future work should investigate strategies to limit correct for extrapolation, consider alternative summaries such as average local effects Epx la m(X) (ALE) (Apley & Zhu, 2020), which avoid these issues.
6. Discussion
In this paper, we have proposed FastPD, an algorithm that consistently estimates model PD functions for tree-based models i with linear time complexity in the number of observations. Under additional assumptions, it also consis- tently estimates the ground truth function of the condi-
*xGBoost uses default max_depth of 6, warn- ing that deeper trees may increase memory usage ag- gressively: https:// /ttps:///x/oott.o/ rer.ocsttrraateebbss...oo io/en/ stable/parameter html

---

Fast Estimation of Partial Dependence Functions Using Trees
Acknowledgments
chine Learning Repository, 1996.
DOI:
https//bbi.rr//002444///XXX
Niklas Pfister was supported by a research grant (0069071)
from Novo Nordisk Fonden. Marvin Wright is supported by the German Research Foundation (DFG) under the grants 437611055 1051 and 459360854. Tessa Steensgaard and Munir Hiabu are supported by the project framework InterAct.
Bischl, B., Casalicchio, G., Feurer, M., Gijsbers, P., Hutter, F, Lang, M., Mantovani, R. G., van Rijn, J. N., and Vanschoren, J. Openml benchmarking suites, 2021. URL
https://arxiv.om org/ hhtp://bs///a /ttp//108.0131
Impact Statement
Bordt, S. and von Luxburg, U. From shapley values to generalized additive models and back. In Ruiz, F., Dy, J., and van de Meent, J.-W. (eds.), Proceedings of The 26th International Conference on Artificial Intelligence and Statistics, volume 206 of Proceedings of Machine Learning Research, pp. 709-745. PMLR, 25-27 Apr 2023. URL htw.t: http://poceeen..ee/ee mlr. mlr.press/ v206 2206/bordt23a. html.
This paper contributes to the field of explainable Al (XAT) by providing novel algorithm, FastPD, for estimating Partial Dependence (PD)-based explanations of tree-based models. XAI is critical field focused on making complex machine learning models more transparent, trustworthy, and accountable. Our work addresses a key challenge in XAI: the need for computationally efficient methods that provide consistent and comprchensive explanations. While methods like SHAP are popular, some widely-used implementations can yield inconsistent results, potentially undermining trust. By enabling fast and consistent estimation of PD functions, FastPD allows for more reliable application of PD-based exaplanations, including SHAP and full functional decom- positions. This enhanced reliability and the ability to disen- tangle main effects from interactions contribute directly to core XAI goals: fostering user trust, enabling robust model debugging, facilitating the identification and mitigation of bias for improved faimess, and supporting scientific discov- ery by providing deeper insights into model behavior.
Chastaing, G., Gamboa, F., and Prieur, C. Generalized Hoeffding- decomposition for dependent variables application to sensitivity analysis. Electronic Journal of Statistics, 6:2420 2448, 2012. doi: 10. 10.1214/12- URL https: / httdddiioroogg org...rgg/// 1214 12-11//111111-4
Chen, H., Janizek, D., Lundberg, S., and Lee, S.-L. True to the Model or True the Data? arXiv preprint arXiv:2006. 16234, 2020.
Chen, T. and Guestrin, C. XGBoost: A Scalable Tree Boosting System. In Krishnapuram, B. and Shah, M. (eds.), Proceedings of the 22nd ACM SIGKDD Interna- tional Conference on Knowledge Discovery and Data Mining, KDD '16, pp. 785-794, ACM, 2016. ISBN 978- 4803--40032232-22 URL http: / dooi acm. org/ 29397785
References
Adadi, A. and Berrada, M. Peeking Inside the Black- Box: A Survey on Explainable Artificial Intelligence (XAI). IEEE Access, 6:52138-52160, 2018. doi: ALL10//ACCESS.2111.200
10.
Chernozhukov, V., Chetverikov, D., Demirer, M., Duflo, E., Hansen, C., Newey, W., and Robins, J. Double/debiased machine learning for treatment and structural param- eters. The Econometrics Journal, 21(1):C1-668, 01 2018. ISSN 1368-4221. URL 10. www.1111eect........ 12097. 12097..
Ancona, M., Oztireli, C., and Gross, M. Explaining Deep Neural Networks with a Polynomial Time Algo- rithm for Shapley Value Approximation. In Chaud- huri, K. and Salakhutdinov, R. (eds.), Proceedings of the 36th International Conference on Machine Learn- ing, volume 97 of Proceedings of Machine Learn- ing Research, pp. 272-281. PMLR, 09-15 Jun 2019. httpttts: http:/poceeddiii ngs. mlr. ptwpss/99// anconal 9a. html.
Fischer, S. F., Feurer, L. H. M., and Bischl, B. OpenML- CTR23 - a curated tabular regression benchmark- ing suite. In AutoML Conference 2023 (Workshop), 2023. URL httpt:httt//ppteteee http:////epeeeeeeeeeeeee nwt.//eee/teeee forum? id He-AAA0044494.
Apley, D. W. and Zhu, J. Visualizing the effects of predictor variables in black box supervised learning models. Jour- nal of the Royal Statistical Society Series B: Statistical Methodology, 82(4):1059-1005 06 2020. ISSN 1369- 7412. URL https:://// http:://doooooooooo http:///...111ss...... 12377.
Friedman, J. H. Greedy Function Approximation: A Gra- dient Boosting Machine. The Annals of Statistics, 29 18):1189-1222 2001. ISSN 00905364, 21688966. URL http: /ttp:///wwww. http://wwww.ttt...ss///ss jstor wwt. htrggstaale///a 2699986.
Fumagalli, F., Muschalik, M., Hüllermeier, E., Hammer, B., and Herbinger, J. Unifying feature-based explana- tions with functional ANOVA and cooperative game
Becker, B. and Kohavi,
Adult.
UCI Ma-
10

---

Fast Estimation of Partial Dependence Functions Using Trees
theory. In The 28th International Conference on Arti- ficial Intelligence and Statistics, 2025. URL https: //openreview.nee net/ ferest/oormmiidddd -ttpormtmiddttpttttttt
Kennedy, E. H., Ma, Z., McHugh, M. D., and Small, D. S. Non-parametric methods for doubly robust estimation of continuous treatment effects. Journal of the Royal Statistical Society Series B: Statistical Methodology, 79 (4):1229- 09 2016. ISSN 1369-7412. URL https: / [/doi.orgie 10. hall1/rrsb... 12212.
Harsanyi, C. A Simplified Bargaining Model for the n-Person Cooperative Game. International Economic Review, 4(2):194-220, 1963.
Lengerich, B., Tan, S., Chang, C.-H., Hooker, G., and Caruana, R. Purifying Interaction Effects with the Functional ANOVA: An Efficient Algorithm for Re- covering Identifiable Additive Models. In Chiappa, S. and Calandra, R. (eds.), Proceedings of the Twenty Third International Conference on Artificial Intelligence and Statistics, volume 108 of Proceedings of Machine Learning Research, pp. 2402-2412. PMLR, 26-28 Aug 2020. WtRLhhtts:///pree
Hastie, T., Tibshirani, R., Friedman, J. H., and Friedman, H. The Elements of Statistical Learning: Data Mining, Inference, and Prediction, volume Springer, 2009.
Herren, A. and Hahn, P. R. Statistical aspects of shap: Functional anova for model interpretation. arXiv preprint arXiv:2208.0 09970, 2022.
Hiabu, M., Meyer, J. T., and Wright, M. N. Unifying local and global model explanations by functional de- composition of low dimensional structures. In Ruiz, F., Dy, J., and van de Meent, J.-W. (eds.), Proceed- ings of the 26th International Conference on Artificial Intelligence and Statistics, volume 206 of Proceedings of Machine Learning Research, pp. 7040-7060. PMLR, 2023.
ngs ml press!!
Lundberg, S. M., Erion, G., Chen, H., DeGrave, A., Prutkin, J. M., Nair, B., Katz, R., Himmelfarb, J., Bansal, N., and Lee, S.-I. From local explanations to global under- standing with explainable AI for trees. Nature machine intelligence, 2(1):56-67, 2020.
/ httpproceeddnngg mlr. press/ 2206/hiabu200 html.
Molnar, C., Freiesleben, T., König, G., Herbinger, J., Reisinger, TT, Casalicchio, G., Wright, M. N., and Bischl, B. Relating the Partial Dependence Plot and Permuta- tion Feature Importance the Data Generating Process. In Longo, L. (ed.), World Conference Explainable Artificial Intelligence, pp. 456-479. Springer, 2023.
Hooker, G. Generalized Functional ANOVA Diagnostics for High-Dimensional Functions of Dependent Variables. Journal of Computational and Graphical Staristics, 16(3): 709-732, 2007. ISSN 8600. URL http::/wwwy jstor. org httgg//talllle//ee /7599227..
Janzing. D., Minorics, L., and Blocbaum, P. Feature rel- evance quantification in explainable AI: A causal prob- lem. In Chiappa, S. and Calandra, R. (eds.), Proceed- ings of the 23rd International Conference on Artificial Intelligence and Statistics, volume 108 of Proceedings of Machine Learning Research, pp. 2907-2916. PMLR, 2020. httpss.:: /ttp///ooeediccccoooooocooceoocc ngs. mwr.prees v108 / w http://nannnnnn.....a html..
Muschalik, M., Fumagalli, F., Hammer, B., and Hüllermeier, E. Beyond TreeSHAP: Efficient Computation of Any- Order Shapley Interactions for Tree Ensembles. In Wooldridge, M., Dy, J., and Natarajan, S. (eds.), Pro- ceedings of the 38th AAAI Conference on Artificial Intel- ligence (Technical Track 13), pp. 14388-14580 14396, 2024.
Rota, G.-C. On the foundations of combinatorial theory I.
Theory of Möbius functions. Zeitschrift für Wahrschein- lichkeitstheorie und Verwandte Gebiet, pp. 340-368, 1964.
Jethani, N., Sudarshan, M., Covert, I. C., Lee, S.-I., and Ran- ganath, R. FastSHAP: Real-time shapley value estimation. In International Conference on Learning Representations, 2022. URL https: http:///pppeeeeeeee///te net/ forum? id=zq2G_VTVSS VTV53T.
Stone, C. The Use of Polynomial Splines and Their Tensor Products in Multivariate Function Estimation. The Annals of Statistics, 22(():118 171, 1994. doi: 10. 14/aos/ 1176325361. URL https: ///////oio///////////o/ /ttp://oooooo.../////// / aos /1//111662536111
Ke, G., Meng, Q., Finley, T., Wang, T., Chen, W., Ma, W., Ye, Q., and Liu, T.-Y. LightGBM: A highly efficient gra- dient boosting decision tree. In Guyon, I., Luxburg. U. V., Bengio. S., Wallach, H., Fergus, R., Vishwanathan. S., and Garnett, R. (eds.), Advances in Neural Information Processing Systems, volume 30. Curran Associates, Inc., 2017. URL https::/proceddingg newrips. ce cc/paper he//paper_flles/speeeeeeeeeeeeee B44SENaa0EE434466590000555331 Paper.pdf.
Strumbelj, E. and Kononenko. I. An efficient explanation of individual classifications using game theory. The Journal of Machine Learning Research, 11:1-18, 2010.
Taufiq, M. F., Blöbaum, P., and Minorics, L. Manifold Restricted Interventional Shapley Values. In Ruiz, F,, Dy, J., and van de Meent, J.-W. (eds.), Proceedings

---

Fast Estimation of Partial Dependence Functions Using Trees
of the 26th International Conference on Artificial In- felligence and Statistics, volume 206 of Proceedings of Machine Learning Research, pp. 5079-5106. PMLR, 2023. URL https: pptpceeeenngg mww. prespress/ v206/ 1006//auffgge tawfia23a..aa.aaaaaaa.aaa html.
Wang, G., Chuang, Y.-N., Du, M., Yang, F., Zhou, Q.. Tripathi, P., Cai, X., and Hu, X. Accelerating shap- ley explanation via contributive cooperator selection. In Chaudhuri, K., Jegelka, S., Song, L., Szepesvari, C.. Niu, G., and Sabato, S. (eds.), Proceedings of the 39th International Conference on Machine Learn- ing. volume 162 of Proceedings of Machine Learn- ing Research, pp. 22576-22590. PMLR, 17-23 Jul 2022. http:t / v1 62 wwbgg22b html..
:tttppp:://rroeeee /pppcceeedings..cc/ press//
Yang. Fast TreeSHAP: Accelerating SHAP Value Compu- tation for Trees. arXiv preprini urXiv:2006. 16234, 2022.
Yu, P. Bifet, A., Read, J., and Xu, Linear tree shap. In Koyejo, S., Mohamed, S., Agarwal, A., Belgrave, D., Cho, K., and Oh, A. (eds.), Advances in Neural Information Processing Systems, vol- ume 35, pp. 25818-25828, Curran Associates, Inc., 2022. URL https: http:::://ppcceeddnng./ neurips. cc/paper_fiiee wtt.ff/ls//pppee/22///ii.. 3bathiee7952200d1128888233333 Paper-Conferrece.ppo pdf.
7em, A., Broelemann, K., and Kasneci, G. Interventional SHAP Values and Interaction Values for Piecewise Linear Regression Trees. In Williams, B., Chen, Y., and Neville, (eds.), Proceedings of the 37th AAAI Conference on Artificial Intelligence (Technical Track 9), pp. 11 11164-
11173, 2023.
12

---

Fast Estimation of Partial Dependence Functions Using Trees
A. Additional Details on Simulations
In this section we provide additional details on the numerical experiments shown Figures 2-5 the main text. We first state the two data generating processes (DGPs) we used in the experiments. Afterwards, we will give details on the figures and specify which of the two DGPs has been used in each figure.
1
0.3
DGP 1:
We consider the covariate distribution Px = (0, Σ) with covariance matrix E
and the target
0.3
1
function m IR? I. is defined for all i E TR2
m(x) = + 42 + 2x
We generate independent samples from (X by first sampling X Px and then Y (mm(X).
DGP 2: We consider the covariate distribution Px = (0, Σ) with Σ = 3 17 + J/. where 17 E R denotes the identity matrix and J7 E RR7 denotes the antidiagonal identity matrix (entries of ones going from lower left corner to upper right corner, rest being zero). The target function m I²- + R is defined for all æ E R by
mx) 3 sin(x1) + 2.5 cos((....2) + 1.12ng + sin(x445) + 0.7
We generate independent samples from (x, Y) by first sampling X Px and then Y (m)()), 0.11. In Figure 3, this DGP is modified to have 6 additional covariates that are not used by the target function.
All numerical experiments were conducted using R-4.4.1 or Python-3. 12 on a dedicated cluster with 2 Intel Xeon Gold 6302@2.1 GHz CPUs and 192 GB of memory. We modified the existing R package glex to compute the PD functions using FastPD and the path-dependent algorithm which is due to (Friedman, 2001) but also reproduced Algorithm in (Lundberg et al., 2020). The code of the experiments can be found in the attached Git repository. codefors corffoorsmmaatt inauuaaaiionn ons. Finally, we used PD-10 to emulate the SHAP values that would have been computed by TreeSHAP-int since they are equivalent.
A.1. Estimation Error Comparison Figure 3
For this numerical experiment we generated iid datasets ((x(1, y(.))....,,( (x(m)) y(m))} over B = 100 repetitions with sample sizes n = 500 and n = 5000 from DGP 1. Multiple XGBoost models (m²) were trained in each of the 100 repetitions with 5-fold cross-validation and their out-of-fold mean squared prediction error (MSPE) was computed. We ran the cross-validation with random search and 50 evaluations to tune the hyperparameters: nrounds E .I, 1000) eta E [0.01, 0.3] and max depth (2.3, ........00 6). Following optimization, the best hyperparameter configuration was used to fit XGBoost model on all observations, and the SHAP value Φ1 estimated for all n observations using the different methods. The generated samples were also used as background samples. The SHAP MSEs were then computed as 1 each method.
(X()) 21 fm (X(+))) where 2fth the estimate of the SHAP value & for the target function mi. for
1. ADDITIONAL SIMULATIONS ON FIGURE 3
We have furthermore conducted the same experiment with 8 covariates with pairwise correlation and 0.7, respectively, We notice that TreeSHAF-par - performs noticeably worse in high-correlation scenarios, while PastPD method is robust.
TreeSHAP- path can reliably estimate SHAP values when covariates are uncorrelated. However, its error remains higher than FastPD because it does not leverage all available samples. At each inner node, it conditions on the path taken by the evaluation point, significantly reducing the effective sample size. This effect evident in the Figure for n = 500.
A.2. Inconsistency of TreeSHAP-patal Figure 2
For this numerical experiment, we followed the same procedure as in Section A.1. We selected the repetition where the MSE of FastPD matched its median MSE across all trials for the FastPD plot, and similarly, the repetition where TreesHAP-pal -path had median MSE for its plot.
13

---

Fast Estimation of Partial Dependence Functions Using Trees
0.10000
Method
0.01000
FastP
FaaPID-500
MSE
0.00100
FasPD-100 100
FaaPPD50
TrosSHAP-iat
0.00010
TreeSHIAP
0.00001
500
5000
Figure 7. Reproduced Figure 3 with 8 covariates under the same target function DGP except that the covariates have pairwise correlation of 0.7.
1000000
Method
FasPD
0.00100
FasPPD-500
MSE
FasPDD100
FastPD-50
Tro:SHAP-int
TreeSHAPP pash
0.00001
500
5000
Figure 8. Reproduced Figure 3 with 8 covariates under the same target function as DGP 1, except that the covariates are uncorrelated.
1. ADDITIONAL SIMULATIONS ON FIGURE
We also conducted the same experiment under DGP with pairwise correlations of {0, 0.1, 0.3, 0.7). The results indicate that TreeSHAP-pat:: produces biased estimates of Model SHAP, whereas PastPD does not.
A.3. Inconsistency Friedman-path Figure 5
For this numerical experiment we followed the same procedure as in Section A.I. We selected the single repetition for which the MSE of the FastED mi compoonttttt corresponded to the median MSE across all trials. In order to reproduce the Friedman hatt plot, we used the R package glex which provided the path-dependent functional decomposition that computed using the algorithm Friedman (2001). Furthermore, have modified the package implement FastPD algorithm, which then used to compute the functional decomposition for the FastPD plot.
A.3.1. ADDITIONAL. SIMULATIONS ON FIGURE 5
We also conducted the same experiment under DGP with pairwise correlations 0.3 and 0.7. The median performing run shown in Figure 10. We see that F Pa performs well in estimating the model functional component but by doing so differs (and more so with increasing correlation) from the ground truth functional component.
14

---

Fast Estimation of Partial Dependence Functions Using Trees
TreeSHAP- -path vs FastPD for different correlation values
500 samples Gaussian covariates.
For each plot, the median result of that method with corresponding correlation is shown
TreeSHAP -path cor 0
FastPP corr = 0
TreeSHAP-Path cor = 0.1
FastPD corr = 0.1
21
TreeSHAP -path corr 0.3
FastPD coIr = 0.3
2
21
21
TreeSHAP-Pate corr 0.7
FastFD coir 0.7
10
(R))
451
251
Type
Medel SHAP
Felimate
Figure 9. Reproduced Figure 2 with different pairwise correlations under DGP 2.
15

---

Fast Estimation of Partial Dependence Functions Using Trees
Correlation = 0.3
Correlation = 0.7
2.5
Estimated ma (x)
Estimated my
0.0
2.55
Line
FastDD Pasc20-100--
Ground TTruth
Figure 10. Plot of median FastPD and Friedman path estimated component over 150 runs for correlations 0.3 and 0.7.
Figure 10 shows that in high-correlation settings the machine learning model can struggle to recover the true regression function; consequently, even though FastPD accurately estimates the model component, it does not target the underlying ground truth.
A.4. Runtime Comparison Figure 4
For this numerical experiment, we generated single dataset of size n = 8000 using DGP 2 and fitted an XGBoost model with 20 trees and max-depth 5, with the rest being the default XGBoost parameters. We performed no hyperparameter tuning here only wish to examine the runtime when is varied. Both the model in and the dataset saved, then evaluated the runtime follows
1. For computing the functional components using FastPD: We implemented the algorithm in R (4.4.1) with Repp bindings to compute the all PD functions using FastPD.
2. For computing the SHAP values using TreeSHAP-ints We used Python-3. 3.12 12 and modified the shap package to compute the SHAP explanations for all features using arbitrary many background samples.
3. For computing the SHAP values using Zem et al. (2023): We used Python 3.12 and the plt reeshap package to compute the SHAP explanations for all features using arbitrary many background samples
4. For computing the SHAP values using Vani aPDD: wrote a simple script in Python 3.12 which repeatedly calls the predict function of XGBoost on the synthetic samples created from the background samples and the evaluation points, the predictions are then averaged to obtain the partial dependence functions.
For all E {1000, 2000..... 8000), we took subset D the original dataset of size and used it both as background and evaluation data (i.e., n4 n, k). We then all methods 100 times obtain SHAP values for ne evaluation points using ni background samples.
B. Evaluation Algorithm
"GitHub: https: : //gi thub. wttp:/c/////sscceeeeeeeeee
'GitHuh: https: : //gitttttuut wtt.ccoo/cccccc
16

---

Fast Estimation of Partial Dependence Functions Using Trees
Algorithm 2 PastPD evaluation step calculate (2s)) To be applied after augmenting the as Algorithm 1.
input Query point x, feature set S, tree structure, path features 7, path data P output (xg) PD-function evaluated on zs
1: Un TU,T;)
2: if (xr)) calculated before then
3: return (IV)
4: end if
5: function G()
6: if j is leaf then
7: U3 U UnT
8: Extract D from P3
9:
P+ length D ( mm/mm
10: return Uy P
11: end if
12:
if' dy E U then
13:
if ma I t then
14:
return G((,)
else
return G(r,)
17: end if
18: else
19:
return G(2,) G(r,)
20: end if
21: end function
by (xv) +- G(1)
23: (x(5)) t by (xv)
C. Proofs
C.1. Proof of Proposition 3.1
Proof. We show (i) and (ii) via example, let Px be distribution over X
X2) R² such that
500/2500 0.2
if (0,0),
250/2500
0.1
if (0,0.4),
Px = x)
250/2500 0.1
if (0.7.0),
1500/2500
0.6
if (0.7,0.4) otherwise.
0
Next, assume ny = 2500 observations sampled from from Px. There a non-zero probability that Dn, satisfies
500
if x (0, 0),
2500
250
if (0, 0.4),
Σ 1(x10 x)
250
if x (0.7,0).
1500
if = (0.7, 0.4)
otherwise.
We now consider two decision trees i and m B as depicted in Figure 11. The leaves, going from left to right, are labeled Tn to La and are identical for both trees, implying that they are functionally equivalent, i.e., mA(r)) = m B (x:) for all æ.
17

---

Fast Estimation of Partial Dependence Functions Using Trees
0.5
0.3
< 0.3
< 0.3
< 0.5
< 0.5
L1: 500
L2: 250
L3: 250
L4: 1500
L1: 500
Lx: 250
L2: 250
La: 1500
(a) First Decision Tree m
(b) Second Decision Tree ri
Figure 11. The two trees have the same leaves hence predict the same values, but their explanations differ when obtained via PreeSsAP-pathl The number on each leaf is the number of observations landing in that leaf. The left-branch is followed when the split condition is true.
We first prove (7) by showing that their SHAP values differ when they are computed using the path-dependent algorithm on Dng Indeed, let = (0.1,0.2 be the observation to be explained and VN be the value of leaf N. We follow the left branch if the split condition is satisfied. Assume that Vi 10, V/2 -5, V/3 - and Va 10. In the following denote by ith and w the estimates the path-dependent PD functions (used in both TreesHAP -aaeea-PPPatt-- -path and Friedman-pathh For
the first tree, ug ( using Dn, equals
1
(500
Vi 250 V22 + 250 V/3 1500 V4) 7,
2500
1
(x1) 750
(500-
V1 + 250 1/2) 5,
(x2) 2500
(750
Vi + 1750 V/2) -0.5,
01.2 (3), x2) = V1 10.
For the second tree, 6g (xs), using Dn, equals
agni
(500 V 250 V3 + 250 1/2 1500 VT) 7,
2500
Gm (x1) =
Vi + 1750 1/2) 00., (750
2500
am B (x2)
1
(500 V, + 250 1/)) = 5,
750
1,2 (I1, x2) = V11 = 10.
Finally, the TreeSHAP -pee--aaap path estimates of the SHAP value for feature 2 in both trees are given as
Inn
2 1112 (01,
(22) +
2 U1 (x1)
100 4.25,
(x2) + J. " (II)
21 T2)
02
-1.20 -1.25.
The computed SHAP values not only differ but have opposite signs! We observe that for m feature ø1 has positive attribution, whereas the attribution is negative for B We can also compute the empirical PD functions for both trees as follows
00
(500 V, + 250 1/2 250 V/3 + 1500 V4) 7,
2500
0, (x1) 2500
(750
Vi 1750 V22) 00.5
8y (x(2)) 2500
(750
Vi 1750 V/) -0.5,
(2,, 22)
V1 10.
18

---

Fast Estimation of Partial Dependence Functions Using Trees
And thus, the empirical SHAP estimate is given as
1 2 1,2 (2), (x2) + 01 (01) 1/ 1.5,
which is the same for both n and n. Furthermore, by construction, the empirical SHAP estimate is equal to the population SHAP value, δ1 = o o
We now show (fi). Let # L denote the number of observations that fall in leaf N. The path-dependent approximations of the PD functions for the first tree can be alternativlyy written as
1 (# L1 V1 +
1/2 + # L3 V/2 + #L4 V4), L
T1b
1
(I1)
(#L Vi + L V22,,
#L, + #L2
02 (x2)
4
1 -(##11 + #2) 1/2 + (#L3 + #La) Va),
Y1b
(2) + = V1= = 10.
12
By the strong law of large numbers holds that #L nb " E as n4, - oo, so therefore iig a., 7 and
E L2) we have the following ont (x2) an -0.5. However, since (#L1 # #22 ui e Px
E
#L1
Px(X E L1)
= - = 500/750,
#L + L2
Px(X E (1) + Px(X E (2)
#12
Px(X T22)
0.1/0.3 250/750.
#L1 + #L2
Px E L1) + Px E L2)
Hence "f (x1) 0.6, 5, implying that 2% 0.3, 4.25, which is not the same as the population SHAP, which was 1.5.
C.2. Proof of Proposition 3.2
Proof. First, observe that the PD function ug (xt) = Epxlm((s, Xs)] of m exists with respect to any S C [d] since m is bounded.
For the first part of the statement, we fix S [d]. Since FastPD exactly evaluates the empirical PD function, it holds for all E X that
1 ns
S
(xs)
Σ nb
m x )
Therefore, since m is bounded the strong law of large numbers implies that
a.s. 1. (xs) for ni + 0c.
For the second part of the statement, again fix S C [d] and let min be a uniformly consistent estimate of m trained on Dn.. observations. Then by applying the triangle inequality it readily follows for all x E X that
Men Us (IS) - "'S' (1ss)
miy (xs,, (x.s)(
x( - Epx [m(xs, Xs)
Tbb
1 m 7hh
1 2b
mo (IS, x())
m m x (0))
m(xs, x!)) - Epx (m(2s, Xs)l
+
Tbb
70h
==1
mmg x5)) mm(ss x
+
1
m(Is. X (0)) Epx [m(xs, Xs)l
146
Thh
sup
mxx) |m mz (x) mm))
+
1
m x!()
Ep [m(xs, Xg)] as 0.
Tbb
where the convergence follows from using uniform consistency of may and the consistency of the empirical PD function which follows from the strong law of large numbers as above.
19

---

Fast Estimation of Partial Dependence Functions Using Trees
D. Experiments on Real Data
D.I. Adult Dataset- Figure 6
To reproduce Figure 6, we used the Adult dataset where we first removed all observations with missing values and applied ordinal encoding to every categorical feature. We then performed randomized grid search (25 evaluations) with 5-fold cross-validation to tune the following XGBoost hyperparameters:
max depth (3,4,5,6, 6, 7), nrounds (100, 200, 300),
eta E {0.01, 0.05, 0.1},
min child weight E (1,3,5),
subsample (0.6, 0.8, 1.0),
colsample.bytree E {0.6, 0.8, 1.0).
The best-performing model had the following hyperparameters:
subsample 0.8, min.childdweight 1,
max_depth = 6,
eta 0.05,
colsample.bytee = 0.6,
nrounds = 100
D.2. FastPD vs Path-dependent (Regression)
We summarize the results of the experiments comparing FastPD with path-dependent methods in the regression setting below. We used the OpenML-CTR23 Regression Task Collection, which contains 35 varied regression datasets. On each dataset, we ran 5-fold cross-validation, tuning the following hyperparameters for the XGBoost model over 100 randomly sampled settings:
eta [0, 0.5], nrounds E 200), col colsample.bytrer [0.5, 1],
depth E [1,5],
subsample E [0.5, 1].
The best-performing model was then selected based on the cross-validated MSE, and used by FastPD, FastPD (50), FastPD (100) and the path-dependent method (Friedman -pantn) compute the functional components (see Figure 5). We then applied the auriable-importane measure for each component S as
(mportance(mss)
Σ n
1
ms mss (x
This enables to rank the importance of each component. We present the importances of all components that were ranked in the top five by any method for each dataset. Using Pa as the reference method, the relative differences
Importance((ther) )
Importancc(FasttP)
of each method with respect to FastPD are reported in parentheses. Finally, for each dataset we report in parentheses the number of observations n, the feature dimension p and the maximum depth of the XGBoost model D. Note that FastPD
and the path-dependent method trivially equivalent for those datasets where the tuned XGBoost model had D 1.
20

---

Fast Estimation of Partial Dependence Functions Using Trees
Variablle/Itteraction
FutPD
(Reference)
(50)
(100)
dependent
Noneyball 1n: 1232, 15, D: 21
Intercepe
713.8
6314
715.5
713.8
2.7%
(10.25)
(025)
SLG
35.96
35.91
36.12
37.11
(-0.1%
(+0.45)
(+3.255)
OBP
28.36
28.45
28.2
28.5
(+0.5%
(-0,8%
(+0.5%)
15.94
15.52
(+4.5%
(+1.7%%)
14.05%
RA
11:05
(+0.9%
(++11156)
(-
054R, 1ish toxicity (n: 369. p: 7, D:
Intercept
4.066
4.143
4.125
4.066
(+1.9%% )
(+1.55%)
(026)
MLOGP
0.4468
0.4085
0.4514
0.5736
(-8.6%
(+11%) )
(+29.755)
SM1_De
0.4279
0.4339
0.4323
(+-1.4%)
0,2865
0.2880
(+44.4%)
(+44.6%)
(-6.455)
CICO
0.1907
0.1681
0.2047
0.1703
11..9%
(477335)
10.75))
abalone (n: 4177, 9, B:
Intercepe
9.934
10.08
10.17
9.901
(+1.5%
(+22445)
(088)
shell weight
1,388
1.33
1.368
1.206
(-4.2%
(-2.25)
(-13.15)
sbadkod weight
1.340
1.311
0.8631
(+7.11%
(++45) }
(-31.5%)
stradee weight wiooe. weight
0.8577
0.8526
0.8685
0.4454
(-0.6%
(+0.115)
(- 18.115))
whole weight
0.7164
0.7108
0.6896
0.4368
(-0.8% )
(-2.35%
(-20)
airfoit, roise (n: 1583, B: 6, D::
Intercepe
124.8
121.9
125.1
124.8
(+0.1%
(+0.25%)
(056)
Progancc
3,335
3,380
3.554
(+1.0%
(++6.6%)
1856
deplaccement thicckneerrieeeeeecyyy
(-4.8%
(-8.9%%
15.55))
chord leuguiafyy
1.837
1.953
1.93
1.821
(1..00%
(45.115)
(- 0.996)
ehord length
1.678
1,437
1.998
1.643
(-2.4%
[+11..1%)
(-8%)
puction_verilfone fication In: 2843, p: 8, Dr
Intercepe
7337
75332
8131
7037
(+8.1%
[+13.5%
(080)
process. impocity
5548
4625
(-17.7%
(-1.8%)
(-3%)
property producs.22
(++1.9%
(+1.75%.)
(+24.05))
process.i hy pnpurityyprrpeetyyoooot..
2329
2642
2176
2627
(-13.4%
( (-6.6%)
(+12,895)
property produce..6 pmogertyyprouuee..
2219
2020
1060
1667
(-8.%%
(- 11.7%
24.0%%
houses (n: 18992. p:
Intercepe
5487
5821
4867
5487
(+6.11
[-11.3%
(+4.22%
(-6.4%)
(036)
1320
1364
1282
(+3333%
(-2.0%)
(086)
bathroota
620.8
570.2
185
620.8
(-7.2%
(4..2%
(056)
furniture.uok furuishe
312.1
336.5
314.8
312.1
(+7.8%
(+0.95%
housing (n: 28648, p: 9,
Intercepe
207700
218700
(+0.4%
(+5.88%)
(055)
latitude
90150
(-3.6%
(-3.65)
(- 42.88
latitudelooggtude
82130
83870
$33.30
48730
(++1.6%
(+11.75)
00.756)
21

---

Fast Estimation of Partial Dependence Functions Using Trees
longitude
80700
(+28%)
(+7.2%)
(-30.9%)
medianlucome
42210
40500
43100
48430
3.8%
(42.335
(114775)
cars In: 884, p: 18, D: 51
Intercepe
21330
19870
23:90
21330
(-6.8%
[+10.19%
(053)
Cylinder
2701
2088
3164
2969
0..5%
(+16.8%%
(+6.225)
Cadillae
3285
34
[+21.8%%
(-3.45)
Saab
2253
2231
(-0.9%
(-1.8%))
(+5.995
Milkage
1168
1152
1213
1164
(-1.2% )
(+-495 )
(-0.250)
concreto, compressiiee consrete_compresssive strength (n: 1838, p: 9, 0: 4)
Intercepe
35.82
38.
36.2
35.82
(+6.6%
(+1.15%)
(055)
8.087
8.365
7.96
7.641
(+3.4%
(-1.6%)
(-2.8%
(-0.45%
4.093
4.145
4,104
3.914
(+-1.3%
(-10.35)
[- 4.495)
blast furnace slag
2.702
2,623
2.722
2.129
(-2.0%
(++0.7%6
10.15))
cpstiwagee (h: 20135, p: 7, D: 4)
Intercepe
604
546.8
622
604
(-9.5%
(+-19
(050)
100
128.4
125.6
(-10.9%
(+-5%)
[+2.66))
experience
121.1
123.8
114.5
(+11.5%
(+3.88%)
15)
parttime you
45.72
65.15
43.05
46.77
42..0%
(-5.8%)
(+2.355)
38.99
33.19
39.36
37.16
(-10.3%%
(+6.45))
(+0.550)
cou activity 8192, 22, D: 5)
Intercept.
83.81
94.37
83.07
(-0.2%
(+0.55%
freeswapp
20.6%%
(-7.9%)
(-0.15))
vêt
2.359
2.58
2,325
3.304
(19.4%
( (-.45)
(1 40.15))
solll
1.597
1,808
1.708
1.687
(-13.2%%
(+-755 )
(+5.665)
pst
0.8853
0.9213
0.959
0.8976
(+4.11%
(+8.35%
(+1.450)
33940, P: 28, D: 5)
Intercept
3940
4278
(+0.4%)
(+8.88%
(056)
2400
(-0,3%
(+39% ) (-3.45)
1242
1351
1289
1616
(+8.8%
(+3.88%)
(+30.115)
clarity V822
407.1
418.2
374.4
300.5
(4.2.7%
(-805 }
26.25%)
363.3
303.3
29d
274.3
(-16.0%
[-19.1%)
(-24.5%%
xcy
(+17.8%%
[+-99.0%)
18.156)
caratty
293.8
318.2
308.1
611.9
(+8.3%
(++1.0%)
++110.5%)
VVSI
243.8
410.G
248.8
181
(+2.11%
25.88%)
mergyyefficcengg (n: 768, pr. 9, D: 5)
Intercepe
22.31
21.34
22.67
22.31
(-4.3%
(+1.6%)
(080)
ovesall height
10.05
10.56
8.634
(-0.2%
(+44.9%)
15.35%)
glacing grns
2.044
(+7.2%
(-5.8%)
(-0.35)
ovesal heightsrllttive compactness
1.502
1,585
1.386
1.6
(-0.4%
[-12 (-1..95)
(+0.55)
22

---

Fast Estimation of Partial Dependence Functions Using Trees
relative compactarss
1.491
(-0.1%
22.9%
(+44.0%)
rifa In: 19178, 29, D: 3)
Intercept
9020
15910
9500
9020
76.4%
(43.225)
(026)
overnll
7061
11630
7564
7775
(-1.5%)
[+1.555)
skell ball control
884.3
900.9
1112
899.5
(+.577%%
(+1.735)
cationality name, England
908.5
497.6
855
(-1.8%)
(-5.5%)
attacking heading aceuracy
448-7
377.7
200.3
(++.22%
[-11.5%
(-6.450
skill dribbling
250.7
441.7
412.6
220.7
(-66
[-66.00%]
(-8.450)
defendinng saadling tackle
237.1
710.9
531.9
262.1
(-41.25%
(+10.5%))
forest fires 317, 13, D:
Intercepe
3.536
5,088
4.289
3.636
43..9%
(1221.3%)
temp
2.576
1000 776
(+2.995)
DMC
0.3103
0.3302
0.3246
0.2826
(1.6.4%
(14.66%)
8.9963
Eay.satteemp
0.1736
0.1705
0.1967
0.1701
[-1.8%
[+13.3%%
(-2%)
0.1575
0.1124
0.1394
0.1575
[-12.15 )
(050)
day.snt
0.1403
0.1796
0.1499
(-8.8%
(-16%%
(-2.650
benchwark 24824, D:
Intercepe
113
121.5
(-8.6%
(-1.756)
(056)
GameSerziigg.mxx
15.59
15.16
15.67
15.50
(-2.8%
(+00.5%)
[096)
GameName battlefieldd
9.174
11:48
8:28
9.267
(-21
[-12.85
(-2,2%)
Game Tarke
6.357
5.871
8.087
31%
(-3633%
(-2535
GameNamee Kadinlioe
7,265
(-15.3%)
[- (8.777%
(-2.655
GameName Braaaammegggallllaa grandTheffAutot
6.011
11.18
7.967
26: 2%
[1 1/.33.5)
(- 2.356)
7.022
6.575
7,271
6.611
(-6.4%
(+-3.5%)
(+222.656)
GamaeNamae.e GanaeNanas.aposLeggnil
6.837
10.33
9.649
6.594
89%
[+41.11%
(-3.650
GameeName.desttny22
11.93
6.69
5.311
+117.4%6
[++21.0%)
(-3,25)
peographical www.orrignn.ooo music (n: 1059,
Intercepe
26.11
29.12
26.05
(-2.1%
(+9.115))
(0RC)
V/2
2.66
2,702
2,486
2,521
(+3.8% ) (-6.5%)
(+9.89))
V02
2.016
2,229
2.013
1.627
(-10.6%)
[ (-0.1%)
10.35))
Vd
1.53
1.787
1.48
1.288
(-16.8%
(-4.6%)
15.8%)
1.287
(-7.%%
(-5.8%)
(+5.555)
V91
1.173
1.358
0.9099
1.141
(-15. 5%
[-22 d%
(-2,750
grid stabllity In: 18800, p: 13, D: 5)
Intercepe
0.01074
0.010044
0.01687
0.01574
-30.5%
(+7.25)
(056)
tan2
0.01023
0,01183
0.009302
0.01084
(-15.6%
(-9.15%)
(+1.15)
twrs4
0.01021
0.005866
0.008845
0.01017
113.9%%
[-13.15)
(-0.4%)
0.01014
0.006637
0.0108
0.01014
(-14.6%
(+6.55%)
(05))
tao]
0.01008
0,01042
0.01186
0.01015
(+3.44%
[+17.75%
(+0.7%)
23

---

Fast Estimation of Partial Dependence Functions Using Trees
83
0.009094
0,00422
(+0.0%
(+1.15%) (-2.4%)
102
0.009225
0.00844
0.01086
0.000254
1..9% y
(1177.7%
(+0.355)
gd
0.009084
0.009958
0.009668
0.006025
(+9.6%
(+-495 }
(-0.655)
health Insurance (a: 22272, p: 12, 5)
Intercepe
20.6
24.86
27.17
26.5
(-2.5%
(+46.5%)
yes
8.041
8,252
8.687
(+2.6%
(-2.3%)
(+8%)
experience
3.489
3.612
3.239
3,002
(+1.1%%
(-6.0%%
(- (-10.9%)
kiclsht
2.547
2.504
2.407
2.142
(+1.4%) )
(-2.3%)
(- 16.2%)
bumby
1.637
1.711
1.434
1.636
(+4.5%
[-12.05)
(-0.156)
kinöna (e: 8192, p:
Intercepe
0.714
0.7176
0.7396
0.714
(40.5%
(433.6%)
theta3
0.1331
15.5%%
(45.33%)
0.995)
thetaä
0.05947
0.06424
0.06806
0.06835
(1.85)
( 2.45%
(-195
tbetadrllaeaa
0.04003
0.01000
0.03762
0.0403
[+100%
(-655 )
[+0.755)
thetadrtiaead
0.03869
0.04127
0.03474
0.08879
[+6.9%
(-10%%
(+0.58)
thota6
0.03624
0.02404
0.04615
0.03586
(-31 2%
[+27.35
(-15)
thcta7
0.033544
0.09077
0.04927
0.00534
(-13.2%
(-39%%
(-0.3%)
kings county (n: 23613, 22, D: 51
Intercept
540100
548000
586000
640100
(+1.33% )
(+8.55%
[056)
Int
106400
106800
100600
105700
(-1.5%
(+33% )
(-0.75)
*g living
64680
65630
67860
74220
(+1100%
(++5.15)
(+14.95%)
grnde
59060
70090
(-3.2%
[+:14.7%
(4.24.15)
long
28700
27500
35610
278:00
(-2.8%
(1124.15)
(-3%)
viand housing (n: 13932, p: 16, D: 4)
Intercepe
399900
$36400
410000
309900
[+90.1%)
(+2.55%
(033)
TOT ING. AREA
66800
69330
67920
52580
[+3.99
(+1.75%)
(+23.8%)
OCEAN DIST
39470
422900
36080
48600
(++8.6%
(-8.6%)
(+233.15) 156)
CNTR DIST
490020
42160
397300
(-30.19)
(+11.95%
(+5.45))
SURCNTR DI
35880
37150
36230
34710
(+3.33%
(+-1%) )
(-3.9%%
LONGITUDE
34240
37200
329-00
25080
(18.0%%
(-3.8%%
266.4%)
navell propulsion plant (n: 11934, p: 15,
Intercepe
0.975
0.9752
0.9745
0,975
(0%)
(-0.1%)
by turbine excit preeeure couppeessor outles air temperature
0.01705
0.01608
(+2.3%
(-0.45))
53.155
0.01571
0.01587
0.0157
0.01368
(+15%)
(-0.156)
(-11 11.6%%
turbinc shaft tarque
0.00GU27
0,004503
0.005038
0.003663
(-0.7% )
(+40.25)
1-228.15 155)
gas turbine exhaust gas premoure
0.004985
0.005062
0.005043
0,003119
(+1.5%
(+1.25))
(-37, (37.
gt compesssor outlet, air temperotuuccpp turbine exit pressure 0.003217
0/002212
0.002262
0,004006
(-0.2%
(+11.4%)
(+52.555)
physiochenicll_prse prottein (n: 43738, 10,
Intercept
6.079
7.72
(-9.9%
(-0.65))
(056)
F6
5.889
5,986
5.691
3,506
(+1.6%
(-3.45%
456)
24

---

Fast Estimation of Partial Dependence Functions Using Trees
5.25
2,763
(-1.9)
(-0.1%)
(-47.8%)
F1
5.012
4.725
5,066
1.384
(-3.7%
(41.15)
72.455)
TEF7
3.202
3.031
3,009
1,453
(-3.2%) )
(-695 }
53. (5.7
punadyn32nh 8192, 33, D: 5)
tousttheet5
0.0183
0.01843
0,01848
0,0188
(40.7%
(4-19 >
(038)
tars4
0.01223
0.01181
0.01267
0,01299
(-349%
(+3.0%)
(+0.59))
thctaa
0.0009914
0,0029077
0.002525
0.001023
( (-163%)
(+255.95))
(+3.225)
thitn3
0.00006446000005
0.0005821
0. 0,0064466
(-16%)
(-9.7%)
(05C)
0.00059790.00000
0.0005471
0. 0.0009662
(-13.99%
(-8.5%)
(-0,5%)
tuml
0.00039740.00000
0.0005881
0.0003911
6.0%%
48%
(-166363
Intercepe
-0.000355
0,002155-000000000
0.0000868
(+572%)
58.7%
(056)
red Nina [n: 1599, 12, D: 5)
Intercepe
5.636
5.675
5.635
5.636
(10.7% )
(0%) )
(026)
alcohul
0.2392
0.2238
0.2418
0,2608
(-6.4%
(+1.15)
(+00%
salplates
0.1847
0.2005
0.1901
0.1821
(+8.6%
(+2.95%)
(-1.45)
volatilo_ neddity
0.1125
0.1325
0.1058
0.1349
(-17.9%
(-695 }
(+10.9%)
total sulfar dioudd
0.1319
0.1233
0.1221
0.06211
(-10.220 200.2%%
(+9.1155)
(-17.756)
sarcos (n: 48933, p: 22, D: 5)
V15
14.41
14.24
14.46
10.4
(-1.2%)
(+0.33%)
27.85%
Intercepe
13.66
14.2
16.06
13.66
(+-1%)
(+2.95%)
[050)
VI
4.656
4.730
4.823
3,966
(+118%
(+33.6%)
14.8%)
4.125
(-8.5%
(-2.8%)
(4:3%)
V18
3.735
3,858
3,556
3.261
(+33.3%
(-4.8%)
12.75))
socrab (e: 1156, p: 6, D: 3)
counts for sone first oncuupstin
18.73
15.00
19.92
18.32
(-10.0%
(+46.45)
(-2.255)
Intercepe
18.21
10.91
20.8
18:21
[+13.15%
(050)
counts for sams first cccupationssos nccupation Monager
3.427
2,238
3.326
3.207
(-32 9%%
(-2.9%)
(-6.45)
SecupationMannger
2.876
1.611
3,047
2.704
(-10%)
(+6%) )
(-5.9%)
race white
2,812
2,402
3.054
2,812
(-14.6%
(+88.6%)
[096)
Banily structureeuuccm. nonintact
2.621
2.557
2.712
2.541
(-2.4%
(433553)
[- 2.955
Gumily_ www.cturuuuuuuc www.anacttaaee...oooooooooo white
2:254
2,618
2.401
2.254
(-16.1%%
(+18.3%)
solar Flare 1066, D:
Intercept
0.2313
(4.9%)
23.7%
(033)
Class.E
0.1198
0.1201
0.09706
0.1138
(+5.3%
[-14.7% )
(066)
spot Iistribation.I
0.07133
0,0777
0.05889
0.07133
(+8.0%
(-.44%%
(033)
activity:2
0.06658
0.0545
0.05299
0.05058
(-3.7%
(-6.3%)
(080)
largest spot sioc.K
0.03674
0,04309
0.02733
0.03674
(-18.6%
[-25.8%
(056)
space_ga (n: 3187,
honeeespogg
0.3214
0.318
0.3215
0.1254
(-1.1%
(0%%
(-50. (00.75))
0.3201
0.3200
0.3205
0.1387
(+0.2% ) (+0.15%)
56. 256)
25

---

Fast Estimation of Partial Dependence Functions Using Trees
0,2810
(+5.8%
(-1.4%)
444.5%)
Income
0.08183
0,73806
0.07767
0 040229
4.0% )
( 5.116)
48. 356
ycoord
0.07958
0.08196
0.07673
0,07708
(+33%)
(-3.6%)
(-3.15)
education
0.07769
0,07654
0.08197
0,00851
(45.63%)
Interceps
-0.5702
(+-4%)
(-6.15)
(050)
student_ www.orrorffceeeccc (n: 649,
Intercepe
12.21
11.57
(+2.6%
(-2.8%%
(056)
Billures
0.6817
0.6033
0.7445
0.7618
(+2.1%) )
(+9.22%)
(+11.8%)
MS
0.4585
0,4219
0.0602
0.4582
(-3.1%
(+4.9%%
(+4.8%)
higher yoe
0.2677
0.2675
0.2621
0.2744
(0%)
(-2.1%)
(+22655)
gooect
(-446)
[+-10.8%
(-2.355)
Fedu
0.2105
0.195
0.2354
0.2159
77.4%
[1-11.9%)
(122663))
stadytine
0.2069
0.2611
0.2139
0.2157
(+-3.456
[+44,355]
superconductivity (n: 21263, p: 82, D:
Intercepe
34.41
37.6
37.25
34.41
(+9.3%
(+8.39%)
(050)
range ThermnlCanducctviiy
10.24
9.074
(-17.6%)
(+4.22%)
(+-44.7%)
tange atomie radius
6.994
6.612
(-19.9%%
(+13.39%) ) (+72.065)
wtd Vilemon
5.257
5,338
5.441
4.771
(+1.33% )
(+-3.5%)
(-9.255)
tange aatumic radiuewtid ginean Valence
3.253
3.765
3.064
2.133
(-15.7%%
(-6.45)
(-25.2%)
mwan wwwannaseTnollaaaa TheernolCnndnesiitt
1,693
2,248
-3..0%
(++3.4%%
(+-20.156)
video 68784,
Intercepe
13.5
5.713
(-333 1%
[-12.8%
(036)
height
7.135
8,856
7.088
7.133
19%
(-0.7%)
(026)
voole, h261
6.294
9.054
5.311
5.611
[+-44%
[-15.0%)
10.0%)
rodec.ha6t.. height
5.193
6.577
4.903
4.649
(-5.6%)
10.55%)
Nitrateoo
2.417
2.647
16. 46.
1-1088%
(-2.35))
72000,
D:
Intercepe
3760000
37540000
3740000
3260000
(-0.2%
(-0.5%))
(05))
energe4
23330
25030
23670
23620
(+7.2%
(+1.44%)
(+1.29%)
enesy
23100
22790
22400
22170
(-1.3%
(-2.6%)
(-4%)
emegge
23030
23490
235.00
22100
(4.2%)
(+2.15)
456)
umeggy?
(+1.09%
(+0.7%)
3.655)
emegy112
22730
22310
23030
22170
(-1.8%
(+1.35%)
(-2.550
15
22600
22340
22700
226-00
(-1.4%
(+0.25)
(053)
emsgy10
22640
22440
21820
22570
(-0.9%
(-3.6%)
(-0.35)
enggy14
22020
23320
(+0.5%
(+33.6%)
5-555%)
mmnzy5
23210
(+3.8%
(+69%)
(-1,450
white wine (n: 4898, 12, D: 5)
Intereept
5.878
5,844
5.89
5.878
(-0.6%
(+0.25%
[095)
26

---

Fast Estimation of Partial Dependence Functions Using Trees
abcohul
(+5.3% )
(-3,5%)
(-5%)
density
0.1584
0.1409
0.1628
0.1123
9..0% )
(44.15)
28.256)
resiinal sugae
0.1461
0.1617
0.1613
0.1179
(-10.78)
[+100.9]]
10.35)
volatile aridity
0.1275
0.1271
0.1218
0.1296
(-4.5%6
(+1.665)
sulfor dioxide
(+12%)
[+24.5%
(-0.65)
27

---

Fast Estimation of Partial Dependence Functions Using Trees
D.3. FastPD vs Path-dependent (Classification)
For the classification experiments, we utilized the OpenML-CCI8 Task Collection. The experimental procedure, including 5-fold cross-validation and XGBoost hyperparameter tuning, mirrored that of the regression setting. However, datasets with over 500 features were excluded since they primarily were image-based datasets.
For multiclass problems (where the number of classes K > 2), the default approach of XGBoost involves training K separate binary classification models (fk), each outputting raw score and distinguishing one class from the rest (one-vs-rest). The interpretability methods were then applied to the sum these raw scores, f Ek= fx-
28

---

Fast Estimation of Partial Dependence Functions Using Trees
Dataset
Variable/Inuteraction
FutPD
FutPD
(Reference)
(GB)
(100)
dependent
sestur/fhaessgeentatiforoossssss (n: 9873, 33, D: 5)
X26
0.5959
0.6367
0,616
0.5459
6.688%
(4.44% )
( 8.44%)
X28
0.3427
0.0583
0.5007
0.5737
(+21.35
(+0.7%
(+5.7%)
X6
0.3989
0.6222
0.5049
0.3717
(+28 2%
(-5.0%)
X25
0.371
0.4044
0.324
0.324
(-09%
(- (-12.79
(-12.7%)
X11
0.3238
0.3388
0.3637
0.3915
(+-16%%
(+9.2%
(+20.99%
Inseropi
-3.879
33.606
-4.097
-3.879
( (-796
(45.00%
(0%6
MiceProtein (e: 1086, D: 78, 0: 2)
SODI_N
1.752
1,784
1,702
1.875
(+1.8%
(+2.3%
(+7%)
POAMKIL_N
1.226
1:307
1,303
1.196
(16.0%%
6.6633%
2.4%%
GlaR3
0,9862
(+0.33%
(-0.8%
(-6.9%)
BRAF N
1,057
1,121
1.06
1,077
616.11%
6..0.3% )
( 0.9%%
CaNA_
0,9113
1.015
0,9779
1.001
6+7.6%
(43.00%
(+9.55%)
Inseroept
-25.18
-24.91
24.92
-26.18
(-1.1%
(-19
(0%)
Phishinghibsitts
URL of D
3.328
3.581
3,313
3.187
6+7.6%
(-0.5%
(-4,256
SSLfinol State.1
3,044
3.45
3.026
3,591
(+13.3%%
(-0.6% )
(+31.15))
URI. of 1nchoe.1
2,065
2.351
2.615
2,373
(-11.8%%
(-1.9% )
(-11 --115
URL of O Amtbor. Author.1
1.646
1,358
1.705
1.642
(-17.8%%
(+3.6%
(-0.2%%
Interoppt
0.07477
0.4047
-0.8664
0.07477
(+-441.3%%
1419.25
(0%)
adult 48842, 15,
age
0.7777
0.6176
(+10..5%
(-0.1%
(+5.1%)
capital.ain
0.6253
0.48n4
0.5007
0,0000
29.6%
(- 19.4%
(15.6%%
soniitll status movried
0.6014
0.6082
0.6356
D.6055
(+1.1%%
(+5.7%
(4-0.75)
oducatioo.urm
0.3962
0.3975
0.3969
0.4742
(+0.6%
(+0.9%
(+2055)
Interoept
-2.074
2.672
2.074
(+30.29 2%0
(-24%
(0%)
Melkcntdeta_suthossttt (n: 841, pr 71,
tiee
1.519
1.626
1,519 (096)
(-0.3%
(+0.1%
1.326
1.37
1.325
1.326 (095)
(+3.33%
(-0.1%) )
bee
1.1
1.102
1.11
1.1 (0%)
(10.22%)
(.0.9% )
should
0.8523
0.8532
0.837
0,6523
(+0.1%
(-1.8%
(05)
Inseroept
-8.48
-8.448
(0%)
(+0.6%
(+2%)
analcwnddtta.dht deft (e: 797, 5, Dr 1)
Insercept
0.4323
0.5592
0.3711
0.4323
(+22. 4%%
1-14
(0%)
Ethnie White
0.2147
0.2171
0,2132
0.2147
(41.1%
(-0.7%
(0%)
Ethnie Dark
0.119
0.1117
0.1234
0.119 (0%))
(-6.15
(+3.7%
DMPT. Begin.
0.03935
0.05373
0.03803
0.08885
(+30.5%%
(-3.4%)
(0%)
DMFT.
0.03664
0.03443
0.6418
0.00664
(-6%)
(+14.19%
(05)
balance- scale (n: 625. 5, D: 2)
left distannn
0.9896
1.081
1.005
D.5896
(+9.2%
(+1.6% )
(0%)
29

---

Fast Estimation of Partial Dependence Functions Using Trees
weigts
0.5251
(-12.9%
(+4.7%)
(0%)
right distamoerright weigght.
0.4602
0-4382
0,4807
0.4602
4.8%
6..6.4%
(0%)
left dis.onnceeeiiiiaa distannce
0.441
0.4379
0,4528
0.441 (0%)
(-0.7%
(+2
Inseroept
-2.739
22.46
22,42
- -27700
(-796
(40.1%
(0%)
nark-eari
Incercept
(0%6)
6+5.7%
(+2.8%%
V12
1.267
1,272
1.202
1.287
(+1.2%
(-2%)
(+2.4%)
VS mnlmown
0.5507
0.3857
0.669
0.6091
(-305
(+21.55
(+20.995)
V1I may
0.3244
0.3225
0,3487
0,2573
(-0.6%
(+7.5%
(-20.7%)
V12 VO. unknown
0.2655
0.2475
0.2858
0.308
6.8%
747704%
(1:1933)
0,2515
1+1.1%
(-32.8%
(-34.656)
balinots-authanticatiinn (n: 1372, 8: 5,
V1
5.301
5.392
6,242
5,306
(11.1%
(-1.7% )
( (-0.55%
V2
4.571
4.792
4.696
4.399
643.55%
(+2 79%
(-3.8%)
V3
4.31
4.305
4.218
3.78
(-0.1%%
(-2
(-12.35)
Interorpt
2.143
1.427
3.174
2.143 (095)
(-33 4%%
(+48 19t
V2.V3
0.8987
0.8774
0,8601
0,8824
(-2.4%
(-5.1%)
(-1.8%)
bloed transfusion service center (n: 748, 5, D: 1)
Intrrop
1.907
2.078
1.9
1.907 (055)
(-99%
(-0.4% )
V1
0.6442
0.6461
0.6447
0.6412
(0%
(+0.1%
(0%)
0.4394
0.4455
0.4394
(+1.4%
(+0.6%
(0%)
0.3667
0.3667
(+9.8%
(+6.7%)
(0%)
V3
0.156
0.1586
0.1526
0.136 (0%)
04..4%
(+:1%) )
breast-u (n: 609, p: 16, D: 13
Incercepi
2.177
1.714
1,977
2.177 (0%))
( -1.13%%
(-9
Bare. Naclei
1.122
1.147
1.139
1.122 (0%%)
(42.22%
(+1.5%
Cell Sire Uniformity
0.879
0.9123
0.8081
0,879 (095)
(+3.8%
1..6%
Clamp Tatekness
0.6338
0.5865
0.6206
006338
(-7.5%
(-2.1%
(05)
Cell _haae___auaaee Unikormity
0.6229
0.6393
0.627
D.6229
(+2.00%
(+0.7% )
(0%)
Blond
Blond_CCmmnntiin Clromatiin
0.5479
0.5897
0.5402
0.5479
(17.6%%
( (-14%%
(0%6
car (8: PI 7,
Jug. boot big
1.925
1,703
1.702
1.741
(-11.0%
(-8.0%
(-9.6%)
buying
1.324
(0%6)
23.7%%
(-4.5%%
maics low
1.282
1,389
1.156
1.253
6+8.3%
(-9.8%
(-2.3%)
buying med
1.274
1.551
1.415
1.364
(+21.
(+ 11.1%.9%
(47.15%)
maint med
1.097
1.179
1.15
1.011
6+7.5%
(+1.8% 5.%%%
(-5.1%)
daors.Gmere
1.029
0.9159
(+2.1%
(+ 12.7%
(-1156)
Interoept
-10.95
-10.55
110.96
(-0.9%
(+2.6%
(05)
chum (n: 5938, p: 21, D: 5)
Intereept
3.316
3.108
3.528
3.316 (175)
(-6.3%
(+6.4%
30

---

Fast Estimation of Partial Dependence Functions Using Trees
tocal day.dddy_aaaae charge
0,4808
(+14.8%
(- 18.5%%
(+12%%)
Internstional plon.]
0.3634
0.3857
0.314
0.3782
61.619
13.4%%
(13.2%)
International plan. litotal ind calls
0.2522
0.5301
0,2674
0.2373
(+330 09%
(-69%
(-5.9%)
aumber customer service, colla.4
0.226
0.2262
0.1726
0.2172
(40.1%
23.7%
(-3.9%)
0.1968
(+41.45) 41%
(+9.5%
(-2.4%)
sumber calls.t
0.1109
0.2977
0.1737
0.1203
(+ (+144.9%
(+ (+488.00
(+2.95%)
Clinate: www.l www.lationn..oommm crashes (n: $48, 19, D: 2)
veonse_ earT
1.001
0.8057
1,028
1.002
(-10.7%
(+2.79)
(+0.115)
VeSuIsS 2
0.9863
0.9107
1.023
0.9713
(-5.8%
(+5.99
(+0.55%)
CORVIct 0007
0.7112
0.6609
0.7399
0.7023
7.1%%
(4-47
1.3%%
bulegnal vdel
0,4785
0,4794
( 12.12.00
(+4.22%
(4-0.255)
2iveonss
0.4772
0.465
0,4816
0,4723
2.6%
(.0.99% )
(0%)
Inzercepi
-3.572
-3.907
-3.505
-3.572
(+11.095
(-1.9% )
(0%)
CRC (n: 1473, p: 10, 31
Inseroept
0,2812
0.311
0.2436
0,2812
(+10.69%
(-13.49
(0%)
Number of chikdron over bor
0.1651
0.1626
0.1678
0.1881
(-1.5%
(+1 6%
(+19.995)
Wifes education.d
0.1497
0.1487
0.1324
0.135
(-29)
(- 11.169
(-9.8%)
Wifes 3
0.1128
0.1067
0.1117
0.1061
(-656
(-0.5% )
(-8.4%)
Wifer
0.1016
0.1022
0.1076
0.08718
(+0.6%
(+5.9%
(-14.2%)
connect-4 67537, 43, D: 5)
81.1
0.3457
0.405
0.3675
0.3166
(+17.2%%
(4...4%
(-8.7%)
0,2835
1355
( (-206676
(-6.5%)
62.2
0.1752
0.1971
0.161
0.1005
(4 (1.12.5%
6 8-1%
8.4%)
d1.2
0.1502
0.1682
0.1451
0.1834
(+11255
(-3.4%)
(+22.155)
02.1
0.1377
0.1346
0.1748
0.1006
(-2.3%
(+26
(-26.95)
Interoept
-0.6306
0.7717
0.3144
0.6300
(+-22 4%%
(-50.19
(0%)
credit-spprowe D:
Ab.
1.600
1.63
1.71
(+2.1%
(+0.9%
(+0.55%)
A15
0.489
0.4922
0.5062
0.6344
(+0.7%
(+3.3% )
(+9.3%%
A8
0.338
0.3257
0.3266
0.351
3.6%%
( (-3.7%
(13.8%%
All
0.2694
0.2514
0.2939
0,3002
(-6.7%
(+9.1%
(+11.45)
Inserpept
0.1468
-0.02637
0.1466
( 13.2%
( 118%
(0%)
credit-g (n: 1688, 21, 4)
Insereept
1.708
1-481
1.744
1.208 (0%5)
(-13.3%
(+2 1%
cheeking status mocchlin
0.6726
0L7092
0.67442
0,7091
(+5-4%
(+0.2%%
(45-4%)
duration
0.3661
0.3587
0.3834
0.3751
(-2.6%
(+4.7%
(+2.5%)
credit ansomm
0.3079
0.3182
0.3174
0.2082
(+3.33%
(43.1%
(-3.2%)
creditt iistory other ardtinnnnnngg.eeeeei
0.2568
0.27225
0.2447
(-11.1%
(+6. 19
(-4.7%)
credit ancomtscuration
0.2043
0.251
0.2363
0.1479
(+22.9%%
(+ (+13.2%
(-27.0%)
31

---

Fast Estimation of Partial Dependence Functions Using Trees
cylinder D:
prese, speed
0,4688
(-1.8%
(+2.3%
(+1.4%)
Luk
0.3341
0.3537
0.3564
0.3298
(15.99%
6.66.7%
(- 1.3%%)
ESA Vollage
0.3306
0.5192
0,3274
0.3402
(-344%
(-19E
(+2.9%%)
prese type:woolloc00
0.2944
0.2937
0.3007
0.2755
(4-2.1%
(-6.4%)
solveus, pes
0.2894
0.2735
0.285
(-6.4%
(-5.5%
(-1.5%)
Inseroept
-0.45583
0.0608
0.3207
-045533
(-81.8%
(- 19.19%
(05)
diabetes (n: 768, D: 1)
Interopp
1.378
1.536
1.231
1.578 (055)
(+11.5%%
(-10.7%)
plas
0.7338
0,7202
0,7330
0.7336
(-1.8%
10%
(0%)
70166
0.5288
0.5546
0.5056
0.5288
(14.9%%
( (-4.4%
(0%6)
9ge
0.3541
0:3541
(-0.5%
(-0.6%
(0%)
pedi
0,1807
0.1833
0.1787
0,1807
111.4%
( (-1.1% )
(0%)
dna Lo: 3186, p:: 181, D: 51
A92
0.9427
0.9694
0,9219
1.137
(4-2.8%)
(-2.2%
(+200635)
A64
0.51
0.4448
0.5111
0U6308
(-12.8%%
(+0.2%%
(+20.455
A80
0.488
0.4896
0.5167
0.8322
(+0.1%
(45.9%
(+700.5%)
A93
0.4782
0.4634
0.3601
0.4856
(-2.9%
(-21.9%
(+1.5%)
AB5
0.4725
0.4878
0.4933
0.49488
(+3322%
(+-4.4% )
(4-4.7%)
A94
0.4435
0.4853
0.417
0.453
(-99%
(+0.8%
(+2.11%)
Interoppt
-3.09
-2.01
-3.245
-3 3.00 (0%)
(-6.8%
dresses-sales D:
Inserrept
0.7038
0,7038
(+17.77%
(43388%)
(0%)
V6.Spring
0.2741
0.2689
0.2785
0.2741
9.2%%
(+1.6%% )
(0%)
210.
0.1774
0.1663
0,1841
0,1774
(-6.3%
(4-3.8%)
(0%)
V10.cotton.
0.1542
0.1557
0.1546
0,1542
(-196
(+0.3%
(05)
0.126
0.1367
0.1246
0.125 (095)
6+8.5%
(-1.1%)
electricity 43312, 0:
aswprtee
3.821
3,881
3,923
3.188
(+1.6%
(+2.7% )
(-10.6%)
date
2.297
2.688
2,224
1.796
(+17%% )
(-3.2% )
(- (-2188))
dnterneopriie
1.55
1.407
1.503
1.327
(-9.2%%
(-3%)
11.44%)
zewprioerveppiice
0.3858
0.6725
0.1202
0,3269
(+74.3%%
(+8.9%%
16.3%%
vieprice
0.3618
0,4026
(4-19.6%)
<+ 13.39
(+119995)
Insercept
-0.5675
-1.507
-0.5729
-0.5675
(+176.195
(+19% )
(056)
sucalyptus th: 736, p: 28, D: 21
Vig
0.6912
0.6582
0.6811
0.7452
(-48%%
(-1.5%%
(47.8%)
0.3565
0.5429
0.5752
0.5935
(-2.3%
(+6.8%))
Fmn
0.423
0.4562
(-2.2%
(+4%
(+7.0%%)
Surv
0.3368
0.3786
0.3338
(+12.89%
(+ 18%
(-0.8%)
Interoept
-4.664
4,236
4,205
4.564
(-7.2%
(-5.9% )
(0%)
32

---

Fast Estimation of Partial Dependence Functions Using Trees
first order theoren
proving provinn
6118, 52,
0.2757
0.3417
0:2213
(-13.3%
(+7.4%
(-30.4%)
V23
0.2871
0.2907
0,2853
0:3112
(...3%%
( 0.0%% )
(18.4%)
V38
0.2435
0.2327
0,221
0.1989
(-4.4%)
(-9.2%
(-10.156)
V25
0,2168
0.2039
0.216
0.1875
(-696
(-0.4%
(-1666%)
0.1941
(+24.5%
(-2.5%
(-56%)
V4
0.1925
0.2139
0.2219
0.121
(+11.19%
(+ (+155,5%)
[-37.1%)
VI1
0.1
0.1181
0.2226
0.1219
--25. 7%6
(+40.19%
(-23335)
Insercept
-1.776
-4,022
-1.835
-1.776
(-3.2%
(0%)
In: 583, p: 11,
Interoept
1.609
1.48
1.614
1,600 (095)
(-856
(-9.9%
0,2843
0.2812
0,2843
(-0.6%
(0%%
(0%)
V4
0.2712
0.2474
0,2507
0:2712
8.8%%
( (-5.3% )
(0%)
V6
0.2215
0.2427
0,2218
0.2215
(+9.66%)
(+0.1%% )
(0%)
V3
0,1898
0.1787
0.1847
0,1898
(-5.8%
(-2.7%
(05)
jal 10885, p: 22,
Interoppt
2.17
2.035
2,217
2.17 (076)
(-6.2%%
(+2.2%%
10Blan
0.3761
0.3452
0,3802
0,3056
(-898
(+1.4%
(-18.5%)
loc
0.3712
0.3421
0.3687
0.45-2
(-7.8%
(-3.4% )
(+22.44%)
1OHankktotal Op
0.1968
0.1761
0,201
0.07208
(-11.5%%
(+2 1.%.
[- (-63. (-6..4%)
wa
0.188
0.2046
0.1004
0.1033
(+8.8%
(+1.33%
46.1%%
Oped
0.1595
0.1797
0.1506
0,1035
(+12..7%
(-5.6%)
[- (-35.156)
0.1185
0.1251
0.118
0,1152
(+.5.6%
< 0..4% )
(-2.8%%
jungle_ choss, chesss_pps_sscccce____ raw, endgawwcooppleeee complete (n: 44819, p: 7, D: 5)
white._ pptcee_eeeeaeeee rank
2.148
2.007
2.14
2.18
(+12.195
( (-0.4%
(41.65%)
black piecoO rank
1.811
2,105
1.64
1.835
2%%
(-9
(+1.3%)
bllck_ppioon piec00 ranksbluck picecl_strennth
0.9972
1.041
0.9096
0.9841
(+4.4%%
(-8.8%
(-1.3%%
black pleceO rankwhite piecel) strength
0.7611
0.7064
0.782
0,776
(-7.3%
(+2.7%)
(+25)
bilck_ pioco_pooo___ee strengthewhireeegg plennt) strength
0.7242
0.8054
0.789
0,7168
(+11.2%)
(+8.9%% )
(-1.2%)
Interoept
-1.976
-4.064
-4.663
-4 1.9976
( 10.3%
( 6.33%
(0%)
kcl (n: 2189, p: 22,
Insercept
2,684
2.129
2,509
2,081 (095)
(-9%%
(-2.4%
10Code
0.3139
0.2703
0.3206
-1.4%%
( 7.2%%
(-6.8%)
0.2808
0.2559
0.2707
0.2074
(-8.9%
(-1.8%
1-26.15)
loc
0.2543
0.2248
0.2668
0.2418
(-11.6%)
(+0.6%
(-4.9%)
0.1886
0.149
0.1908
0.2447
(-21%)
(+4.3%
[+20.775 75))
0.1677
0.1581
0.1052
(-5.7%
12.8%
(-34.9%)
kc2 (n: 522, 22,
Interoept
2.356
2.195
2.279
2.336 (096)
(-6.8%
(-3.5% )
0.4001
0.399
0.3871
0.2766
(-0.3%
(-3.2%
(-31.1%)
33

---

Fast Estimation of Partial Dependence Functions Using Trees
0.3854
0.3627
(-1.3%)
(-3%)
(-5.9%)
1ppd
0.3419
0.5641
0.3712
0:3244
(1.6.9%
(48.00% }
(-656)
10Blaa
0.1861
0.1726
0,2096
0.1824
(-7.3%
(+ (+-12.6%
(-255)
unig Op
0,1085
0.1759
0.1678
0.1498
(4-1196
(45..9%
(-5.6%)
kr-vs-kp
bocgeeu
3.097
3.647
(-4.3%
1-1.2%
(-1.45)
wkask.f
3.488
3.916
2.624
3.346
(+12.29%
(+3.9%
(-4.1%)
Himmx.f
3.124
2.326
2,886
3.066
(-25. 5%%
(-7.6% )
(-2.2%)
blodbq,F
1.345
1,837
1.109
1,371
8%%
(+4.9%%
(+2.11%)
Interoept
0.7884
0.8956
0.9633
0.7984
(4:22.2%
67%
(0%6)
latter (e: 20088, 17,
X.ege
4.107
(-8.7%
(-1.8%)
(-10.995)
y.ege
3.441
3.427
2,505
3.13 (-99%)
< 0.4%
<- 14.77%
Negvy
2.07
2,589
2.68
2,408
(-396
(+0.4% )
(-0.8%)
xybar
2.45
2.085
2.143
2.3
(-14.9%
(-12.5%)
(-0.15%)
x2ybr
2,286
2.37
2,258
1.983
(+3.7%
(-
[- 12.8%))
Inseroept
-131.2
-128.6
-22822
131.2
(-298
(-2.3%
(05)
#ffat factors (n: 2609, 217, D: 2)
1.213
1,191
1.187
1.222
(-1.8%
(-2.1% )
(+0077%
0t:181
0.9014
0.052
0.9638
0.9899
(+5.6%%
(+5.8%
(+9.8%)
40137
0.7361
0.8493
0.8563
0.5063
(+15..4%
(+ 16.3%
(+27.2%%
Mt:194
0.6111
0.6215
0.6017
(-1.7%
(-3.1%)
(-5.7%)
at:213
0.6099
0.6938
0.0082
0.6421
(4 13.8%%
949000%
(45.33%
Innercept
-37.82
-38.57
-37.7
-37.82
(4-255
(-0.3%
(0%)
ffeet-fourier (n: 2000, p: 77, D: 51
nt:7
0.9039
0.8697
0.805
1.171
(-1.9%
(-10.9%
(+20.5%%)
A11773
0.8344
0.8686
0.891
1.116
(+3.9%
(+6.8%
(+:33775))
0.8338
0.9625
0,7984
0,7766
(+3.5%
(-12%%
(-0.8%)
A1:76
0.5861
0.4715
0.5696
D.6126
(-19.4%
(-2.8% )
(+-4.7%)
0.54
0.601
0.6122
0.6534
(+11.3%
(+ 13.4%
(121333
Insercept
-26.19
-20.55
-20.44
-26.
(+1.4%%
(+1%%
(0%)
Kaarh-nn 2880, D:
1-469
L.522
(+5.4%%
(-2%)
(43.66%)
a5:3
1.318
1,248
1.301
1.487
(-5.5%
(-1 -11.%%%)
[+12.895)
6154
1.126
1,338
1.064
1.133
(+18.8%)
(-5.5%
(40066%)
nt:7
0.9811
1.109
0.9505
1.126
(+139% )
(-2.50 -.2.5%
(+14.8%)
0.8707
0.7276
1.049
0.9546
(-17%)
(+ 19.7%
(+9.9%)
Interoept
-33.11
-33.07
33.2
-33.11
(+1.7%
(+0.3%
(05)
#foat morpphologgcal....ccc...o.o..... In: 2690, p: 7, D: 31
atc4
3.748
3.794
3,589
3.748 (175)
(+1.2%%
(-1.2%)
34

---

Fast Estimation of Partial Dependence Functions Using Trees
4002
3,308
3,318 (0%)
(-3.2%
(-0.3%)
ato]
2.913
2:857
2.921
2.915 (05))
1..9%
(4..5% )
A556
1.96
1.964
1,967
1.96 (0%)
(+0.2%
(-0.2% )
Inseroopt
-21.72
23.36
22.37
-21772
(47.6%
(+33%
(0%)
*feat-plize 2006,
0t:162
0.4647
0.6221
(-39%
(-11.7%
(+33.99%)
0.4553
0.3895
0.4203
0.507
(-14.5%)
(-3.5%
(+11.4%))
W1157
0.4189
0.4877
0.4943
0.4012
(+16.49%
(-18%)
(-4.2%)
a0:214
0.412
0.4125
0.4054
0.4574
(+0.15%
(-1.8%
(+1186)
At:153
0.3907
0-4174
0.442
0.686
(46.88
13.1%
(1 (1.49.7%)
M1s113
0.3706
0.5165
(+9.7%
(-3 1..1%
(4-39.456)
Inseroept
-31.93
-31.29
-31.89
-31.50
( 296
C 0.11% )
(0%)
#feat-zernike (n: 2088, p: 48, D: 21
A1:29
1.137
1.151
1,141
1.186
1+1.2%%
(+0.4%
(44433%)
RE:33G
1.029
1.11
1.068
1.028
6+7.9%
(+3.8%
(-0.155)
457:45
0.9863
0.9713
0.9926
0.7626
(-1.5%
(40.6%
[-22, 750)
A5:43
0.9G8
1,018
0.9376
0,5098
(+6.3%
1%%
(+
A1/33
0.9426
0.9923
0.9413
0,8801
(+5.33%
(-0.1% )
(-8.6%)
at:19
0.8309
0.8354
0.7189
1.071
(+0.55%
(- 13.5%%
(+28.99%)
Interoept
-26.83
22.25
27.78
96.83
(-2.2%
(0%)
34465,
V6
1.002
0.0554
1.065
1.403
(-4.7%
(+6633%
(4-4056)
V50
0.8547
0.7582
0,8026
0.9458
11.5%%
(+1.00% )
(4 (+10.77%)
V1
0.6198
0.6085
0.6263
0.676
(-1.6%)
(-19% }
(49.115)
V100.3
0.5639
0.5918
0.0077
0,8028
(+-1.9%)
(+7
(+58.35)
0.6574
0.593
0.6077
0.4243
(+6.4%
(-99%)
[-23.95) (93.99)
V1
0.5561
0.5659
0.6561
0.8947
(+1.8%
(-18%
(+00.0%)
Interoept
-3.003
2.635
1.991
- 33.033
(-14%)
(-37%
(0%)
nunera128.6 (e: 96328, 9: 22, D: 2)
Inseroept
0.4794
0.4686
0.4916
0.4794
2.3%%
(+2.5%%
(0%)
ataribute 13
0.03296
0.03171
0.03317
0.03044
(-3.8%
(+0.0%
(-7.9%)
ataributte
0.02981
0.02930
0.02958
(0%
( (-1.5%
(-0.7%)
atcribute 11
0.02838
0.02989
0.02848
0.02714
(+4.66%)
(+0.00 )
(-4.4%)
attaibute 16
0.0274
0.02482
0.02681
D.0282
(-9.1%)
( (-2.22 2.2%%
(42.9%)
aptdigits In: 5820, p: 65, D: 41
1uput27
1.119
1.15
1.181
1.396
(42.8%%
55%
(+24.8%)
Impoti53
0.8117
0.8321
0.7816
0.9487
(+2.3%
(-3.7%
(+16.9%)
37pot37
0.7809
0.8531
(-8%
(+9.25
(+355.5%)
Impot54
0.7408
0.8106
0.7068
0.8397
(+9.4%
(-1.6%
(+13.45%
35

---

Fast Estimation of Partial Dependence Functions Using Trees
Inserrept
-35.07
(-1.7%
(-0.8%
(0%)
ozone- Bor [n: 2534, 73, D: 4)
Inseroept
4.69
4.52
4.385
4.60 (055)
< 3.6%%
< 6.5%
V11
0.4146
0.0809
0,4333
0,3055
(+1695
(+4.5% )
(-26.350
V56
0,3559
0.3246
0.3718
0.351
(44.5%%
(-1.4%)
0.2362
0,1838
(+11.390
(45.5%
[-22 255)
V13
0.149
0.1463
0.1651
0.1626
(-1.8%
(+ (+110.8%
(+9.11%)
V66
0.1438
0.1736
0.1455
0.1416
(+20. 79%
(+1.2%%
(-1.5%)
Pel (n: 1189, p: 22, 31
Insercept
3,846
3,877
3.84
3,846 (0%)
(+0.8%
(-0.2%
OBBank
0.544
0.5678
0.5165
0.6763
(14.4%
( (-6.1%
(45.8%)
1OBankk00Coumeet
0,2128
(-1.3%
(+0.66%
(-36.955)
I
0.3226
0.3427
0,3041
0.3716
(1.6.2%
C 5.7%
(+17777%)
unig Opnd
0.1911
0.1854
0.19
0.1207
(-395
(-0.6%
(-36.856)
LoeCodeAndConnne
0.1888
0.2501
0.17661
0.1863
(+32.55%
(-6.7%
(-1.9%)
po3 p::
Interoept
3.426
3.613
3.425 (095)
(+9.22%
(+2.55%
LOC. BLANK
0.6884
0.6975
0.6662
0U6823
(+-1.4%
(-2%
(-5.4%)
HALSTRAD_COOEE
0.4008
0.4438
0.3974
0.3645
(+10.7%%
(-0.8% )
(-9.1%%
JUM_UNNQUEECC UNIQUELOPERANDS OPERANDS
0.1638
0.1682
0.1578
0.2197
(+2.77%
(-3.8%
(+31.1%)
HALSTEAD, LEVEL
LEVEL
0.1263
0.1239
0.09466
1.%%%%
6.99%
29.9%%
LOC_ LOC_C COMMENT
0.1227
(+-17.4%
(44.8%%
(-7.6%%
NUMBER OF LINES
0.1213
0.08314
0.1474
0.1031
( 23.2%
(- 16563
pod Lo: 1458, p:: 38, D: 31
Inzercepi
4.275
3.854
4.38
4.275 (0%))
(- (-14.5%
(+
LOC. CODE AND COMMENT
1.507
1.59
1.471
1.64
(+5.55%
(-2
(+8.8%)
CYCLOMATIC DENSITY
0.3314
0.3703
0.3038
0.3219
(+11.7%
-..5%%
(-2.9%)
LOC. BLANK
0.3061
0.3236
0.3011
0.2824
(+5.7%
(-1.6%
(-7.7%)
CONDITION COUNT
0.2766
0.2901
0.2684
0,2782
(+4.9%
( 6-6.0%
(+-0.0%)
pandigits In: 18992, p: 17. D: 5)
16pot16
1.334
1.501
1.309
1.176
(+19.33%
( (.19% )
(1.8%)
Iuput14
1.067
1.08
1.077
1.1
(+1.2%
(+0.9%%
(+-3.1%)
Imput2
1.06
(+10
( (-2.6%
(4-29.75)
10
1.009
1.028
1.004
0.9265
(+1.7%
(-0.5%
1-8.2%)
Impat7
0.7728
0.8284
0.9029
1.026
(47.3%
(+ 16.0%
(+332885)
Inseroept
-36.75
-37.21
336.0
-30.75
6+1.3%
(-0.3%
(0%)
phonese 5484, p: 5)
Interoept
2.866
2,834
2,896 (075)
(-13.9%
(-1.1%%
1.115
1.271
(-14%) )
(-1 1.8%%
(+145)
V1
0.9520
1.166
0.8947
0.994
(+22.4%%
6-6.1%
(+4.33%)
36

---

Fast Estimation of Partial Dependence Functions Using Trees
0.789
(-30. 4%
(-12.7%
(+10.25)
V2
0.7639
0.062
0,8119
0.6761
< 13.35%
6.66.3% )
11.6%)
V1:V3
0.7302
0.7381
0.0831
0.6196
(+1.1%
(-6.5%
(-15.156)
qsar- baar-biddeg 1055, 42, D: 4)
Inseroopt
1.872
2.18
1.829
1,872 (0%%)
(+16.66
(-2.5%
0.6763
0,5709
0,7878
(-15%
(+0.5%
(+366,95)
V38
0.6006
0.4735
0.4765
0.493
(-5.4%
(-1.8% )
(-1.5%)
V1
0.2923
0.304
0.3126
0.3063
(-19%
(+6.99% )
(+4.88%)
V22
0.247
0.2705
0.2252
0.2285
(+11.1%%
(-8.8%
(-7.5%%
V27
0.2454
0.253
0.9646
0.2768
(+.3.1%
(+-12.4%)
satleage 6438,
Flaatar
1,031
1,022
0:3268
(+112.6%
(-0.9%%
[- 1.66)156
DiGater
0.6645
0.7351
0.6434
0,6967
(+-10.6%
( 3.3.2% )
16.2%%
C15attr
0.6832
0.5949
0.6421
0.3713
(- 10.35%
(-3
(-4450)
Polnttr
0.6431
0.6393
0.0657
0.4263
(-0.6%
(+3.55
[-33.
Rilattr
0.604
0.5513
0.4987
0.5466
(-8.7%
(- 396
[+56..75 750)
Böatte
0.6692
0.5816
0.6756
0.6131
(+2.2%%
(+1.1%%
(-9.9%)
A7azz
0.5248
0.5368
0.4983
0.5058
(+2.33%
(-6% )
(-9%%
Inseroepi
-13.01
-13.86
-18.5
-13.61
(+1.8%
(-0.7%
(0%)
segnent (n: 2310, 17, 3)
Imn mean
1.577
1.472
1.673
1.686
(-6,7%
( (-0.3%
(46.99%)
Interasity anesa
1,038
L.174
(+-2055
(+4.11%)
(+13.115)
sawgreen
0.9083
1,009
0.9038
0.089
(+11..1%
(42.88% )
24.15)
cowbluue mnscm
0.7008
0.5916
0.6722
0.6068
(-15.6%
(-4.1%
[- 13344456)
sawred.mcom
0.6926
0.7473
0.7109
0.3800
(+8.1%%
(+3.7%
(-11.15)
exgreen mosn
0.6305
0.7868
0.5637
0.5051
(+24.8%%
10.69
(-19.990
Insereept
-16.76
-16.55
16.73
16.77
(-0.7%
(-0.2%
(05)
sencion (n: 1593, 257, D: 3)
V16
0.5414
0.5967
0.5165
0.4268
(+10.2%%
(-4.8% )
(-21.4%)
V229
0.4821
0.5406
0.4654
0.5530
(4-1455
( (-3.5%)
(+14.8%)
V77
0.4651
0.5043
0.4314
0.5012
(+8.4%
(-7.2%
(+9.44%)
0.4304
0.3961
0.6107
(4122.1%
(-8.2%
(4-41.955)
V15
0.4267
0.0812
0353
0,3625
(+12.89%
(+29% )
(-15%)
V177
0.417
0.4086
0.425
0.4731
(-255
(+
(+13.53))
Insercept
-30.14
22.002
-00.4
-30.14
(-0.7%
(+0.3%
(0%)
sparbase 4681, p: 38, 4)
Interoept
1.827
1.727
1.397
1.827 (075)
(-5.3%
(-23.5%
word. feop_pporge
0.9277
0.8711
0.9642
1.023
(-6.1%
(+3.9%
(+10.3%)
word. froo hp
0.8724
0.8814
0.8138
0.9647
(-1%%
(-6.7%
(+9.4%)
37

---

Fast Estimation of Partial Dependence Functions Using Trees
(-0.1%)
(+6.7%
(+77.29%)
capiial_run wew___/eeeslllllnneee longet
0.4376
0.4276
0.426
0,4318
2.3%
< 2.7% )
( (-1.3%)
captaa run length total
0.4237
0.4077
0,4321
0,4258
(-3.8%
(-29%) }
(+0.65%)
chor froq .24
0.372
0.3631
0.3414
0.6447
2.4%
( (-8.2%
(473.355
61,
ataribute
0.7944
0.8258
0.7655
(-2.9%
(-49%
(-3.6%)
attribute 31.
0.7998
0.7031
0.7679
0.7462
(-7.5%
(-0.3%
(-1.9%)
attribute 30.C
0.7277
0.6369
0.708
0.722
1-11..5
(+9.77 79.
(-0.8%)
ataribute T
0.6851
0.811
0.6216
0.6703
(+21.99%
(-6.5%
(++0.8%)
attribate 32.0
0.6008
0.6695
0.4642
0.6048
(+9.0%
23.9%
0.0.%
ataribute
0,6011
0.6066
( (-1899%
(+6.9%%
(42.66%)
Inseroept
-3.997
-4.027
-3.588
-3.007
< 1255
( 0.3%% )
(0%)
stell-platss-futt (n: 1941, p: 28, D: St
V14
0.8236
0.810
0.7835
1.881
(-0.9%
(-4.9%
[+-43.45) 456)
V25
0.8116
0.6918
0.9687
0.8836
(-14.8%%
49%
(+8.9%%
0.7111
0.8872
0.7183
0.7845
(+24.8%%
(+1%
(+10.95))
V12
0.6505
0.6896
0.7197
0.7353
(+5.9%
(+ 10.6%
(+135))
Inserorpt
-13.17
-13.88
13.3
-13.57
(+2.3%%
(-2%
(0%)
texture (n: 5088, p: 41, 0: 31
V23
2.455
2.464
2,369
2,505
(-0.4%
(-2.7%
(+25))
V10
2,006
2.037
1.87
1,206
(+1.9%%
( (-6.8%
(- (-39.9%)
1.91
1,209
(+4.99%
(-0.9%%
(-36.7%)
V30
1.63
1,515
1.658
L.083
( (-1.%%
( (-.4%% )
( 32.66%)
V6
0.8912
0.8164
0,8066
1.208
(-84%%
(-9.5%
(+-35.53))
Inseroept
-39.86
-1.00
40.01
-39.80
6+-4.6%
(+0.4%
(0%)
tac toe 958, 19. D: 3)
middlle.midddl... aquire.
2.332
2,278
2,198
2,122
(-2.3%
(-5.7%
(-13.756)
madddle.mdddle..o wwwddaddddddd.... sqquarexx.
1.882
1,981
1,764
1.544
(+5.3%
(-6.3%%
(- (-17.1%)
tog left. aquure.a
1.71
1,696
1.635
1.501
(-0.8%
(- 10.2%
(- (-12.2%)
bottom right aquare.o
1.705
1.827
1.872
1.438
(47.3%
(+9.9%% )
11.6%%)
top sop.tighissquuee..
1.506
1.6
1.643
1.349
(+6.2%%
(+9.1%
[- 1-1.4%)
Insercept
-1.818
-1.818
(4 (4-66.2%
26.22%
(0%)
vehicle (n: 846, 19, 2)
ELONGATEDNESS
1.544
1.088
1.596
L.326
(-4.9%
(+33 4%
(-16.15)
DISTANCE, DISTANC_CCIRCUAARIT CIRCULARITY
1.1G8
1.185
1.209
0.9147
6+1.5%
(+3.5%%
(-21.75)
MAN RECTANGULARITY
1.02
1.063
0.9021
1.032
(+4.22%
(-8
(+1.2%%
KURTOSIS, ABOUT MINOR
0.6494
0.7477
0.7149
0.3895
(+15.1%
(+ 10.19
(- 405)
SKEWNESS ABOUT MAJOR
ABOUT_MAJOR
0.6262
0.5876
0.6585
0.6867
(-6.2%%
(+5 2.5.%%%)
(+9.7%)
Interoept
-5.312
5.066
- 552206
5.332
(-4.0%%
(-0.3%
(0%)
38

---

Fast Estimation of Partial Dependence Functions Using Trees
vowel
(-2%
(+4.1%%
(-9.2%)
Retare__4 4
2.426
2:455
2,400
2-429
11.2
(40.9% )
(10.11%)
Peature 1
2.308
2.45
2,433
2,592
(+3.5%
(+2.7%
(+0.55%)
Feature D.Peature.
1.517
1.478
1,302
1.273
(-2.%%
(-8.2%
(-1111%)
Incercept
-39.77
-39.77
(-2.2%
(+1.5%
(0%)
xall sall-robet- navigation (n:
V15
2,281
2.427
2,300
3,076
6+6.4%
(+2.5%
(+30,99%)
V14.V16
1.024
1,621
1.777
1.087
(-0.2%
(+9.49% )
(-33. 150)
V1d
1.381
1,701
1.53
1.387
(+
(+-0.4%)
1.246
1.238
1.367
(-0.0%
10.4%
(49.7%)
(-1.7%
(-8%%
(+-67.650)
Insereept
-10.77
-11.5
-10.79
-10777
7..33%
0..0.2% )
(0%)
wdbc In: 569, p:: 31, 5)
Inseroepi
2.367
3.815
3,279
2.967 (0%5)
(+61
(+-38 59%
V28
1.124
1.109
1.1
1.502
(-1.3%
(+333.05)
1.469
(+1.3%
(- 6%
1--48 (+-08.950)
V8
0.9086
0.8257
0.8677
1.189
(-9.1%
(-1.%%
(+300,9%)
V22
0.7937
0.8219
0.7633
0.8532
(+3.0%
(-3.8%
(47.5%%
V23
0.7138
0.6631
0.6834
0.9698
(-7.1%
(-1.3%
(+-35.9%)
In: p:
Interoopt
6.613
6.194 (095)
6+6.8%
(-1.2%
Mesn, G-Mean
(-0.8%
(-0.5%
[- (-36.256)
Mean
2,903
3,068
2,981
2.175
6+.6.6%
(42.77% )
26.1%))
Mesn__R R
2.829
2,775
2,851
1.73
(-1.9%
(+0.89%
[-38 896)
SD Plaa
0.3629
0.3611
0.3608
0.3064
(-0.3%
(-2.55 2.2.5%
(-1.8%)
0.3257
(+2.4%
(43.4%%
(-8.65)
39


Reviews:
[{"review_id": "a7hPM7L1Hj", "reviewer": "Reviewer_KVb7", "summary": "This paper introduces FastPD, a novel tree-based algorithm for estimating partial dependence (PD) functions, which are central for interpreting machine learning models via SHAP values. The paper identifies a critical limitation in the commonly used TreeSHAP-path method: while TreeSHAP-path is theoretically exact under feature independence, its reliance on conditioning on the decision path can lead to biased SHAP estimates when features are correlated. FastPD overcomes this issue by decoupling the computation into an augmentation phase—where the full background dataset is used to precompute empirical probabilities—and an efficient evaluation phase that reuses these values to compute the PD functions exactly. The result is a method that is both computationally efficient and robust to feature correlations.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your comments, we address them below.\n\n1. **Real-world application:**\n\n   - **Benchmark:**  \n     For an additional comparison between `FastPD`, `FastPD-100`, `FastPD-50`, and the path-dependent method, we have now added a benchmark considering 33 regression and 29 classification datasets from the OpenML-CTR23 Task Collection and the OpenML-CC18 Task Collection. For a set $S \\subseteq \\{1,\\dots,d\\}$, we compute the importance measure of the estimated functional component $\\hat{m}_S$ by $\\hat{E}[|m_S(x)|]$, where $\\hat{E}[\\cdot]$ denotes the empirical mean.  \n     For each dataset, we record the importance of the components $\\hat{m}_S$ that are ranked in the top 5 by any of the four methods (`FastPD`, `FastPD-100`, `FastPD-50`, path-dependent).  \n     The results are available here:  \n     [https://www.dropbox.com/scl/fi/ujdikzu8iyrn53ha03qc4/summary.pdf?rlkey=ypnkbdhjwgylr38ylj16ugjzj&st=f97bol4d&dl=0](https://www.dropbox.com/scl/fi/ujdikzu8iyrn53ha03qc4/summary.pdf?rlkey=ypnkbdhjwgylr38ylj16ugjzj&st=f97bol4d&dl=0)  \n     We observe that while results are consistent across methods for some datasets, substantial differences can emerge in others. In several cases the differences are >100%, i.e., the feature importance values from the path-dependent method are more than twice as high as those from FastPD.\n\n   - **Concrete example:**  \n     We highlight one dataset (`adult`) where `FastPD` and the path-dependent method gave qualitatively different insights. The following plot based on a single model obtained from one run of hyperparameter search. We have plotted the interaction effect between age and relationship status:  \n     $m_{\\text{age,relationship}}$.  \n     [https://www.dropbox.com/scl/fi/8wx0xj34rglhst6tm8eb0/adult.pdf?rlkey=nrgz74xjrd1udoxgrhohwx6au&st=yclm7q6o&dl=0](https://www.dropbox.com/scl/fi/8wx0xj34rglhst6tm8eb0/adult.pdf?rlkey=nrgz74xjrd1udoxgrhohwx6au&st=yclm7q6o&dl=0)\n     The path-dependent method indicates no significant interaction effect during working age (35–60), suggesting that age affects husbands and wives similarly in this range. Outside this interval, it estimates a more positive effect for wives than for husbands.  \n     Using `FastPD`, we get a different picture: Between ages 30 and 65, the effect is more positive for husbands than for wives, with the advantage reversing outside this age range.  \n\n2. **Flowcharts/diagrams:**  \n   We agree that a flowchart and illustration would help in explaining the algorithm, and will add it for the camera-ready version.\n\n3. **Reproducibility details:**  \n   We will modify the beginning of the experiments section to include an overview of our hyperparameter settings.\n\n4. **Package implementation:**  \n   A package implementation exists, including automated plot functions. However, due to anonymity, we prefer not to share it at this point, but it will be made available in the camera-ready version.", "claims_and_evidence": "Overall, the claims are well supported by clear theoretical derivations and convincing simulation experiments, though real-world dataset experiments would further strengthen the evidence.", "methods_and_evaluation_criteria": "The proposed method is specifically designed for tree-based models, leveraging the structure to precompute necessary statistics (augmentation) and then efficiently evaluate PD functions. This two-step approach is both novel and well-motivated.\nOverall, the methods and evaluation criteria make sense for the problem of model interpretability, although additional experiments on real-world datasets would enhance the evaluation.", "theoretical_claims": "The theoretical arguments are well-presented and, at a high level, correct. Minor technical details may need further scrutiny, but no major issues were identified.", "experimental_designs_or_analyses": "The experiments are based on simulated data with controlled levels of feature correlation using tree-based models (e.g., XGBoost). The comparisons include multiple methods (VanillaPD, TreeSHAP-path, TreeSHAP-int, and FastPD) and evaluate both accuracy (MSE) and computational runtime. The experimental analyses are sound and clearly demonstrate that FastPD achieves lower estimation error and faster computation. While the designs are rigorous for simulation studies, incorporating additional experiments on real-world datasets would further validate the approach in practical settings.", "relation_to_broader_scientific_literature": "In general this is a good contribution to the literature on interpretability, and shapley values, particularly addressing the bias with existing approaches like TreeSHAP.", "essential_references_not_discussed": "N/A.", "other_strengths_and_weaknesses": "Strengths:\nNovel decoupling of augmentation and evaluation steps to compute exact PD functions.\nClear mathematical exposition linking SHAP values, partial dependence functions, and the issues with conditioning in TreeSHAP-path.\nComprehensive simulation experiments that highlight both estimation accuracy and runtime improvements.\n\nWeakness:\nLack of evaluation on real-world datasets limits the demonstration of practical applicability.\nThe algorithm’s presentation could benefit from more intuitive diagrams or flowcharts to aid understanding.\nReproducibility details (e.g., explicit hyperparameter settings) could be more thoroughly described in the main text.", "other_comments_or_suggestions": "Consider including experiments on real-world datasets to validate the method’s practical relevance. For example, what can practitioners expect to see in practice? Is the bias of TreeSHAP really that bad? Can it lead to \"catastrophic\" situations where you attribute significantly less importance to something with with TreeSHAP vs FastPD (we now know it is consistent, but can you make a case for why this is so bad in practice in a relatively varied collection of benchmark datasets?). I think these extensions will really increase the impact of the paper, which is why I want to be transparent and tell the authors that I will definitely raise my score from a 3, if a stronger set of experiments with interpretations on the differences between FastPD and other methods is presented! I think it will really convince readers to use this method!  \n\n\nAdditional illustrative diagrams of the augmentation and evaluation steps in FastPD would improve clarity.", "questions_for_authors": "Code seems a little messy, do the authors intend to release an easy-to-use package? I think it would be helpful for people to adopt this approach.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "N2YE6L9bji", "reviewer": "Reviewer_XQ8h", "summary": "This paper proposes FastPD, an efficient, and consistent algorithm for estimating Partial Dependence (PD) functions in tree-based models. FastPD addresses computational inefficiencies present in existing methods by decomposing the estimation process into a two-step procedure: a tree augmentation step leveraging background data and an evaluation step for calculating PD functions. The authors show that FastPD improves computational complexity from quadratic to linear with respect to the number of samples. Furthermore, the paper demonstrates that existing methods, like TreeSHAP-path, are inconsistent when features are correlated. Experimental results indicate that FastPD provides more accurate estimates of PD-based interpretations, including SHAP values, compared to competing algorithms.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your comments, we address them below.\n\n1. **Real-world applications:**  \n   We have addressed this in our reply to the first reviewer. Please see **1. Real-world application** in our response to R1.\n\n2. **High-complexity models:**  \n   The computational complexity of both our algorithm and the baselines scales exponentially with depth, which motivates the use of smaller depth values. Additionally, increasing depth captures higher-order interactions that are inherently less interpretable. Gradient Boosting methods such as XGBoost fit shallow trees well; we therefore recommend using a depth of 5 or less in practice. We will clarify this point in the paper.\n\n3. **Referencing SHAP scalability:**  \n   We will add some references for non-tree-based SHAP algorithms in our introduction.\n\n4. **Distinguishing model PD and ground-truth PD:**  \n   We will add the following to the application section:  \n   *\"We see that FastPD is well-suited for estimating the model PD, though it may deviate from the underlying ground truth PD. In practical settings where accurately capturing the relationship between the predictors and response is crucial, we recommend using FastPD as an initial visualization tool, which is to be complemented by other statistical methods for estimating continuous treatment effects, see e.g. Kennedy, E. H., Ma, Z., McHugh, M. D., & Small, D. S. (2017). Non-parametric methods for doubly robust estimation of continuous treatment effects. Journal of the Royal Statistical Society Series B: Statistical Methodology, 79(4), 1229-1245.\nand Chernozhukov, V., Chetverikov, D., Demirer, M., Duflo, E., Hansen, C., Newey, W., & Robins, J. (2018). Double/debiased machine learning for treatment and structural parameters. The Econometrics Journal, 21(1),  C1–C68..\"*\n\n5. **Choice of background sample size:**  \n   We now note in the main text that in practice, one should use all available data as background. If this is not feasible, a smaller background sample size can be chosen (e.g., $n_b = 100$) and compared to a slightly larger one (e.g., $n_b = 150$). If the resulting PD functions are similar, then the smaller sample size is likely sufficient.\n\n   As a remark, in our experiments we have observed that augmenting the tree is typically fast, so the computational cost of using more background samples is low. The majority of computation time is instead spent on evaluating the explanation points.", "claims_and_evidence": "The claims regarding the computational efficiency and consistency of FastPD are convincingly supported through theoretical propositions and experimental validations. However, the claim about significant differences between FastPD and TreeSHAP-path estimates is primarily demonstrated through simulation experiments with moderate feature correlation. Additional real-world experiments could further strengthen these claims.", "methods_and_evaluation_criteria": "The methods and evaluation criteria used in the paper are appropriate for the stated problem of estimating PD functions efficiently. The paper clearly outlines how the complexity and consistency of the algorithms are assessed. Using simulations with varying levels of feature correlation and benchmarking against established methods (TreeSHAP-path, TreeSHAP-int, VanillaPD) is sensible and effective.", "theoretical_claims": "The theoretical claim regarding the inconsistency of TreeSHAP-path is clearly stated, and its correctness is supported by a formal proof provided in the supplementary material. I reviewed the proof in the appendix and found it sound and convincing.", "experimental_designs_or_analyses": "The experimental design and analysis presented in the paper appear sound. The experiments include comparisons of computational runtime, estimation error (mean squared error), and consistency across different correlation settings. No immediate issues were found in the methodology or its implementation.", "relation_to_broader_scientific_literature": "The paper positions itself clearly within the broader scientific literature, notably referencing foundational work on SHAP values, PD functions, and functional decomposition. It builds directly on TreeSHAP-related methods and addresses previously identified limitations (such as inconsistency due to feature correlation).", "essential_references_not_discussed": "The paper is comprehensive in its citations; however, additional references exploring the scalability of SHAP (beyond tree-based methods) would enhance context for readers interested in broader applicability. \n\n- Jethani, Neil, et al. \"Fastshap: Real-time shapley value estimation.\" International conference on learning representations. 2021.\n- Wang, Guanchu, et al. \"Accelerating shapley explanation via contributive cooperator selection.\" International Conference on Machine Learning. PMLR, 2022.", "other_strengths_and_weaknesses": "Strengths include clear contributions, detailed and rigorous mathematical arguments regarding algorithmic complexity, and practical relevance for model interpretability in correlated feature settings. The experimental validation is thorough and clearly presented.\n\nA weakness is that the scalability discussion focuses primarily on moderately deep trees and relatively simple experimental settings. The impact of scaling to larger, real-world datasets and more complex tree-based ensembles (deeper or broader models) is less clear.", "other_comments_or_suggestions": "The paper could be clearer in distinguishing the practical implications of model-based PD functions versus ground-truth PD functions in realistic scenarios. \n\nMinor typo found: \"adddition\" should be \"addition\" (Section 1.1).", "questions_for_authors": "How does FastPD scale when applied to larger and deeper tree-based ensembles (e.g., random forests or deep gradient boosting trees)? Clarifying this could strengthen practical applicability. \n\nCould you provide guidance or recommendations on choosing the optimal number of background samples for FastPD in different practical scenarios?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {"ethical_review_concerns": "N/A"}}, {"review_id": "hiDSEw3ih5", "reviewer": "Reviewer_P6Hy", "summary": "This paper proposes an efficient tree-based algorithm called FastPD for estimating partial dependence (PD) functions and extracting functional decompositions. The authors highlight how it can unify PD plots, Shapley values, and higher-order interaction effects under one consistent framework. By carefully caching and reusing computations, the method achieves gains over naive or path-dependent approaches, particularly when features are correlated. The main thrust is that FastPD provides both improved consistency and reduced computational complexity, compared to commonly used baselines like path-dependent TreeSHAP. In practice, it claims a user to quickly derive multiple interpretability artifacts from a single model-agnostic foundation.", "questions": "", "limitations": "", "rating": 2, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your comments.\n\nIn our response to the first reviewer we have conducted further experiments of our method on real data (see **1. Real-world application** in our response to R1). In our response to the second reviewer, we advice using shallow trees to reduce complexity for large-scale datasets with many features (see **2. High-complexity models** in our response to R2). We believe that our method is still very applicable even in high-dimensions, provided that the model is not overly complex. We have not encountered numerical instability in our experiments with real data; since with many samples, the estimation of partial dependence (PD) functions becomes more reliable, not less. Moreover, practitioners are typically interested in marginal effects and low-order interactions, because they are easier to interpret. In all cases, the actual quantity being computed is a just an empirical average, for which `FastPD` can provide a speed-up when the model is a tree-based model. \n\nWe agree that not all functional components could or should be acquired, since for higher-order interactions the effects of these components are usually very close to zero, we will make this point more clear in the paper. This further motivates the use of shallow trees, as they limit the fitting of higher-order interactions and preserve interpretability.\nAs a remark, in the package implementation of our method, we provide the user the option to specify the order of the effect they wish to acquire. \n\nFor the reference that we had inadvertently missed, perhaps it was the following you meant?\n\n> Fumagalli, F., Muschalik, M., Hüllermeier, E., Hammer, B., & Herbinger, J. (2024). *Unifying Feature-Based Explanations with Functional ANOVA and Cooperative Game Theory*. arXiv preprint arXiv:2412.17152.\n\nWe have already cited Hiabu et al. multiple times, but were not yet aware of Fumagalli et al. and will ensure it is cited in the revised version.\n\nFinally, we agree that the paper title could be clearer. A possible revision would be:  \n**\"Fast Estimation of Partial Dependence Functions using Trees\"**,  \nwhich more accurately reflects the tree-based nature of our approach.", "claims_and_evidence": "The authors study the inconsistency of TreeSHAP, more specifically when features are correlated.", "methods_and_evaluation_criteria": "The evaluation is quite weak; FastPD is only compared with two versions of TreeSHAP in one experiment of adding more background samples over a relatively small XGBoost. The evaluation is merely over the synthetic data sets; We cannot draw any conclusion from the experimetns provided in the article.", "theoretical_claims": "The paper studies the consistency of the proposed method and the inconsistency of TreeSHAP.", "experimental_designs_or_analyses": "While the authors include simulation studies—showing that the method works better on synthetic data compared to TreeSHAP—they do not examine any real-world datasets or large-scale industrial applications. This can make it difficult to assess how well FastPD copes with real-world issues such as messy data distributions, large numbers of features with intricate dependencies, and numeric stability in high-dimensional spaces. This is particularly important because the proposed method is more an algorithmic innovation.", "relation_to_broader_scientific_literature": "There are several studies at the intersection of functional decomposition and PDP. FastPD could be a worthy added value to the literature, but it definitely requires far more experiments and probably more theoretical studies on the behavior of the method.", "essential_references_not_discussed": "References are fine, but a recent study is missed:\nUnifying Feature-Based Explanations with Functional ANOVA and Cooperative Game Theory\n\n\n[1] Hiabu, M., Meyer, J. T., and Wright, M. N. Unifying\nlocal and global model explanations by functional decomposition of low dimensional structures", "other_strengths_and_weaknesses": "I think the most crucial problem is the experimental setup; the evaluative measures are not rigorous, and the experiments are only conducted on small, limited synthetic data sets. TreeSHAP is also the only benchmarks. More comprehensive experiments are required, where FastPD is compared over real data sets according to some meaningful evaluative measures.\nThis is particularly important for FastDP because it puts forward more an algorithmic way of computing PD than a rigorous theoretical one.", "other_comments_or_suggestions": "The title could be more informative by mentioning that FastPD is particularly for tree-based machine learning models.", "questions_for_authors": "The author should discuss that not all the functional components could be acquired as there are exponentially many such components.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: The paper introduces FastPD, a novel algorithm for estimating partial dependence (PD) functions for tree-based models. These functions are crucial in computing, for instance, SHAP explanations or partial dependence plots. FastPD has a low complexity and, unlike other fast algorithm such as path-dependent TreeSHAP, it is exact in the case of feature correlation.

The strengths of the paper have been listed by the reviewers as the clear motivation, novelty, and soundness of the proposed method, the clear mathematical derivation, the link drawn in the paper between SHAP and PD functions, and the good empirical performance in terms of estimation quality and computing times

All reviewers expressed concerns about the lack of experiments on real-world datasets. In their response, the authors have carried out new experiments on a large set of real-world benchmarks to demonstrate the difference between FastPD and path-dependent TreeSHAP, as suggested by reviewer KVb7. They also identified a dataset where FastPD and TreeSHAP lead to different conclusions. While reviewer P6Hy still believes more experiments are necessary, reviewer KVb7 has increased their score.

Although the reviewers don’t unanimously recommend acceptance, I believe incorporating the new experiments will strengthen the paper that already presents a solid algorithmic and theoretical contribution and is well-presented. Therefore, I recommend acceptance. I trust the authors to incorporate the new results into the paper and make all other changes promised in their responses. I agree with reviewer P6Hy that the title of the paper should explicitly mention that the approach is focused on tree-based models.
Decision: Accept (poster)