From global to local MDI variable importances for random forests and when they are Shapley values

1 Motivation . While research in machine learning ( ML ) often focuses on predictive accuracy , another important topic concerns the interpretation of ML models and their predictions . Interpreting a model helps to uncover the mechanisms it captures ( e.g. , biomarkers useful to diagnose a disease ) , and to explain its predictions ( e.g. , why a particular patient is diagnosed healthy or sick ) . The latter becomes essential when a ML prediction may impact one ’ s life , and as a way of checking the trustworthiness of a ML model ( e.g. , to identify unwanted biases ) . Common interpretation tools include variable importance measures that assess which , and to which extent , variables are important for a model . They help to understand how the model works , and to gain insight on the underlying modelled mechanism . In tree-based methods , such as Random forests [ Breiman , 2001 ] , feature importance scores can be derived as a low-cost by-product of the learning step . Given their extensive use in applied research , tree-based importance measures have been studied both empirically ( see , e.g. , Strobl et al . [ 2007 ] , Archer and Kimes [ 2008 ] , Genuer et al . [ 2010 ] , Auret and Aldrich [ 2011 ] ) and theoretically ( see , e.g. , Ishwaran et al . [ 2007 ] , Louppe et al . [ 2013 ] , Louppe [ 2014 ] , Sutera et al . [ 2018 ] , Li et al . [ 2019 ] , Sutera [ 2019 ] , Scornet [ 2020 ] ) . Assuming a sufficiently large learning set and number of trees , these works showed that importance measures have desirable properties , such as consistency with respect to the notion of feature relevance . They also analyzed the impact of learning meta-parameters ( e.g. , randomization level , tree depth , ensemble size ) on such properties . While standard importance measures evaluate the global importance of a feature at the level of a dataset , several works proposed new approaches based on Shapley values to derive local scores reflecting the importance of a feature for a given prediction ( e.g. , Neto and Paulovich [ 2020 ] , Lundberg et al . [ 2020 ] , Izza et al . [ 2020 ] ) . The contribution of the present work in this context is two-fold . First , we show that the standard mean decrease of impurity ( MDI ) measure when derived from totally randomized trees and in asymptotic ∗Corresponding author . Email address : sutera.antonio @ gmail.com 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . conditions ( similar to those used to show the consistency with respect to the relevance ) are Shapley values , and therefore have the same properties as any other importance measures based on these values ( Section 3 ) . Secondly , we propose a new local MDI measure for tree-based ensemble models to measure feature relevance locally , which naturally derives from global MDI and corresponds to Shapley values in the same conditions ( Section 4 ) . Global and local MDI are compared against other Shapley-value based scores , both conceptually ( Section 5 ) and empirically ( Section 6 ) . 2 Background . In what follows , we consider a standard supervised learning setting and denote by V = { X1 , . . . , Xp } the set of p input variables , and by Y the output . Game theory and Shapley value . We only remind here concepts and results that are useful later in the paper . Notations below are mostly adapted from [ Besner , 2019 ] . In game theory , a TU-game ( V , v ) ( i.e. , a cooperative game with transferable utilities ) is defined by a finite set of players V = { X1 , . . . , Xp } 2 and a characteristic ( or coalition ) function v ∈ V : 2V → R , with v ( ∅ ) = 0 , that maps a coalition ( i.e. , a set ) of players to a real number representing the outcome or gain of a game ( see , e.g. , [ van den Brink et al. , 2015 ] ) . A TU-game ( V , v ) is monotonic if v ( S ) ≤ v ( T ) if S ⊆ T ⊆ V . Let us denote by V −m the set V \ { Xm } . The marginal contribution MCvm ( S ) of player Xm ∈ V for S ⊆ V −m is defined by MCvm ( S ) = v ( S ∪ { Xm } ) − v ( S ) . A player Xm ∈ V is called a null player if MCvm ( S ) = 0 for all S ⊆ V −m . Two players Xi and Xj are said to be symmetric whenever v ( S ∪ { Xi } ) = v ( S ∪ { Xj } ) for all coalitions S ⊆ V −i , j . A TU-value φv : V → R is a function that assigns to any player Xm ∈ V and any function v ( ∈ V is omitted in the rest ) a value , denoted φv ( Xm ) ∈ R , also known as its payoff , reflecting its contribution in the game ( V , v ) . Several properties or axioms for TU-values have been defined in the literature that are expected to be satisfied in practical contexts ( see [ Besner , 2019 ] for a more exhaustive list ) : Efficiency : For all v , ∑ Xm∈V φv ( Xm ) = v ( V ) . The TU-value divides the total gain ( i.e. , gain when all players are involved ) among all players in an additive way . Symmetry : For all v , φv ( Xi ) = φv ( Xj ) if players Xi and Xj are symmetric . Two players of equal contributions in every game ( i.e. , with every coalition S ) should get the same value . Null player : For all v , φv ( Xm ) = 0 if Xm is a null player . A null player should get a zero payoff . Strong monotonocity3 : For all v , w and Xm ∈ V such that MCvm ( S ) ≥ MCwm ( S ) for all S ⊆ V −m , we have φv ( Xm ) ≥ φw ( Xm ) . If a player ’ s marginal contributions are greater ( or equal ) in a game than in another in all coalitions , then its payoff in this game should not be lower than in the other . Specific forms of TU-value have been studied in the literature from the point of view of which axioms they satisfy and how uniquely they are defined by these axioms . As one of the most prominent results , it has been shown [ Young , 1985 ] that the only TU-value that satisfies Efficiency , Symmetry , Null player4 , and Strong monotonicity is the Shapley value ϕShv defined by [ Shapley , 1953 ] : ϕShv ( Xm ) = ∑ S⊆V −m |S| ! ( p− |S| − 1 ) ! p ! MCvm ( S ) . ( 1 ) Other equivalent formulations of the Shapley value , as well as other axiomatisations of this value , have been proposed in the literature . Axiomatisations of other , typically more general , sets of TU-values are also available ( see [ Besner , 2019 ] for a recent and exhaustive discussion of this topic ) . Feature relevance . In the feature selection literature , a common definition of the relevance of a feature is as follows [ Kohavi et al. , 1997 ] : A variable Xm ∈ V is relevant to Y ( with respect to V ) iff ∃B ⊂ V : Xm ⊥̸ Y |B . A variable Xm is irrelevant if it is not relevant . Relevant variables can be further divided into two categories according to their degree of relevance [ Kohavi et al. , 1997 ] : A variable Xm is strongly relevant 2We use the same notations for the ( set of ) players as for ( set of ) input features , as the two will coincide later . 3In the ML literature , strong monotonicity is often called consistency [ Lundberg and Lee , 2017 ] . 4Actually , the Null player property is not required as it can be derived from strong monotonicity . to Y ( with respect to V ) iff Y ⊥̸Xm|V −m . A variable X is weakly relevant if it is relevant but not strongly relevant . Strongly relevant variables thus convey information about the output that no other variable ( or combination of variables ) in V conveys . Decision trees and forests . Each interior node of a decision tree [ Breiman et al. , 1984 ] is labelled with a test based on some input and each leaf node is labelled with a value of the output . The tree is typically grown from a learning sample of size N drawn from P ( V , Y ) using a procedure that recursively partitions the samples at each node t into two child nodes ( tL and tR ) . The test st used to partition the samples at node t is the one that maximises the mean decrease of some node impurity measure i ( · ) ( e.g. , the Shannon entropy , the Gini index or the variance of Y ) : ∆i ( s , t ) = i ( t ) − p ( tL ) p ( t ) i ( tL ) − p ( tR ) p ( t ) i ( tR ) , where p ( tL ) and p ( tR ) are the proportions of samples that fall in nodes tL and tR respectively . Single decision trees suffer from a high variance that is very efficiently reduced by building instead an ensemble of randomized trees and aggregating their predictions . Popular methods are Breiman [ 2001 ] ’ s Random Forests that build each tree from a different bootstrap sample with a local random selection of K ( ≤ p ) variables at each node from which to identify the best split , and Geurts et al . [ 2006 ] ’ s Extra-Trees which skip bootstrapping and additionally randomly select the split values . Following Geurts et al . [ 2006 ] , Louppe et al . [ 2013 ] , ensemble of randomized trees grown with the value of the randomization parameters K set to 1 will be called Totally randomized trees . Mean decrease impurity importance . Given an ensemble of trees , several methods have been proposed to evaluate the ( global ) importance of variables for predicting the output [ Breiman et al. , 1984 , Breiman , 2001 ] . This paper focuses on the Mean Decrease of Impurity ( MDI ) importance . Given a tree T , the MDI importance of a variable Xm for predicting the output Y is defined as : Imp ( Xm , T ) = ∑ t∈T : ν ( st ) =Xm p ( t ) ∆i ( st , t ) , ( 2 ) where the sum is over all interior nodes t in T , ν ( st ) denotes the variable tested at node t , and p ( t ) is the fraction of samples reaching node t. Imp ( Xm , T ) is thus the ( weighted ) sum of impurity decreases over all nodes where Xm is used to split . The MDI importance of Xm derived from forests of NT trees is then the average of Imp ( Xm , T ) over all trees : Imp ( Xm ) = 1 NT ∑ T Imp ( Xm , T ) . ( 3 ) While this measure was initially proposed as a heuristic , Louppe et al . [ 2013 ] characterise it theoretically under the following conditions : ( 1 ) all input variables and the output are categorical ( not necessarily binary ) ( 2 ) trees use so-called exhaustive splits5 , and ( 3 ) impurity is measured by Shannon entropy6 . Later , we will refer to these conditions collectively as the categorical setting . In the categorical setting , Louppe et al . [ 2013 ] ( Thm . 1 ) show that for totally randomized trees ( i.e. , K = 1 ) and in asymptotic conditions ( i.e. , assuming NT → ∞ and a learning sample of infinite size ) , the MDI importance , denoted Imp∞ , is given by : Imp∞ ( Xm ) = p−1∑ k=0 1 Ckp 1 p− k ∑ B∈Pk ( V −m ) I ( Y ; Xm|B ) , ( 4 ) where Pk ( V −m ) is the set of subsets of V −m of cardinality k , and I ( Y ; Xm|B ) is the conditional mutual information of Xm and Y given the variables in B . They also show that the sum of the MDI importances of all variables is equal to the mutual information between all input features and the output [ Louppe et al. , 2013 , Thm . 2 ] : p∑ m=1 Imp∞ ( Xm ) = I ( Y ; V ) . ( 5 ) A direct consequence of Equation 4 is that a variable Xm is irrelevant iff Imp∞ ( Xm ) = 0 , which makes the MDI importance a sensible measure to identify relevant variables . 5Each node is split into |Xi| sub-trees , one for each of the |Xi| different values of the split variable Xi . 6A short introduction to information theory and the related notations used in the paper is provided in Appendix G. 3 Global MDI importances are Shapley values In this section , we revisit the theoretical analysis of Louppe et al . [ 2013 ] and Sutera [ 2019 ] in the light of TU-games and TU-values , focusing on the categorical setting and asymptotic conditions adopted by these authors . We show in Section 3.1 that MDI importances computed from totally randomized trees can be interpreted as the Shapley value for a particular TU-game and we then discuss in Section 3.2 the case of non totally randomized trees ( i.e. , K > 1 ) .

Summary: The paper’s contributions are on variable importance measures in tree and tree ensembles. In particular, the paper provides an interpretation of the “mean decrease of impurity” measure (MDI) of totally randomised trees as Shapley values where the characteristic function of the underlying game is the mutual information between a feature variable set and the target variable. Moreover, a novel local variant of MDI measure is introduced that quantifies the impact of variable for the classification of a specific input point, which can again be interpreted as Shapley values in the case of totally randomised trees.