Joint Shapley values: a measure of joint feature importance

The Shapley value is one of the most widely used measures of feature importance partly as it measures a feature ’ s average effect on a model ’ s prediction . We introduce joint Shapley values , which directly extend Shapley ’ s axioms and intuitions : joint Shapley values measure a set of features ’ average contribution to a model ’ s prediction . We prove the uniqueness of joint Shapley values , for any order of explanation . Results for games show that joint Shapley values present different insights from existing interaction indices , which assess the effect of a feature within a set of features . The joint Shapley values provide intuitive results in ML attribution problems . With binary features , we present a presence-adjusted global value that is more consistent with local intuitions than the usual approach . 1 INTRODUCTION . Game theory ’ s Shapley value partitions the value arising from joint efforts among individual agents ( Shapley , 1953 ) . Specifically , denote by N = { 1 , . . . , n } a set of agents , and by GN the set of games on N , where a game is a set function v from 2N to R with v ( ∅ ) = 0 . Then v ( S ) is the worth created by coalition S ⊆ N . When there is no risk of confusion , we omit braces to indicate singletons ( e.g . i rather than { i } ) and denote a set ’ s cardinality by the corresponding lower case letter ( e.g . s = |S| ) . For any agent i , Shapley ’ s value is then ψi ( v ) ≡ ∑ S⊆N\ { i } s ! ( n− s− 1 ) ! n ! [ v ( S ∪ i ) − v ( S ) ] . ( 1 ) This is the average worth that i adds to possible coalitions S , weighted as follows : if the set of agents , S , has already ‘ arrived ’ , draw the next agent , i , to arrive uniformly over the remaining N\S agents . Shapley ’ s value is widely used in explainable AI ’ s attribution problem , partitioning model predictions among individual features ( q.v . Štrumbelj & Kononenko ( 2014 ) ; Lundberg & Lee ( 2017 ) ) after the model has been trained . Evaluating the prediction function at a specific feature value corresponds to an agent ’ s presence ; evaluating it at a reference ( or baseline ) feature value corresponds to the agent ’ s absence . A feature ’ s Shapley value is its average marginal contribution to the model ’ s predictions . When features are correlated , individual measures of importance may mislead ( Bhatt et al. , 2020 ; Patel et al. , 2021 ) , as measures of individual significance such as the t-test do . Thus , Shapley ’ s value has been extended to sets of features ( Grabisch & Roubens , 1999a ; Marichal et al. , 2007 ; Alshebli et al. , 2019 ; Dhamdhere et al. , 2020 ) . As these extensions introduce axioms not present in Shapley , they do not preserve the Shapley value ’ s intuition . We extend Shapley ’ s axioms to sets of features , randomizing over sets rather than individual features . The resulting joint Shapley value thus directly extends Shapley ’ s value to a measure of sets of feature importance : the average marginal contribution of a set of features to a model ’ s predictions . Our approach ’ s novelty is seen in our extension of the null axiom : in Shapley ( 1953 ) , a null agent contributes nothing to any set of agents to which it may belong ; here , a null set contributes nothing to any set to which it may belong . By contrast , interaction indices ( Grabisch & Roubens , 1999a ; Alshebli et al. , 2019 ; Dhamdhere et al. , 2020 ) recursively decompose sets into individual elements , retaining the original Shapley null axiom . Thus , these indices measure sets ’ contributions relative to their constituent elements — so are complementary to the joint Shapley value . The generalised Shapley value ( Marichal et al. , 2007 ) is closer to our work , but differs in a number of respects : our probabilities are independent of the size of the set of features under consideration1 ; our efficiency axiom is fully joint , while theirs are based on singletons and pairs ; our symmetry axiom provides uniqueness without reliance on recursion or partnership axioms . To illustrate , consider a null coalition , T , whose individual members contribute positively to coalitions that they join . Interaction indices assign a negative value to T , as the individual members act discordantly together . However , from a joint feature importance viewpoint , T should be assigned value 0 . The joint Shapley value matches this intuition . In the movie review application presented below , the joint Shapley values reveal contributions of collections of words , including grammatical features such as negation ( { disappointed } versus { won ’ t , disappointed } ) , and adjectives ( { effort } versus { terrific , effort } ) . Like the Shapley-Taylor interaction index ( Dhamdhere et al. , 2020 ) , our efficiency axiom depends on a positive integer k , the order of explanation , which limits the number of joint Shapley values to those for subsets up to cardinality k. As there are 2N − 1 non-empty subsets of N , the full set of joint Shapley values rapidly becomes unmanageable otherwise . In practice , k should be set to trade off insight ( favouring higher k ) against computational cost ( favouring lower k ) . For each k , the extended axioms yield a unique joint Shapley value . Unlike in Shapley ( 1953 ) , the joint anonymity and symmetry axioms are not interchangeable : each imposes distinct restrictions . Section 2 presents and extends the original Shapley axioms . Section 3 introduces joint Shapley values , deriving them as the unique solution to the extended axioms . Section 4 illustrates joint Shapley values in the game theoretical environment and applies them to the Boston housing ( Harrison & Rubinfeld , 1978 ) and Rotten Tomatoes movie review ( Pang & Lee , 2005 ) datasets , comparing them to interaction indices and presenting a sampling technique to facilitate calculation . Section 5 concludes . The Appendix collects proofs and other supplemental material . 2 EXTENDING SHAPLEY ’ S AXIOMS . For a game v ∈ GN , and a permutation σ on N , denote a permuted game by σv ∈ GN such that σv ( σ ( S ) ) = v ( S ) , ∀S ⊆ N , where σ ( S ) = { σ ( i ) : i ∈ S } . An index φ ( v ) of the game v ∈ GN is any real-valued function on 2N . The original axioms uniquely satisfied by the ( standard ) Shapley value ψ , are : LI linearity : ψ is a linear function on GN , i.e . ψ ( v + w ) = ψ ( v ) + ψ ( w ) and ψ ( av ) = aψ ( v ) for any v , w ∈ GN and a ∈ R. NU null : An agent that adds no worth to any coalition has no value , i.e . if v ( S ∪ { i } ) = v ( S ) for all S ⊆ N \ { i } , then ψi ( v ) = 0 . This axiom is sometimes called dummy . EF efficiency : The sum of the values of all agents is equal to the worth of the entire set , i.e . for all v ∈ GN , ∑n i=1 ψi ( v ) = v ( N ) . AN anonymity : For any σ on N and any v ∈ GN , ψi ( v ) = ψσ ( i ) ( σv ) , for all i ∈ N . 1Measures of joint significance ( e.g . F -tests ) and model selection ( e.g . BIC or AIC ) penalize larger feature sets to avoid overfitting . As joint Shapley values are calculated after model training , this problem does not arise . SY symmetry : If two agents add equal worth to all coalitions that they can both join then they receive equal value : if v ( S ∪ { i } ) = v ( S ∪ { j } ) ∀S ⊆ N \ { i , j } then ψi ( v ) = ψj ( v ) . This is strictly weaker than anonymity . Now extend each of these axioms in natural ways to conditions on sets rather than singletons . Below , φS ( v ) denotes an index for coalition S on game v. JLI joint linearity : φ is a linear function on GN , i.e . φ ( v + w ) = φ ( v ) + φ ( w ) and φ ( av ) = aφ ( v ) for any v , w ∈ GN and a ∈ R. ( This axiom has not been modified . ) JNU joint null : A coalition that adds no worth to any coalition has no value , i.e . if v ( S ∪T ) = v ( S ) for all S ⊆ N \ T , then φT ( v ) = 0 . JEF joint efficiency : The sum of the values of all coalitions up to cardinality k is equal to the worth of the entire set , i.e . for all v ∈ GN , ∑ ∅6=T⊆N : |T |≤k φT ( v ) = v ( N ) . JAN joint anonymity : For any σ on N and any v ∈ GN , φT ( v ) = φσ ( T ) ( σv ) , for all T ⊆ N . JSY joint symmetry : If two coalitions perform equally when joining coalitions that they can both join and for other coalitions they add no worth then they receive an equal value , i.e . if 1. v ( S ∪ T ) = v ( S ∪ T ′ ) for all S ⊆ N \ ( T ∪ T ′ ) , 2. v ( S ∪ T ) = v ( S ) for all S ⊆ N \ T such that S ∩ T ′ 6= ∅ , 3. v ( S ∪ T ′ ) = v ( S ) for all S ⊆ N \ T ′ such that S ∩ T 6= ∅ , then φT ( v ) = φT ′ ( v ) . Axiom JSY only equates the joint Shapley values for coalitions T and T ′ if they contribute identically to coalitions that they may both join , and contribute nothing to the other coalitions . Axioms JLI , JEF and JAN are all also used in Dhamdhere et al . ( 2020 ) . Our joint null and joint symmetry notions appear to be new : they reflect our interest in a set of features ’ contribution to a model ’ s predictions , so that the set ’ s cardinality should not play a role in determining its value . 3 JOINT SHAPLEY VALUES . Our main result is that there is a unique solution to axioms JLI , JNU , JEF , JAN and JSY , the joint Shapley value . The uniqueness is up to the kth order of explanation ; we say nothing about |T | > k. Theorem 1 . For each order of explanation k ∈ { 1 , . . . , n } , there is a unique ( up to the kth order of explanation ) index φJ which satisfies axioms JLI , JNU , JEF , JAN and JSY . It has the form φJT ( v ) = ∑ S⊆N\T q|S| [ v ( S ∪ T ) − v ( S ) ] for each ∅ 6= T ⊆ N with |T | ≤ k , where ( q0 , . . . , qn−1 ) uniquely solves the recursive system q0 = 1∑k i=1 ( n i ) , qr = ∑r−1s= ( r−k ) ∨0 ( rs ) qs∑k∧ ( n−r ) s=1 ( n−r s ) ; ( 2 ) for all r ∈ 1 , . . . , n− 1 . When k = 1 , the joint Shapley values coincide with Shapley ’ s values.2 When k = n , we have qr = r∑ j=0 ( r j ) ( −2 ) r−j 2n−j − 1 , ∀ r ∈ { 0 , . . . , n− 1 } . For each k , the constants qr are non-negative and satisfy ∑n−1 s=n−k ( n s ) qs = 1 . Further , as the joint Shapley value is similar in form to equation ( 1 ) ’ s standard Shapley value , the value of coalition T 2The Shapley-Taylor interaction index , φST , also has this property ( Dhamdhere et al. , 2020 ) . depends on its marginal contribution to other coalitions ; unlike interaction indices , it does not depend on the worth of its constituent agents . As with the Shapley value , the joint Shapley value can be seen as the worth brought by ‘ arriving ’ agents but , rather than arriving one at a time , they can now also arrive in coalitions . We develop this interpretation in Appendix A . We show here the implication of each of the joint axioms introduced above and prove Theorem 1 . It is already known that joint linearity restricts measures to be linear combinations of worths : Lemma 1 ( Grabisch & Roubens ( 1999a ) , Proposition 1 ) . If φ satisfies JLI , then for every ∅ 6= T ⊆ N there exists a family of real constants { aST } S⊆N such that for every v ∈ GN , φT ( v ) = ∑ S⊆N aTSv ( S ) . Axiom JNU then constrains the values of the constants { aTS } : Lemma 2 . Suppose φ satisfies JLI and JNU and let { aST } S⊆N , ∅ 6=T⊆N be the constants from Lemma 1 . Then for every ∅ 6= T ⊆ N and ∅ 6= S ⊆ N \ T , aTS = −aTS∪T . Further , for every ∅ 6= T ⊆ N , S ⊆ N \ T , and ∅ 6= H ( T , aTS∪H = 0 . SCombining these two lemmas yields : Proposition 1 . Suppose φ satisfies JLI and JNU . Then there exist constants { pT ( S ) } that depend on T and S such that for every ∅ 6= T ⊆ N and v ∈ GN φT ( v ) = ∑ S⊆N\T pT ( S ) [ v ( S ∪ T ) − v ( S ) ] . ( 3 ) Now establish a condition on the { pT ( S ) } values which must be satisfied under JEF : Proposition 2 . For each order of explanation k , φ satisfies axioms JLI , JNU and JEF if and only if for every ∅ 6= T ⊆ N with |T | ≤ k and v ∈ GN , φT ( v ) = ∑ S⊆N\T p T ( S ) [ v ( S ∪ T ) − v ( S ) ] with { pT ( S ) } satisfying δN ( S ) = ∑ ∅ 6=T⊆S : |T |≤k pT ( S \ T ) − ∑ ∅6=T⊆N\S : |T |≤k pT ( S ) , ( 4 ) for all ∅ 6= S ⊆ N , where δN ( S ) equals 1 if S = N and 0 otherwise . Recall that symmetry axiom SY is strictly weaker than anonymity axiom AN ( Malawski , 2020 ) . This is not the case for their joint counterparts : each imposes a different constraint on the constants { pT ( S ) } , as we shall see here . First , consider the effect of imposing JAN . Proposition 3 . For each order of explanation k , φ satisfies axioms JLI , JNU , JEF and JAN if and only if for every ∅ 6= T ⊆ N and v ∈ GN , φT ( v ) = ∑ S⊆N\T p T ( S ) [ v ( S ∪ T ) − v ( S ) ] with { pT ( S ) } satisfying ( 4 ) and pT ( S ) = pT ′ ( S′ ) ∀∅ 6= T , T ′ ⊆ N , S ⊆ N \ T , S′ ⊆ N \ T ′ s.t . s = s′ , t = t′ . ( 5 ) The analogous result for JSY is : Proposition 4 . For each order of explanation k , φ satisfies axioms JLI , JNU , JEF and JSY if and only if for every ∅ 6= T ⊆ N and v ∈ GN , φT ( v ) = ∑ S⊆N\T p T ( S ) [ v ( S ∪ T ) − v ( S ) ] with { pT ( S ) } satisfying ( 4 ) and pT ( S ) = pT ′ ( S ) ∀∅ 6= T , T ′ ⊆ N , S ⊆ N \ ( T ∪ T ′ ) . ( 6 ) Combining Propositions 2–4 completes the proof of Theorem 1 . The computational complexity of deriving the ( q0 , . . . , qn−1 ) is O ( nk2 ) . Once these have been determined , the joint Shapley values can be calculated ; the complexity of doing so isO ( 3n∧ ( 2nnk ) ) .

Summary: The paper proposes an extension of the Shapley values, namely, the joint Shapley values which measure a set of features' average effect of a models prediction. The uniqueness of the joint Shapley values is proved. Moreover, training details and tuning parameters are provided in the accompanying code.