J OINT S HAPLEY VALUES : A MEASURE OF JOINT FEATURE IMPORTANCE
Anonymous authors
Paper under double-blind review
A BSTRACT
The Shapley value is one of the most widely used measures of feature importance partly as it measures a feature’s average effect on a model’s prediction.
We introduce joint Shapley values, which directly extend Shapley’s axioms and intuitions: joint Shapley values measure a set of features’ average contribution to a model’s prediction.
We prove the uniqueness of joint Shapley values for any order of explanation.
Results for games show that joint Shapley values present different insights from existing interaction indices which assess the effect of a feature within a set of features.
The joint Shapley values provide intuitive results in ML attribution problems.
With binary features, we present a presence-adjusted global value that is more consistent with local intuitions than the usual approach.
I NTRODUCTION
Game theory ’s
Shapley value partitions the value arising from joint efforts among individual agents (Shapley, 1953).
Speciﬁcally, denote by N = { 1 , . . . , n } a set of agents, and by G N the set of games on N , where a game is a set function v from 2 N to R with v ( ∅ ) = 0 .
Then v ( S ) is the worth created by coalition S ⊆ N .
When there is no risk of confusion, we omit braces to indicate singletons (e.g. i rather than { i } ) and denote a set’s cardinality by the corresponding lower case letter (e.g. s = | S | ).
For any agent i , Shapley’s value is then
[Equation]
This is the average worth that i adds to possible coalitions S , weighted as follows: if the set of agents,
S , has already ‘arrived’, draw the next agent, i , to arrive uniformly over the remaining N \ S agents.
Shapley’s value is widely used in explainable AI ’s attribution problem , partitioning model predictions among individual features (q.v. Štrumbelj & Kononenko (2014); Lundberg & Lee (2017)) after the model has been trained.
Evaluating the prediction function at a speciﬁc feature value corresponds to an agent’s presence; evaluating it at a reference (or baseline) feature value corresponds to the agent’s absence.
A feature’s Shapley value is its average marginal contribution to the model’s predictions.
When features are correlated, individual measures of importance may mislead (Bhatt et al., 2020;
Patel et al., 2021), as measures of individual signiﬁcance such as the t -test do.
Thus, Shapley’s value has been extended to sets of features (Grabisch & Roubens, 1999a; Marichal et al., 2007;
Alshebli et al., 2019; Dhamdhere et al., 2020).
As these extensions introduce axioms not present in Shapley, they do not preserve the Shapley value’s intuition.
We extend Shapley’s axioms to sets of features, randomizing over sets rather than individual features.
The resulting joint Shapley value thus directly extends Shapley’s value to a measure of sets of feature importance: the average marginal contribution of a set of features to a model’s predictions.
Our approach’s novelty is seen in our extension of the null axiom: in Shapley (1953), a null agent contributes nothing to any set of agents to which it may belong; here, a null set contributes nothing to any set to which it may belong.
By contrast, interaction indices (Grabisch & Roubens, 1999a;
Alshebli et al., 2019; Dhamdhere et al., 2020) recursively decompose sets into individual elements, retaining the original Shapley null axiom.
Thus, these indices measure sets’ contributions relative to their constituent elements — so are complementary to the joint Shapley value.
The generalised Shapley value (Marichal et al., 2007) is closer to our work, but differs in a number of respects: our probabilities are independent of the size of the set of features under consideration 1 ; our efﬁciency axiom is fully joint, while theirs are based on singletons and pairs; our symmetry axiom provides uniqueness without reliance on recursion or partnership axioms.
To illustrate, consider a null coalition, T , whose individual members contribute positively to coalitions that they join.
Interaction indices assign a negative value to T , as the individual members act discordantly together.
However, from a joint feature importance viewpoint, T should be assigned value 0.
The joint Shapley value matches this intuition.
In the movie review application presented below, the joint Shapley values reveal contributions of collections of words, including grammatical features such as negation ({disappointed} versus {won’t, disappointed}), and adjectives ({effort} versus {terriﬁc, effort}).
Like the Shapley-Taylor interaction index (Dhamdhere et al., 2020), our efﬁciency axiom depends on a positive integer k , the order of explanation , which limits the number of joint Shapley values to those for subsets up to cardinality k .
As there are 2 N − 1 non-empty subsets of N , the full set of joint Shapley values rapidly becomes unmanageable otherwise.
In practice, k should be set to trade off insight (favouring higher k ) against computational cost (favouring lower k ).
For each k , the extended axioms yield a unique joint Shapley value.
Unlike in Shapley (1953), the joint anonymity and symmetry axioms are not interchangeable: each imposes distinct restrictions.
Section 2 presents and extends the original Shapley axioms.
Section 3 introduces joint Shapley values, deriving them as the unique solution to the extended axioms.
Section 4 illustrates joint Shapley values in the game theoretical environment and applies them to the Boston housing (Harrison &
Rubinfeld, 1978) and Rotten Tomatoes movie review (Pang & Lee, 2005) datasets, comparing them to interaction indices and presenting a sampling technique to facilitate calculation.
Section 5 concludes.
The Appendix collects proofs and other supplemental material.
E XTENDING S HAPLEY ’ S AXIOMS
For a game v ∈ G N , and a permutation σ on N , denote a permuted game by σv ∈ G N such that σv ( σ ( S ))
= v ( S ) ,
∀ S ⊆ N , where σ ( S ) =
{ σ ( i ) : i ∈ S } .
An index φ ( v ) of the game v ∈ G N is any real-valued function on 2 N .
The original axioms uniquely satisﬁed by the (standard) Shapley value ψ , are:
LI linearity : ψ is a linear function on G N ,
i
e. ψ ( v + w ) =
ψ ( v ) + ψ ( w ) and ψ ( av ) =
aψ ( v ) for any v, w ∈ G N and a ∈ R .
NU null :
An agent that adds no worth to any coalition has no value, i.e. if v ( S ∪ { i } ) = v ( S ) for all S ⊆ N \ { i } , then ψ i ( v ) = 0 .
This axiom is sometimes called dummy .
EF efﬁciency : The sum of the values of all agents is equal to the worth of the entire set, i.e. for all v
∈
G N
(cid:80) ni
=
ψ i ( v ) = v ( N ) .
AN anonymity :
For any σ on N and any v ∈ G N , ψ i ( v ) =
ψ σ ( i ) ( σv ) , for all
i ∈ N .
SY symmetry : If two agents add equal worth to all coalitions that they can both join then they receive equal value: if v ( S ∪ { i } )
=
v ( S ∪ { j } ) ∀ S ⊆
N \ { i, j } then ψ i ( v ) =
ψ j ( v ) .
This is strictly weaker than anonymity.
Now extend each of these axioms in natural ways to conditions on sets rather than singletons.
Below, φ S ( v ) denotes an index for coalition S on game v .
JLI
joint linearity : φ is a linear function on G N , i.e. φ ( v + w ) =
φ ( v ) + φ ( w ) and φ ( av ) =
aφ ( v ) for any v, w ∈ G N and a ∈ R .
(This axiom has not been modiﬁed.)
JNU joint null :
A coalition that adds no worth to any coalition has no value, i.e. if v ( S ∪ T )
=
v ( S ) for all S ⊆ N \ T , then φ T ( v ) = 0 .
JEF joint efﬁciency : The sum of the values of all coalitions up to cardinality k is equal to the worth of the entire set, i.e. for all v ∈ G N , (cid:88) ∅ (cid:54) =
T ⊆ N : | T |≤ k φ T ( v ) =
v ( N ) .
JAN joint anonymity :
For any σ on N and any v ∈ G N , φ T ( v ) =
φ σ ( T ) ( σv ) , for all T ⊆ N .
JSY joint symmetry : If two coalitions perform equally when joining coalitions that they can both join and for other coalitions they add no worth then they receive an equal value, i.e. if
v ( S ∪ T )
= v ( S ∪ T (cid:48) ) for all S ⊆ N \ ( T ∪ T (cid:48) ) , 2.
v ( S ∪ T )
= v ( S ) for all S ⊆ N \ T such that S ∩ T (cid:48) (cid:54) = ∅ , 3.
v ( S ∪ T (cid:48) )
= v ( S ) for all S ⊆ N \ T (cid:48) such that S ∩ T (cid:54) = ∅ ,
then φ T ( v ) =
φ T (cid:48) ( v ) .
Axiom JSY only equates the joint Shapley values for coalitions T and T (cid:48) if they contribute identically to coalitions that they may both join, and contribute nothing to the other coalitions.
Axioms JLI, JEF and JAN are all also used in Dhamdhere et al.
Our joint null and joint symmetry notions appear to be new: they reﬂect our interest in a set of features’ contribution to a model’s predictions, so that the set’s cardinality should not play a role in determining its value.
J OINT S HAPLEY VALUES
Our main result is that there is a unique solution to axioms JLI, JNU, JEF, JAN and JSY, the joint
Shapley value.
The uniqueness is up to the k th order of explanation; we say nothing about | T | > k .
Theorem 1.
For each order of explanation k ∈ { 1 , . . . , n } , there is a unique (up to the k th order of explanation) index φ J which satisﬁes axioms JLI, JNU, JEF, JAN and JSY.
It has the form
[Equation]
for each ∅ (cid:54)
= T ⊆ N with | T | ≤ k , where ( q 0 , . . .
, q n − 1 ) uniquely solves the recursive system
[Equation]
for all r ∈ 1 , . . . , n − 1 .
When k = 1 , the joint Shapley values coincide with Shapley’s values.
When k = n , we have
[Equation]
For each k , the constants q r are non-negative and satisfy (cid:80) n − 1 s = n − k (cid:0) ns (cid:1)
q s = 1 .
Further, as the joint
Shapley value is similar in form to equation (1)’s standard Shapley value, the value of coalition T depends on its marginal contribution to other coalitions; unlike interaction indices, it does not depend on the worth of its constituent agents.
As with the Shapley value, the joint Shapley value can be seen as the worth brought by ‘arriving’ agents but, rather than arriving one at a time, they can now also arrive in coalitions.
We show here the implication of each of the joint axioms introduced above and prove Theorem 1.
It is already known that joint linearity restricts measures to be linear combinations of worths:
Lemma 1 (Grabisch & Roubens (1999a), Proposition 1) .
If φ satisﬁes JLI, then for every ∅ (cid:54) = T ⊆
N there exists a family of real constants { a ST } S ⊆ N such that for every v ∈ G N ,
[Equation]
Axiom JNU then constrains the values of the constants { a TS } :
Lemma 2.
Suppose φ satisﬁes JLI and JNU and let { a ST } S ⊆ N, ∅ (cid:54)
=
T ⊆
N be the constants from
Lemma 1.
Then for every ∅ (cid:54)
=
T ⊆ N and ∅ (cid:54) =
S ⊆ N \ T , a TS = − a TS ∪ T .
Further, for every ∅ (cid:54) =
T ⊆ N , S ⊆ N \ T , and ∅ (cid:54) =
H (cid:40) T , a TS ∪ H = 0 .
We collect proofs in the Appendix.
Combining these two lemmas yields:
Proposition 1.
Suppose φ satisﬁes JLI and JNU.
Then there exist constants { p T ( S ) } that depend on
T and S such that for every ∅ (cid:54) =
T ⊆ N and v ∈ G N
[Equation]
Now establish a condition on the { p T ( S ) } values which must be satisﬁed under JEF:
Proposition 2.
For each order of explanation k , φ satisﬁes axioms JLI, JNU and JEF if and only if for every ∅ (cid:54) =
T ⊆ N with | T | ≤ k and v ∈ G N , φ T ( v ) =
(cid:80) S ⊆
N \ T p T ( S )
[ v ( S ∪ T ) − v ( S )] with (cid:8) p T ( S ) (cid:9) satisfying
[Equation]
=
[Equation]
for all ∅ (cid:54)
=
S ⊆ N , where δ N ( S ) equals 1 if S = N and 0 otherwise.
Recall that symmetry axiom SY is strictly weaker than anonymity
axiom AN (Malawski, 2020).
This is not the case for their joint counterparts: each imposes a different constraint on the constants { p T ( S ) } , as we shall see here.
First, consider the effect of imposing JAN.
Proposition 3.
For each order of explanation k , φ satisﬁes axioms JLI, JNU, JEF and JAN if and only if for every ∅ (cid:54) =
T ⊆ N and v ∈ G N , φ T ( v ) =
(cid:80) S ⊆
N \ T p T ( S )
[ v ( S ∪ T ) − v ( S )] with (cid:8) p T ( S ) (cid:9) satisfying (4) and
[Equation]
The analogous result for JSY is:
Proposition 4.
For each order of explanation k , φ satisﬁes axioms JLI, JNU, JEF and JSY if and only if for every ∅ (cid:54) =
T ⊆ N and v ∈ G N , φ T ( v ) =
(cid:80) S ⊆
N \ T p T ( S )
[ v ( S ∪ T ) − v ( S )] with (cid:8) p T ( S ) (cid:9) satisfying (4) and
[Equation]
Combining Propositions 2–4 completes the proof of Theorem 1.
The computational complexity of deriving the ( q 0 , . .
, q n − 1 ) is O ( nk 2 ) .
Once these have been determined, the joint Shapley values can be calculated; the complexity of doing so is O (3 n ∧ (2 n n k )) .
E XPERIMENTS 4. G AME THEORETICAL
We present two game theoretical models from Dhamdhere et al.
(2020) with known ‘ground truths’.
For each, we compare the joint Shapley value, the Shapley interaction index (Grabisch & Roubens, 1999b), the generalised Shapley value (Marichal et al., 2007), the added-value index (Alshebli et al.,
, and the Shapley-Taylor interaction index (Dhamdhere et al., 2020), respectively:
[Equation]
[Table]
[Equation]
[Table]
[Equation]
[Table]
[Equation]
[Table]
[Equation]
[Table]
[Equation]
[Table]
[Equation]
[Table]
[Equation]
[Table]
[Equation]
[Table]
The
[Table]
and
[Table]
[Table] [Table]
−3 /
[Table] [Table]
[Table] [Table]
i
[Table] [Table]
[Table] [Table]
− 1 / 12 c  5 /
[Table] [Table]
[Table]
i, j 1 / 3 c 1 / 2 (4 + c ) − 1 / 6 c 1 / 3 c 1 / 18 (8 + c ) 1 / 21 (8 + c ) N 3 + c c 3 + c 3 / 4 c c 3 / 21 (3 + c )
In the n = 3 majority game, Table 1 shows that, while Shapley-Taylor assigns 0 to singletons, the joint Shapley value recognises that singletons contribute positively when joining existing coalitions.
The Shapley-Taylor and Shapley Interaction indices assign negative values to N (for discordant interaction between members); the joint Shapley rewards N for adding worth in its own right.
In the linear model with crosses, joint Shapley values’ signs can again differ from those of interaction indices: let c = − 2 , which sets φ J 1 = 0 : if i = 1 joins the empty coalition or either singleton, it adds unit worth; however, when i = 1 joins { 2 , 3 } , it subtracts unit worth.
Averaging (according to the arrival order), gives a net contribution of 0 : coalition 1 does not contribute any worth in expectation.
By contrast, the Shapley-Taylor value always (for k > 1 ) assigns value to singletons equal to their worth, as it is not (by design) capturing information about expected contributions of features.
4. T HE AI/ML ATTRIBUTION PROBLEM
Following Štrumbelj & Kononenko (2010), let f be a prediction function, and x = ( x 1 , . . .
, x n ) ∈
A an instance from the feature space.
For a set of features S ⊆ N , deﬁne the prediction difference v x ( S ) when only features in S are known, as
[Equation]
where τ ( x , z , S ) ∈
A is deﬁned as τ ( x , z , S ) i equal to x i for i ∈ S and z i otherwise.
Thus v x ( S ) is the difference between the expected prediction when only the feature values of x in S are known, and the expected prediction when no feature values are known.
Then for each T ⊆ N , φ JT ( v x ) is the contribution of features T to the prediction f ( x ) .
To estimate φ J T ( v x ) , let X be a random variable whose law coincides with the law of the number of agents already present when T arrives in the arrival interpretation (so that the law of X depends on t , the size of T ):
[Equation]
for each i ∈ { 0 , . . .
, n } and S be a random variable uniform on the set of subsets of N \ T of size
X .
Once X is sampled, we can generate the set of agents already arrived, called S , by choosing it uniformly from all subsets of N \ T of size precisely X .
Then
[Equation]
Thus, by the law of large numbers, we can estimate this expectation by taking an average of v i ( T ∪ S i ) − v i ( S i ) where S i has the same distribution as S and v i ( S ) =
f ( τ ( x , z i , S ))
− f
( z i ) , where z i is chosen uniformly from the feature space.
Refer to φ JT ( v x ) as the local joint Shapley value of features T at instance x .
We can combine local values to obtain, for each feature, a global joint Shapley value.
We do so in two ways.
The ﬁrst is the standard methodology (q.v.
Lundberg & Lee (2017)), which averages the absolute values of locals.
We introduce the second for models with exclusively binary variables by considering presence/absence of features T in x .
This presence-adjusted global joint Shapley value is deﬁned as
[Equation]
where 1 x ( T )
=1 is one if all features T are present in x , and 0 otherwise.
Covert et al.
(2020) introduced SAGE (Shapley Additive Global Importance), a more sophisticated treatment of global inﬂuence measures that maintains efﬁciency at the global level.
All experiments are run on a single Intel(R) Core(TM) i7-6820HQ CPU.
Training details and tuning parameters are provided in the accompanying code.
4.2. S IMULATED DATA
Consider three features, x 1 , x 2 and x 3 , each uniformly drawn from (0 , 1) and a dataset ofobservations.
We ﬁrst consider independently drawn features; we then investigate correlation by ﬁxing x 2 = 1 − x 1 .
We use simpliﬁed ‘ML models’, f ( x ) , to obtain exact global joint Shapley values, averaging the absolute values of local joint Shapley values derived from equation (7).
Tabledisplays the results.
[Table]
Independent variables As only x 1 inﬂuences f 1 , only coalitions including it receive value; the more diluted its role, the lower the value; x 2 and x 3 are symmetrically irrelevant.
For f 2 , x 1 and x 2 play equal roles [Table] and receive equal values (up to variance due to the sample size).
For f 2 and f 3 , { x 1 , x 2 } receives similar value [Table] and more than in f 1 , where only x 1 inﬂuenced model predictions.
Comparing f 2 and f 3 shows that values only differ for coalitions containing both x 1 and x 2 .
Correlated features f 2 now exhibits what we term a cancellation effect between x 1 and x 2 , assigning a value of zero to coalitions containing both.
Similarly, f 3 reveals an enhancement effect : values assigned to coalitions with both x 1 and x 2 are larger than in the independent case.
Enhancement and cancellation effects do not uniquely identify underlying phenomena.
To illustrate, let x 1 , x 2 and x 3 be independent Bernoulli(0.5) random variables.
Setting k = 3 and letting n → ∞ , we obtain the exact presence-adjusted global joint Shapleys in Table 3.
[Table]
The joint Shapley values containing x 1 and x 2 are larger for models f 2 and f 4 than for model f 1 , a different sort of enhancement effect from that above.
However, the joint Shapley value for x 1 is smaller in f 4 than it is in f 1 or f 2 , again a different type of cancellation effect.
Finally, joint Shapley values can provide insight into the black-box model’s structure.
If, for example, f can be decomposed as f ( x ) =
g ( x 1 ) + h ( x 2 , . . .
, x n ) with independent
Bernoulli(0.5) random variables, then the presence-adjusted global joint Shapley value of x 1 is 1 / 2 ( g (1) − g (0))
(cid:80) n − 1 s =0 (cid:0) n − 1 s (cid:1) q s = 5 / 21 , as shown in the table.
If the joint Shapley value deviates from this, we reject the decomposition, as in the case of f 4 .
4.2. B OSTON HOUSING DATA
For comparability, we follow Dhamdhere et al.
(2020) by training a random forest on the Boston housing dataset (Harrison & Rubinfeld, 1978), computing global values using the ﬁrst notion discussed above.
Table 4 presents the largest and smallest global joint Shapley and Shapley-Taylor values.
[Table]
For k = 1 , the joint Shapley values are the classical Shapley values.
The unimportance of CHAS seems to reﬂect its low variance (only 35 of 506 units back onto the Charles).
For k = 2 , LSTAT and RM still have the largest joint Shapley values, with {LSTAT, RM} having third largest.
The top pairs all involve LSTAT or RM, indicating that these variables also contribute to explaining house prices jointly with other variables.
By contrast, {LSTAT, RM} is the 16th largest Shapley-Taylor interaction value.
The k = 2 singleton joint Shapley values are fairly evenly distributed throughout the pairs; by contrast, the singleton k
=
Shapley-Taylor values outrank all the pairs.
This is consistent with our extension of axiom NU to not favour singletons.
The largest joint
Shapley value involving NOX is {NOX, LSTAT}, which is about ﬁve times as large as {NOX, DIS} and {NOX, RAD}.
This is consistent with Harrison and Rubinfeld’s observation that NOX offset
RAD and DIS, but reinforced LSTAT: knowing the values of NOX and LSTAT adds a lot of predictive power; knowing the values of NOX and RAD or DIS tends to wash out additional predictive power.
For k = 3 , singletons again dominate: the largest triple is {AGE, LSTAT, RM}, the three individually most important features.
Continuing the above analysis, the joint Shapley value of {DIS, RAD} is about 0.015, while that of {DIS, NOX, RAD} is about 0.008, and that of {DIS, LSTAT, RAD} is about 0.022.
We understand this to mean {DIS, NOX, RAD} contains the same sort of information as {DIS, RAD}, but with NOX offsetting the other variables, while LSTAT brings novel socio-economic information to the distance variables DIS and RAD, which it continues to partially offset.
Similarly, the least important joint features include the individually unimportant CHAS and its variants —- echoing the role played by RM and LSTAT at the top of the list.
While {DIS, LSTAT} has a ShapleyTaylor value of 1.33 (indicating a large interaction between the pair but not its overall importance), its joint Shapley is 0.06 (ranked 18th) — directly assigning a value of importance.
Table 4’s values are exact.
Figure 1 demonstrates the estimation procedure above, showing convergence of the sampled values as the iterations increase.
As with other sampling procedures (e.g.
Štrumbelj & Kononenko (2014)), this cuts the complexity to order n to a linear power of k ; as k → n , the complexity approaches 2 n times a polynomial in n (compared to 3 n for the exact φ J ).
[Figure]
r
[Figure] [Figure]
between consecutive φ J samples averages converges to zero; k =movie review #
4.2. M OVIE REVIEWS
We train a fully connected neural network (two hidden layers, 16 units per layer, ReLU activations) on the binary movie review classiﬁcations in Pang & Lee (2005).
From the full set of reviews, we remove a test block of 100 (picked to include those in Table 1 of Dhamdhere et al.
for analysis.
We encode reviews as the 1000 most common words in the corpus, augmented by key words in
Table 1 of Dhamdhere et al.
(2020) to aid comparison of measures, for a total of 1004.
(The DSA features are drawn from positive reviews: {won’t}, {disappointed}, {both}, {inspiring}, {a}, {crisp}, {excellent}, {youthful}, {John}, {terriﬁc}.)
Binary accuracy is typically c. 76% after four epochs.
Table 5’s local joint Shapley values are tiny as each of the 2 1004 − 1 joint features has a tiny effect on the probability of a positive review.
In the vein of Dhamdhere et al.
, we identify intuitive effects:
1. negation : the negative local joint Shapley values for {disappointed} and {be, disappointed} become positive when negated by adding {won’t} to the coalition.
enhancement : the positive local joint Shapley values for {both} and {and} are enhanced by the positive {and, both}.
context : the sign of local joint Shapley values involving {well} depend on the context; while {you, well} is positive, {left, well} is negative.
lost potential : the positive local joint Shapley values involving {fascinating} become negative when {been} is added.
adjectival : while {director} and {effort} are individually negative, they become positive when qualiﬁed with the adjectives {winning} and {terriﬁc}, respectively.
For global values, as the features are binary, we compute the presence-adjusted global joint Shapleys, ˜ φ J .
Comparing the DSA features’ global ˜ φ J across the test reviews for k = 1 and k = 2 shows
[Table]
{terriﬁc} to be the largest
[Table]
followed by {both} then {excellent}.
This reﬂects prevalence in the training set, where {terriﬁc} appears four times as often in positive reviews.
Similarly, on the negative ˜ φ J side, {John} is followed by {disappointed}: {John} appears thrice as often in negative reviews.
The largest pairs are {both, inspiring} followed by {excellent, youthful}.
All of the pairs, except {John, terriﬁc}, have a positive sign: these pairs typically do not appear outside the positive reviews from which they were drawn.
As our encoding discards sequential information, the pairs are smaller than the singletons: a coalition merely indicates co-occurrence in a review, rather than a bigram.
As the 10 DSA features are a very small subset of all 1004, we cannot asses whether JEF holds.
However, as a health check note: the average presence-adjusted local joint Shapleys over the positive reviews is about 20% larger than that over the negative reviews.
They are both negative: as these features are drawn from positive reviews, but largely absent in any given review, their effect is negative.
Further, Table 6 indicates that the ranking of single features is largely preserved as k increases from 1 to 2 : the exception is {a, crisp}: in the corpus, {crisp} is typically preceded by {a}.
Figure 2 shows the estimated values’ convergence.
[Table]
C ONCLUSIONS
The joint Shapley value directly extends Shapley’s value to measure the effect of a set of features on a model’s predictions.
Further work is needed to maintain properties like global efﬁciency (Covert et al., 2020) and to make sampling more efﬁcient (Williamson & Feng, 2020; Mitchell et al., 2021).
We believe that understanding complex models is labor intensive.
Nevertheless, we envisage a common workﬂow that computes k = 1 Shapley values, followed by k = 2 to identify strong pairwise effects; analyses for k ≥ 3 then respond to the analyst’s evolving questions about the model’s functioning.
E THICS STATEMENT
This paper contributes to the literature on explainable AI, an important component of the Fairness,
Accountability and Transparency research agenda for ethical AI.
It does not use human subjects; it only draws on publicly available datasets; the work has not been sponsored, and does not seek to promote any third organisations; none of the authors face any conﬂict of interest.
R EPRODUCIBILITY STATEMENT
Our proofs and source code are available in the accompanying supplemental material; all data are taken from the public domain.
A A PPENDIX
See attached supplemental material for proofs and source code.