SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Algorithm 4 Welford Algorithm for Mean and Variance [45]
Require:  $\mu ,s2,n,\Delta$    
1:  $n\gets n + 1$    
2:  $\Delta_1\gets \Delta -\mu$    
3:  $\mu \leftarrow \mu +\Delta /n$    
4:  $\Delta_{2}\gets \Delta -\mu$    
5:  $s2\gets s2 + \Delta_1\Delta_2$    
6: return  $\mu ,s2$
For the interested reader, we provide a more detailed view on our empirical evaluation of Section 5. We give descriptions and pseudocode of the baseline algorithms approximating the three considered interaction indices SII, STI, and FSI, formal definitions of our synthetic games, and finally further obtained results that we omitted in the main part due to space constraints.
D.1 Baseline Algorithms for SII, STI, and FSI
In this section, we describe our baseline algorithms for SII, STI and FSI. We distinguish between permutation-based approximation (SII and STI) and kernel-based approximation (FSI).
D.1.1 Permutation-based (PB) Approximation
The algorithm for SII is outlined in Algorithm 5. Note that with each permutation only  $d - s + 1$  interaction estimates of order  $s$  are updated.
Algorithm 5 Permutation-based sampling for SII for all orders up to  $s_0$  [40]  
Require: maximum interaction order  $s_0$ , interaction set  $S_{s_0}$ , budget  $K$   
1: sum[S]  $\leftarrow$  0 for all  $S \in S_{s_0}$   
2: count[S]  $\leftarrow$  0 for all  $S \in S_{s_0}$   
3: permutationCost  $\leftarrow$  0  
4: for  $s = 1, \ldots, s_0$  do  
5: permutationCost  $\leftarrow$  permutationCost +2s · (d - s + 1) ▷ Evaluate costs per permutation  
6: end for  
7: while  $K \geq$  permutationCost do  
8: π  $\leftarrow$  {i1, ..., id} random permutation of D  
9: for  $s = 1, \ldots, s_0$  do  
10: for  $m = 1, \ldots, d - s + 1$  do  
11: S  $\leftarrow$  {im, ..., im+s-1}  
12: T  $\leftarrow$  {i1, ..., im-1} the set of predecessors of im in π  
13: sum[S]  $\leftarrow$  sum[S] + δS(T) ▷ δS(T) costs 2s evaluations  
14: count[S] = count[S] + 1  
15: end for  
16: end for  
17: K  $\leftarrow$  K-permutationCost ▷ Update budget  
18: end while  
19: SII[S]  $\leftarrow$  sum[S]/count[S] for all  $S \in S_{s_0}$ .  
20: return SII
The sampling-based algorithm for top-order interactions of STI is outlined in Algorithm 6. Note that with each permutation all top-order interaction estimates can be updated. However, the update requires a significant amount (permutationCost) of model evaluations.
D.1.2 Kernel-based (KB) Approximation
Given a budget of  $K$ , we first find the sampling budget by identifying  $k_{0}$  according to Algorithm 2 with weights  $q(t) \coloneqq \mu(t)$  and subtracting the number of subsets used for the deterministic part. We then sample the remaining subsets according to  $p(T) \propto \mu(t)$  according to Algorithm 3.
Algorithm 6 Permutation-based sampling for STI for all orders up to  $s_0$  [39, 40]
Require: maximum interaction order  $s_0$ , interaction set  $\mathcal{S}_{s_0}$ , budget  $K$   
1: sum[S]  $\leftarrow 0$  for all  $S \in S_{s_0}$   
2: count[S]  $\leftarrow 0$  for all  $S \in S_{s_0}$   
3: Compute exact (trivial) lower-order interactions  
4: eval[S]  $\leftarrow 0$  for all  $S \in S_{s_0-1}$   
5: for  $S \in S_{s_0-1}$  do  
6: eval[S]  $\leftarrow \nu(S)$   
7:  $K \leftarrow K - 1$   
8: end for  
9: for  $S \in S_{s_0-1}$  do  
10: for  $L \in \mathcal{P}(S)$  do  
11: SII[S]  $\leftarrow$  SII[S] +  $(-1)^{s-l} \cdot$  eval[L]  
12: end for  
13: end for  
14: Compute sampling-based top-order interaction estimates  
15: permutationCost  $\leftarrow 2^{s_0} \cdot \binom{d}{s_0}$   
16: while  $K \geq$  permutationCost do  
17:  $\pi \leftarrow \{i_1, \ldots, i_d\}$  random permutation of  $D$   
18: for all top-order interactions  $S$  do  
19:  $i_m \leftarrow$  the leftmost element of  $S$  in  $\pi$   
20:  $T \leftarrow \{i_1, \ldots, i_{m-1}\}$  the set of predecessors of  $i_m$  in  $\pi$   
21: sum[S]  $\leftarrow$  sum[S] +  $\delta_S^\nu(T)$   
22: count[S] = count[S] + 1  
23: end for  
24:  $K \leftarrow K-$  permutationCost  
25: end while  
26: STI[S]  $\leftarrow$  sum[S]/count[S] for all top-order interactions.  
27: return STI
Given the collection of  $K$  subsets (deterministic and sampled), we solve an approximated weighted least square objective as
$$
\begin{array}{l} \mathbb {E} _ {T \sim p (T)} \left[ \left(\nu (T) - \sum_ {S \in \mathcal {S} _ {s _ {0}}} \beta (S)\right) ^ {2} \right] \\ \approx \sum_ {T \in \mathcal {T} _ {k _ {0}}} p (T) \left(\nu (T) - \sum_ { \begin{array}{c} S \in \mathcal {S} _ {s _ {0}} \\ S \subseteq T \end{array} } \beta (S)\right) ^ {2} + p (T \in \mathcal {T} _ {k _ {0}}) \mathbb {E} _ {T \sim p _ {k _ {0}} (T)} \left[ \left(\nu (T) - \sum_ { \begin{array}{c} S \in \mathcal {S} _ {s _ {0}} \\ S \subseteq T \end{array} } \beta (S)\right) ^ {2} \right], \\ \end{array}
$$
where  $k_{0}$  is found similar to SHAP-IQ and  $p_{k_0}$  is a probability distribution over  $\mathcal{T}_{k_0}$  with  $p_{k_0}(T) \propto \mu(t)$ , which is related to  $p$  with  $p_{k_0}(T) = p(T) / p(T \in \mathcal{T}_{k_0})$ . The expectation over  $p_{k_0}$  is then found by Monte Carlo integration. Approximating this objective yields a weighted sum of sampled subsets that approximates the weighted least square problem. This approximated least-square problem is then computed explicitly using
$$
\hat {I} ^ {\mathrm {F S I}} = \left(\mathbf {Z} ^ {T} \mathbf {W} \mathbf {Z}\right) ^ {- 1} \mathbf {Z} ^ {T} \mathbf {W} \mathbf {y}, \tag {3}
$$
where  $Z \in \{0,1\}^{K \times d_{s_0}}$  is a matrix that represents a binary encoding for each sampled subset where an entry in column  $S \in S_{s_0}$  is equal to one, if the subset contains  $S$  and zero otherwise. The matrix  $W \in \mathbb{R}^{K \times K}$  contains the weights for each subset on the diagonal, e.g.  $p(T)$  for subsets of the deterministic part or  $p(T \in T_{k_0}) / m$  for subsets of the sampled part, where  $m$  refers to the number of sampled subsets for Monte Carlo integration. The vector  $\mathbf{y}$  consists of all model evaluations  $\nu_0(T)$ , where  $T$  is in the collection of subsets. To include the optimization constraint, we add  $D$  to the
collection with weight set to a high positive constant (mimicking infinite). The algorithm is outlined in Algorithm 7.
Algorithm 7 Kernel-based approximation of FSI [26, 40]
Require: maximum interaction order  $s_0$ , budget  $K$ , high constant  $c_{0} >> 0$ .  
1: Weight vector  $w[T]$  with one row and column per distinct subset  $T$ .  
2: Binary subset matrix  $Z[T, S]$  with one row per distinct subset  $T$  and one column per interaction subset  $S$ .  
3: Model evaluation vector  $y[T]$  with model evaluations  $\nu(T)$  per distinct subset  $T$ .  
4: Index array:  $I$  per distinct subset  $T$   
5: Initialize constraints  
6:  $w \gets \text{APPEND}(w, c_{0})$   
7:  $Z \gets \text{APPENDROW}(Z, 1^{T})$  with  $1^{T}$  of length  $d_{s_{0}}$ .  
8:  $y \gets \text{APPEND}(y, \nu_{0}(D))$ .  
9:  $K \gets K - 1$ .  
10: Deterministic Part  
11: for  $t = 1, \ldots, d - 1$  do  
12:  $p(t) \gets q(t) / \left( \sum_{k=1}^{d-1} q(k) \binom{d}{k} \right)$   
13: end for  
14:  $k_{0} \gets \text{GETSAMPLINGORDER}(q, K)$   
15: for  $T \in \mathcal{T}_{1}$  and  $T \notin \mathcal{T}_{k_{0}}$  do  
16:  $Z \gets \text{APPENDROW}(Z, \text{BINARY}(T))$   
17:  $y \gets \text{APPEND}(y, \nu_{0}(T))$   
18:  $w \gets \text{APPEND}(w, p(t) / \binom{d}{t})$   
19:  $K \gets K - 1$   
20: end for  
21: Sampling Part  
22:  $w_{0} \gets \text{sum}(p(t))$  for  $k_{0} \leq t \leq d - k_{0}$ .  
23: for  $k = 1, \ldots, K$  do  
24:  $T \gets \text{SAMPLE}(p, k_{0})$   
25: if  $T \in I$  then  
26:  $Z \gets \text{APPENDROW}(Z, \text{BINARY}(T))$ .  
27:  $y \gets \text{APPEND}(y, \nu_{0}(T))$   
28:  $w \gets \text{APPEND}(w, 1)$   
29:  $I. ADDINDEX(T)$   
30: else  
31:  $w[I[T]] \gets w[I[T]] + 1$   
32: end if  
33: end for  
34:  $w[I[T]] \gets w[I[T]] \cdot w_{0} / K$  for all  $T \in I$   
35:  $W \gets \text{DIAG}(w)$   
36: FSI  $\longleftarrow$  SOLVEWLS(Z,W,y).  
37: return FSI
D.1.3 Computational Complexity of Baseline Methods
To evaluate one permutation for STI, the PB algorithm requires  $2^{s}$  model evaluations per interaction, i.e. in total  $\binom{d}{s_0} \cdot 2^{s_0}$  for all top-order interactions. With each evaluated permutation all interaction estimates can be updated. For lower order interactions, STI requires to compute model evaluations for all subsets with  $|S| \leq s_0$ . For SII, the complexity is  $(d - s + 1) \cdot 2^{s}$  per permutation and only interaction estimates with  $S \in \pi$  can be updated per permutation, i.e.  $d - s + 1$  interaction estimates with one permutation. This constitutes a significant drawback over the PB approximations for SV, which iterates only once through the permutation requiring  $d - 1$  evaluations to update all estimates of the SV. In contrast, as for SV, the KB approach of FSI allows to update all interaction estimates using one single model evaluation. However, the KB approach of FSI always requires to estimate all interactions with order  $s \leq s_0$  and its computational effort increases non-linear with the number of subsets used, as solving the weighted least square problem requires inverting a  $K \times d_{s_0}$  matrix.
D.2 Further Information about the Models
This section contains a detailed information about the models (SOUM, LM, and ICM) used in our experiments.
D.2.1 Sum of Unanimity Model (SOUM)
Definition D.1 (Sum of Unanimity Model (SOUM)). For  $N$  subsets  $Q_{1},\ldots ,Q_{N}\subseteq D$  and coefficients  $a_1,\dots ,a_N\in \mathbb{R}$  the sum of unanimity model (SOUM) is defined as
$$
\nu (T) := \sum_ {n = 1} ^ {N} a _ {n} \mathbf {1} (Q _ {n} \subseteq T).
$$
For SOUMs, it is possible to efficiently compute the ground-truth values for CII.
Proposition D.2 (Ground-truth values for SOUM). For a SOUM, it holds
$$
I _ {\nu} ^ {m} (S) = \sum_ {n = 1} ^ {N} a _ {n} \omega \left(q _ {n}, | S \cap Q _ {n} |\right),
$$
$$
\omega (q, r) = \sum_ {t = q} ^ {d} \sum_ {k = 0} ^ {k _ {m a x} (r)} \binom {d - q - (s - r)} {t - q - k} \binom {s - r} {k} \gamma_ {s} ^ {m} (t, k + r)
$$
and  $k_{max}(r) \coloneqq \min(t - q, s - r)$
Proof. Due to the linearity of the CII, it suffices to compute the CII for  $\nu_{Q}(T) \coloneqq \mathbf{1}(Q \subseteq T)$ . By Theorem 4.1, we have
$$
\begin{array}{l} I _ {\nu_ {Q}} ^ {m} (S) = \sum_ {T \subseteq D} \mathbf {1} (Q \subseteq T) \gamma_ {s} ^ {m} (t, | T \cap S |) \\ = \sum_ {t = q} ^ {d} \sum_ {k = 0} ^ {k _ {m a x}} \binom {d - q - (s - | S \cap Q |)} {t - q - k} \binom {s - | S \cap Q |} {k} \gamma_ {s} ^ {m} (t, k + | S \cap Q |) \\ =: \omega (q, | S \cap Q |), \\ \end{array}
$$
where we used that  $Q \cap S \subseteq T \cap S$  due to  $\mathbf{1}(Q \subseteq T)$  and  $|T \cap S| = |S \cap Q| + (|T \cap S|) \setminus (|S \cap Q|)$ , where  $k := |(T \cap S) \setminus (S \cap Q)|$  ranges from 0 to  $k_{\max} := \min(t - q, s - |S \cap Q|)$ . Since  $S \cap Q$  is fixed, we need to count the number of subsets  $T$  of size  $t$ , given  $k$ , such that  $|T \cap S| = |S \cap Q| + k$ . We count  $\binom{s-|S \cap Q|}{k}$  ways to choose subsets of elements that are not in  $S \cap Q$  but are in  $S$ . Then  $q - (s - |S \cap Q|)$  elements of  $T$  are fixed. We thus select from  $d - q - (s - |S \cap Q|)$  elements exactly  $t - q - k$  elements, as  $q$  and  $k$  elements are already contained in  $T$ .
Finally, the CII value is given as
where the weights  $\omega$  can be precomputed with  $|S\cap Q_n|\in \{0,\dots ,s\}$
D.2.2 Language Model (LM)
For a language model (LM), we use a fine-tuned version of the DistilBERT $^5$  transformer architecture [34] on movie review sentences from the original IMDB dataset [27] for sentiment analysis, i.e.  $\nu$  has values in  $[-1, 1]$ . The IMDB data stems from the dataset library [22]. In the LM, for a given sentence, different feature coalitions are computed by masking absent features in a tokenized sentence. The implementation is based on the transformers API [47].
5The model can be found at https://huggingface.co/dhlee347/distilbert-imdb.
D.2.3 Image Classifier (ICM)
The image classification model (ICM) is a ResNet18 [19] pre-trained on ImageNet [11] as provided by torch [30]. We randomly sample 50 images from ImageNet [11] and explain the prediction of the highest probability class given the original image. To obtain the prediction of different coalitions, we pre-compute super-pixels with SLIC [1, 44] to obtain a function on  $d = 14$  features and apply mean imputation on absent features.
D.3 Hardware Details and Environmental Impact of the Experiments
Running the experiments required approximately 2000 CPU hours in total. The experiments concerning the approximation quality of SHAP-IQ compared to the baselines were run on an computation cluster on hyperthreaded Intel Xeon E5-2697 v3 CPUs clocking at with  $2.6\mathrm{Ghz}$ . To further increase the efficiency of the experiments, the outputs of the LM and ICM were pre-computed given the powerset of all features. Around 1500 CPU hours were consumed for these experiments on the cluster. Before running the experiments on the cluster, the implementations were validated on a Dell XPS 15 9510 containing an Intel i7-11800H at  $2.30\mathrm{GHz}$ . For this and further small-scale experiments like the n-SII values approximately 500 CPU hours were consumed.
D.4 Further Experimental Results
This section describes the further results and experiments omitted in the main body of the work.
D.4.1 Approximation Quality of top Order Interactions.
We further compute interaction scores for  $s_0 = 1$ ,  $s_0 = 2$ ,  $s_0 = 3$ , and  $s_0 = 4$  of all three interaction indices SII, STI, and FSI on the LM. We plot the MSE and Prec@10 based on  $g = 50$  independent iterations for these settings. All results are summarized in Figure 6. Moreover, we compute interaction scores for  $s_0 = 1$ ,  $s_0 = 2$ ,  $s_0 = 3$ , and  $s_0 = 4$  of all three interaction indices SII, STI, and FSI on the ICM. The MSE and Prec@10 based on  $g = 50$  independent iterations for these settings are shown in Figure 7. Further results to the plots show in Section 5.2 for the SOUM are presented in Figure 8.
D.4.2 n-SII Estimation on Example Sentences.
We further probe the LM with randomly selected sentences from the IMDB dataset and estimate the SII scores up to order 4. For the example sentence "It is a gruesome cannibal movie. But it's not bad. If you like Hannibal, you'll love this." in Section 5.2 all orders of n-SII with  $s_0 = 1, 2, 3, 4$  are illustrated in Figure 9. Another example sentence "I have never forgot this movie. All these years and it has remained in my life." with all orders of n-SII up to the maximum interaction order  $s_0 = 1, 2, 3, 4$  is shown in Figure 10. Lastly, for four sentences n-SII for all orders with  $s_0 = 4$  are visualized in Figure 11.
Figure 6: Approximation Quality for LM with interaction order  $s_0 = 4$  for  $g = 50$  iterations (first row), with interaction order  $s_0 = 3$  for  $g = 50$  iterations (second row), with interaction order  $s_0 = 2$  for  $g = 50$  iterations (third row), and with interaction order  $s_0 = 1$  (Shapley Value) for  $g = 50$  iterations (fourth row).
Figure 7: Approximation Quality for ICM with interaction order  $s_0 = 4$  for  $g = 50$  iterations (first row), with interaction order  $s_0 = 3$  for  $g = 50$  iterations (second row), with interaction order  $s_0 = 2$  for  $g = 50$  iterations (third row), and with interaction order  $s_0 = 1$  (Shapley Value) for  $g = 50$  iterations (fourth row).
Figure 8: Approximation Quality for SOUM order  $s_0 = 2$  (first row) and  $s_0 = 1$  (Shapley value, second row) for  $g = 50$  iterations on the SOUM with  $N = 100$  interactions,  $d = 30$  features, and  $\ell_{\mathrm{max}} = 30$ .
Figure 9: Estimated SII values with orders  $s = 1,2,3,4$  for the sentence "It is a gruesome cannibal movie. But it's not bad. If you like Hannibal, you'll love this." ( $d = 23$ ) provided to the LM. The plots show all orders of n-SII with maximum interaction order  $s_0 = 1$  (top left),  $s_0 = 2$  (top right),  $s_0 = 3$  (bottom left), and  $s_0 = 4$  (bottom right).
Figure 10: Estimated SII values with orders  $s = 1,2,3,4$  for the sentence "I have never forgot this movie. All these years and it has remained in my life." ( $d = 16$ ) provided to the LM. The plots show all orders of n-SII with maximum interaction order  $s_0 = 1$  (top left),  $s_0 = 2$  (top right),  $s_0 = 3$  (bottom left), and  $s_0 = 4$  (bottom right).
Figure 11: Estimated n-SII values with  $s_0 = 4$  for the sentences provided to the LM: "Absolutely nothing is redeeming about this total piece of trash." ( $d = 12$ , top left), "I actually liked the ending even though it did not make a lot of sense." ( $d = 15$ , top right), "However, I really liked the ending so much, I actually smiled and cried tears of joy. I felt good." with ( $d = 19$ , bottom left), and "It is a gruesome cannibal movie. But it's not bad. If you like Hannibal, you'll love this." ( $d = 23$ , bottom right).
E Further theoretical Results for the Shapley Value
In this section, we give two additional results for the special case of the SV. We explicitly state the inverse of the covariance matrix from [8] and further present a simplified representation of SHAP-IQ, if applied on the SV, which aligns with Theorem 4.4.
E.1 Explicit Inverse of Covariance Matrix of Unbiased KernelSHAP
The covariance matrix has been explicitly computed in [8, Appendix A]. In this section, we provide the explicit form of the inverse  $A^{-1}$ . This inverse admits the same structure and is a central element in the explicit calculation of U-KSH as a weighted sum, which is linked to SHAP-IQ.
Proposition E.1 (Explicit covariance matrix [8]). For the covariance matrix it holds
$$
A := \mathbb {E} [ Z Z ^ {T} ] = \mu_ {2} \mathbf {J} + (\mu_ {1} - \mu_ {2}) \mathbf {I},
$$
with  $i,j\in D$  and constants
$$
A _ {i i} := \mu_ {1} := \mathbb {P} (Z _ {i} = 1) = \frac {1}{2} \quad A _ {i j} := \mu_ {2} := \mathbb {P} (Z _ {i} = Z _ {j} = 1) = \frac {1}{d (d - 1)} \frac {\sum_ {k = 2} ^ {d - 1} \frac {k - 1}{d - k}}{\sum_ {k = 1} ^ {d - 1} \frac {1}{k (d - k)}}.
$$
Proof. The proof is given in [8, Appendix A].
We have for  $\sum_{k=1}^{d-1} \frac{1}{k(d-k)} = 2h_{d-1}$ , where  $h_n$  is the  $n$ -th harmonic number. Furthermore  $\mu_1 - \mu_2 = \frac{1}{2h_{d-1}}$ . We now give the explicit form of  $A^{-1}$ .
Proposition E.2 (Explicit inverse of covariance matrix). Let  $A \coloneqq \mathbb{E}[ZZ^T]$ . Then, we have an explicit form of the inverse  $A^{-1}$  as
$$
A ^ {- 1} = \tilde {\mu} _ {2} \mathbf {J} + (\tilde {\mu} _ {1} - \tilde {\mu} _ {2}) \mathbf {I}
$$
with constants
$$
(A ^ {- 1}) _ {i i} := \tilde {\mu} _ {1} = 2 h _ {d - 1} \frac {\mu_ {1} + (d - 2) \mu_ {2}}{\mu_ {1} + (d - 1) \mu_ {2}} \qquad (A ^ {- 1}) _ {i j} := \tilde {\mu} _ {2} = 2 h _ {d - 1} \frac {- \mu_ {2}}{\mu_ {1} + (d - 1) \mu_ {2}}
$$
Proof. The proof follows directly from Proposition E.1 and Lemma B.2.  $\square$
E.2 Simplified Representation of SHAP-IQ for the Shapley Value
In this section, give an explicit form of SHAP-IQ for the SV that admits a similar form as the SV representation in Theorem 4.4. We consider the SV weights  $m(t) \coloneqq \frac{(d - t - 1)!t!}{d!}$ .
Proposition E.3 (SHAP-IQ for SV). For SHAP-IQ with  $p(T) \propto \mu(t)$  and sampling order  $k_0 = 1$ , it holds
$$
\hat {I} _ {1} ^ {m} (i) = c _ {1} (i) + \frac {2 h _ {d - 1}}{K} \sum_ {k = 1} ^ {K} \nu_ {0} (T _ {k}) \left[ {\bf 1} (i \in T _ {k}) - \frac {t _ {k}}{d} \right],
$$
where  $h_n$  is the  $n$ -th harmonic number.
Proof. Recall the definition of SHAP-IQ of order 1
$$
\hat {I} _ {1} ^ {m} (i) := c _ {1} (i) + \frac {1}{K} \cdot \sum_ {k = 1} ^ {K} \nu (T _ {k}) \frac {\gamma_ {s} ^ {m} \left(t _ {k} , \mid T _ {k} \cap \{i \} \mid\right)}{p \left(T _ {k}\right)}.
$$
with  $p(T_k) \coloneqq \mu(t_k) / R \propto \mu(t_k)$ . We proceed by rewriting  $\gamma_s^m(t_k, |T_k \cap \{i\}|) = \gamma_s^m(t_k, \mathbf{1}(i \in T_k))$  for  $T_k \in \mathcal{T}_1$  as
$$
\begin{array}{l} \gamma_ {s} ^ {m} \left(t _ {k}, \mathbf {1} (i \in T _ {k})\right) = (- 1) ^ {1 - \mathbf {1} (i \in T _ {k})} \frac {(d - t - 1 + \mathbf {1} (i \in T _ {k})) ! (t - \mathbf {1} (i \in T _ {k})) !}{d !} \\ = \mu (t) \frac {1}{d} [ \mathbf {1} (i \in T _ {k}) (d - t _ {k}) - \mathbf {1} (i \notin T _ {k}) t _ {k} ] \\ = \mu (t) \left[ \mathbf {1} (i \in T _ {k}) - \frac {t _ {k}}{d} \right]. \\ \end{array}
$$
$$
\begin{array}{l} \hat {I} _ {1} ^ {m} (i) := c _ {1} (i) + \frac {1}{K} \cdot \sum_ {k = 1} ^ {K} \nu \left(T _ {k}\right) \frac {\mu (t) \left[ \mathbf {1} \left(i \in T _ {k}\right) - \frac {t _ {k}}{d} \right]}{p \left(T _ {k}\right)} \\ = c _ {1} (i) + \frac {R}{K} \cdot \sum_ {k = 1} ^ {K} \nu \left(T _ {k}\right) \left[ \mathbf {1} \left(i \in T _ {k}\right) - \frac {t _ {k}}{d} \right]. \\ \end{array}
$$
For the normalizing constant, we have
$$
R = \sum_ {T \in \mathcal {T} _ {1}} \mu (t) = \sum_ {t = 1} ^ {d - 1} \mu (t) \left( \begin{array}{c} d \\ t \end{array} \right) = \sum_ {t = 1} ^ {d - 1} \frac {d}{t (d - t)} = \sum_ {t = 1} ^ {d - 1} \left(\frac {1}{t} + \frac {1}{d - t}\right) = 2 h _ {d - 1},
$$
F Approximation Methods for the SV
There are two prominent representations of the SV, which are used for sampling-based approximation. Both allow to update all SVs simultaneously with one sample as well as maintaining the efficiency property.
F.1 Permutation-based (PB) Approximation
Permutation-based (PB) approximation was introduced for SV [5]. It is based on the observation that the marginal contributions  $\delta_i^\nu (T) = \nu (T\cup \{i\}) - \nu (T)$  can be computed by using permutations  $\pi \in \mathfrak{S}_D$  of  $D$  and  $\nu (u_{i}^{-}(\pi)) - \nu (u_{i}^{+}(\pi))$ , where  $u_{i}^{-}(\pi), u_{i}^{+}(\pi)$  are the sets that consist of all elements preceding  $i$  in  $\pi$  with and without  $i$ , respectively. For each subset  $T\subseteq D\setminus \{i\}$  of size  $t$  there are exactly  $t!(d - t - 1)!$  permutations with  $T = u_{i}^{-}(\pi)$  and thus
$$
I ^ {\mathrm {S V}} (i) = \frac {1}{d !} \sum_ {\pi \in \mathfrak {S} _ {D}} \delta_ {i} ^ {\nu} (u _ {S} ^ {-} (\pi)) = \mathbb {E} _ {\pi \sim \mathrm {u n i f} (\mathfrak {S} _ {D})} [ \delta_ {i} ^ {\nu} (u _ {S} ^ {-} (\pi)) ].
$$
This expectation can be efficiently approximated by sampling  $\pi \sim \mathrm{unif}(\mathfrak{S}_D)$  and using a Monte Carlo estimate for the expectation. As  $\sum_{i\in D}\delta_i^\nu (u_i^- (\pi)) = \nu (D) - \nu (\emptyset)$  for arbitrary permutations  $\pi$ , the efficiency constraint is maintained throughout the sampling procedure. The Monte Carlo estimates allows to apply well-established statistical results to obtain bounds on the approximation error [4].
F.2 Kernel-based (KB) Approximation
Kernel Shapley Additive Explanation Values [26], short KernelSHAP (KSH), and Unbiased KernelSHAP (U-KSH) [8] make use of the representation of the SV as the solution to a constrained quadratic optimization problem [6]
$$
I ^ {\mathrm {S V}} = \arg \min  _ {\beta} \sum_ {T \in \mathcal {T} _ {1}} \mu (t) \left(\nu_ {0} (T) - \sum_ {i \in T} \beta_ {i}\right) ^ {2} \tag {4}
$$
$$
\text {s . t .} \sum_ {i \in D} \beta_ {i} = \nu_ {0} (D)
$$
with  $\nu_0(T) \coloneqq \nu(T) - \nu(\emptyset)$ ,  $\mathcal{T}_k \coloneqq \{T \subseteq D : k \leq t \leq d - k\}$  and  $\mu(t) \coloneqq \frac{1}{d-1} \binom{d-2}{t-1}^{-1}$ . This quadratic optimization problem can be solved explicitly using the weighted least square solution
$$
I ^ {\mathrm {S V}} = \left(\mathbf {Z} ^ {T} \mathbf {W} \mathbf {Z}\right) ^ {- 1} \mathbf {Z} ^ {T} \mathbf {W} \mathbf {y}, \tag {5}
$$