SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Note that this resembles eq. (D.0.1), only with coefficients that are shared between all subsets with the same cardinality. Due to each attribution being a linear combination of the predictions, we can associate a matrix  $A \in \mathbb{R}^{d \times 2^d}$  with each semvalue, similar to Proposition 1. Furthermore, we can let  $A$  be a linear combination of basis elements corresponding to each cardinality. We define the basis matrix  $A^{(k)} \in \mathbb{R}^{d \times 2^d}$  for each cardinality  $k = 1, \ldots, d$  as follows:
$$
A _ {i S} ^ {(k)} = \left\{ \begin{array}{l l} \binom {d - 1} {| S | - 1} ^ {- 1} & \text {i f} i \in S \text {a n d} | S | = k \\ - \binom {d - 1} {| S |} ^ {- 1} & \text {i f} i \notin S \text {a n d} | S | = k - 1 \\ 0 & \text {o t h e r w i s e .} \end{array} \right.
$$
With this, we can write the matrix associated with any semvalue as a function of the cardinality weights  $\alpha$ :
$$
A (\alpha) = \sum_ {k = 1} ^ {d} \alpha_ {k} A ^ {(k)}.
$$
An important insight is that by composing a linear function with the convex spectral norm function [11], we can see that  $\| A(\alpha)\| _2$  is a convex function of  $\alpha$ . This lets us find both the upper and lower bound of  $\| A(\alpha)\| _2$  subject to  $\alpha$  being a valid probability distribution.
For the lower bound, we refer readers to Theorem C.3 from Wang and Jia [64], who prove that the Banzhaf value achieves the minimum with  $\alpha^{*}$  such that  $\alpha_{k}^{*} = \binom{d-1}{k-1}/2^{d-1}$  and spectral norm  $\|A(\alpha^{*})\|_{2} = 1/2^{d/2-1}$ . Notice that this is a valid probability distribution because we have  $\sum_{k=1}^{d} \binom{d-1}{k-1} = 2^{d-1}$ . Also note that the proof in [64] parameterizes  $\alpha$  slightly differently.
For the upper bound, we can leverage convexity to make the following argument. The fact that spectral norm is convex with  $\alpha$  representing probabilities lets us write the following for any  $\alpha$  value:
$$
\sigma_ {\max } \left(A (\alpha)\right) = \sigma_ {\max } \left(\sum_ {k = 1} ^ {d} \alpha_ {k} \cdot A ^ {(k)}\right) \leq \sum_ {k = 1} ^ {d} \alpha_ {k} \cdot \sigma_ {\max } \left(A ^ {(k)}\right) \leq \max  _ {k} \left\{\sigma_ {\max } \left(A ^ {(k)}\right) \right\}.
$$
Finding the upper bound therefore reduces to finding the basis element with the largest spectral norm. For each basis element  $A^{(k)}$ , we can derive the spectral norm by first finding the maximum eigenvalue of  $A^{(k)}A^{(k)^{\top}} \in \mathbb{R}^{d\times d}$ . In deriving this matrix, we first consider the diagonal entries:
$$
\begin{array}{l} \left(A ^ {(k)} A ^ {(k) ^ {\top}}\right) _ {i i} = \sum_ {S: i \in S, | S | = k} \binom {d - 1} {| S | - 1} ^ {- 2} + \sum_ {S: i \notin S, | S | = k - 1} \binom {d - 1} {| S |} ^ {- 2} \\ = \binom {d - 1} {k - 1} \binom {d - 1} {k - 1} ^ {- 2} + \binom {d - 1} {k - 1} \binom {d - 1} {k - 1} ^ {- 2} \\ = 2 \binom {d - 1} {k - 1} ^ {- 1}. \\ \end{array}
$$
Next, we consider the off-diagonal entries:
$$
\begin{array}{l} \left(A ^ {(k)} A ^ {(k) ^ {\top}}\right) _ {i j} = \sum_ {S: i \in S, j \in S, | S | = k} \binom {d - 1} {| S | - 1} ^ {- 2} + \sum_ {S: i \notin S, j \notin S, | S | = k - 1} \binom {d - 1} {| S |} ^ {- 2} \\ = \binom {d - 2} {k - 2} \binom {d - 1} {k - 1} ^ {- 2} + \binom {d - 2} {k - 1} \binom {d - 1} {k - 1} ^ {- 2} \\ = \frac {k - 1}{d - 1} \binom {d - 1} {k - 1} ^ {- 1} + \frac {d - k}{d - 1} \binom {d - 1} {k - 1} ^ {- 1} \\ = \left( \begin{array}{c} d - 1 \\ k - 1 \end{array} \right) ^ {- 1}. \\ \end{array}
$$
To summarize, the entries of this matrix are the following:
$$
\big (A ^ {(k)} A ^ {(k) ^ {\top}} \big) _ {i j} = \left\{ \begin{array}{l l} 2 \binom {d - 1} {k - 1} ^ {- 1} & \text {i f} i = j \\ \binom {d - 1} {k - 1} ^ {- 1} & \text {o t h e r w i s e .} \end{array} \right.
$$
We therefore see that  $A^{(k)}A^{(k)^{\top}}$  is equal to a rank-one matrix plus an identity matrix, or  $A^{(k)}A^{(k)^{\top}} = b\mathbf{1}_d\mathbf{1}_d^{\top} + (c - b)I_d$ , with values for  $b = \left(A^{(k)}A^{(k)^{\top}}\right)_{ij}$  and  $c = \left(A^{(k)}A^{(k)^{\top}}\right)_{ii}$  given above. Recognizing that  $b > 0$  and therefore that  $\mathbf{1}_d$  is the leading eigenvector, we have the following expression for the maximum eigenvalue:
$$
\begin{array}{l} \lambda_ {\max } \left(A ^ {(k)} A ^ {(k) ^ {\top}}\right) = c + (d - 1) b \\ = \left(A ^ {(k)} A ^ {(k) ^ {\top}}\right) _ {i i} + (d - 1) \cdot \left(A ^ {(k)} A ^ {(k) ^ {\top}}\right) _ {i j} \\ = (d + 1) \binom {d - 1} {k - 1} ^ {- 1}. \\ \end{array}
$$
Finally, we must consider how to maximize this as a function of  $k$ . Setting  $k = 1$  or  $k = d$  achieves the same value of  $d + 1$ , which yields the following spectral norm:
$$
\| A ^ {(d)} \| _ {2} = \sqrt {\lambda_ {\max} \big (A ^ {(d)} A ^ {(d)} ^ {\top} \big)} = \sqrt {d + 1}.
$$
The case with  $k = d$  corresponds to leave-one-out (see Lemma 7), and  $k = 1$  corresponds to an analogous leave-one-in approach that has been discussed in prior work [15, 17].
Proof. Following Monderer and Samet [46], solution concepts satisfying linearity, boundedness and efficiency are random-order values (or quasvalues). This means that the attributions are the average marginal contributions across a distribution of feature orderings. Reasoning about such orderings requires new notation, so we use  $\pi$  to denote an ordering over  $[d]$ , and  $\pi(i)$  to denote the position of index  $i$  within the ordering. We also let  $\operatorname{Pre}(\pi, i)$  denote the set of all elements preceding  $i$  in the ordering, or  $\operatorname{Pre}(\pi, i) \equiv \{j : \pi(j) < \pi(i)\}$ . Similarly, we let  $\operatorname{Pre}(\pi, i, j)$  be an indicator of whether  $j$  directly precedes  $i$  in the ordering, or  $\operatorname{Pre}(\pi, i, j) \equiv \mathbb{I}\{\pi(j) = \pi(i) - 1\}$ . Finally, we use  $\Pi$  to denote the set of all orderings, where  $|\Pi| = d!$ .
Now, given a distribution over orderings  $p(\pi)$ , we can write the attributions for any random-order value as follows:
$$
\begin{array}{l} \phi_ {i} (f, x) = \sum_ {\pi \in \Pi} p (\pi) \left(f \left(x _ {\operatorname {P r e} (\pi , i) \cup \{i \}}\right) - f \left(x _ {\operatorname {P r e} (\pi , i)}\right)\right) \\ = \sum_ {S \subseteq [ d ] \backslash \{i \}} \Big (\sum_ {\pi : \operatorname {P r e} (\pi , i) = S} p (\pi) \Big) \Big (f (x _ {S \cup \{i \}}) - f (x _ {S}) \Big). \\ \end{array}
$$
Based on this formulation, we can associate a matrix  $A \in \mathbb{R}^{d \times 2^d}$  with each random-order value based on the coefficients applied to each prediction. The entries are defined as follows:
$$
A _ {i S} = \left\{ \begin{array}{l l} \sum_ {\pi : \operatorname {P r e} (\pi , i) = S \setminus \{i \}} p (\pi) & \text {i f} i \in S \\ - \sum_ {\pi : \operatorname {P r e} (\pi , i) = S} p (\pi) & \text {o t h e r w i s e .} \end{array} \right.
$$
Furthermore, we can view  $A$  as a linear combination of basis elements  $A^{(\pi)} \in \mathbb{R}^{d \times 2^d}$  corresponding to each individual ordering. We define the basis matrix  $A^{(\pi)}$  associated with each  $\pi \in \Pi$  as follows:
$$
A _ {i S} ^ {(\pi)} = \left\{ \begin{array}{l l} 1 & \text {i f P r e} (\pi , i) = S \setminus \{i \} \\ - 1 & \text {i f P r e} (\pi , i) = S \\ 0 & \text {o t h e r w i s e .} \end{array} \right.
$$
With this, we can write the linear operator  $A$  corresponding to a distribution  $p(\pi)$  as the following linear combination:
$$
A (p) = \sum_ {\pi \in \Pi} p (\pi) \cdot A ^ {(\pi)}.
$$
By composing a linear operation with the convex spectral norm function [11], we can see that  $\| A(p)\| _2$  is a convex function of the probabilities  $p(\pi)$  assigned to each ordering. This will allow us to derive a form for both the upper and lower bound on  $\| A(p)\| _2$ .
Beginning with the upper bound, we can use convexity to write the following:
$$
\sigma_ {\max } \big (A (p) \big) = \sigma_ {\max } \Big (\sum_ {\pi \in \Pi} p (\pi) \cdot A ^ {(\pi)} \Big) \leq \sum_ {\pi \in \Pi} p (\pi) \cdot \sigma_ {\max } \big (A ^ {(\pi)} \big) \leq \max  _ {\pi \in \Pi} \Big \{\sigma_ {\max } \big (A ^ {(\pi)} \big) \Big \}.
$$
Finding the upper bound therefore reduces to finding the basis element with the largest spectral norm. For an arbitrary ordering  $\pi$ , we can derive the spectral norm via the matrix  $A^{(\pi)}A^{(\pi)^{\top}}$ , whose entries are given by:
$$
\left(A ^ {(\pi)} A ^ {(\pi) ^ {\top}}\right) _ {i j} = \left\{ \begin{array}{l l} 2 & \text {i f} i = j \\ - 1 & \text {i f} i \neq j \text {a n d} \operatorname {P r e} (\pi , i, j) \\ - 1 & \text {i f} i \neq j \text {a n d} \operatorname {P r e} (\pi , j, i) \\ 0 & \text {o t h e r w i s e .} \end{array} \right.
$$
Within these matrices, the diagonal entries are always equal to 2, most rows contain two  $-1$ 's, and two rows (corresponding to the first and last element in the ordering  $\pi$ ) contain a single  $-1$ . We can see that the matrices  $A^{(\pi)}A^{(\pi)^{\top}}$  are identical up to a permutation of the rows and columns, or that they are similar matrices. They therefore share the same eigenvalues.
Without loss of generality, we choose to focus on the ordering  $\pi'$  that places the indices in their original order, or  $\pi'(i) = i$  for  $i \in [d]$ . The corresponding matrix  $A^{(\pi')} A^{(\pi')^\top}$  has the following form:
$$
A ^ {(\pi^ {\prime})} A ^ {(\pi^ {\prime})} ^ {\top} = \left( \begin{array}{c c c c c c} 2 & - 1 & 0 & 0 & \dots & 0 \\ - 1 & 2 & - 1 & 0 & \dots & 0 \\ 0 & - 1 & 2 & - 1 & \dots & 0 \\ 0 & 0 & \ddots & \ddots & \ddots & \vdots \\ \vdots & \vdots & \ddots & - 1 & 2 & - 1 \\ 0 & \dots & \dots & 0 & - 1 & 2 \end{array} \right)
$$
This is a tridiagonal Toeplitz matrix, a type of matrix whose eigenvalues have a known closed-form solution [49]. The maximum eigenvalue is the following:
$$
\begin{array}{l} \lambda_ {\max } \left(A ^ {(\pi^ {\prime})} A ^ {(\pi^ {\prime}) ^ {\top}}\right) = 2 + 2 \sqrt {(- 1) \cdot (- 1)} \cos \left(\frac {\pi}{d + 1}\right) \\ = 2 + 2 \cdot \cos \left(\frac {\pi}{d + 1}\right). \\ \end{array}
$$
We therefore have the following upper bound on the spectral norm for a random-order value:
$$
\| A \| _ {2} \leq \sqrt {2 + 2 \cdot \cos \left(\frac {\pi}{d + 1}\right)}.
$$
This value tends asymptotically towards 2, and it is achieved by any random-order value that places all its probability mass on a single ordering. Such a single-order value approach has not been discussed in the literature, but it is a special case of a proposed variant of the Shapley value [26].
Next, we consider the lower bound. We now exploit the convexity of  $\| A(p)\| _2$  to argue that a specific distribution achieves a lower spectral norm than any other distribution.
Consider an arbitrary distribution  $p(\pi)$  over orderings, with the associated linear operator  $A(p)$  and spectral norm  $\| A(p)\| _2$ . Our proof technique is to find other distributions that result in different linear operators  $A$  that share the same spectral norm. For any ordering  $\pi^{\prime}\in \Pi$ , we can imagine permuting the distribution  $p(\pi)$  in the following sense: we generate a new distribution  $p_{\pi '}(\pi)$  such that  $p_{\pi '}(\pi) = p(\pi (\pi '))$ , where  $\pi (\pi^{\prime})$  is a modified ordering with  $\pi (\pi^{\prime})(i) = \pi (\pi^{\prime}(i))$ . That is, the probability mass is reassigned based on a permutation of the indices.
Crucially, the linear operator  $A(p_{\pi'})$  is different from  $A(p)$  but shares the same spectral norm. This is because  $A(p_{\pi'})A(p_{\pi'})^\top$  is similar to  $A(p)A(p)^\top$ , or they are equal up to a permutation of the rows and columns. We therefore have  $\| A(p_{\pi'})\|_2 = \| A(p)\|_2$ , and due to convexity we have
$$
\left\| A \left(p / 2 + p _ {\pi^ {\prime}} / 2\right) \right\| _ {2} \leq \frac {1}{2} \| A \left(p _ {\pi^ {\prime}}\right) \| _ {2} + \frac {1}{2} \| A (p) \| _ {2} = \| A (p) \| _ {2}.
$$
This does not imply that  $p / 2 + p_{\pi'} / 2$  is a minimizer, but we can now generalize this technique to identify a lower bound. Rather than generating a single permuted distribution, we propose generating all possible permuted distributions. That is, we consider the set of all  $d!$  permutations  $\pi' \in \Pi$  and generate the corresponding distributions  $\{p_{\pi'} : \pi' \in \Pi\}$ . Notice that when we take the mean of these distributions, the result is identical regardless of the original distributions  $p(\pi)$ , and that it assigns uniform probability mass to all orderings, because we have:
$$
\frac {1}{d !} \sum_ {\pi^ {\prime} \in \Pi} p _ {\pi^ {\prime}} (\pi) = \frac {1}{d !} \sum_ {\pi^ {\prime} \in \Pi} p \left(\pi^ {\prime}\right) = \frac {1}{d !}.
$$
We can then exploit convexity to write the following inequality:
$$
\left\| A \left(\sum_ {\pi^ {\prime} \in \Pi} p _ {\pi^ {\prime}} / d!\right) \right\| _ {2} \leq \| A (p) \| _ {2}.
$$
We can make this argument for any original distribution  $p$ , so this provides a global lower bound. The random-order value with uniform weighting is the Shapley value, whose spectral norm was derived in Lemma 7. We therefore conclude with the following lower bound for random-order values, which is achieved by the Shapley value:
$$
\left\| A \right\| _ {2} \geq \sqrt {\frac {2}{d}}.
$$
Proof. The result follows from the spectral norm result in Lemma 7, and the fact that the Shapley value is the only solution concept to satisfy linearity, boundedness, symmetry and efficiency [46].  $\square$
E Proofs: main results
We now re-state and prove our results from Section 4.
$$
\left\| \phi (f, x) - \phi (f, x ^ {\prime}) \right\| _ {2} \leq g (\text {r e m o v a l}) \cdot h (\text {s u m m a r y}) \cdot \| x - x ^ {\prime} \| _ {2},
$$
Proof. Given two inputs  $x$  and  $x'$ , the results in Lemma 1 and Lemma 3 imply the following bound for any feature set  $\mathbf{x}_S$ ,
$$
\left| f \left(x _ {S} ^ {\prime}\right) - f \left(x _ {S} ^ {\prime}\right) \right| \leq L ^ {\prime} \cdot \left\| x _ {S} - x _ {S} ^ {\prime} \right\| _ {2},
$$
where we have  $L' = L$  when using the baseline or marginal approach and  $L' = L + 2BM$  when using the conditional approach. This bound depends on the specific subset, we also have a bound across all subsets because  $\| x_S - x_S' \|_2 \leq \| x - x' \|_2$  for all  $S \subseteq [d]$ . In terms of the corresponding vectors  $v, v' \in \mathbb{R}^{2^d}$ , this implies the following bound:
$$
\| v - v ^ {\prime} \| _ {\infty} = \max  _ {S \subseteq [ d ]} | v _ {S} - v _ {S} ^ {\prime} | \leq L ^ {\prime} \cdot \| x - x ^ {\prime} \| _ {2}.
$$
Next, Lemma 7 shows that we have the following bound on the difference in attributions:
$$
\| \phi (f, x) - \phi (f, x ^ {\prime}) \| _ {2} = \| A v - A v ^ {\prime} \| _ {2} \leq \| A \| _ {1, \infty} \cdot \| v - v ^ {\prime} \| _ {\infty},
$$
where the norm  $\| A\|_{1,\infty}$  depends on the chosen summary technique. Combining these bounds, we arrive at the following result:
$$
\| \phi (f, x) - \phi (f, x ^ {\prime}) \| _ {2} \leq \| A \| _ {1, \infty} \cdot L ^ {\prime} \cdot \| x - x ^ {\prime} \| _ {2}.
$$
Substituting in the specific values for  $L^{\prime}$  and  $\| A\|_{1,\infty}$  completes the proof.
$$
\left\| \phi (f, x) - \phi \left(f ^ {\prime}, x\right) \right\| _ {2} \leq h (\text {s u m m a r y}) \cdot \| f - f ^ {\prime} \|,
$$
Proof. Given two models  $f$  and  $f'$ , the results in Lemmas 5 and 6 imply the following bound for any feature set  $x_{S}$ ,
$$
\left| f \left(x _ {S}\right) - f ^ {\prime} \left(x _ {S}\right) \right| \leq \| f - f ^ {\prime} \|,
$$
where the specific norm is  $\| f - f' \|_{\infty}$  when using the baseline or marginal approach and  $\| f - f' \|_{\infty, \mathcal{X}}$  when using the conditional approach. In terms of the corresponding vectors  $v, v' \in \mathbb{R}^{2^d}$ , this implies the following bound:
$$
\left\| v - v ^ {\prime} \right\| _ {\infty} = \max  _ {S \subseteq [ d ]} \left| v _ {S} - v _ {S} ^ {\prime} \right| \leq \| f - f ^ {\prime} \|.
$$
Next, Lemma 7 showed that we have the following bound on the difference in attributions:
$$
\left\| \phi (f, x) - \phi \left(f ^ {\prime}, x\right) \right\| _ {2} = \left\| A v - A v ^ {\prime} \right\| _ {2} \leq \left\| A \right\| _ {1, \infty} \cdot \left\| v - v ^ {\prime} \right\| _ {\infty},
$$
$$
\| \phi (f, x) - \phi (f ^ {\prime}, x) \| _ {2} \leq \| A \| _ {1, \infty} \cdot \| f - f ^ {\prime} \|.
$$
Substituting in the specific values for  $\| f - f \|$  and  $\| A \|_{1,\infty}$  completes the proof.
Next, we present a corollary that considers the case of simultaneous input and model perturbations. While this case has received less attention in the literature, we include this result for completeness because it is easy to analyze given our previous analysis.
Corollary 1. The robustness of removal-based explanations to simultaneous input and model perturbations is given by the following meta-formula,
$$
\left\| \phi (f, x) - \phi \left(f ^ {\prime}, x ^ {\prime}\right) \right\| _ {2} \leq g (\text {r e m o v a l}) \cdot h (\text {s u m m a r y}) \cdot \| x - x ^ {\prime} \| _ {2} + h (\text {s u m m a r y}) \cdot \| f - f ^ {\prime} \|,
$$
where the functional distance and factors associated with the removal and summary technique are given in Theorems 1 and 2.
Proof. This result follows from triangle inequality:
$$
\left\| \phi (f, x) - \phi \left(f ^ {\prime}, x ^ {\prime}\right) \right\| _ {2} \leq \left\| \phi (f, x) - \phi \left(f, x ^ {\prime}\right) \right\| _ {2} + \left\| \phi \left(f, x ^ {\prime}\right) - \phi \left(f ^ {\prime}, x ^ {\prime}\right) \right\| _ {2}.
$$
The bounds for the first and second terms are given by Theorem 1 and Theorem 2, respectively.
F The role of sampling when removing features
In this section, we present results analyzing the impact of sampling when removing features. Sampling is often required in practice to integrate across the distribution  $q(\pmb{x}_S)$  for removed features (see eq. (1)), and this can create further discrepancy between attributions for similar inputs.
F.1 Theory
Here, we provide a probabilistic bound for the difference between the model prediction  $f(x_{S})$  with a subset of features  $x_{S}$  and its estimator  $\hat{f}(x_{S})$  with  $m$  independent samples from  $q(\boldsymbol{x}_{\bar{S}})$ .
Lemma 12. Let  $\hat{f}(x_S) \equiv \frac{1}{m} \sum_{i=1}^{m} f(x_S, x_{\bar{S}}^{(i)})$  be the empirical estimator of  $f(x_S)$  in eq. (1), where  $x_{\bar{S}}^{(i)} \stackrel{i.i.d.}{\sim} q(\pmb{x}_{\bar{S}})$ . For any  $\delta \in (0,1)$  and  $\varepsilon > 0$ , if the sample size  $m$  satisfies  $m \geq \frac{2B^2 \log(2/\delta)}{\varepsilon^2}$ , then  $\mathbb{P}\left(|\hat{f}(x_S) - f(x_S)| \leq \varepsilon\right) \geq 1 - \delta$ .
Proof. With Assumption 2, we have  $-\frac{B}{m} \leq \frac{f(x_S, x_{\bar{S}}^{(i)})}{m} \leq \frac{B}{m}$ . Also,  $\mathbb{E}[\hat{f}(x_{\bar{S}})] = f(x_{\bar{S}})$  because  $x_{\bar{S}}^{(i)}$  are sampled from  $q(x_{\bar{S}})$ . Applying Hoeffding's inequality [35], we obtain the probabilistic bound
$$
\begin{array}{l} \mathbb {P} \left(\left| \hat {f} \left(x _ {S}\right) - f \left(x _ {S}\right) \right| \geq \varepsilon\right) \leq 2 \exp \left(- \frac {2 \varepsilon^ {2}}{\sum_ {i = 1} ^ {m} \left(\frac {2 B}{m}\right) ^ {2}}\right) \\ = 2 \exp \left(- \frac {m \varepsilon^ {2}}{2 B ^ {2}}\right) \leq 2 \exp \left(- \frac {\left(\frac {2 B ^ {2} \log (2 / \delta)}{\varepsilon^ {2}}\right) \varepsilon^ {2}}{2 B ^ {2}}\right) = \delta , \\ \end{array}
$$
and hence for the complement event we have
$$
\mathbb {P} \left(\left| \hat {f} \left(x _ {S}\right) - f \left(x _ {S}\right) \right| \leq \varepsilon\right) \geq 1 - \delta .
$$
Lemma 12 can be applied to bound the difference between the original and perturbed attributions when the integral in eq. (1) is estimated via sampling. For example, under input perturbations, the difference  $|\hat{f}(x_S) - \hat{f}(x_S')|$  decomposes into  $|f(x_S) - f(x_S')| + |f(x_S) - f(x_S')| + |f(x_S') - \hat{f}(x_S')|$  by the triangle inequality. The first and last terms can be bounded with high probability using Lemma 12 and the union bound, and the middle term is bounded using results from Section 3.1 (e.g., Lemma 1 or Lemma 3). Intuitively, by combining probabilistic bounds over all the feature subsets, it is possible to derive a final probabilistic upper bound with the following meta-formula:
$$
\left\| \phi (f, x) - \phi (f, x ^ {\prime}) \right\| _ {2} \leq g (\text {r e m o v a l}) \cdot h (\text {s u m m a r y}) \cdot \| x - x ^ {\prime} \| _ {2} + g _ {\text {s a m p l i n g}} (\text {r e m o v a l}), \tag {F.1.1}
$$
where  $g_{\mathrm{sampling}}(\mathrm{removal})$  is a function of  $\delta, \varepsilon, m$  when removal = marginal or conditional, and  $g_{\mathrm{sampling}}(\mathrm{baseline}) = 0$  because no sampling is needed. Similarly, for our bound under model perturbations, the term  $g_{\mathrm{sampling}}(\mathrm{removal})$  can be included to account for sampling in feature removal.
F.2 Empirical observations
In eq. (F.1.1), the sampling-dependent term  $g_{\mathrm{sampling}}(\mathrm{removal})$  does not depend on the input perturbation strength  $\| x - x' \|_2$ , so we empirically estimate  $g_{\mathrm{sampling}}(\mathrm{removal})$  with the intercept of the empirical upper bound (as illustrated in Figure 3) when  $\| x - x' \|_2 = 0$ . As shown in Figure 13 (left),  $g_{\mathrm{sampling}}(\mathrm{baseline})$  is indeed zero, whereas  $g_{\mathrm{sampling}}(\mathrm{marginal})$  and  $g_{\mathrm{sampling}}(\mathrm{conditional})$  empirically decrease with increasing sample size for feature removal as expected. We observe that the sampling error tends to be lower for the conditional approach compared to the marginal approach, potentially due to lower variances in the conditional versus marginal distributions. Finally, Figure 13 (right) confirms that the perturbation-dependent term  $g(\mathrm{removal}) \cdot h(\mathrm{summary}) \cdot \| x - x' \|_2$  does not depend on the removal sample size.
Figure 13: Intercepts and slopes of empirical and theoretical upper bounds with respect to the input perturbation norm, for Occlusion, Banzhaf, Shapley, RISE, and LIME attribution differences in the synthetic experiment in Appendix A.1, and with varying sample size for estimating feature removal.
G Feature grouping
Rather than generate attribution scores for each feature  $x_{i}$ , another option is to partition the feature set and generate group-level scores. This is common in practice for images [12, 16, 53], where removing individual pixels is unlikely to produce meaningful prediction differences, and reducing the number of features makes attributions less computationally intensive. For example, LIME pre-computes superpixels using a basic segmentation algorithm [53], and ViT Shapley uses grid-shaped patches defined by a vision transformer [16].
When computing group-level attributions, we require a set of feature index groups that we denote as  $\{G_1,\ldots ,G_g\}$ , where each group satisfies  $G_{i}\subseteq [d]$ . We assume that the  $g$  groups are non-overlapping and cover all the indices, so that we have  $G_{i}\cap G_{j} = \{\}$  for  $i\neq j$  and  $G_{1}\cup \dots \cup G_{g} = [d]$ . Under this approach, the attributions are a vector of the form  $\phi (f,x)\in \mathbb{R}^g$ , and we compute them by querying the model only with subsets that are unions of the groups. For this, we let  $T\subseteq [g]$  denote a subset of group indices and denote the corresponding union of groups as  $G_{T} = \bigcup_{i\in T}G_{i}\subseteq [d]$ . Using this notation and taking the Shapley value as an example, the attribution for the  $i$ th group  $G_{i}$  is defined as
$$
\phi_ {i} (f, x) = \frac {1}{g} \sum_ {T \subseteq [ g ] \setminus \{i \}} \binom {g - 1} {| T |} ^ {- 1} \left(f (x _ {G _ {T} \cup \{i \}}) - f (x _ {G _ {T}})\right),
$$
where  $f(x_{G_T})$  is defined based on the chosen feature removal approach.