SlowGuess/NeurIPS_2023_OCR huggingface dataset:
$$
\begin{array}{r} \pmb {\Sigma} _ {n} = \left[ \begin{array}{c c c c} \frac {1}{n} \sum_ {i} \pi (\mathbf {z} _ {i} ^ {\prime}) (\mathbf {z} _ {i 1} ^ {\prime}) ^ {2} + \frac {\lambda}{n} & \frac {1}{n} \sum_ {i} \pi (\mathbf {z} _ {i} ^ {\prime}) \mathbf {z} _ {i 1} ^ {\prime} \mathbf {z} _ {i 2} ^ {\prime} & \dots & \frac {1}{n} \sum_ {i} \pi (\mathbf {z} _ {i} ^ {\prime}) \mathbf {z} _ {i, 1} ^ {\prime} \mathbf {z} _ {i d} ^ {\prime} \\ \frac {1}{n} \sum_ {i} \pi (\mathbf {z} _ {i} ^ {\prime}) \mathbf {z} _ {i 1} ^ {\prime} \mathbf {z} _ {i 2} ^ {\prime} & \frac {1}{n} \sum_ {i} \pi (\mathbf {z} _ {i} ^ {\prime}) (\mathbf {z} _ {i 2} ^ {\prime}) ^ {2} + \frac {\lambda}{n} & \dots & \frac {1}{n} \sum_ {i} \pi (\mathbf {z} _ {i} ^ {\prime}) \mathbf {z} _ {i 2} ^ {\prime} \mathbf {z} _ {i d} ^ {\prime} \\ \vdots & \vdots & \ddots & \vdots \\ \frac {1}{n} \sum_ {i} \pi (\mathbf {z} _ {i} ^ {\prime}) \mathbf {z} _ {i 1} ^ {\prime} \mathbf {z} _ {i d} ^ {\prime} & \frac {1}{n} \sum_ {i} \pi (\mathbf {z} _ {i} ^ {\prime}) \mathbf {z} _ {i 2} ^ {\prime} \mathbf {z} _ {i d} ^ {\prime} & \dots & \frac {1}{n} \sum_ {i} \pi (\mathbf {z} _ {i} ^ {\prime}) (\mathbf {z} _ {i d} ^ {\prime}) ^ {2} + \frac {\lambda}{n} \end{array} \right] \end{array}
$$
By taking  $n\to \infty$  , we have
$$
\boldsymbol {\Sigma} _ {n} \rightarrow \boldsymbol {\Sigma} = \left(\alpha_ {1} - \alpha_ {2}\right) \mathbf {I} + \alpha_ {2} \mathbf {1 1} ^ {\top}
$$
$$
\begin{array}{l} \alpha_ {1} = \mathbb {E} _ {\mathbf {z} ^ {\prime} \sim \operatorname {U n i} (\{0, 1 \} ^ {d})} [ \pi (\mathbf {z} ^ {\prime}) z _ {i} ^ {\prime} ] = \mathbb {E} _ {\mathbf {z} ^ {\prime} \sim \operatorname {U n i} (\{0, 1 \} ^ {d})} [ \pi (\mathbf {z} ^ {\prime}) (z _ {i} ^ {\prime}) ^ {2} ] \\ = \sum_ {k = 0} ^ {d} e ^ {(k - d) / \sigma^ {2}} \mathbb {P} \left(z _ {i} ^ {\prime} = 1 | \| \mathbf {z} ^ {\prime} \| _ {0} = k\right) \mathbb {P} (\| \mathbf {z} ^ {\prime} \| _ {0} = k) \\ = \sum_ {k = 0} ^ {d} e ^ {(k - d) / \sigma^ {2}} \frac {k \left(\frac {d}{k}\right)}{d 2 ^ {d}} \\ = \sum_ {k = 0} ^ {d} e ^ {(k - d) / \sigma^ {2}} \frac {\binom {d - 1} {k - 1}}{2 ^ {d}} \\ = \sum_ {k = 0} ^ {d} e ^ {(k - 1) / \sigma^ {2}} e ^ {(1 - d) / \sigma^ {2}} \frac {\binom {d - 1} {k - 1}}{2 ^ {d}} \\ = e ^ {(1 - d) / \sigma^ {2}} \frac {\left(1 + e ^ {\frac {1}{\sigma^ {2}}}\right) ^ {d - 1}}{2 ^ {d}} = \frac {\left(1 + e ^ {- \frac {1}{\sigma^ {2}}}\right) ^ {d - 1}}{2 ^ {d}} \\ \end{array}
$$
$$
\begin{array}{l} \alpha_ {2} = \mathbb {E} _ {\mathbf {z} ^ {\prime} \sim \operatorname {U n i} (\{0, 1 \} ^ {d})} [ \pi (\mathbf {z} ^ {\prime}) z _ {i} ^ {\prime} z _ {j} ^ {\prime} ] \\ = \frac {1}{Z} \sum_ {k = 0} ^ {d} e ^ {(k - d) / \sigma^ {2}} \mathbb {P} \left(z _ {i} ^ {\prime} = 1, z _ {j} ^ {\prime} = 1 | \| \mathbf {z} ^ {\prime} \| _ {0} = k\right) \mathbb {P} \left(\| \mathbf {z} ^ {\prime} \| _ {0} = k\right) \\ = \sum_ {k = 0} ^ {d} e ^ {(k - d) / \sigma^ {2}} \frac {k (k - 1)}{d (d - 1)} \frac {\binom {d} {k}}{2 ^ {d}} \\ = \sum_ {k = 0} ^ {d} e ^ {(k - d) / \sigma^ {2}} \frac {\binom {d - 2} {k - 2}}{2 ^ {d}} \\ = \sum_ {k = 0} ^ {d} e ^ {(k - 2) / \sigma^ {2}} e ^ {(2 - d) / \sigma^ {2}} \frac {\binom {d - 2} {k - 2}}{2 ^ {d}} \\ = e ^ {(2 - d) / \sigma^ {2}} \frac {\left(1 + e ^ {\frac {1}{\sigma^ {2}}}\right) ^ {d - 2}}{2 ^ {d}} = \frac {\left(1 + e ^ {- \frac {1}{\sigma^ {2}}}\right) ^ {d - 2}}{2 ^ {d}} \\ \end{array}
$$
By Sherman-Morrison formula, we have
$$
\begin{array}{l} \boldsymbol {\Sigma} ^ {- 1} = \left(\left(\alpha_ {1} - \alpha_ {2}\right) \mathbf {I} + \alpha_ {2} \mathbf {1} \mathbf {1} ^ {\top}\right) ^ {- 1} = \frac {1}{\alpha_ {1} - \alpha_ {2}} \left(\mathbf {I} + \frac {\alpha_ {2}}{\alpha_ {1} - \alpha_ {2}} \mathbf {1} \mathbf {1} ^ {\top}\right) ^ {- 1} \\ = \frac {1}{\alpha_ {1} - \alpha_ {2}} (\mathbf {I} - \frac {\frac {\alpha_ {2}}{\alpha_ {1} - \alpha_ {2}} \mathbf {1 1} ^ {\top}}{1 + \frac {\alpha_ {2}}{\alpha_ {1} - \alpha_ {2}} d}) = (\beta_ {1} - \beta_ {2}) \mathbf {I} + \beta_ {2} \mathbf {1 1} ^ {\top} \\ \end{array}
$$
$$
\beta_ {1} = \frac {\alpha_ {1} + (d - 2) \alpha_ {2}}{(\alpha_ {1} - \alpha_ {2}) (\alpha_ {1} + (d - 1) \alpha_ {2})}, \quad \beta_ {2} = - \frac {\alpha_ {2}}{(\alpha_ {1} - \alpha_ {2}) (\alpha_ {1} + (d - 1) \alpha_ {2})}
$$
In the following, we aim to establish the concentration of  $\hat{\mathbf{w}}^{\mathrm{LIME}}$ .
Concentration of  $\Sigma_{n}$ . Considering  $0 \leq \pi(\cdot) \leq 1$  and  $\mathbf{z}_i \in \{0,1\}^d$ , each element within  $\Sigma_{n}$  resides within the interval of  $[0,2]$ . Moreover, as
$$
\frac {1}{2 ^ {d}} \leq \alpha_ {1} = \frac {(1 + e ^ {- \frac {1}{\sigma^ {2}}}) ^ {d - 1}}{2 ^ {d}} \leq \frac {2 ^ {d - 1}}{2 ^ {d}} = \frac {1}{2}
$$
$$
\frac {1}{2 ^ {d}} \leq \alpha_ {2} = \frac {(1 + e ^ {- \frac {1}{\sigma^ {2}}}) ^ {d - 2}}{2 ^ {d}} \leq \frac {2 ^ {d - 2}}{2 ^ {d}} = \frac {1}{4}
$$
$$
\frac {e ^ {- 1 / \sigma^ {2}}}{2 ^ {d}} \leq \alpha_ {1} - \alpha_ {2} = e ^ {- \frac {1}{\sigma^ {2}}} \frac {(1 + e ^ {- \frac {1}{\sigma^ {2}}}) ^ {d - 2}}{2 ^ {d}} \leq \frac {1}{4}
$$
The elements within  $\pmb{\Sigma}$  are within the range of  $[0, \frac{1}{4}]$ . Consequently, the elements in  $\pmb{\Sigma}_n - \pmb{\Sigma}$  fall within the range of  $[- \frac{1}{4}, 2]$ .
Referring to the matrix Hoeffding's inequality [31], it holds true that for all  $t > 0$
$$
\mathbb {P} (\| \pmb {\Sigma} _ {n} - \pmb {\Sigma} \| _ {2} \geq t) \leq 2 d \exp (- \frac {n t ^ {2}}{3 2 d ^ {2}})
$$
Bounding  $\| \Sigma^{-1}\| _F^2$
$$
\| \boldsymbol {\Sigma} ^ {- 1} \| _ {F} ^ {2} = d \beta_ {1} ^ {2} + (d ^ {2} - d) \beta_ {2} ^ {2}
$$
$$
\frac {d}{2 ^ {d}} \leq \alpha_ {1} + (d - 1) \alpha_ {2} \leq \frac {2 + (d - 1)}{4} = \frac {d + 1}{4}
$$
$$
\frac {d - 1}{2 ^ {d}} \leq \alpha_ {1} + (d - 2) \alpha_ {2} \leq \frac {2 + (d - 2)}{4} = \frac {d}{4}
$$
$$
\left| \beta_ {1} \right| = \left| \frac {\alpha_ {1} + (d - 2) \alpha_ {2}}{(\alpha_ {1} - \alpha_ {2}) (\alpha_ {1} + (d - 1) \alpha_ {2})} \right| \leq \left| \frac {1}{\alpha_ {1} - \alpha_ {2}} \right| \leq 2 ^ {d} e ^ {1 / \sigma^ {2}}, \beta_ {1} ^ {2} \leq 2 ^ {2 d} e ^ {2 / \sigma^ {2}}
$$
$$
| \beta_ {2} | = \left| - \frac {\alpha_ {2}}{(\alpha_ {1} - \alpha_ {2}) (\alpha_ {1} + (d - 1) \alpha_ {2})} \right| = \left| e ^ {1 / \sigma^ {2}} \frac {1}{(\alpha_ {1} + (d - 1) \alpha_ {2})} \right| \leq d ^ {- 1} 2 ^ {d} e ^ {1 / \sigma^ {2}},
$$
$$
\beta_ {2} ^ {2} \leq d ^ {- 2} 2 ^ {2 d} e ^ {2 / \sigma^ {2}}
$$
$$
\| \pmb {\Sigma} ^ {- 1} \| _ {F} ^ {2} = d \beta_ {1} ^ {2} + (d ^ {2} - d) \beta_ {2} ^ {2} \leq d 2 ^ {2 d} e ^ {2 / \sigma^ {2}} + (d ^ {2} - d) d ^ {- 2} 2 ^ {2 d} e ^ {2 / \sigma^ {2}} \leq 2 d 2 ^ {2 d} e ^ {2 / \sigma^ {2}}
$$
Concentration of  $\Gamma_{n}$ . With  $|f|\leq 1$ , all elements within both  $\Gamma_{n}$  and  $\Gamma$  exist within the range of [0, 1]. According to matrix Hoeffding's inequality [31], for all  $t > 0$ ,
$$
\mathbb {P} (\| \boldsymbol {\Gamma} _ {n} - \boldsymbol {\Gamma} \| \geq t) \leq 2 d \exp \left(- \frac {n t ^ {2}}{8 d}\right)
$$
Concentration of  $\hat{\mathbf{w}}^{\mathrm{LIME}}$ . When  $\| \boldsymbol{\Sigma}^{-1}(\boldsymbol{\Sigma}_n - \boldsymbol{\Sigma})\| \leq 0.32$  [8], we have
$$
\| \boldsymbol {\Sigma} _ {n} ^ {- 1} \boldsymbol {\Gamma} _ {n} - \boldsymbol {\Sigma} ^ {- 1} \boldsymbol {\Gamma} \| \leq 2 \| \boldsymbol {\Sigma} ^ {- 1} \| _ {F} \| \boldsymbol {\Gamma} _ {n} - \boldsymbol {\Gamma} \| _ {2} + 2 \| \boldsymbol {\Sigma} ^ {- 1} \| _ {F} ^ {2} \| \boldsymbol {\Gamma} \| \| \boldsymbol {\Sigma} _ {n} - \boldsymbol {\Sigma} \|
$$
$$
\| \boldsymbol {\Sigma} ^ {- 1} (\boldsymbol {\Sigma} _ {n} - \boldsymbol {\Sigma}) \| \leq \| \boldsymbol {\Sigma} ^ {- 1} \| \| \boldsymbol {\Sigma} _ {n} - \boldsymbol {\Sigma} \| \leq 2 ^ {1 / 2} d ^ {1 / 2} 2 ^ {d} e ^ {1 / \sigma^ {2}} \| \boldsymbol {\Sigma} _ {n} - \boldsymbol {\Sigma} \|
$$
Exploiting the concentration of  $\Sigma_{n}$ , where  $n \geq n_1 = 2^7 d^3 2^{2d} e^{2/\sigma^2} \log(4d/\delta)$  and  $t = t_1 = 5^{-2} 2^{2.5} d^{-0.5} 2^{-d} e^{-1/\sigma^2}$ , we have
$$
\mathbb {P} (\| \pmb {\Sigma} _ {n} - \pmb {\Sigma} \| _ {2} \geq t) \leq 2 d \exp \left(- \frac {n t ^ {2}}{3 2 d ^ {2}}\right) \leq 2 d \exp \left(- \frac {n t ^ {2}}{3 2 d ^ {2}}\right) \leq \frac {\delta}{2}
$$
Therefore, with a probability of at least  $1 - \frac{\delta}{2}$ , we have
$$
\| \boldsymbol {\Sigma} ^ {- 1} (\boldsymbol {\Sigma} _ {n} - \boldsymbol {\Sigma}) \| \leq \| \boldsymbol {\Sigma} ^ {- 1} \| \| \boldsymbol {\Sigma} _ {n} - \boldsymbol {\Sigma} \| \leq 2 ^ {1 / 2} d ^ {1 / 2} 2 ^ {d} e ^ {1 / \sigma^ {2}} \| \boldsymbol {\Sigma} _ {n} - \boldsymbol {\Sigma} \| \leq 0. 3 2
$$
For  $n \geq n_2 = 2^8\epsilon^{-2}d^22^{2d}e^{2/\sigma^2}\log(4d/\delta)$  and  $t_2 = 2^{-2.5}d^{-0.5}2^{-d}e^{-1/\sigma^2}\epsilon$ , the following concentration inequality holds:
$$
\mathbb {P} (\| \boldsymbol {\Gamma} _ {n} - \boldsymbol {\Gamma} \| \geq t _ {2}) \leq 2 d \exp \left(- \frac {n _ {2} t _ {2} ^ {2}}{8 d}\right) \leq \frac {\delta}{2}
$$
In this context, with a probability of at least  $1 - \frac{\delta}{2}$ , we have
$$
\| \boldsymbol {\Sigma} ^ {- 1} \| \| \boldsymbol {\Gamma} _ {n} - \boldsymbol {\Gamma} \| \leq \frac {\epsilon}{4}
$$
Considering  $\| \Gamma \| \leq \sqrt{d}$ , we select  $n \geq n_3 = 2^9\epsilon^{-2}d^52^{4d}e^{4 / \sigma^2}\log (4d / \delta)$  and  $t_3 = 2^{-3}d^{-1.5}2^{-2d}e^{-2 / \sigma^2}\epsilon$ , leading to
$$
\mathbb {P} (\| \pmb {\Sigma} _ {n} - \pmb {\Sigma} \| _ {2} \geq t _ {3}) \leq 2 d \exp \left(- \frac {n _ {3} t _ {3} ^ {2}}{3 2 d ^ {2}}\right) \leq 2 d \exp \left(- \frac {n _ {3} t _ {3} ^ {2}}{3 2 d ^ {2}}\right) \leq \frac {\delta}{2}
$$
With a probability at least  $1 - \delta /2$  , we have
$$
\| \boldsymbol {\Sigma} ^ {- 1} \| ^ {2} \| \boldsymbol {\Gamma} \| \| \boldsymbol {\Sigma} _ {n} - \boldsymbol {\Sigma} \| \leq \frac {\epsilon}{4}
$$
In summary, we choose  $n \geq \max \{n_1, n_2, n_3\}$ , and then for all  $\epsilon > 0, \delta \in (0,1)$
$$
\mathbb {P} \left(\left\| \boldsymbol {\Sigma} _ {n} ^ {- 1} \boldsymbol {\Gamma} _ {n} - \boldsymbol {\Sigma} ^ {- 1} \boldsymbol {\Gamma} \right\| \geq \epsilon\right) \leq \delta
$$
B.5 Proof of Theorem 4.2 and Corollary 4.3
Theorem B.2. Suppose  $\mathbf{z}'\sim \mathcal{P}$  such that the largest eigenvalue of  $\mathbf{z}'(\mathbf{z}')^\top$  is bounded by  $R$  and  $\mathbb{E}[\mathbf{z}'(\mathbf{z}')^\top ] = (\alpha_1 - \alpha_2)\mathbf{I} + \alpha_2\mathbf{1}\mathbf{1}^\top$ ,  $\| Var(\mathbf{z}'(\mathbf{z}')^\top)\|_2\leq \nu^2$ ,  $|(z'f(\mathbf{z}))_i|\leq M$  for some  $M > 0$ .  $\{\mathbf{z}_i'\}_{i = 1}^n$  are i.i.d. samples from  $\mathcal{P}$  and are used to compute GLIME explanation  $\hat{\mathbf{w}}^{\mathrm{GLIME}}$ . For any  $\epsilon >0,\delta \in (0,1)$ , if  $n = \Omega (\epsilon^{-2}M^{2}\nu^{2}d^{3}\gamma^{4}\log (4d / \delta))$  where  $\gamma^2 = d\beta_1^2 +(d^2 -d)\beta_2^2,\beta_1 = (\alpha_1+$ $(d - 2)\alpha_{2}) / \beta_{0},\beta_{2} = -\alpha_{2} / \beta_{0},\beta_{0} = (\alpha_{1} - \alpha_{2})(\alpha_{1} + (d - 1)\alpha_{2}))$ , we have  $\mathbb{P}(\| \hat{\mathbf{w}}^{\mathrm{GLIME}} - \mathbf{w}^{\mathrm{GLIME}}\|_2 < \epsilon)\geq 1 - \delta$ .  $\mathbf{w}^{\mathrm{GLIME}} = \lim_{n\to \infty}\hat{\mathbf{w}}^{\mathrm{GLIME}}$ .
Proof. The proof closely resembles that of Theorem 4.1. Employing the same derivation, we deduce that:
$$
\boldsymbol {\Sigma} = \left(\alpha_ {1} + \lambda - \alpha_ {2}\right) \mathbf {I} + \alpha_ {2} \mathbf {1 1} ^ {\top}, \quad \boldsymbol {\Sigma} ^ {- 1} = \left(\beta_ {1} - \beta_ {2}\right) \mathbf {I} + \beta_ {2} \mathbf {1 1} ^ {\top}
$$
$$
\beta_ {1} = \frac {\alpha_ {1} + \lambda + (d - 2) \alpha_ {2}}{(\alpha_ {1} + \lambda - \alpha_ {2}) (\alpha_ {1} + \lambda + (d - 1) \alpha_ {2})}, \quad \beta_ {2} = - \frac {\alpha_ {2}}{(\alpha_ {1} + \lambda - \alpha_ {2}) (\alpha_ {1} + \lambda + (d - 1) \alpha_ {2})}
$$
Given that  $\lambda_{\max}(z'(z')^\top) \leq R$  and  $\|\mathrm{Var}(\mathbf{z}'(\mathbf{z}')^\top)\|_2 \leq \nu^2$ , according to the matrix Hoeffding's inequality [31], for all  $t > 0$ :
$$
\mathbb {P} \left(\left\| \boldsymbol {\Sigma} _ {n} - \boldsymbol {\Sigma} \right\| _ {2} \geq t\right) \leq 2 d \exp \left(- \frac {n t ^ {2}}{8 \nu^ {2}}\right)
$$
Applying Hoeffding's inequality coordinate-wise, we obtain:
$$
\mathbb {P} (\| \boldsymbol {\Gamma} _ {n} - \boldsymbol {\Gamma} \| \geq t) \leq 2 d \exp \left(- \frac {n t ^ {2}}{8 M ^ {2} d ^ {2}}\right)
$$
$$
\left\| \boldsymbol {\Sigma} ^ {- 1} \right\| _ {F} ^ {2} = d \beta_ {1} ^ {2} + (d ^ {2} - d) \beta_ {2} ^ {2} = \gamma^ {2}
$$
By selecting  $n \geq n_1 = 2^5\gamma^2\nu^2\log (4d / \delta)$  and  $t_1 = 2^3 5^{-2}\gamma^{-1}$ , we obtain
$$
\mathbb {P} (\| \boldsymbol {\Sigma} _ {n} - \boldsymbol {\Sigma} \| _ {2} \geq t _ {1}) \leq 2 d \exp \left(- \frac {n _ {1} t _ {1} ^ {2}}{8 \nu^ {2}}\right) \leq \frac {\delta}{2}
$$
with a probability of at least  $1 - \delta / 2$ .
$$
\left\| \boldsymbol {\Sigma} ^ {- 1} (\boldsymbol {\Sigma} _ {n} - \boldsymbol {\Sigma}) \right\| \leq \left\| \boldsymbol {\Sigma} ^ {- 1} \right\| \cdot \left\| \boldsymbol {\Sigma} _ {n} - \boldsymbol {\Sigma} \right\| \leq \gamma t _ {1} = 0. 3 2
$$
Letting  $n \geq n_2 = 2^5 \epsilon^{-2} M^2 d^2 \gamma^2 \log (4d / \delta)$  and  $t_2 = 2^{-2} \epsilon \gamma^{-1}$ , we have
$$
\mathbb {P} (\| \boldsymbol {\Gamma} _ {n} - \boldsymbol {\Gamma} \| \geq t _ {2}) \leq 2 d \exp \left(- \frac {n _ {2} t _ {2} ^ {2}}{8 M ^ {2} d ^ {2}}\right) \leq \frac {\delta}{2}
$$
$$
\| \boldsymbol {\Sigma} \| \| \boldsymbol {\Gamma} _ {n} - \boldsymbol {\Gamma} \| \leq \gamma t _ {2} \leq \frac {\epsilon}{4}
$$
As  $\| \mathbf{T} \| \leq M$ , by choosing  $n \geq n_3 = 2^5 \epsilon^{-2} M^2 \nu^2 d \gamma^4 \log(4d / \delta)$  and  $t_3 = 2^{-2} \epsilon M^{-1} d^{-0.5} \gamma^{-2}$ , we have
$$
\mathbb {P} (\| \pmb {\Sigma} _ {n} - \pmb {\Sigma} \| _ {2} \geq t _ {3}) \leq 2 d \exp \left(- \frac {n _ {3} t _ {3} ^ {2}}{2 \nu^ {2}}\right) \leq \frac {\delta}{2}
$$
and with a probability of at least  $1 - \delta /2$
$$
\| \boldsymbol {\Sigma} ^ {- 1} \| ^ {2} \| \boldsymbol {\Gamma} \| \| \boldsymbol {\Sigma} _ {n} - \boldsymbol {\Sigma} \| \leq \gamma^ {2} M d ^ {0. 5} t _ {3} = \frac {\epsilon}{4}
$$
Therefore, by choosing  $n = \max \{n_1,n_2,n_3\}$  , we have
Corollary B.3. Suppose  $\{\mathbf{z}_i^{\prime}\}_{i = 1}^{n}$  are i.i.d. samples from  $\mathbb{P}(\mathbf{z}',\| \mathbf{z}'\| _0 = k) = e^{k / \sigma^2} / (1 + e^{1 / \sigma^2})^d,k = 1,\ldots ,d$  are used to compute GLIME-BINOMIAL explanation. For any  $\epsilon >0,\delta \in (0,1)$ , if  $n = \Omega (\epsilon^{-2}d^{5}e^{4 / \sigma^{2}}\log (4d / \delta))$ , we have  $\mathbb{P}(\| \hat{\mathbf{w}}^{Binomial} - \mathbf{w}^{Binomial}\|_2 <   \epsilon)\geq 1 - \delta .\mathbf{w}^{Binomial} = \lim_{n\to \infty}\hat{\mathbf{w}}^{Binomial}$ .
Proof. For GLIME-BINOMIAL, each coordinate of  $\mathbf{z}'(\mathbf{z}')^\top$  follows a Bernoulli distribution, ensuring the bounded variance of both  $\mathbf{z}'(\mathbf{z}')^\top$  and  $(\mathbf{z}'f(\mathbf{z}'))_i$ . Additionally, we have
$$
\| \boldsymbol {\Gamma} \| \leq \sqrt {d},
$$
$$
\begin{array}{l} \alpha_ {1} = \mathbb {E} \left[ (z _ {i} ^ {2}) ^ {\prime} \right] = \mathbb {E} [ z _ {i} ^ {\prime} ] = \frac {e ^ {1 / \sigma^ {2}}}{1 + e ^ {1 / \sigma^ {2}}} \\ = \sum_ {k = 0} ^ {d} \mathbb {P} \left(z _ {i} ^ {\prime} = 1 | \| \mathbf {z} ^ {\prime} \| _ {0} = k\right) \mathbb {P} \left(\| \mathbf {z} ^ {\prime} \| _ {0} = k\right) \\ = \sum_ {k = 0} ^ {d} \frac {k}{d} \frac {\binom {d} {k} e ^ {k / \sigma^ {2}}}{(1 + e ^ {1 / \sigma^ {2}}) ^ {d}} \\ = \sum_ {k = 0} ^ {d} \frac {\binom {d - 1} {k - 1} e ^ {k / \sigma^ {2}}}{(1 + e ^ {1 / \sigma^ {2}}) ^ {d}} \\ = \frac {\left(1 + e ^ {1 / \sigma^ {2}}\right) ^ {d - 1}}{\left(1 + e ^ {1 / \sigma^ {2}}\right) ^ {d}} e ^ {1 / \sigma^ {2}} = \frac {e ^ {1 / \sigma^ {2}}}{1 + e ^ {1 / \sigma^ {2}}} \\ \end{array}
$$
$$
\begin{array}{l} \alpha_ {2} = \mathbb {E} \left[ z _ {i} ^ {\prime} z _ {j} ^ {\prime} \right] = \frac {e ^ {1 / \sigma^ {2}}}{1 + e ^ {1 / \sigma^ {2}}} \\ = \sum_ {k = 0} ^ {d} \mathbb {P} \left(z _ {i} ^ {\prime} = 1, z _ {j} ^ {\prime} = 1 | \| \mathbf {z} ^ {\prime} \| _ {0} = k\right) \mathbb {P} \left(\| \mathbf {z} ^ {\prime} \| _ {0} = k\right) \\ = \sum_ {k = 0} ^ {d} \frac {k (k - 1)}{d (d - 1)} \frac {\binom {d} {k} e ^ {k / \sigma^ {2}}}{(1 + e ^ {1 / \sigma^ {2}}) ^ {d}} \\ = \sum_ {k = 0} ^ {d} \frac {\binom {d - 2} {k - 2} e ^ {k / \sigma^ {2}}}{(1 + e ^ {1 / \sigma^ {2}}) ^ {d}} \\ = \frac {(1 + e ^ {1 / \sigma^ {2}}) ^ {d - 2}}{(1 + e ^ {1 / \sigma^ {2}}) ^ {d}} e ^ {2 / \sigma^ {2}} = \frac {e ^ {2 / \sigma^ {2}}}{(1 + e ^ {1 / \sigma^ {2}}) ^ {2}} = \alpha_ {1} ^ {2} \\ \end{array}
$$
$$
\begin{array}{l} | \beta_ {1} | ^ {2} = | \frac {\alpha_ {1} + \lambda + (d - 2) \alpha_ {2}}{(\alpha_ {1} + \lambda - \alpha_ {2}) (\alpha_ {1} + \lambda + (d - 1) \alpha_ {2})} | ^ {2} \\ \leq \left| \frac {1}{\alpha_ {1} + \lambda - \alpha_ {2}} \right| \leq \frac {1}{\left| \alpha_ {1} - \alpha_ {2} \right|} = e ^ {- 1 / \sigma^ {2}} \left(1 + e ^ {1 / \sigma^ {2}}\right) ^ {2} \leq 4 e ^ {1 / \sigma^ {2}} \\ \end{array}
$$
$$
\begin{array}{l} | \beta_ {2} | ^ {2} = | - \frac {\alpha_ {2}}{(\alpha_ {1} + \lambda - \alpha_ {2}) (\alpha_ {1} + \lambda + (d - 1) \alpha_ {2})} | ^ {2} \\ \leq \frac {\alpha_ {2} ^ {2}}{(\alpha_ {1} - \alpha_ {2}) (\alpha_ {1} + \lambda + (d - 1) \alpha_ {2}) ^ {2}} \\ = \frac {\alpha_ {1} \alpha_ {2}}{(1 - \alpha_ {1}) (\alpha_ {1} + (d - 1) \alpha_ {2}) ^ {2}} \\ \leq \frac {\alpha_ {1} \alpha_ {2}}{(1 - \alpha_ {1}) ((d - 1) \alpha_ {2}) ^ {2}} = \frac {e ^ {- 1 / \sigma^ {2}} (1 + e ^ {1 / \sigma^ {2}}) ^ {2}}{(d - 1) ^ {2}} \leq \frac {2 ^ {2} e ^ {1 / \sigma^ {2}}}{(d - 1) ^ {2}} \\ \end{array}
$$
$$
d \beta_ {1} ^ {2} + (d ^ {2} - d) \beta_ {2} ^ {2} \leq d e ^ {1 / \sigma^ {2}} + e ^ {1 / \sigma^ {2}} \frac {d}{d - 1} \leq d e ^ {1 / \sigma^ {2}}
$$
B.6 Formulation of SmoothGrad
Proposition B.4. SmoothGrad is equivalent to GLIME formulation with  $\mathbf{z} = \mathbf{z}' + \mathbf{x}$  where  $\mathbf{z}' \sim \mathcal{N}(\mathbf{0}, \sigma^2\mathbf{I})$ ,  $\ell(f(\mathbf{z}), g(\mathbf{z}')) = (f(\mathbf{z}) - g(\mathbf{z}'))^2$  and  $\pi(\mathbf{z}) = 1$ ,  $\Omega(\mathbf{v}) = 0$ .
The explanation returned by GLIME for  $f$  at  $x$  with infinitely many samples under the above setting is
$$
\mathbf {w} ^ {*} = \frac {1}{\sigma^ {2}} \mathbb {E} _ {\mathbf {z} ^ {\prime} \sim \mathcal {N} (\mathbf {0}, \sigma^ {2} \mathbf {I})} [ \mathbf {z} ^ {\prime} f (\mathbf {z} ^ {\prime} + \mathbf {x}) ] = \mathbb {E} _ {\mathbf {z} ^ {\prime} \sim \mathcal {N} (\mathbf {0}, \sigma^ {2} \mathbf {I})} [ \nabla f (\mathbf {x} + \mathbf {z} ^ {\prime}) ]
$$
which is exactly SmoothGrad explanation. When  $\sigma \to 0$ ,  $\mathbf{w}^{*} \to \nabla f(\mathbf{x} + \mathbf{z})|_{\mathbf{z} = 0}$ .
Proof. To establish this proposition, we commence by deriving the expression for the GLIME explanation vector  $\mathbf{w}^*$ .
Exact Expression of  $\Sigma$ : For each  $i = 1, \dots, n$ , let  $\mathbf{z}_i' \sim \mathcal{N}(\mathbf{0}, \sigma^2\mathbf{I})$ . In this context,
$$
\hat {\boldsymbol {\Sigma}} _ {n} = \left[ \begin{array}{c c c} \frac {1}{n} \sum_ {k} (z _ {k 1} ^ {2}) ^ {\prime} & \dots & \frac {1}{n} \sum_ {k} z _ {l 1} ^ {\prime} z _ {k d} ^ {\prime} \\ \vdots & \ddots & \vdots \\ \frac {1}{n} \sum_ {k} z _ {k d} ^ {\prime} z _ {k 1} ^ {\prime} & \dots & \frac {1}{n} \sum_ {k} (z _ {k d} ^ {2}) ^ {\prime} \end{array} \right]
$$
$$
\boldsymbol {\Sigma} = \mathbb {E} _ {\mathbf {z} ^ {\prime} \sim \mathcal {N} (\mathbf {0}, \sigma^ {2} \mathbf {I})} [ \mathbf {z} ^ {\prime} (\mathbf {z} ^ {\prime}) ^ {\top} ] = \left[ \begin{array}{c c c} \sigma^ {2} & \dots & 0 \\ \vdots & \ddots & \vdots \\ 0 & \dots & \sigma^ {2} \end{array} \right]
$$
$$
\boldsymbol {\Sigma} ^ {- 1} = \left[ \begin{array}{c c c} \frac {1}{\sigma^ {2}} & \dots & 0 \\ \vdots & \ddots & \vdots \\ 0 & \dots & \frac {1}{\sigma^ {2}} \end{array} \right]
$$
$$
\mathbf {w} ^ {*} = \boldsymbol {\Sigma} ^ {- 1} \boldsymbol {\Gamma} = \frac {1}{\sigma^ {2}} \mathbb {E} _ {\mathbf {z} ^ {\prime} \sim \mathcal {N} (\mathbf {0}, \sigma^ {2} \mathbf {I})} [ \mathbf {z} ^ {\prime} f (\mathbf {x} + \mathbf {z} ^ {\prime}) ] = \mathbb {E} _ {\mathbf {z} ^ {\prime} \sim \mathcal {N} (\mathbf {0}, \sigma^ {2} \mathbf {I})} [ \nabla f (\mathbf {x} + \mathbf {z} ^ {\prime}) ]
$$
The final equality is a direct consequence of Stein's lemma [17].