SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Definition 8 (EPAC-PAC learnability). For any  $\delta \in (0,1),\tau >0$  the sample complexity of  $(\delta ,\tau ,\gamma)$  - EPAC learning of  $\mathcal{H}$  with respect to a surrogate loss  $\phi$  denoted  $m(\delta ,\tau ,\gamma ;\mathcal{H},\phi)$  is defined as the smallest  $m\in \mathbb{N}$  for which there exists a learning rule  $\mathcal{A}$  such that every data distribution  $\mathcal{D}_{\mathcal{X}}$  over  $\mathcal{X}$ , with probability at least  $1 - \delta$  over  $S\sim \mathcal{D}^m$
$$
\phi (\mathcal {A} (S), \mathcal {D}) \leq \inf  _ {h \in \mathcal {H}} \phi (h, \mathcal {D}) + \tau
$$
$$
\operatorname {e r r} _ {\mathcal {D}} (\mathcal {A} (S)) \leq \inf  _ {h \in \mathcal {H}} \operatorname {e r r} _ {\mathcal {D}} (h) + \gamma .
$$
If no such  $m$  exists, define  $m(\delta, \tau, \gamma; \mathcal{H}, \phi) = \infty$ . We say that  $\mathcal{H}$  is EPAC-PAC learnable in the agnostic setting with respect to a surrogate loss  $\phi$  if  $\forall \delta \in (0,1), \tau > 0$ ,  $m(\delta, \tau, \gamma; \mathcal{H}, \phi)$  is finite.
G A Generalization Bound in the Realizable Setting
In this section, we assume that we are in the doubly realizable [2] setting where there exists  $h^* \in \mathcal{H}$  such that  $\mathrm{err}_{\mathcal{D}}(h^*) = 0$  and  $\phi(h^*, \mathcal{D}) = 0$ . The optimal classifier  $h^*$  lies in  $\mathcal{H}$  and also achieve zero expected explanation loss. In this case, we want to output a hypothesis  $h$  that achieves both zero empirical risk and empirical explanation risk.
Theorem G.1 (Generalization bound for the doubly realizable setting). For a hypothesis class  $\mathcal{H}$ , a distribution  $\mathcal{D}$  and an explanation loss  $\phi$ . Assume that there exists  $h^* \in \mathcal{H}$  that  $\mathrm{err}_{\mathcal{D}}(h^*) = 0$  and  $\phi(h^*, \mathcal{D}) = 0$ . Let  $S = \{(x_1, y_1), \ldots, (x_n, y_n)\}$  is drawn i.i.d. from  $\mathcal{D}$  and  $S_E = \{x_1', \ldots, x_k'\}$  drawn i.i.d. from  $\mathcal{D}_X$ . With probability at least  $1 - \delta$ , for any  $h \in \mathcal{H}$  that  $\mathrm{err}_S(h) = 0$  and  $\phi(h, S_E) = 0$ , we have
$$
\operatorname {e r r} _ {D} (h) \leq R _ {n} (\mathcal {H} _ {\phi , \varepsilon_ {k}}) + \sqrt {\frac {\ln (2 / \delta)}{2 n}}
$$
$$
\varepsilon_ {k} = 2 R _ {k} (\mathcal {G}) + \sqrt {\frac {\ln (2 / \delta)}{2 k}}
$$
when  $\mathcal{G} = \{\phi (h,x)\mid h\in \mathcal{H},x\in \mathcal{X}\}$
Proof. We first consider only classifiers that have low empirical explanation loss and then perform standard supervised learning. From Rademacher-based uniform convergence, for any  $h \in \mathcal{H}$ , with probability at least  $1 - \delta / 2$  over  $S_{E}$
$$
\phi (h, \mathcal {D}) \leq \phi (h, S _ {E}) + 2 R _ {k} (\mathcal {G}) + \sqrt {\frac {\ln (2 / \delta)}{2 k}}
$$
when  $\mathcal{G} = \{\phi(h, x) \mid h \in \mathcal{H}, x \in \mathcal{X}\}$ . Therefore, for any  $h \in \mathcal{H}$  with  $\phi(h, S_E) = 0$ , we have  $h \in \mathcal{H}_{\phi, \varepsilon_k}$  with probability at least  $1 - \delta/2$ . Now, we can apply the uniform convergence on  $\mathcal{H}_{\phi, \varepsilon_k}$ . For any  $h \in \mathcal{H}_{\phi, \varepsilon_k}$  with  $\mathrm{err}_S(h) = 0$ , with probability at least  $1 - \delta/2$ , we have
$$
\operatorname {e r r} _ {\mathcal {D}} (h) \leq R _ {n} (\mathcal {H} _ {\phi , \varepsilon_ {k}}) + \sqrt {\frac {\ln (2 / \delta)}{2 n}}.
$$
Therefore, for  $h \in \mathcal{H}$  that  $\phi(h, S_E) = 0$ ,  $\operatorname{err}_S(h) = 0$ , we have our desired guarantee.
We remark that, since our result relies on the underlying techniques of the Rademacher complexity, our result is on the order of  $O\left(\frac{1}{\sqrt{n}}\right)$ . In the (doubly) realizable setting, this is somewhat loose, and more complicated techniques are required to produce tighter bounds. We leave this as an interesting direction for future work.
H Rademacher Complexity of Linear Models with a Gradient Constraint
We calculate the empirical Rademacher complexity of a linear model under a gradient constraint.
Figure 7: Illustration of different value of a function  $f(v)$ .
Theorem H.1 (Empirical Rademacher complexity of linear models with a gradient constraint). Let  $\mathcal{X}$  be an instance space in  $\mathbb{R}^d$ , let  $\mathcal{D}_{\mathcal{X}}$  be a distribution on  $\mathcal{X}$ , let  $\mathcal{H} = \{h : x \to \langle w_h, x \rangle \mid w_h \in \mathbb{R}^d, \|w_h\|_2 \leq B\}$  be a class of linear model with weights bounded by some constant  $B > 0$  in  $\ell_2$  norm. Assume that there exists a constant  $C > 0$  such that  $\mathbb{E}_{x \sim \mathcal{D}_{\mathcal{X}}}[\|x\|_2^2] \leq C^2$ . Assume that we have an explanation constraint in terms of gradient constraint; we want the gradient of our linear model to be close to the gradient of some linear model  $h'$ . Let  $\phi(h, x) = \theta(w_h, w_{h'})$  be an explanation surrogate loss when  $\theta(u, v)$  is an angle between  $u, v$ . For any  $S = \{x_1, \ldots, x_n\}$  is drawn i.i.d. from  $\mathcal{D}_{\mathcal{X}}$ , we have
$$
R _ {S} \left(\mathcal {H} _ {\phi , \tau}\right) = \frac {B}{n} \mathbb {E} _ {\sigma} \left[ \| v \| f (v) \right].
$$
when  $v = \sum_{i=1}^{n} x_i \sigma_i$  and
$$
f (v) = \left\{ \begin{array}{l l} 1 & \text {w h e n} \theta (v, w ^ {\prime}) \leq \tau \\ \cos (\theta (v, w ^ {\prime}) - \tau) & \text {w h e n} \tau \leq \theta (v, w ^ {\prime}) \leq \frac {\pi}{2} + \tau \\ 0 & \text {w h e n} \theta (v, w ^ {\prime}) \geq \frac {\pi}{2} + \tau . \end{array} \right.
$$
For the proof, we refer to Appendix H for full proof. We compare this with the standard bound on linear models which is given by
$$
R _ {S} (\mathcal {H}) = \frac {B}{n} \mathbb {E} _ {\sigma} [ \| v \| ].
$$
Figure 8: Benefits of an explanation constraint also depend on the data distribution. We represent data points  $x_{i}$  with red squares (Left). The possible regions for  $v = \sum_{i=1}^{n} x_{i} \sigma_{i}$  are the shaded areas (Right). When the data is highly correlated with  $w'$ ,  $v$  would lie in a region where  $f(v)$  is large (Top) and this implies that the gradient constraints provide less benefits. On the other hand, when the data is almost orthogonal to  $w'$ ,  $v$  would lie in a region with a small value of  $f(v)$  (Bottom) which leads to more benefits from the gradient constraints
The benefits of the explanation constraints depend on the underlying data distribution; in the case of linear models with a gradient constraint, this depends on an angle between  $v = \sum_{i=1}^{n} x_i \sigma_i$  and  $w'$ . The explanation constraint reduces the term inside the expectation by a factor of  $f(v)$  depending on  $\theta(v, w')$ . When  $\theta(v, w') \leq \tau$  then  $f(v) = 1$  which implies that there is no reduction. The value of  $f(v)$  decreases as the angle between  $\theta(v, w')$  increases and reaches  $f(v) = 0$  when  $\theta(v, w') \geq \frac{\pi}{2} + \tau$ . When the data is concentrated around the area of  $w'$ , the possible regions for  $v$  would be close to  $w'$  or  $-w'$  (Figure 8 (Top)). The value of  $f(v)$  in this region would be either 1 or 0 and the reduction would be  $\frac{1}{2}$  on average. In essence, this means that the gradient constraint of being close to  $w'$  does not actually tell us much information beyond the information from the data distribution. On the other hand, when the data points are nearly orthogonal to  $w'$ , the possible regions for  $v$  would lead to a small  $f(v)$  (Figure 8 (Bottom)). This can lead to a large reduction in complexity. Intuitively, when the data is nearly orthogonal to  $w'$ , there are many valid linear models including those not close in angle to  $w'$ . The constraints allow us to effectively shrink down the class of linear models that are close to  $w'$ .
Proof. (Proof of Theorem H.1) Recall that  $\mathcal{H}_{\phi, \tau} = \{h : x \to \langle w_h, x \rangle \mid w_h \in \mathbb{R}^d, \|w_h\|_2 \leq B, \theta(w_h, w_{h'}) \leq \tau\}$ . For a set of sample  $S$ , the empirical Rademacher complexity of  $\mathcal{H}_{\phi, \tau}$  is given by
$$
\begin{array}{l} R _ {S} \left(\mathcal {H} _ {\phi , \tau}\right) = \frac {1}{n} \mathbb {E} _ {\sigma} \left[ \sup  _ {h \in \mathcal {H} _ {\phi , \tau}} \sum_ {i = 1} ^ {n} h \left(x _ {i}\right) \sigma_ {i} \right] \\ = \frac{1}{n}\mathbb{E}_{\sigma}\left[\sup_{\substack{\| w_{h}\|_{2}\leq B\\ \theta (w_{h},w_{h^{\prime}})\leq \tau}}\sum_{i = 1}^{n}\langle w_{h},x_{i}\rangle \sigma_{i}\right] \\ = \frac{1}{n}\mathbb{E}_{\sigma}\left[\sup_{\substack{\| w_{h}\|_{2}\leq B\\ \theta (w_{h},w_{h^{\prime}})\leq \tau}}\langle w_{h},\sum_{i = 1}^{n}x_{i}\sigma_{i}\rangle \right]. \\ \end{array}
$$
For a vector  $w' \in \mathbb{R}^d$  with  $\| w' \|_2 = 1$ , and a vector  $v \in \mathbb{R}^d$ , we will claim the following,
1. If  $\theta(v, w') \leq \tau$ , we have
$$
\sup_{\substack{\| w\|_{2}\leq B\\ \theta (w,w^{\prime})\leq \tau}}\langle w,v\rangle = B\| v\| .
$$
2. If  $\frac{\pi}{2} +\tau \leq \theta (v,w^{\prime})\leq \pi$  , we have
$$
\sup_{\substack{\| w\|_{2}\leq B\\ \theta (w,w^{\prime})\leq \tau}}\langle w,v\rangle = 0.
$$
3. If  $\tau \leq \theta(v, w') \leq \frac{\pi}{2} + \tau$ , we have
$$
\sup_{\substack{\| w\|_{2}\leq B\\ \theta (w,w^{\prime})\leq \tau}}\langle w,v\rangle = B\| v\| \cos (\theta (v,w^{\prime}) - \tau)
$$
For the first claim, we can see that if  $\theta(v, w') \leq \tau$ , we can pick  $w = \frac{Bv}{\|\boldsymbol{v}\|}$  and achieve the optimum value. For the second claim, we use the fact that  $\theta(\cdot, \cdot)$  satisfies a triangle inequality and for any  $w$  that  $\theta(w, w') \leq \tau$ , we have
$$
\theta (v, w) + \theta (w, w ^ {\prime}) \geq \theta (v, w ^ {\prime})
$$
$$
\theta (v, w) \geq \theta (v, w ^ {\prime}) - \theta (w, w ^ {\prime})
$$
$$
\theta (v, w) \geq \frac {\pi}{2} + \tau - \tau = \frac {\pi}{2}.
$$
This implies that for any  $w$  that  $\theta(w, w') \leq \tau$ , we have  $\langle w, v \rangle = \|w\| \|v\| \cos(\theta(v, w)) \leq 0$  and the supremum is given by 0 where we can set  $\|w\| = 0$ . For the third claim, we know that  $\langle w, v \rangle$  is maximum when the angle between  $v, w$  is the smallest. From the triangle inequality above, we must have  $\theta(w, w') = \tau$  to be the largest possible value so that we have the smallest lower bound  $\theta(v, w) \geq \theta(v, w') - \theta(w, w')$ . In addition, the inequality holds when  $v, w'$ ,  $w$  lie on the same plane. Since we do not have further restrictions on  $w$ , there exists such  $w$  and we have
as required. One can calculate a closed form formula for  $w$  by solving a quadratic equation. Let  $w = \frac{B\tilde{w}}{\|\tilde{w}\|}$  when  $\tilde{w} = v + \lambda w'$  for some constant  $\lambda > 0$  such that  $\theta(w, w') = \tau$ . With this we have an equation
$$
\frac {\langle \tilde {w} , w ^ {\prime} \rangle}{\| \tilde {w} \|} = \cos (\tau)
$$
$$
\frac {\langle v + \lambda w ^ {\prime} , w ^ {\prime} \rangle}{\| v + \lambda w ^ {\prime} \|} = \cos (\tau)
$$
Let  $\mu = \langle v, w' \rangle$ , solving for  $\lambda$ , we have
$$
\frac {\mu + \lambda}{\sqrt {\| v \| ^ {2} + 2 \lambda \mu + \lambda^ {2}}} = \cos (\tau)
$$
$$
\mu^ {2} + 2 \mu \lambda + \lambda^ {2} = \cos^ {2} (\tau) \left(\| v \| ^ {2} + 2 \lambda \mu + \lambda^ {2}\right)
$$
$$
\sin^ {2} (\tau) \lambda^ {2} + 2 \sin^ {2} (\tau) \mu \lambda + \mu^ {2} - \cos^ {2} (\tau) \| v \| ^ {2} = 0
$$
$$
\lambda^ {2} + 2 \mu \lambda + \frac {\mu^ {2}}{\sin^ {2} (\tau)} - \cot^ {2} (\tau) \| v \| ^ {2} = 0
$$
Solve this quadratic equation, we have
$$
\lambda = - \mu \pm \cot (\tau) \sqrt {\| v \| ^ {2} - \mu^ {2}}.
$$
Since  $\lambda > 0$ , we have  $\lambda = -\mu + \cot(\tau) \sqrt{\|v\|^2 - \mu^2}$ . We have
$$
\begin{array}{l} \tilde {w} = v + \lambda w ^ {\prime} \\ = v + (- \mu + \cot (\tau) \sqrt {\| v \| ^ {2} - \mu^ {2}}) w ^ {\prime} \\ = v - \langle v, w ^ {\prime} \rangle w ^ {\prime} + \cot (\tau) w ^ {\prime} \sqrt {\| v \| ^ {2} - \mu^ {2}}. \\ \end{array}
$$
With these claims, we have
$$
\begin{array}{l} R_{S}(\mathcal{H}_{\phi ,\tau}) = \frac{1}{n}\mathbb{E}_{\sigma}\left[\sup_{\substack{\| w_{h}\|_{2}\leq B\\ \theta (w_{h},w_{h^{\prime}})\leq \tau}}\langle w_{h},\sum_{i = 1}^{n}x_{i}\sigma_{i}\rangle \right] \\ = \frac {B}{n} \mathbb {E} _ {\sigma} \left[ \| v \| 1 \left\{\theta \left(v, w ^ {\prime}\right) \leq \tau \right\} + \| v \| 1 \left\{\tau \leq \theta \left(v, w ^ {\prime}\right) \leq \frac {\pi}{2} + \tau \right\} \cos \left(\theta \left(v, w ^ {\prime}\right) - \tau\right) \right] \\ = \frac {B}{n} \mathbb {E} _ {\sigma} \left[ \| v \| f (v) \right]. \\ \end{array}
$$
Theorem H.2 (Rademacher complexity of linear models with gradient constraint, uniform distribution on a sphere). Let  $\mathcal{X}$  be an instance space in  $\mathbb{R}^d$ , let  $\mathcal{D}_{\mathcal{X}}$  be a uniform distribution on a unit sphere in  $\mathbb{R}^d$ , let  $\mathcal{H} = \{h : x \to \langle w_h, x \rangle \mid w_h \in \mathbb{R}^d, ||w_h||_2 \leq B\}$  be a class of linear model with weights bounded by some constant  $B > 0$  in  $\ell_2$  norm. Assume that there exists a constant  $C > 0$  such that  $\mathbb{E}_{x \sim \mathcal{D}_{\mathcal{X}}}[||x||_2^2] \leq C^2$ . Assume that we have an explanation constraint in terms of gradient constraint; we want the gradient of our linear model to be close to the gradient of some linear model  $h'$ . Let  $\phi(h, x) = \theta(w_h, w_{h'})$  be an explanation surrogate loss when  $\theta(u, v)$  is an angle between  $u, v$ . We have
$$
R _ {n} (\mathcal {H} _ {\phi , \tau}) = \frac {B}{\sqrt {n}} \left(\sin (\tau) \cdot p + \frac {1 - p}{2}\right),
$$
$$
p = \mathrm {e r f} \left(\frac {\sqrt {d} \sin (\tau)}{\sqrt {2}}\right).
$$
Proof. From Theorem H.1, we have that
$$
\begin{array}{l} R _ {n} \left(\mathcal {H} _ {\phi , \tau}\right) = \mathbb {E} \left[ R _ {S} \left(\mathcal {H} _ {\phi , \tau}\right) \right] \\ = \frac {B}{n} \mathbb {E} _ {\mathcal {D}} \left[ \mathbb {E} _ {\sigma} \left[ \| v \| 1 \{\theta (v, w ^ {\prime}) \leq \tau \} + \| v \| 1 \{\tau \leq \theta (v, w ^ {\prime}) \leq \frac {\pi}{2} + \tau \} \cos (\theta (v, w ^ {\prime}) - \tau) \right] \right] \\ = \frac {B}{n} \mathbb {E} _ {\mathcal {D}} \left[ \mathbb {E} _ {\sigma} \left[ \| v \| 1 \left\{\theta \left(v, w ^ {\prime}\right) \leq \frac {\pi}{2} - \tau \right\} + \| v \| 1 \left\{\frac {\pi}{2} - \tau \leq \theta \left(v, w ^ {\prime}\right) \leq \frac {\pi}{2} + \tau \right\} \cos \left(\theta \left(v, w ^ {\prime}\right) - \tau\right) \right] \right] \\ \end{array}
$$
when  $v = \sum_{i=1}^{n} x_i \sigma_i$ . Because  $x_i$  is drawn uniformly from a unit sphere, in expectation  $\theta(v, w')$  has a uniform distribution over  $[0, \pi]$  and the distribution  $\|v\|$  for a fixed value of  $\theta(v, w')$  are the same for all  $\theta(v, w') \in [0, \pi]$ . From Trigonometry, we note that
$$
\cos \left(\frac {\pi}{2} - 2 \tau + a\right) + \cos \left(\frac {\pi}{2} - a\right) = \sin (2 \tau - a) + \sin (a) \leq 2 \sin (\tau).
$$
By the symmetry property and the uniformity of the distribution of  $\theta(v, w')$  and  $\|v\|$ .
$$
\begin{array}{l} \mathbb {E} _ {\mathcal {D}} \left[ \mathbb {E} _ {\sigma} \left[ \| v \| 1 \{\frac {\pi}{2} - \tau \leq \theta (v, w ^ {\prime}) \leq \frac {\pi}{2} + \tau \} \cos (\theta (v, w ^ {\prime}) - \tau) \right] \right] \\ = \mathbb {E} _ {\mathcal {D}} \left[ \mathbb {E} _ {\sigma} \left[ \| v \| 1 \{0 \leq \theta (v, w ^ {\prime}) \leq 2 \tau \} \cos \left(\frac {\pi}{2} + \theta (v, w ^ {\prime}) - \tau\right) \right] \right] \\ = \mathbb {E} _ {\mathcal {D}} \left[ \mathbb {E} _ {\sigma} \left[ \| v \| \left(1 \{0 \leq \theta (v, w ^ {\prime}) \leq \tau \} \cos \left(\frac {\pi}{2} + \theta (v, w ^ {\prime}) - \tau\right) + 1 \{\tau \leq \theta (v, w ^ {\prime}) \leq 2 \tau \} \cos \left(\frac {\pi}{2} + \theta (v, w ^ {\prime}) - \tau\right)\right) \right] \right. \\ = \mathbb {E} _ {\mathcal {D}} \left[ \right. \mathbb {E} _ {\sigma} \left[ \right. \| v \| \left( \right.1 \{0 \leq \theta (v, w ^ {\prime}) \leq \tau \} \cos \left(\frac {\pi}{2} + \theta (v, w ^ {\prime}) - \tau\right) + 1 \{0 \leq 2 \tau - \theta (v, w ^ {\prime}) \leq \tau \} \cos \left(\frac {\pi}{2} - (2 \tau - \theta (v, w ^ {\prime})))\right)\left. \right]\left. \right. \\ = \mathbb {E} _ {\mathcal {D}} \left[ \right. \mathbb {E} _ {\sigma} \left[ \right. \| v \| \left( \right.1 \{0 \leq \theta (v, w ^ {\prime}) \leq \tau \} \cos \left(\frac {\pi}{2} + \theta (v, w ^ {\prime}) - \tau\right) + 1 \{0 \leq \tilde {\theta} (v, w ^ {\prime}) \leq \tau \} \cos \left(\frac {\pi}{2} - \tilde {\theta} (v, w ^ {\prime}))\right)\left. \right]\left. \right] \\ = \mathbb {E} _ {\mathcal {D}} \left[ \right. \mathbb {E} _ {\sigma} \left[ \right. \| v \| \left( \right.1 \{0 \leq \theta (v, w ^ {\prime}) \leq \tau \} \cos \left(\frac {\pi}{2} + \theta (v, w ^ {\prime}) - \tau\right) + 1 \{0 \leq \theta (v, w ^ {\prime}) \leq \tau \} \cos \left(\frac {\pi}{2} - \theta (v, w ^ {\prime}))\right)\left. \right]\left. \right] \\ \leq \mathbb {E} _ {\mathcal {D}} \left[ \mathbb {E} _ {\sigma} \left[ \| v \| 1 \{\frac {\pi}{2} - \tau \leq \theta (v, w ^ {\prime}) \leq \frac {\pi}{2} + \tau \} \sin (\tau) \right] \right] \\ \end{array}
$$
when  $\tilde{\theta} (v,w^{\prime}) = \frac{\pi}{2} -\theta (v,w^{\prime})$  . We have
$$
\begin{array}{l} R _ {n} \left(\mathcal {H} _ {\phi , \tau}\right) \leq \frac {B}{n} \mathbb {E} _ {\mathcal {D}} \left[ \mathbb {E} _ {\sigma} \left[ \| v \| 1 \{\theta (v, w ^ {\prime}) \leq \frac {\pi}{2} - \tau \} + \| v \| 1 \{\frac {\pi}{2} - \tau \leq \theta (v, w ^ {\prime}) \leq \frac {\pi}{2} + \tau \} \sin (\tau) \right] \right] \\ = \frac {B}{n} \mathbb {E} _ {\mathcal {D}} \left[ \mathbb {E} _ {\sigma} [ \| v \| ] \right] (\Pr (\theta (v, w ^ {\prime}) \leq \frac {\pi}{2} - \tau) + \Pr (\frac {\pi}{2} - \tau \leq \theta (v, w ^ {\prime}) \leq \frac {\pi}{2} + \tau) \sin (\tau)) \\ \end{array}
$$
The last equation follows from the symmetry and uniformity properties. We can bound the first expectation
$$
\begin{array}{l} \mathbb {E} _ {\mathcal {D}} \left[ \mathbb {E} _ {\sigma} \| v \| \right] = \mathbb {E} _ {\mathcal {D}} \left[ \mathbb {E} _ {\sigma} \| \sum_ {i = 1} ^ {n} x _ {i} \sigma_ {i} \| \right] \\ \leq \mathbb {E} _ {\mathcal {D}} [ \sqrt {\mathbb {E} _ {\sigma} \| \sum_ {i = 1} ^ {n} x _ {i} \sigma_ {i} \| ^ {2} ]} ] \\ = \mathbb {E} _ {\mathcal {D}} [ \sqrt {\mathbb {E} _ {\sigma} \sum_ {i = 1} ^ {n} \left\| x _ {i} \right\| ^ {2} \sigma_ {i} ^ {2}} ] \\ \leq C \sqrt {n}. \\ \end{array}
$$
Next, we can simply note that, since our data is distributed over a unit sphere, each data has norm no greater than 1. Therefore, we know that  $C = 1$  is indeed an upper bound on  $\mathbb{E}_{x\sim \mathcal{D}_X}[[|x||_2^2]$ . For the probability term, we note that in expectation  $v$  has the same distribution as a random vector  $u$  drawn uniformly from a unit sphere. We let this be some probability  $p$ :
$$
p = \Pr \left(\frac {\pi}{2} - \tau \leq \theta (v, w ^ {\prime}) \leq \frac {\pi}{2} + \tau\right) = \Pr \left(| \langle u, w ^ {\prime} \rangle | \leq \sin (\tau)\right).
$$
We know that the projection  $\langle u, w' \rangle \sim \mathcal{N}(0, \frac{1}{d})$ . Then, we have that  $|\langle u, w' \rangle|$  is given by a Folded Normal Distribution, which has a CDF given by
$$
\begin{array}{l} \Pr \left(| \langle u, w ^ {\prime} \rangle | \leq \sin (\tau)\right) = \frac {1}{2} \left[ \operatorname {e r f} \left(\frac {\sqrt {d} \sin (\tau)}{\sqrt {2}}\right) + \operatorname {e r f} \left(\frac {\sqrt {d} \sin (\tau)}{\sqrt {2}}\right) \right] \\ = \operatorname {e r f} \left(\frac {\sqrt {d} \sin (\tau)}{\sqrt {2}}\right). \\ \end{array}
$$
We then observe that
$$
\begin{array}{l} \Pr \left(\theta (v, w ^ {\prime}) \leq \frac {\pi}{2} - \tau\right) = \frac {1}{2} \left(1 - \Pr \left(\frac {\pi}{2} - \tau \leq \theta (v, w ^ {\prime}) \leq \frac {\pi}{2} + \tau\right)\right) \\ = \frac {1 - p}{2} \\ \end{array}
$$
Plugging this in yields the following bound
$$
p = \operatorname {e r f} \left(\frac {\sqrt {d} \sin (\tau)}{\sqrt {2}}\right).
$$
I Rademacher Complexity for Two Layer Neural Networks with a Gradient Constraint
Here, we present the full proof of the generalization bound for two layer neural networks with gradient explanations. In our proof, we use two results from Ma [24]. One result is a technical lemma, and the other is a bound on the Rademacher complexity of two layer neural networks.
Lemma I.1. Consider a set  $S = \{x_{1},\dots,x_{n}\}$  and a hypothesis class  $\mathcal{F}\subset \{f:\mathbb{R}^d\to \mathbb{R}\}$ . If
$$
\sup  _ {f \in \mathcal {F}} \sum_ {i = 1} ^ {n} f (x _ {i}) \sigma_ {i} \geq 0 \text {f o r a n y} \sigma_ {i} \in \{\pm 1 \}, i = 1, \dots , n,
$$
then, we have that
$$
\mathbb {E} _ {\sigma} \left[ \sup  _ {f \in \mathcal {F}} \left| \sum_ {i = 1} ^ {n} f (x _ {i}) \sigma_ {i} \right| \right] \leq 2 \mathbb {E} _ {\sigma} \left[ \sup  _ {f \in \mathcal {F}} \sum_ {i = 1} ^ {n} f (x _ {i}) \sigma_ {i} \right].
$$
Theorem I.2 (Rademacher complexity for two layer neural networks [24]). Let  $\mathcal{X}$  be an instance space and  $\mathcal{D}_{\mathcal{X}}$  be a distribution over  $\mathcal{X}$ . Let  $\mathcal{H} = \{h : x \mapsto \sum_{i=1}^{m} w_i \sigma(u_i^\top x) | w_i \in \mathbb{R}, u_i \in \mathbb{R}^d, \sum_{i=1}^{m} |w_i| \|u_i\|_2 \leq B\}$  be a class of two layer neural networks with  $m$  hidden nodes with a ReLU activation function  $\sigma(x) = \max(0, x)$ . Assume that there exists some constant  $C > 0$  such that  $\mathbb{E}_{x \sim \mathcal{D}_{\mathcal{X}}}[\|x\|_2^2] \leq C^2$ . Then, for any  $S = \{x_1, \ldots, x_n\}$  is drawn i.i.d. from  $\mathcal{D}_{\mathcal{X}}$ , we have that
$$
R _ {S} (\mathcal {H}) \leq \frac {2 B}{n} \sqrt {\sum_ {i = 1} ^ {n} | | x _ {i} | | _ {2} ^ {2}}
$$
We defer interested readers to [24] for the full proof of this result. Here, the only requirement of the data distribution is that  $\mathbb{E}_{x\sim \mathcal{D}_X}[\| x\| _2^2 ]\leq C^2$ . We now present our result in the setting of two layer neural networks with one hidden node  $m = 1$  to provide clearer intuition for the overall proof.
Theorem I.3 (Rademacher complexity for two layer neural networks ( $m = 1$ ) with gradient constraints). Let  $\mathcal{X}$  be an instance space and  $\mathcal{D}_{\mathcal{X}}$  be a distribution over  $\mathcal{X}$ . Let  $\mathcal{H} = \{h : x \mapsto w\sigma(u^\top x) | w \in \mathbb{R}, u \in \mathbb{R}^d, |w| \leq B, \|u\| = 1\}$ . Without loss of generality, we assume that  $\|u\| = 1$ . Assume that there exists some constant  $C > 0$  such that  $\mathbb{E}_{x \sim \mathcal{D}_{\mathcal{X}}}[\|x\|_2^2] \leq C^2$ . Our explanation constraint is given by a constraint on the gradient of our models, where we want the gradient of our learnt model to be close to a particular target function  $h' \in \mathcal{H}$ . Let this be represented by an explanation loss given by
for some  $\tau > 0$ . Let  $h'(x) = w'\sigma((u')^\top x)$  the target function, then we have
$$
R _ {n} \left(\mathcal {H} _ {\phi , \tau}\right) \leq \frac {\tau C}{\sqrt {n}} \quad i f | w ^ {\prime} | > \tau ,
$$
$$
R _ {n} \left(\mathcal {H} _ {\phi , \tau}\right) \leq \frac {3 \tau C}{\sqrt {n}} \quad i f | w ^ {\prime} | \leq \tau .
$$
Figure 9: Visualization of the piecewise constant function of  $\nabla_{x}h(x) - \nabla_{x}h^{\prime}(x)$  over 4 regions.
Proof. Our choice of  $\phi(h, x)$  guarantees that, for any  $h \in \mathcal{H}_{\phi, \tau}$ , we have that  $\|\nabla_x h(x) - \nabla_x h'(x)\| \leq \tau$  almost everywhere. We note that for  $h(x) = w\sigma(u^\top x)$ , the gradient is given by  $\nabla_x h(x) = w u 1\{u^\top x > 0\}$ , which is a piecewise constant function over two regions (i.e.,  $u^\top x > 0, u^\top x \leq 0$ ), captured by Figure I.
We now consider  $\nabla_{x}h(x) - \nabla_{x}h^{\prime}(x)$ , and we have 3 possible cases.
Case 1:  $\theta (u,u^{\prime}) > 0$
This implies that the boundaries of  $\nabla_x(h)$  and  $\nabla_x h'(x)$  are different. Then, we have that  $\nabla_x h(x) - \nabla_x h'(x)$  is a piecewise constant function with 4 regions, taking on values