SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Max Torop, $^{1*}$  Aria Masoomi, $^{1*}$  Davin Hill, $^{1}$  Kivanc Kose, $^{2}$  Stratis Ioannidis, $^{1}$  Jennifer Dy $^{1}$
$^{1}$  Northeastern University,  $^{2}$  Memorial Sloan Kettering Cancer Center {torop.m, masoomi.a}@northeastern.edu, {dhill, ioannidis, jdy}@ece.neu.edu, {kosek}@mskcc.org
Several recent methods for interpretability model feature interactions by looking at the Hessian of a neural network. This poses a challenge for ReLU networks, which are piecewise-linear and thus have a zero Hessian almost everywhere. We propose SmoothHess, a method of estimating second-order interactions through Stein's Lemma. In particular, we estimate the Hessian of the network convolved with a Gaussian through an efficient sampling algorithm, requiring only network gradient calls. SmoothHess is applied post-hoc, requires no modifications to the ReLU network architecture, and the extent of smoothing can be controlled explicitly. We provide a non-asymptotic bound on the sample complexity of our estimation procedure. We validate the superior ability of SmoothHess to capture interactions on benchmark datasets and a real-world medical spirometry dataset.
As machine learning models are increasingly relied upon in a variety of high-stakes applications such as credit lending [50, 42] medicine [10, 67, 15], or law [38], it is important that users are able to interpret model predictions. To this end, many methods have been developed to assess the importance of individual input features in effecting model output [53, 76, 81, 66, 73]. However, one may achieve a deeper understanding of model behavior by quantifying how features interact to affect predictions. While diverse notions of feature interaction have been proposed [53, 57, 56, 63, 25, 77, 84], in this work, we focus on the intuitive partial-derivative definition of feature interaction [25, 4, 47, 85, 18, 39].
Given a function and point, the Interaction Effect [47] between a given set of features on the output is the partial derivative of the function output with respect to the features; intuitively, it represents the infinitesimal change in the function engendered by a joint infinitesimal change in each chosen feature. The Interaction Effect derives in part from Friedman and Popescu [25], who define the global interaction between a set of features over the data distribution as the expected square of the partial-derivative with respect to those features. As in prior works [47, 39], we eschew the expectation to focus on local interactions occurring around a given point  $x$  and avoid squaring partial-derivatives to maintain the directionality of the interaction. We focus on pair-wise feature interactions, which, in the context of the Interaction Effect, are the elements of the Hessian.
ReLU networks are a popular class of neural networks that use ReLU activation functions [28]. The use of ReLU has desirable properties, such as the mitigation of the vanishing gradient problem [28], and it is the sole activation function used in popular neural network families such as ResNet [32] and VGG [72]. However, ReLU networks are piece-wise linear [58] and thus have a zero Hessian almost everywhere (a.e.), posing a problem for quantifying interactions (see Figure 1(a)).
Figure 1: (a) An exemplar illustration of a simple 5-hidden-layer ReLU network  $f: \mathbb{R}^2 \to \mathbb{R}$ . Note that  $f$  is piece-wise linear and thus has  $\nabla_x^2 f(x) = 0$  a.e. (b) The ReLU network convolved with  $q_{0.3I}: \mathbb{R}^2 \to \mathbb{R}$ , the density function of  $\mathcal{N}(0,0.3I)$ . This function is no longer piece-wise linear and admits non-zero higher order derivatives.
A common approach for estimating the Hessian in ReLU networks is to take the Hessian of a smooth surrogate network which approximates the original: each ReLU is replaced with SoftPlus, a smooth approximation to ReLU [28, 22], before differentiating [39, 20]. However, this approach affords one only coarse control over the smoothing as each internal neuron is smoothed, leading to unwieldy asymmetric effects, as can be seen in Figure 2.
In this work, we propose SmoothHess: the Hessian of the ReLU network convolved with a Gaussian. Such a function is a more flexible smooth surrogate than SoftPlus as smoothing is done on the network output, and the covariance of the Gaussian allows one to mediate the contributions of points based on their Mahalanobis distance. Unfortunately, obtaining the Hessian of the convolved ReLU network is impossible using naïve Monte-Carlo (MC) averaging. However, by proving an extension of Stein's Lemma [78, 49], we show that such a quantity can be efficiently estimated using only gradient calls on the original network. For an illustration of ReLU network smoothing, see Figure 1(b).
- We propose SmoothHess, the Hessian of the Gaussian smoothing of a neural network, as a model of the second-order feature interactions.
- We derive a variant of Stein's Lemma, which allows one to estimate SmoothHess for ReLU networks using only gradient calls.
- We prove non-asymptotic sample complexity bounds for our SmoothHess estimator.
- We empirically validate the superior flexibility of SmoothHess to capture interactions on MNIST, FMNIST, and CIFAR10. We utilize SmoothHess to derive insights into a network trained on a real-world medical spirometry dataset. Our code is publicly available.*
The remainder of this paper is organized as follows: In Sec. 2, we summarize the gradient-based methods for feature importance and interactions that are most related to our work. In Sec. 3, we provide a technical preliminary covering the definitions and techniques underlying both our method and competing works. Next, in Sec. 4, we introduce our method, SmoothHess, explain how to estimate it, and provide sample complexity bounds for our estimation procedure. In Sec. 5, we experimentally demonstrate the ability of SmoothHess to model interactions. Finally, in Sec. 6, we summarize our method and results, followed by a discussion of limitations and possible solutions.
Feature Importance and First-Order Methods: Methods that quantify feature importance fall into two categories: (i) perturbation-based methods (e.g., [53, 66, 17]), which evaluate the change in model outputs with respect to perturbed inputs, and (ii) gradient-based methods (e.g., [72, 76, 81]), which leverage the natural interpretation of the gradient as infinitesimally local importance for a given sample. Most relevant to our work are gradient-based approaches. The saliency map, as defined in [72], is simply the gradient of model output with respect to the input. Several variants are developed to address the shortcomings of the saliency maps. SmoothGrad [76] was developed
*https://github.com/MaxTorop/SmoothHess
Figure 2: Estimated Hessian element between features 1 and 2 at  $x_0 = (0,0)^T$  for a 6-layer ReLU Network  $f:\mathbb{R}^2\to \mathbb{R}$  trained to memorize the Four Quadrant toy dataset. (a) SmoothHess (SH) is estimated with isotropic covariance  $\Sigma = \sigma^2 I$  using granularly sampled  $\sigma^2\in \{1e - 3,\dots ,1\}$ . Aside from at minute  $\sigma^2 < 1e - 2.5$ , where hyper-local noisy behavior is captured, we have  $\nabla^{2}(f*q_{\sigma})(x_{0})_{1,2}\approx (5 + 3 + 12 - 10) / 4 = 2.5$ , the average of the memorized "ground truth" off-diagonal Hessian element over the four quadrants. This indicates a symmetry in the weighting of the contributions from points around  $x_0$  at every level of smoothing. (b) The Hessian of the SoftPlus smoothed function  $f_{\beta}$  (SP Hess) is computed using granularly sampled  $\beta \in \{1e - 1,\dots ,1e4\}$ . The average value of 2.5 is not achieved at any value of  $\beta$ , aside from briefly between  $\log_{10}\beta = 2$  and  $\log_{10}\beta = 3.5$  indicating that SoftPlus fails to incorporate the information around  $x_0$  in a symmetric manner at every level of smoothing.
to address noise by averaging saliency maps (see also Sec. 3), and comes with sample complexity guarantees [3]. Sundararajan et al. [81] introduce Integrated Gradients, the path integral between an input and an uninformative baseline. This is extended to the Shapley framework by Erion et al. [24]. The Grad-CAM line of work [93, 70, 61] is similar in nature to the methods above, with the key distinction that importance is modeled over internal (hidden) layers.
Feature Interactions: A variety of methods have been proposed to estimate higher-order feature interactions, which can again be separated into perturbation-based [54, 82, 86, 57, 92] and gradient-based approaches. Among the latter, Tsang et al. [85] propose Gradient-NID, which estimates feature interaction strength as the corresponding Hessian element squared. Janizek et al. [39] propose Integrated Hessian, which extends Integrated Gradients to use a path-integrated Hessian. Lerman et al. [47] propose Taylor-CAM, a higher-order generalization of Grad-Cam [70]. Cui et al. [18] quantify global interactions for Bayesian neural networks in terms of the expected value of the Hessian over the data distribution. For classification ReLU networks, the CASO and CAFO explanation vectors exploit feature interactions in the loss function using Hessian estimates [74].
Unfortunately, due to their piecewise linearity, existing Hessian-based interaction methods cannot be readily applied to ReLU networks. Janizek et al. [39] replace each ReLU activation with SoftPlus post-hoc, before applying Integrated Hessians. Similarly, Tsang et al. [85] apply their method to networks with the SoftPlus activation instead of ReLU. For regression tasks, Lerman et al. [47] replace ReLU with the smooth activation function GELU [34] before training. Although SoftMax outputs and the cross-entropy loss admit higher order derivatives, pre- or post-hoc smoothing, as above, is necessary for finding interactions affecting logits, internal neurons, or regression outputs. Indeed, while Singla et al. [74] estimate interactions on the original ReLU network, they are only with respect to the loss function.
In contrast, we propose a method for quantifying feature interactions that works with any ReLU network post-hoc without requiring retraining or modifications to the network architecture. It also can be directly estimated with respect to model as well as intermediate layer outputs. Furthermore, our experiments in Sec. 5 show the superior ability of our method to model logit outputs, internal neurons, and SoftMax probabilities as compared to a SoftPlus smoothed network.
3 Technical Preliminary
ReLU Network Background: We denote a ReLU network by  $F: \mathbb{R}^d \to \mathbb{R}^c$ , where  $c = 1$  in the case of regression. We denote the function which we wish to explain as  $f: \mathbb{R}^d \to \mathbb{R}$ , an arbitrary neuron  $f_i^{(l)}$  (the  $i^{th}$  neuron in the  $l^{th}$  layer) in our ReLU network, or a SoftMax Probability
for some class  $k \in \{1, \ldots, c\}$ . We denote sample space  $\mathcal{X} \subseteq \mathbb{R}^d$  and point  $x_0 \in \mathcal{X}$  for which we wish to capture feature interactions affecting  $f(x_0)$ . A general  $L$ -hidden-layer ReLU Network  $F = f^{(L + 1)}: \mathbb{R}^d \to \mathbb{R}^c$  may be defined recursively by [33, 58]:
$$
f ^ {(l)} \left(x _ {0}\right) = W ^ {(l)} g ^ {(l - 1)} \left(x _ {0}\right) + b ^ {(l)}, \text {f o r} l = 1 \dots L + 1, \tag {1a}
$$
$$
g ^ {(l)} \left(x _ {0}\right) = \max  \left(0, f ^ {(l)} \left(x _ {0}\right)\right), \text {f o r} l = 1 \dots L, \tag {1b}
$$
with  $g^{(0)}(x_0) = x_0 \in \mathbb{R}^d$ . For a given layer  $l$  the dimension (number of neurons) is defined as  $n_l \in \mathbb{N}$ , i.e.  $f^{(l)}(x_0), g^{(l)}(x_0) \in \mathbb{R}^{n_l}$  (with  $n_0 = d$ ) with weight and bias  $W^{(l)} \in \mathbb{R}^{n_l, n_{l-1}}$ ,  $b^{(l)} \in \mathbb{R}^{n_l}$ . As stated above, ReLU networks are piecewise linear [58]. Specifically, each neuron  $f_i^{(l)}: \mathbb{R}^d \to \mathbb{R}$ ,  $l \in \{1, \dots, L+1\}$ ,  $i \in \{1, \dots, n_l\}$ , is a piecewise linear function, corresponding to a finite set of  $K \in \mathbb{N}$  convex polytopes  $\mathcal{Q} = \{Q_i\}_{i=1}^K$ ,  $Q_i \subseteq \mathbb{R}^d$  which form a partition of  $\mathbb{R}^d$  [30, 31, 58, 33, 71, 64, 7]. See Figure 1(a) for an example of a ReLU network.
SmoothGrad: SmoothGrad [76] is an extension of the saliency map  $\nabla_x f(x_0)$  [72] developed to reduce noise. SmoothGrad is an average over saliency maps; formally, it is  $\mathbb{E}_{\delta}[\nabla_x f(x_0 + \delta)]$  where  $\delta \sim \mathcal{N}(0,\Sigma)$  and the covariance  $\Sigma \in \mathbb{R}^{d\times d}$  is a hyperparameter usually chosen to be isotropic  $\Sigma = \sigma^2 I,\sigma >0$ . SmoothGrad is estimated by sampling  $n$  perturbation vectors  $\{\delta_i\}_{i = 1}^n$ ,  $\delta_{i}\sim \mathcal{N}(0,\Sigma),\forall i\in \{1,\ldots ,n\}$  and averaging via  $\hat{G}_n^{\mathrm{SG}}(x_0,f,\Sigma) = \frac{1}{n}\sum_{i = 1}^n\nabla_xf(x_0 + \delta_i)$ . Unfortunately, the analogous Hessian average  $\frac{1}{n}\sum_{i = 1}^{n}\nabla_{x}^{2}f(x_{0} + \delta_{i})\approx \mathbb{E}_{\delta}[\nabla_{x}^{2}f(x_{0} + \delta)]$  is not useful for quantifying feature interactions in ReLU networks as  $\mathbb{E}_{\delta}[\nabla_x^2 f(x_0 + \delta)] = 0$ .
SoftPlus: A common approach to assessing higher-order derivatives for ReLU networks is by differentiating a smooth surrogate  $f_{\beta} : \mathbb{R}^{d} \to \mathbb{R}$  where a SoftPlus replaces every ReLU in  $f$  [28, 39, 20, 21]. The SoftPlus function  $s_{\beta}(x_0) = \frac{1}{\beta} \log (1 + e^{\beta x_0})$ ,  $\beta > 0$ , is a smooth approximation of ReLU, where  $\beta$  is a parameter inversely proportional to the level of smoothing [28, 22]. SoftPlus approaches ReLU and  $f_{\beta}$  and  $\nabla f_{\beta}$  approximate  $f$  and  $\nabla f$  with arbitrary accuracy as  $\beta \to \infty$  [20]. Dombrowski et al. [20] establish empirical and theoretical connections between  $\nabla_x f_{\beta}(x_0)$  and SmoothGrad. They observe visual similarities between  $\nabla_x f_{\beta}(x_0)$  and SmoothGrad at appropriate  $\beta$  values. Further, in the simple case of a single neuron with no bias, they prove that replacing ReLU with SoftPlus corresponds to a Gaussian-like smoothing of the gradient. However, this theory does not extend to networks with multiple neurons and layers.
Stein's Lemma: Stein's Lemma [78] is central to our analysis. We relate a variant of Stein's Lemma [51], which extends the results of Stein [78] to hold for multivariate normal distributions with arbitrary covariance matrices:
Lemma 1. (Stein's Lemma [51]) Given  $x_0 \in \mathbb{R}^d$ , covariance matrix  $\Sigma \in \mathbb{R}^{d \times d}$ , multivariate normal random vector  $\delta \in \mathbb{R}^d$  distributed from  $\delta \sim \mathcal{N}(0, \Sigma)$  and almost everywhere differentiable function  $g: \mathbb{R}^d \to \mathbb{R}$  for which  $\mathbb{E}_{\delta}[||[\nabla_x g(x_0 + \delta)]_i|] < \infty$  for each  $i \in \{1, \ldots, d\}$ , then
$$
\mathbb {E} _ {\delta} \left[ \Sigma^ {- 1} \delta g \left(x _ {0} + \delta\right) \right] = \mathbb {E} _ {\delta} \left[ \nabla_ {x} g \left(x _ {0} + \delta\right) \right]. \tag {2}
$$
A zero-th order oracle associated with a function  $g: \mathbb{R}^d \to \mathbb{R}$  is an oracle which, when provided with any input  $x \in \mathbb{R}^d$ , returns  $g(x)$ . Likewise, a first order oracle (or gradient oracle) returns  $\nabla_x g(x)$ . In the context of this work, where  $g$  is a ReLU network, zero-th order and first order oracle calls amount to network forward passes and backpropagation calls, respectively.
Given one has access to  $g$  as a zero-th order oracle, the LHS of Stein's Lemma may be estimated by sampling a set of perturbations  $\{\delta_i\}_{i=1}^n$ ;  $\delta_i \sim \mathcal{N}(0, \Sigma)$ , querying the zero-th order oracle  $g(x_0 + \delta_i)$  for each  $\delta_i$ , and MC-estimating, i.e:
$$
\hat {G} _ {n} \left(x _ {0}, g, \Sigma\right) = \frac {1}{n} \sum_ {i = 1} ^ {n} \Sigma^ {- 1} \delta_ {i} g \left(x _ {0} + \delta_ {i}\right) \approx \mathbb {E} _ {\delta} \left[ \Sigma^ {- 1} \delta g \left(x _ {0} + \delta\right) \right] \stackrel {\text {L e m m a}} {=} ^ {1} \mathbb {E} _ {\delta} \left[ \nabla_ {x} g \left(x _ {0} + \delta\right) \right]. \tag {3}
$$
Such an approach is useful for estimating  $\mathbb{E}_{\delta}[\nabla_{x}g(x_{0} + \delta)]$  in the RHS of Eq. (3) when the gradient of  $g$  is impossible or expensive to obtain but  $g$  itself may be efficiently queried as a zero-th order oracle (see e.g., [44, 48, 60]). Extending Liu [51]'s work, Lin et al. [49] present a second-order variant of Stein's Lemma expressing the expected Hessian in terms of the gradient:
Lemma 2. (First-Order Oracle Stein's Lemma [49]) Given  $x_0 \in \mathbb{R}^d$ , covariance matrix  $\Sigma \in \mathbb{R}^{d \times d}$ , multivariate normal random vector  $\delta \in \mathbb{R}^d$  distributed from  $\delta \sim \mathcal{N}(0, \Sigma)$  and continuously differentiable function  $g(z): \mathbb{R}^d \to \mathbb{R}$  with locally Lipschitz* gradients  $\nabla g: \mathbb{R}^d \to \mathbb{R}^d$ , then
$$
\mathbb {E} _ {\delta} \left[ \Sigma^ {- 1} \delta \left[ \nabla_ {x} g (x _ {0} + \delta) \right] ^ {T} \right] = \mathbb {E} _ {\delta} \left[ \nabla_ {x} ^ {2} g (x _ {0} + \delta) \right]. \tag {4}
$$
Complexity bounds have been derived for similar identities, which express the Hessian using zero-th order information [6, 94, 23]. However, Lemma 2 fails for ReLU networks. This is precisely because ReLU networks are piecewise linear and, therefore, are not continuously differentiable. In the next section, we directly address this through our method for estimating a smoothed Hessian for ReLU networks.
4 SmoothHess
Our main contributions are: (1) We propose SmoothHess, the Hessian of the network convolved with a Gaussian, for modeling feature interactions. (2) We use Stein's Lemma to prove that SmoothHess may be estimated for ReLU networks using only gradient oracle calls. (3) We prove non-asymptotic sample complexity bounds for our SmoothHess estimator.
Gaussian Convolution as a Smooth Surrogate: An alternative smooth-surrogate to  $f_{\beta}$  is  $h_{f,\Sigma}: \mathbb{R}^d \to \mathbb{R}$ , the convolution of  $f$  with a Gaussian:
$$
h _ {f, \Sigma} (x _ {0}) = (f * q _ {\Sigma}) (x _ {0}) = \int_ {z \in \mathbb {R} ^ {d}} f (z) q _ {\Sigma} (z - x _ {0}) d z, \tag {5}
$$
where  $\Sigma \in \mathbb{R}^{d\times d}$  is a covariance matrix,  $q_{\Sigma}(z - x_0) = (2\pi)^{-\frac{d}{2}}|\Sigma|^{-\frac{1}{2}}\exp(-\frac{1}{2}d(x_0,z)_{\Sigma})$  is the density function of the Gaussian distribution  $\mathcal{N}(0,\Sigma)$ ,  $|\cdot|$  is the determinant and  $d(x_0,z)_{\Sigma} = (x_0 - z)^T\Sigma^{-1}(x_0 - z) \in \mathbb{R}$  is the Mahalanobis distance between  $x_0$  and  $z$ .
The Gaussian-smoothed function  $h_{f,\Sigma}$  is infinitely differentiable and does not suffer from the limitations of surrogates obtained from internal smoothing. Here, smoothing is done on the output image of  $f$  and thus the relationship between  $h_{f,\Sigma}(x_0), f(z), z$  and  $x_0$  is made explicit by Eq. (5): the relative contribution of  $f(z)$  to  $h_{f,\Sigma}(x_0)$  is proportional to the exponentiated negative half Mahalanobis distance  $-\frac{1}{2} d(x_0,z)_\Sigma$ . The ability to adjust  $\Sigma$  gives a user fine-grained and localized control over smoothing; the eigenvectors and eigenvalues of  $\Sigma$  respectively encode directions of input space and a corresponding locality for their contribution to  $h_{f,\Sigma}$ .
We define SmoothHess as the Hessian of  $h_{f,\Sigma}$ :
Definition 1. (SmoothHess) Given ReLU network  $f: \mathbb{R}^d \to \mathbb{R}$ , point to explain  $x_0 \in \mathbb{R}^d$ , covariance matrix  $\Sigma \in \mathbb{R}^{d \times d}$  and  $q_{\Sigma}: \mathbb{R}^d \to \mathbb{R}$ , the density function of Gaussian distribution  $\mathcal{N}(0, \Sigma)$ , then SmoothHess is defined to be the Hessian of  $f$  convolved with  $q_{\Sigma}$  evaluated at  $x_0$ :
$$
\nabla_ {x} ^ {2} h _ {f, \Sigma} (x _ {0}) = \nabla_ {x} ^ {2} \left(f * q _ {\Sigma}\right) (x _ {0}) = \int_ {z \in \mathbb {R} ^ {d}} f (z) \nabla_ {x} ^ {2} q _ {\Sigma} (z - x _ {0}) d z. \tag {6}
$$
Well-known properties of the Gaussian distribution may be used to encode desiderata into the convolved function and, accordingly, to SmoothHess through the choice of the covariance. For instance, it is known that as  $d \to \infty$  the isotropic Gaussian distribution  $\mathcal{N}(0, \sigma^2 I_{d \times d})$  converges to  $U(S_{\sigma \sqrt{d}}^{d-1})$ , a uniform distribution over the radius  $\sigma \sqrt{d}$  sphere [88]. Thus, given large enough  $d$ , as is commonly encountered in deep learning datasets, one may choose  $\Sigma = (r / \sqrt{d})I$  to approximately ensure that  $h_{f,\Sigma}(x_0)$  incorporates information from the radius  $r$  sphere around  $x_0$ . We exploit and validate this intuition in our experiments (see Table 1).
Finally, we must highlight the strong connection between SmoothHess and SmoothGrad [76], which Wang et al. [89] prove is equivalent to the gradient of the same smooth surrogate:  $\nabla_{x}h_{f,\Sigma}(x_{0})$ . Thus,
*In its most general form, Lemma 2 holds for functions which are continuously differentiable and have locally ACL gradients. Locally ACL functions are functions which are absolutely continuous on almost every straight line, a mild condition which is satisfied by both locally Lipschitz and continuously differentiable functions [49, 46, 68]. As ReLU networks are locally Lipschitz [29], they are locally ACL.
SmoothGrad and SmoothHess together define a second-order Taylor expansion of  $h_{f,\Sigma}$  at  $x_0$ , which can be used as a second-order model of  $f$  around  $x_0$ .
SmoothHess Computation via Stein's Lemma: We relate our method for estimating SmoothHess for ReLU networks. As stated above, Lemma 2 does not hold for ReLU networks. However, we extend the arguments of Wang et al. [89] and Lin et al. [49] to show that the LHS from Lemma 2 is equivalent to SmoothHess for all Lipschitz continuous functions*:
Proposition 1. Given  $x_0 \in \mathbb{R}^d$ ,  $L$ -Lipschitz continuous function  $g: \mathbb{R}^d \to \mathbb{R}$ , covariance matrix  $\Sigma \in \mathbb{R}^{d \times d}$  and random vector  $\delta \in \mathbb{R}^d$  distributed from  $\delta \sim \mathcal{N}(0, \Sigma)$  with density function  $q_{\Sigma}: \mathbb{R}^d \to \mathbb{R}$ , then
$$
\mathbb {E} _ {\delta} \left[ \Sigma^ {- 1} \delta \left[ \nabla_ {x} g \left(x _ {0} + \delta\right) \right] ^ {T} \right] = \nabla_ {x} ^ {2} \left[ \left(g * q _ {\Sigma}\right) \left(x _ {0}\right) \right] = \nabla_ {x} ^ {2} h _ {g, \Sigma} \left(x _ {0}\right), \tag {7}
$$
where  $*$  denotes convolution.
In other words, even though Lemma 2 does not hold for ReLU networks, Stein's Lemma is indeed evaluating a Hessian: namely, SmoothHess given by Eq. (6). The proof consists of moving the Hessian operator on the RHS of Eq. (7) into the integral defined by  $(g * q_{\Sigma})(x_0)$ . The resulting expression is simplified into a form for which Lin et al. [49] prove is equivalent to the LHS of Eq. (7). The proof is provided in App. B.
Proposition 1 opens up the possibility of an MC-estimate of SmoothHess that only require access to a first-order oracle  $\nabla f$ . This is computed by sampling a set of  $n \in \mathbb{N}$  perturbations  $\{\delta_i\}_{i=1}^n$ ,  $\delta_i \sim \mathcal{N}(0, \Sigma)$ , and for each  $\delta_i$  querying  $\nabla_x f(x_0 + \delta_i)$  before taking the outer product  $\delta_i[\nabla_x f(x_0 + \delta_i)]^T \in \mathbb{R}^{d \times d}$ , Monte-Carlo averaging and finally symmetrizing:
$$
\hat {H} _ {n} ^ {\circ} \left(x _ {0}, f, \Sigma\right) = \frac {1}{n} \sum_ {i = 1} ^ {n} \Sigma^ {- 1} \delta_ {i} \left[ \nabla_ {x} f \left(x _ {0} + \delta_ {i}\right) \right] ^ {T}, \tag {8a}
$$
$$
\hat {H} _ {n} \left(x _ {0}, f, \Sigma\right) = \frac {1}{2} \left(\hat {H} _ {n} ^ {\circ} \left(x _ {0}, f, \Sigma\right) + \hat {H} _ {n} ^ {\circ^ {T}} \left(x _ {0}, f, \Sigma\right)\right). \tag {8b}
$$