SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Alternatively, the user may only have  $k < d$  orthonormal eigenvectors in mind,  $a_1, \ldots, a_k \in \mathbb{R}^d$ , for which they wish to smooth at specific levels  $\sigma_1, \ldots, \sigma_k > 0$ . A procedure such as Gram-Schmidt may be used to find  $a_{k+1}, \ldots, a_d \in \mathbb{R}^d$  which extends  $a_1, \ldots, a_k$  to an orthonormal basis of  $\mathbb{R}^d$ . Again, the smoothing levels along the eigenvectors  $a_{k+1}, \ldots, a_d$  may be chosen according to user preference. For instance, one may select  $\sigma_{k+1} = \ldots = \sigma_d \approx 0$  if minimal smoothing is desired along these directions. Just as above  $\Sigma$  may be set by constructing eigenvector and eigenvalue matrices  $Q = [a_1| \ldots | a_d] \in \mathbb{R}^{d \times d}$  and  $\Lambda = \mathrm{diag}(\sigma_1, \ldots, \sigma_d) \in \mathbb{R}^{d \times d}$  and simply multiplying  $\Sigma = Q\Lambda Q^T$ .
E Experiment Setup
E.1 Datasets and Models
In this work we make use of six datasets, two synthetic datasets (Four Quadrants, Nested Interactions) three benchmark datasets (MNIST, FMNIST, CIFAR10) and a real world medical dataset (Spirometry). Below we describe these datasets and training details.
Four Quadrant. The Four Quadrant dataset consists of points  $x \in \mathbb{R}^2$  sampled from the grid  $[-2, 2] \times [-2, 2]$  with a spacing of 0.008. A 6-layer fully connected ReLU network was trained using RMSProp [83] on the Four Quadrant dataset achieving a final mean-squared-error of  $\approx 1e-4$ . Training lasted for 40,000 iterations with a batch size of 128 and a starting learning rate of  $1e-3$  which was decayed by a factor of  $1e-1$  at iterations 5000, 10,000 and 20,000.
Nested Interactions. The Nested Interactions dataset consists of points  $x \in \mathbb{R}^2$  sampled from the grid  $[-2, 2] \times [-2, 2]$  with a spacing of 0.008. A 6-layer fully connected ReLU network was trained using RMSProp on the Nested Interactions dataset achieving a final mean-squared-error of  $\approx 1e-1$ . Training lasted for 200,000 iterations with a batch size of 64 and a starting learning rate of  $1e-3$  which was decayed by a factor of  $1e-1$  at iterations 40,000, 80,000, 120,000 and 160,000. For more details on Nested Interactions see App. F.3.
MNIST MNIST consists of 70,000 28x28 greyscale images, each corresponding with one of the digits 0-9. There are 60,000 and 10,000 images in the pre-defined train and test sets respectively. We further split the train set into 50,000 images for training and 10,000 for validation. A 5-layer fully connected network with dimensions 500-300-250-250-250 was trained with stochastic gradient descent for 30 epochs, with a batch size of 128 and a starting learning rate of  $1e - 2$  which was decayed by a factor of  $1e - 1$  at iterations 4, 000 and 8, 000. Final accuracies of  $\approx 100\%$  (Train)  $\approx 100\%$  (Val) and  $\approx 98\%$  (Test) were achieved. All images are flattened. For  $\mathcal{P}_{MSE}$  all methods were evaluated on 200 test points. For adversarial attacks all methods were evaluated on 400 test points. The threshold for the quadratic optimization attack was set to  $T = 0.98$ .
FMNIST. FMNIST consists of 70,000 28x28 greyscale images, each corresponding with one of 10 articles of clothing. There are 60,000 and 10,000 images in the pre-defined train and test sets respectively. We further split the train set into 50,000 images for training and 10,000 for validation. Final accuracies of  $\approx 93\%$  (Train)  $\approx 93\%$  (Val) and  $\approx 88\%$  (Test) were achieved. The network and training details are identical to that used for MNIST above. All images are flattened. For  $\mathcal{P}_{MSE}$  all methods were evaluated on 200 test points. For adversarial attacks all methods were evaluated on 400 test points. The threshold for the quadratic optimization attack was set to  $T = 0.98$ .
CIFAR10 CIFAR10 consists of 60,000 3x32x32 RGB color images, each corresponding with an animal or vehicle. There are 50,000 and 10,000 images in the pre-defined train and test sets respectively. We further split the train set into 40,000 images for training and 10,000 for validation. A ResNet-18 [32] was trained on CIFAR10 for 55 epochs using a batch size of 128. The first 5 epochs were used for warmup with a starting learning rate of  $1e - 2$  ending at 0.5. For the rest of training a cosine decay schedule was used [52] decaying down to  $1e - 5$  by the final epoch. Augmentations used for training were (i) Random Horizontal Flip  $(p = 0.5)$  (ii) Color Jitter  $(p = 0.8)$  with brightness, contrast and saturation values equal to 0.4 and hue value 0.1 (iii) Random Grayscale  $(p = 0.2)$ . Final accuracies of  $\approx 85\%$  (Train, Augmentations)  $\approx 96\%$  (Val, No Augmentations) and  $\approx 90\%$  (Test, No Augmentations) were achieved. No augmentations were applied to the validation/test data when evaluating explainers. For  $\mathcal{P}_{MSE}$  all methods were evaluated on 100 test points. For adversarial attacks all methods were evaluated on 200 test points. The threshold for the quadratic optimization attack was set to  $T = 0.8$
Spirometry. The Spirometry dataset uses raw exhalation curves, measured in volume over time, recorded during a spirometry exam. Each spirometry curve is measured in 10ms intervals, which we downsample to 50ms intervals and limit to 15s in total length, resulting in 300 features. We use the UK Biobank dataset, which is a large, population-based study conducted in the United Kingdom. Participant statistics have been previously reported in Sudlow et al. [79]. The UK Biobank records 2-3 exhalation efforts for each participant, using a Vitalograph Pneumotrac 6800 device*. If two efforts are recorded as passing acceptability criteria and are also reproducible ( $\leq 5\%$  difference in Forced Vital Capacity (FVC) and Forced Expiratory Volume in 1 Second  $(\mathrm{FEV}_1)$ ), then the third
*https://biobank.ctsu.ox.ac.uk/crystal/crystal/docs/Spirometry.pdf
Dataset	MNIST	FMNIST	CIFAR10
Function	Class Logit (↓)	Int. Neuron (↓)	Class Logit (↓)	Int. Neuron (↓)	Class Logit (↓)	Int. Neuron (↓)
ε	0.25	0.50	1.00	0.25	0.50	1.00	0.25	0.50	1.00	0.25	0.50	1.00	0.25	0.50	1.00
SH+SG (Us)	4.5e-5	1.8e-4	7.2e-4	4.5e-5	1.8e-4	7.2e-4	4.5e-5	1.8e-4	7.2e-4	4.5e-5	1.8e-4	7.2e-4	1.1e-5	4.6e-5	1.8e-4
SG [76]	4.5e-5	1.8e-4	7.2e-4	4.5e-5	1.8e-4	7.2e-4	4.5e-5	1.8e-4	7.2e-4	4.5e-5	1.8e-4	7.2e-4	1.1e-5	4.6e-5	2.0e-4
SP H + G	400.0	200.0	95.0	400.0	200.0	100.0	350.0	160.0	75.0	360.0	190.0	95.0	11.5	6.0	3.5
SP G	390.0	200.0	100.0	400.0	200.0	100.0	390.0	180.0	95.0	360.0	190.0	95.0	14.0	8.0	4.0
SW (H + G)	190.0	95.0	55.0	190.0	95.0	50.0	170.0	80.0	45.0	170.0	90.0	45.0	8.5	8.5	8.5
SW G	190.0	95.0	55.0	190.0	95.0	50.0	180.0	100.0	65.0	170.0	90.0	50.0	8.5	8.5	8.5
Table 3: Selected values of  ${\sigma }^{2}$  and  $\beta$  ,achieving the lowest average  ${\mathcal{P}}_{MSE}$  on on a held-out validation set.
effort is omitted. A common metric used to evaluate lung health is the Forced Expiratory Volume in 1 Second  $(\mathrm{FEV}_1)$ , which is the maximum volume of air that can be expelled by the participant in 1 second [59]. Note that a participant's  $\mathrm{FEV}_1$  measurement is taken as the maximum  $\mathrm{FEV}_1$  over all recorded exhalation efforts during a single visit.
The spirometer automatically evaluates effort against a number of acceptability criteria. One such criteria is the detection of coughing. In our experiment, we use a subset of exhalation efforts where coughing was detected and train a CNN to predict the participant's final  $\mathrm{FEV}_1$  measurement. This subset contains 8,721 samples, which we split into training (80%) and test (20%) partitions. We follow the preprocessing in Hill et al. [36] to ensure that participants have at least one effort that passes quality control where  $\mathrm{FEV}_1$  can be measured. The trained CNN includes 10 convolution blocks. Each block contains a 1-d convolution of kernel width 200 and 20 channels, batch normalization, dropout  $(p = 0.5)$ , and skip connection. The model is trained using mean squared error (MSE), achieving 0.563 MSE on the train set and 0.547 MSE on the test set.
E.2 Hyperparameters
For ease of reading, we use the following notation below:  $[a:b:c] = \{a,a + c,a + 2c,\ldots ,b - 2c,b - c\} \subseteq \mathbb{R}$  denotes the set of points between  $a$  (inclusive) and  $b$  (exclusive) at intervals of size  $c$ . Here  $a$  and  $b$  are chosen such that  $a < b$  and  $(b - a)\mod c = 0$ . An example is:  $[0.1:1.0:0.1] = \{0.1,0.2,0.3,0.4,0.5,0.6,0.7,0.8,0.9\}$ .
E.2.1  $\mathcal{P}_{MSE}$
For each of the 18 combinations of dataset, function and neighborhood size  $\varepsilon$  the performance of  $\beta$  and  $\sigma$  are validated on a held out set before selection. 200 validation points are used to choose  $\sigma, \beta$  for MNIST and FMNIST and 50 validation points are used for CIFAR10. For SmoothHess + SmoothGrad and SmoothGrad only three values of  $\sigma$  are validated, based on the common sense criterion  $\sigma = \varepsilon / \sqrt{d}$ : given neighborhood size  $\varepsilon$  and dataset with dimension  $d$ ,  $\sigma$  is chosen from  $\sigma \in \{\varepsilon / 2\sqrt{d}, 3\varepsilon / 4\sqrt{d}, \varepsilon / \sqrt{d}\}$ .
The following values of  $\beta$  are checked on a validation set:
$$
\mathbf {M N I S T} \quad \beta \in [ 0. 1: 1: 0. 1 ] \cup [ 1: 2 0: 1 ] \cup [ 2 0: 9 5: 5 ] \cup [ 1 0 0: 8 0 0: 1 0 ]
$$
$$
\mathbf {C I F A R 1 0}: \beta \in [ 0. 1: 1: 0. 1 ] \cup [ 1: 2 0: 0. 5 ] \cup [ 2 0: 9 5: 5 ] \cup [ 1 0 0: 8 0 0: 1 0 ]
$$
The values of  $\sigma$  for SH and  $\mathrm{SH} + \mathrm{SG}$ , and  $\beta$  for  $\mathrm{SPH} + \mathrm{G}$  and SP G which achieve the lowest  $\mathcal{P}_{MSE}$  on the validation data are shown in Table 3. The results for SoftPlus  $\beta$  are interesting: (i) We see that MNIST and FMNIST results, for both class logit and interior neuron, are consistent for fixed  $\varepsilon$ . Further we see that CIFAR10 results between class logit and interior neuron are consistent for fixed  $\varepsilon$ . (ii) The optimal value of  $\beta$  seems to be approximately proportional to the value of  $\varepsilon$ . While our results in Table 1 show that SmoothHess is better at capturing local interactions than the SoftPlus Hessian, the results in Table 3 indicate that the relationship between  $f$ ,  $f_{\beta}$  and  $\beta$  warrants further exploration.
E.2.2 Adversarial Attacks
200 validation points were used to choose  $\beta$  and  $\sigma^2$  for MNIST and FMNIST and 50 validation points were used to choose  $\beta$  and  $\sigma^2$  for CIFAR10.
Dataset	MNIST	FMNIST	CIFAR10
Attack Magnitude ε	0.25	0.50	0.75	1.25	1.75	0.25	0.50	0.75	1.25	1.75	0.1	0.2	0.3	0.4	1.0
SH+SG (Us)	1e-3	5e-3	3e-2	5e-2	1.5e-1	1e-3	4e-2	3e-2	8e-2	3e-2	5e-5	1e-4	4e-4	4e-4	7.5e-4
SG [76]	1e-3	5e-3	2e-2	6e-2	8e-2	5e-3	4e-2	8e-2	1.5e-1	3e-1	5e-5	3e-4	5e-4	7.5e-4	3e-3
SP H + G	8	18	17	10	6	20	25	13	9	6	5	4	3	2	2
SP G	10	12	10	11	3	30	9	11	8	3	5	4	3	2	2
Table 4: Selected values of  ${\sigma }^{2}$  and  $\beta$  ,achieving the lowest post-hoc accuracy of adversarial attacks on a held-out validation set.
The following values of  $\sigma^2$  were validated for adversarial attacks:
$$
\mathbf {M N I S T} \text {a n d F M N I S T :} \sigma^ {2} \in \{0. 0 0 1, 0. 0 0 5 \} \cup [ 0. 0 1: 0. 1: 0. 0 1 ] \cup [ 0. 1 5: 1. 0 0: 0. 0 5 ]
$$
$$
\mathbf {C I F A R 1 0}: \sigma^ {2} \in \{5 e - 0 5, 7. 5 e - 0 5 \} \cup [ 0. 0 0 0 1: 0. 0 0 0 6: 0. 0 0 0 1 ] \cup \{0. 0 0 0 7 5 \} \cup [ 0. 0 0 1: 0. 0 0 6: 0. 0 0 1 ]
$$
The following values of  $\beta$  were validated for adversarial attacks:
$$
\mathbf {M N I S T} \text {a n d F M N I S T :} \beta \in [ 1: 2 0: 1 ] \cup [ 2 0: 1 0 0: 5 ] \cup [ 1 0 0: 2 1 0: 1 0 ].
$$
$$
\mathbf {C I F A R 1 0}: \beta \in [ 1: 1 0: 1 ] \cup [ 1 0: 4 5: 5 ]
$$
The values of  $\sigma^2$  and  $\beta$  that achieve lowest validation post-hoc accuracy are shown in Table 4. Following intuition, it is generally the case that parameters corresponding with increased smoothing (larger  $\sigma^2$  and smaller  $\beta$ ) achieve better results (lower post-hoc accuracy) for large  $\varepsilon$ , and parameters corresponding to less smoothing (smaller  $\sigma^2$  and larger  $\beta$ ) achieve better results for small  $\varepsilon$ .
F.1  $\mathcal{P}_{\mathcal{MS}\mathcal{E}}$
Comparison with Swish: In Table 1,  $\mathcal{P}_{MSE}$  results are presented for five methods: the first (SmoothGrad) and second (SmoothHess + SmoothGrad) order Taylor expansions of the ReLU network  $f$  convolved with a Gaussian, the first and second order Taylor expansions of the SoftPlus smoothed network and the vanilla (unsmoothed) Gradient. We present Table 5, a version of Table 1 which includes additional results comparing with Swish [65] smoothed networks. Swish, an alternative smooth activation to SoftPlus, is formally defined as  $\mathrm{Sw}_\beta(x) = x$  sigmoid  $(\beta x)$  where sigmoid  $(x) = \frac{1}{1 + \exp(-x)}$  and  $\beta$  is a hyperparameter determining the level of smoothing. It can be seen in Table 5 that Swish is generally less effective than SoftPlus, and is outperformed by our method at each combination of dataset and locality.
Standard Deviation: We report the standard deviation of the  $\mathcal{P}_{MSE}$  for each method, dataset, function and neighborhood size  $\varepsilon$  in Table 6. Of the  $18\mathcal{P}_{MSE}$  results, SmoothHess + SmoothGrad attains the lowest standard deviation for 15 and ties with SoftPlus Hessian + SoftPlus Gradient for 2. The standard deviation of SoftPlus Hessian + SoftPlus Gradient is the lowest for FMNIST internal neuron at  $\varepsilon = 0.5$ , achieving  $2.4e-7$  while SmoothHess + SmoothGrad achieves  $2.5e-7$ .
ResNet101: We repeat our  $\mathcal{P}_{MSE}$  experiment for the predicted class logits of CIFAR10 using a ResNet101, reporting results in Table 7. It can be seen in the leftmost column that our method,  $\mathrm{SH} + \mathrm{SG}$ , achieves superior performance to the competing methods at each locality, indicating that SmoothHess can generalize to larger network architectures. The standard deviation of  $P_{MSE}$ , as well as choices of  $\sigma^2 / \beta$  (as selected from a validation set) are reported in the center and righthmost columns, respectively.
F.2 Adversarial Attacks
Comparison with Vanilla Hessian: One may use the vanilla Hessian of the predicted SoftMax probability, which admits higher order derivatives, to construct adversarial attacks. Table 8 is an updated version of Table 2 which includes results for attacks using the vanilla Hessian + vanilla Gradient  $(\mathrm{H} + \mathrm{G})$ , in the third to last row.
We see that inclusion of the vanilla Hessian generally results in more effective attacks then use of the vanilla gradient alone.  $\mathrm{H} + \mathrm{G}$  ties  $\mathrm{SH} + \mathrm{SG}$  and  $\mathrm{SPH} + \mathrm{G}$  for lowest post-hoc accuracy at the
Dataset	MNIST	FMNIST	CIFAR10
Function	Class Logit (↓)	Int. Neuron (↓)	Class Logit (↓)	Int. Neuron (↓)	Class Logit (↓)	Int. Neuron (↓)
ε	0.25	0.50	1.00	0.25	0.50	1.00	0.25	0.50	1.00	0.25	0.50	1.00
SH+SG (Us)	9.6e-7	7.8-6	6.7e-5	4.9e-8	4.0e-7	3.3e-6	6.5e-7	4.0e-6	4.3e-5	2.0e-8	1.8e-7	1.6e-6
SG [76]	4.5e-6	4.1e-5	3.9e-4	2.1e-7	1.7e-6	1.5e-5	3.0e-6	2.7e-5	2.6e-4	1.0e-7	9.0e-7	7.0e-6
SP (H + G)	1.2e-6	9.6e-6	8.1e-5	5.5e-8	4.4e-7	3.7e-6	9.6e-7	7.5e-6	6.5e-5	3.0e-8	2.1e-7	1.8e-6
SP G	4.6e-6	4.1e-5	3.9e-4	2.1e-7	1.7e-6	1.5e-5	3.2e-6	2.8e-5	2.6e-4	1.0e-7	8.5e-7	7.2e-6
SW (H + G)	2.4e-6	2.0e-5	1.9e-4	1.0e-7	8.3e-7	7.3e-6	2.1e-6	1.7e-5	1.8e-4	5.0e-8	4.3e-7	3.7e-6
SW G	5.6e-6	5.0e-5	4.9e-4	2.4e-7	2.0e-6	1.8e-5	3.9e-6	3.5e-5	3.5e-4	1.1e-7	9.6e-7	8.3e-6
G [73]	4.2e-3	1.7e-2	6.7e-2	2.0e-3	7.0e-3	2.9e-2	3.8e-3	1.5e-2	6.0e-2	1.0e-4	4.0e-4	1.8e-3
3.0e-1	1.2e-0
Table 5: Average  $\mathcal{P}_{MSE}$  results at three radii  $\varepsilon$ , with the inclusion of vanilla Gradient (G). Other methods include: SmoothHess + SmoothGrad (SH+SG,Us) SmoothGrad (SG) SoftPlus Grad (SP G) SoftPlus Hessian + Gradient (SP H + G). Results are provided for the predicted class logit, and the penultimate neuron maximally activated by the "three," dress and cat classes for MNIST, FMNIST and CIFAR10 respectively.
Dataset	MNIST	FMNIST	CIFAR10
Function	Class Logit (↓)	Int. Neuron (↓)	Class Logit (↓)	Int. Neuron (↓)	Class Logit (↓)	Int. Neuron (↓)
ε	0.25	0.50	1.00	0.25	0.50	1.00	0.25	0.50	1.00	0.25	0.50	1.00	0.25	0.50	1.00
SH+SG (Us)	9.9e-7	7.3-6	6.4e-5	3.9e-8	2.7e-7	2.1e-6	1.0e-6	7.7e-6	6.4e-5	2.8e-8	2.5e-7	1.9e-6	4.7e-3	1.1e-1	4.3e-1	1.5e-6
SG [76]	7.0e-6	7.4e-5	7.1e-5	2.2e-7	1.6e-6	1.2e-5	5.5e-6	5.1e-5	4.6e-4	2.0e-7	1.8e-6	1.3e-5	4.7e-2	1.9e-1	7.5e-1	4.2e-5
SP (H + G)	1.2e-6	9.5e-6	7.7e-5	4.4e-8	2.9e-7	2.1e-6	2.4e-6	1.5e-5	1.2e-4	3.1e-8	2.4e-7	1.9e-6	6.5e-3	1.3e-1	6.4e-1	5.7e-5
SP G	7.1e-6	7.4e-5	7.2e-4	2.2e-7	1.6e-6	1.2e-5	6.1e-6	5.4e-5	4.7e-4	2.0e-7	1.8e-6	1.4e-5	4.8e-2	1.9e-1	7.6e-1	2.6e-4
SW (H+ G)	2.6e-6	2.0e-5	1.9e-4	6.5e-8	4.6e-7	4.2e-6	7.2e-6	3.3e-5	3.7e-4	5.3e-8	4.5e-7	4.0e-6	5.0e-2	1.7e0	4.3e1	3.6e-4
SW G	7.3e-6	7.2e-5	7.4e-4	2.1e-7	1.5e-6	1.2e-5	6.6e-6	6.2e-5	6.2e-4	1.5e-7	1.1e-6	9.0e-6	5.0e-2	2.1e-1	8.6e-1	2.8e-4
G [73]	2.1e-3	8.4e-3	3.4e-2	7.9e-5	3.0e-4	1.2e-3	2.7e-3	1.1e-2	4.3e-2	9.1e-5	4.0e-4	1.4e-3	2.4e-1	9.6e-1	3.7e-0	9.0e-4
Table 6: Standard deviation of  ${\mathcal{P}}_{MSE}$  at three radii  $\epsilon$  . SmoothHess + SmoothGrad (SH+SG,Us) SmoothGrad (SG) SoftPlus Grad (SP G) SoftPlus Hessian + Gradient (SP H + G) Vanilla gradient (G). Results are provided for the predicted class logit, and the penultimate neuron maximally activated by the "three," dress and cat classes for MNIST, FMNIST and CIFAR10 respectively.
smallest magnitude  $(\varepsilon = 0.25)$  for the simplest dataset (MNIST). However, as no smoothing is done, the attacks generated from the  $\mathrm{H} + \mathrm{G}$  are generally significantly weaker than those generated using smooth surrogates. In fact, aside from MNIST with  $\varepsilon = 0.25$ , the second order vanilla  $\mathrm{H} + \mathrm{G}$  attacks achieve higher post-hoc accuracy than first-order method SmoothGrad for all datasets and values of  $\varepsilon$ . This is especially apparent for CIFAR10, the most complex dataset.
F.3 Nested Interactions
We use the Nested Interactions dataset to highlight SmoothHess's ability to capture different interactions occurring at various localities around a given point. In this experiment we measure interactions around the origin  $x_0 = (0,0)^T \in \mathbb{R}^2$ .
Just like the Four Quadrant dataset, the Nested Interactions dataset consists of points  $x \in \mathbb{R}^2$  sampled uniformly from  $[-2, 2] \times [-2, 2] \subset \mathbb{R}^2$  with a spacing of 0.008. We establish different interactions occurring around the origin  $x_0$ , based upon the distance from  $x_0$ . Specifically, we set the label for a given point  $x$  by:  $x \in B_{0.6}(x_0) \Rightarrow y(x) = \frac{1}{2} x_1^2 + x_1 x_2$ ,  $x \in B_{1.2}(x_0) \backslash B_{0.6}(x_0) \Rightarrow y(x) = x_1 x_2$ ,  $x \in \mathbb{R}^2 \backslash B_{1.2}(x_0) \Rightarrow y(x) = -5x_1 x_2$ .
In words, the interaction between features  $x_{1}$  and  $x_{2}$  is 1 inside the radius-1.2 ball around  $x_{0}$  and is -5 outside of this ball. The interaction between  $x_{1}$  and itself is 1 inside the radius-0.6 ball around  $x_{0}$  and 0 outside of this ball. The interaction between  $x_{2}$  and itself is 0 over all of  $\mathbb{R}^2$ .
We train a 6-layer neural network on the Nested Interactions dataset and estimate SmoothHess and SoftPlus Hessian for  $\sigma^2 \in \{1e-6, \dots, 1e1\}$  and  $\beta \in \{1e-1, \dots, 4.0\}$  respectively. The interaction results for  $x_1$  with itself,  $x_1$  with  $x_2$  and  $x_2$  with itself as a function of the level of smoothing ( $\sigma$  or  $\beta$ ) are reported in Figure 5.
As the target function  $y(x)$  is discontinuous, it is not possible for a network to memorize the Nested Interactions dataset. Thus, there very well may be interactions occurring in the network which are not described as above; the interactions we know occur in the data are not a pure "gold-standard". That being said, Figure 5 shows that SmoothHess captures the interactions as we know occur in the data, and the SoftPlus Hessian does not. This suggests that, to a large extent, both (i) the network has
Value	PMSE	PMSE Std	σ2,β used
ε	0.25	0.50	1.00	0.25	0.50	1.00	0.25	0.50	1.00
SH+SG (Us)	1.3e-4	9.5e-4	6.9e-3	1.7e-4	1.1e-3	1.1e-2	1.1e-5	4.6e-6	1.8e-4
SG	5.3e-4	4.3e-3	3.3e-2	1.0e-3	8.3e-3	6.7e-2	1.1e-5	4.6e-6	1.8e-4
SP (H + G)	3.7e-4	2.6e-3	1.7e-2	6.6e-4	2.7e-3	3.1e-2	41.0	16.0	11.5
SP G	6.2e-4	5.0e-3	3.7e-2	1.1e-3	8.8e-3	6.9e-2	55.0	22.5	11.5
Swish (H + G)	1.5e-3	1.2e-2	4.5e-2	2.5e-3	8.3e-3	2.8e-2	27.5	17.5	1.5
Swish G	8.8e-4	7.0e-3	5.3e-2	1.4e-3	1.1e-2	8.3e-2	60.0	1.0e6	1.0e6
G	3.8e0	1.5e+1	6.0e+1	3.3e0	1.3e1	5.2e1	n/a	n/a	n/a
Table 7: Using a ResNet101 trained on CIFAR10, the average  $\mathcal{P}_{MSE}$  achieved by SmoothHess + SmoothGrad (SH + SG, Us), SmoothGrad (SG), SoftPlus Hessian + SoftPlus Gradient (SP (H+G)), SoftPlus Gradient (SP G), Swish Hessian + Swish Gradient (Swish (H+G)), Swish Gradient (Swish G) and Vanilla Gradient (G), is evaluated as a proxy for explainer quality. Results are reported at three radii  $\varepsilon$ , for the predicted class logit. Left:  $\mathcal{P}_{MSE}$  results are reported. The lowest value in each column is bolded. Middle: The standard deviation of  $\mathcal{P}_{MSE}$  is reported. The lowest value in each column is bolded. Right: The smoothing hyperparameter  $(\sigma^2$  for SmoothGrad and SmoothHess + SmoothGrad,  $\beta$  for SoftPlus and Swish) used is reported. Our method, SH + SG, achieves the lowest  $\mathcal{P}_{MSE}$  for each of the three radii  $\varepsilon$ . This indicates that the interactions SmoothHess captures improve the model of network behaviour, even for large networks such as ResNet101.
Dataset	MNIST	FMNIST	CIFAR10
Attack Magnitude ε	0.25	0.50	0.75	1.25	1.75	0.25	0.50	0.75	1.25	1.75	0.1	0.2	0.3	0.4	1.0
SH+SG (Us)	93.0	80.3	48.0	10.5	2.0	79.5	46.8	25.0	3.5	0.0	62.5	38.5	26.5	15.0	4.5
SG [76]	93.3	81.8	48.8	11.3	2.8	79.5	49.3	26.3	4.0	0.0	65.0	42.0	27.5	17.0	0.0
SP (H + G)	93.0	81.8	51.5	15.8	7.5	79.8	51.0	27.5	5.3	0.8	64.5	42.0	31.0	23.5	7.5
SP G	93.3	82.3	53.8	16.3	5.0	79.8	51.5	29.5	7.8	1.0	66.5	47.5	36.0	29.5	8.5
H + G	93.0	81.8	55.3	19.0	11.8	80.0	50.0	30.3	9.5	2.0	68.0	51.5	40.5	32.5	22.0
G [73]	93.3	82.8	56.0	18.5	8.8	80.3	52.3	31.8	11.0	2.5	69.0	51.5	41.0	34.0	21.5
Random	99.8	99.5	99.0	99.0	98.8	99.3	98.0	97.3	95.5	93.8	100.0	99.5	99.0	98.5	96.5
Table 8: Post-hoc accuracy of adversarial attacks performed on the predicted SoftMax probability, at five attack magnitudes  $\epsilon$ , with the inclusion of the vanilla Hessian + vanilla Gradient (H + G). Lower is better. Other methods include: SmoothHess + SmoothGrad (SH + SG, Ours), SmoothGrad (SG), SoftPlus Gradient (SP G), SoftPlus Hessian + SoftPlus Gradient (SP (H + G)) and vanilla Gradient (G). First order attack vectors are constructed by scaling the normalized gradient by  $\epsilon$  and subtracting from the input. Second order attack vectors are found by minimizing the corresponding second-order Taylor expansions.
memorized the data and (ii) SmoothHess captures the network behaviour while the SoftPlus Hessian does not.
F.4 Qualitative Comparison
We present a visual comparison of the interactions found by SmoothHess with those from other methods. Namely, we consider methods that can be interpreted as the quadratic term in a second-order Taylor expansion around a smooth surrogate network: SoftPlus Hessian and Swish Hessian.
We show interactions found between super-pixels of CIFAR10 test images. To this end, we utilize the Simple Linear Iterative Clustering (SLIC) [2] algorithm to segment the image into 20-25 super-pixels. We sum interactions between each pair of features in each pair of super-pixels, before visualization.
Results are shown in Figure 6 for the predicted class logit of a ResNet18 model for three CIFAR10 test images. Here, each row corresponds to a separate image. Test images are visualized in column 1. Columns 2-4 correspond to the three methods. For each image, interactions between one chosen