Use perturbations when learning from explanations
Authors: Juyeon Heo
Vihari Piratla
Matthew Robert Wicker
Adrian Weller
Venue: NeurIPS 2023 poster (2023)
Keywords: Learning from explanation
Robustness
Interpretability
Shortcuts
Explanations
TL;DR: We recast Machine learning from explanations (MLX) as a robustness problem and show both theoretically and empirically how this approach is better suited for learning from explanations.

Abstract: Machine learning from explanations (MLX) is an approach to learning that uses human-provided explanations of relevant or irrelevant features for each input to ensure that model predictions are right for the right reasons. Existing MLX approaches rely on local model interpretation methods and require strong model smoothing to align model and human explanations, leading to sub-optimal performance. We recast MLX as a robustness problem, where human explanations specify a lower dimensional manifold from which perturbations can be drawn, and show both theoretically and empirically how this approach alleviates the need for strong model smoothing. We consider various approaches to achieving robustness, leading to improved performance over prior MLX methods. Finally, we show how to combine robustness with an earlier MLX method, yielding state-of-the-art results on both synthetic and real-world benchmarks.

Use Perturbations when Learning from Explanations
Juyeon Heo University of Cambridge jh2324@cam ac. uk
Vihari Piratla' University Cambridge vp4210cam. ac uk
Matthew Wicker Alan Turing Institute
Adrian Weller
Alan Turing Institute and University of Cambridge
Abstract
Machine learning from explanations (MLX) is an approach to learning that uses pooaid-r explanations of relevant or irrelevant features for each input to ensure that model predictions are right for the right reasons. Existing MLX ap- proaches rely on local model interpretation methods and require strong model smoothing to align model and human explanations, leading to sub-optimal per- formance. We recast MLX as a robustness problem, where human explanations specify lower dimensional manifold from which perturbations can be drawn, and show both theoretically and empirically how this approach alleviates the need for strong model smoothing. We consider various approaches to achieving robustness, leading to improved performance over prior MI.X methods. Finally, we show how combine robustness with earlier MI.X method, yielding state-of-the-art results on both synthetic and real-world benchmarks. Our implementation can be found at: https:///gtthne wtt.://aaar/orooutttooooo
Introduction
Deep neural networks (DNNSs) display impressive capabilities, making them strong candidates for real-wold deployment. However, numerous challenges hinder their adoption in practice. Several major deployment challenges have been linked the fact that labelled data often under-specifies the task (D' Amour et al., 2020). For example, systems trained on chest x-rays were shown to generalise poorly because they exploited dataset-specific incidental correlations such as hospital tags for diagnosing pneumonia (Zech et al.) 2018 DeGrave et al. 2021). This phenomenon of learning unintended feature-label relationships referred as shoricut learning (Geirhos et al. (020) and critical challenge to solve for trustworthy deployment of machine learning algorithms. A common remedy to avoid shortcut learning is to train on diverse data (Shah et al. 2022) from multiple domains, demographies, etc, thus minimizing the underspecification problem, but this may be impractical for many applications such as in healtheare.
Enriching supervision through human- providdrovided explanations of relevant and irrelevant re- gions/features per example is an appealing direction toward reducing under-specification. For instance, (human-provideds facts such as hospital tag as irrelevant features. Learning from such human- prroiddd explanations (MLX) has been shown to avoid known shortcuts (Schramowski et al.) 2020). Ross al. (2017) pioneered an MI.X approach based on regularizing DNNs, which was followed by several others (Schramowski et al.) 2020; Rieger al. 2020; Stammer et al., 2021: Shao et al. (021). Broadly, existing approaches employ a model interpretation method to obtain pxr-example feature saliency, and
explanation for chest x-ray classification may highlight scanning arti-
'Equal Contribution
37th Conference on Neural Information Processing Systems (NcurIPS 2023).

---

regularize such that model and human- -uman--rovvied explanations align. Since saliency is unbounded for relevant features, many approaches simply regularize the salience of irrelevant features. In the same spirit, we focus on handling specification of irrelevant features, which we refer to as an explanation hereafter.
Existing techniques for MLX employ local interpretation tool and augment the loss with a term regularizing the importance of irrelevant region. Regularization-baase critical concern stemming from their dependence on a local interpretation method. Regularization of local, i.e. globally, i.e. over the entire input domain (see Figure TT. As we demonstrate both analytically and empirically that previous MLX proposals, which are all regularization-base smoothing in order to be globally effective at reducing shortcut learning.
approaches suffer from
explanations may not have the desired effect of reducing shortcuts
require strong model
In this work, we explore MIX using various robust training methods with the objective of training models that are robust to perturbations of irrelevant features. We start by framing the provided human explanations as specifications of local, diveesininniinaal manifold from which perturbations are drawn. We then notice that a model whose prediction is invariant to perturbations drawn from the manifold ought also to be robust to irrelevant features. Our perspective yields considerable advantages, Posing MLX as robustness task enables us to leverage the considerable body of prior work in robustness. Further, we show in Section 4.1 that robust training can provably upper bound the deviation on model value when irrelevant features are perturbed without needing to impose model smoothing. However, when the space of irrelevant features is high- high-dimensional. robust-training may not fully suppress irrelevant features as explained in Section 4.2 Accordingly, we explore combining both robustness- based and regularization-baeee methods, which achieves the best results. We highlight the following contributions:
We theoretically and empirically demonstrate that existing MLX methods require strong model smoothing owing to their dependence on local model interpretation tools.
We study learning from explanations using robust training methods. To the best of our knowledge, we are the first analytically and empirically evaluate robust training methods for MLX.
We distill our insights into our final proposal of combining robustness and regularization- based methods, which consistently performed well and reduced the error rate over the previous best by 20-90%.
2 Problem Definition and Background
We assume access to a training dataset with N training examples, D7 = {(x(') ,)),, with (i) E Rd and ( label. In the MLX setting, human expert also specifies input mask (") for an example (") where non-zero values of the mask identify irrelevant features of the input x(*). An input mask is usually designed to negate a known shortcut feature that classifier is exploiting. [ example, a mask in the ISIC dataset highlights a patch that was found to confound with non-cancerous images. With the added human specification, the augmented dataset contains triplets of example, label and mask, Dr ((x(9) ((()), m ] The task therefore observations well while not exploiting any features that are identified by the mask m.
some examples of masks for the datasets that used for evaluation. For
learn model [(x;; 0) that fits
The method of Ross et al. (017)) which we call Grad-Reg (short for ReaiiinntnRgullliizatiionno and also other similar approaches (Sha et al. 2021 Schramowski et al. 2020) employ an local interpretation method (E) to assign importance scores to input features: (x()), which is then regularized with an R0) term such that irrelevant features are not regarded as important. Their training loss takes the form shown in Equation I for an appropriately defined task-specific loss t.
1S(x) A E(x, f(x; 0)).
R(0)= 2 [[( (1) c (11
n=1
1 0- = arg min Σ 8 ( +XR(0) 311100]]
(I)
2

---

We use @ to denote element-wist product throughout. CDEP (Rieger et al. 2020 is slightly different. They instead use an explanation method that also takes the mask as an argument to estimate the contribution of features identified by the mask, which they minimize similarly.
About obtaining human specification mask. Getting manually specifying explanation masks can be impractical. However, the procedure can be automated if the nuisance/irreleant feature occurs systematically or if it easy to recognize, which may then be obtained automatically using procedure similar Liu et al. (202T) Rieger et al. (2020). A recent effort called Salient- used neuron activation maps to scale curation of such human- seeeaa--dpe masks to Imagenet- scale (Singla Feizi 2021] Singla et al., (022). These efforts may be seen as a proof-of proof-of-concepp for obtaining richer
annotations beyond content labels, and towards better defined tasks.
3
Method
Our methodology is based on the observation that an ideal model must be robust to perturbations to the irrelevant features. Following this observation, we reinterpret the human-provicn mask as specification of lower-dimensional manifold from which perturbations are drawn and optimize the following objective.
E ((((()))))))))) +o max e (f(x(()) + (E (e))) (ii) 0
0" = arg min
(2)
The above formulation uses weighting trade off between the standard task loss and perturbation loss and κ > 0 is a hyperparameter that controls the strength of robustness. We can leverage the many advances in robustness order to approximately solve the inner maximization. We present them below.
Avg-Ex: We can approximate the inner-max with the empirical average of loss averaged K samples drawn from the neighbourhood training inputs. [Singla et al. (2022) adopted this straightforward baseline for supervising using human-provided saliency maps on the Imagenet dataset. Similar to K, we use a to control the noise in perturbations as shown below.
0" arg min Σ Ele (1(0)))))
K Σ
a
e (s(x + (0)mm ())))))))) (00200)022))
()))))
K
PGD-Ex: Optimizing for an estimate of worst perturbation through projected gradient descent (PGD) (adry al.) 2017) popular approach from adversarial robustness. We refer to the approach of using PGD to approximate the second term of our loss as PGD-Ex and denote by ('((()) 0, m (n)) the perturbation found by PGD at x()), Given the non-convexity of this problem, however, guarantees be made about the quality of the approximate solution x".
0" arg mini Σ (1(x()))), al (f(x(m) + (x ,000,,,,n0 0. 1):0), y())
IBP-Ex: Certified robustness approaches, on the other hand, minimize a certifiable upper-boune of the second term. A class of certifiable approaches known as interval bound propagation methods (IBP) (Mirman et al., 2018 Gowal et al. (2188) propagate input intervals to function value intervals that are guaranteed to contain true function values for any input in the input interval.
We define an input interval for (n) as (x()
(n) (") + km) where k is defined Eqn. We then use bound propagation techniques to obtain function value intervals for the corresponding input interval: [(x) (0) which are ranges over class logits. Since we wish to train model that correctly classifies an example irrespective of the value of the irrelevant features, we wish to maximize the minimum probability assigned to the correct class, which is obtained by combining minimum logit for the correct class with maximum logit for incorrect class: f(x("), (()), 1(0), u(m); 0) و ((m) o (()) + (n)) o (1- - (()) where (m)) E (0, 1) denotes the one-hot transformation of the label (m) into c-length vector for classes. We refer to this version of the loss as IBP-Ex, summarized

---

below.
1(m) (m)) = 1 |BP(f(;; (x(m) - xx m (") x(1) x(())
x x m (n))
f(x(") tyln), 1 u 0) - ((m) ) u c y(m))
0" arg ming Σe (fx((m; 0): at (f(xlm) y(m) , 0) 0))
Despite its computational efficiency, IBP is known to suffer from scaling issues when the model is too big. Consequently, it is better to use IBP-Ex only when the model is small (less than four layers of CNN or feed-fore reed-forwar)) and if computational efficiency is desired. On the other hand, we do not anticipate any scaling issues when using PGD-Ex.
IBP-Ex+Grad-Rey We combine robustness and regularization by simply combining their respective loss terms. We show the objective for Eee--x-----eeee below, PGD-Ex+Grad-Ree follows similarly.
Combined robustness and regularization:
0" arg min Σ (1(x())))) + al (f(x!"), (y Lu;0), (R(O).
XR(O) and a, f are as defined in Eqn. 5 and Eqn. respectively. In Section 4 5.2 we demonstrate
the complementary strengths of robustness and regularization-based methods.
Theoretical Motivation
(a) Toy data
(b) Grad-Rec Reg with 3=0
(c) Grad Reg with 8=1 (d) IBP-Ex with B=0
Figure I: Illustration of the uneasy relationship between Grad-Reg and smoothing strength. (b) The decision boundary nearly vertical (zero gradient wrt nuisance y-axis value) for all training points and yet varies as function of value when fitted using Grad-Reg with B = 0. (c) Grad-Reg requires strong model smoothing y-coordinate. (d) IBP-Ex, on the other hand, fits vertical pair of lines without any model smoothing.
1) in order to translate local insensitivity
global robustness to
In this section, we motivate the merits and drawbacks of robustness- based over regularization based methods. Through non-pane non-parametric analysis in Theorems [ we argue that (a) regularization methods are robust to perturbations of irrelevant features (identified by mask) only when the underlying model is sufficiently smoothed, thereby potentially compromising performance, (b) robust training upper-bounds deviation in function values when irrelevant features are perturbed, which can be further suppressed by using a more effective robust training. Although our analysis is restricted to nonparametric models for the case of analysis, we empirically verify our claims with parametric neural network optimized using gradient-based optimizer. We then highlight limitation of robustness-baat methods when the number irrelevant features large through Proposition !
4.1 Merits of Robustness-based methods
Consider two-dimensional regression task, i.e. (*) E A' and y R. Assume that the second feature the shortcut that the model should not use for prediction, and denote by x)"
(n) the jth dimension of th point. We infer regression function from Gaussian process prior f ~ GP(F; K) with a squared exponential kernel where k(c, = exp( Σ, (#.-2,)² As result, we have two hyperparameters 0,, 02 which length scale parameters for the first and second dimensions respectively. Further, impose Gamma prior over the length scale: 0, 2 ((a,
07

---

Theorem 1 (Grad-Reg), We infer a funnesiion f from a GP prior as described above with the additional supervision of (x) /0r2llss / = O, vi E [1, N. Then the function value deviations
to perturbations irrelevant feature are lower bounded by a value proportional to the perturbation strength as shown below.
28a 3
f(x + (xx+ ((x)
(x] +5x3a))
(5)
Full proof of Theorem D is in Appendix A] we provide the proof outline below.
Proof sketch. We exploit the flexibility of GPs to accommodate observations transformed function values if the transformation is closed under linear operation (Hennig et al. 2022) (see Chapter 4). Since gradient is linear operation (contributors 2022), simply revise the observations y to include the partial derivative observations for each training point with appropriately defined kernel. We then express the inferred function, which is the posterior f(x Dr) in closed form.
We then derive a bound on function value deviations to perturbation on the second feature and simplify using simple arithmetic and Bernoulli's inequality to derive the final expression.
We observe from Theorem that if we wish to infer a function that robust to irrelevant feature perturbations, we need to set to very small value. Since the expectation of gamma distributed inverse-square length parameter is TE(0-2] = 3. which we wish to set very small, we are, in effect, sampling functions with very large length scale parameter i.e. strongly smooth functions. This result brings us to the intuitive takeaway that regularization using Grad-Reg applies globally only when the underlying family functions is sufficiently smooth. The result may also hold for any other decal-interpretation methods that is closed under linear operation. One could also argue that can simply use different priors for different dimensions, which would resolve the wver-smoothing issue. However, we do not have access to parameters specific to each dimension in practice and especially with DNNs, therefore only overall smoothness may be imposed such as with parameter norm regularization in Eqn. [
We now look properties of function fitted using robustness methods and argue that they bound deviations in function values better. In order to express the bounds, we introduce numerical quantity called coverage (C) to measure the effectiveness of a robust training method. We first define a notion of inputs covered by a robust training method as & - {x x E X. (xf(x;
< o} C X' for small positive threshold á on loss. We define coverage as the maximum distance along second coordinate between any point in X' and its closest point in X, i.e. C å maxxex min [x² -2/l- We observe that C is small if the robust training is effective. In the extreme case when training minimizes the loss for all points in the input domain, i.c. X = X' then C=0.
Theorem 2. When the following property.
robustness algorithm regularize the network, the fitted function has
If(x + 10, f(x)) s 2C
3 mmaz
8maz and fina are maximum values of Ax, and f(x) in the input domain (X) respectively.
Proof sketch. We begin by estimating the function deviation between an arbitrary point and a training instance that only differ I2, which is bounded by the product of maximum slope (maxx (xxx) and Ary. We then estimate the maximum slope (lipschitz constant) for a function inferred from GP. Finally, function deviation between two arbitrary points is twice the maximum deviation between an arbitrary and a training instance that is estimated the first step. O
Full proof is in Appendix B The statement shows that deviations in function values are upper bounded by a factor proportional to C, which can be dampened by employing an effective robust training method. We can therefore control the deviations in function values without needing to regress q (i.e. without over-smoothing).
Further remarks on sources of over-smoothing in regularization-ben methods. We empiri- cally observed that the term R0) (of Eqn. []. which supervises explanations, also has smoothing effect on the model when the importance scores (IS) are not well normalized, which is often the case. This is because reducing IS(x) everywhere will also reduce saliency of irrelevant features.
5

---

Empirical verification with a toy dataset. For empirical verification of our results, we fit a 3-layer ferd-forward network on a two-dimensional data shown in Figure[[aa where color indicates the label. We consider fitting model that is robust to changes in the second feature shown y-axis. In Figures (b), (c), we show the Grad-Reg fitted classifier using gradient (0f/0rr for our case) regularization for two different strengths of parameter smoothing (0 and respectively). With weak smoothing, we observe that the fitted classifier locally vertical (zero gradient along y-axis), but curved overall (Figure
[ (b)), which is fixed with strong smoothing (Figure (c). On the other hand, IBP-Ex fitted classifier is nearly vertical without any parameter regularization as shown in (d). This example illustrates the need for strong model smoothing when using a regularization -baed method.
4.2 Drawbacks Robustness-based methods
Although robust training is appealing in low dimensions, their merits do not transfer well when the space of irrelevant features is high-dimensional owing to difficulty in solving the inner maximization of Eqn. 2 Sub-par estimation of the maximization term may learn parameters that still depend on irrelevant features. We demonstrate this below with simple exercise.
Proposition 1. Consider a regression task with + 1-dimensional inputs where the first dimensions are irrelevant, and assume they are Ta = y, d E [1, D] while ED+1 ~ NN, 1/K). The MAP estimate of linear regression parameters f(xx > D+1 WaTd when fitted using Avg-Ex Ex are as follows: Wa = 1/(D + K), d E [1, and wD+1 K K/(K + D).
We present the proof in Appendix C We observe that as D increases, the weight of the only relevant feature (xD+1) diminishes. On the other hand, the weight of the average feature: 1 Ed Dd which is D/(D + K) approaches as D increases. This simple exercise demonstrates curse of dimensionality for robustness-bast combining robustness methods with a regularization method especially when the number of irrelevant features large such in the case of Decoy-MNIST dataset, which is described in the section.
methods. For this reason, we saw major empirical gains when
Experiments
We evaluate different methods four datasets: syn- thetic and three real-world. The synthetic dataset is sim-
ilar to decoy-MNIST of Ross et al. (2017) with induced
shortcuts and is presented in Section [.2 For evalua-
tion on practical tasks, we evaluated
plant pheno-
typing (Shao et al., 202T task in Section 5.3 skin cancer detection (Rieger et al. 2020) task presented Section 5.4 and object classification task presented in Section 5.5
phenotycing
All
a
the datasets contain known spurious feature, and were used in the past for evaluation of MLX methods. Figure 2 summarises the three datasets, notice that we additionally require in the training dataset the specification of mask identifying irrelevant features of the input; the patch for ISIC dataset, background for plant dataset, decoy half for Decoy-MNISS images, and label-specific irrelevant region approved by humans for Salient- Salient-Imagenet.
MNIST
Salient
Muld-clas
Figure 2: Sample images and masks for different datasets. The shown mask cor- responds to the image from the second column.
5.1 Setup
5.1.1
Baselines
We denote by ERM the simple minimization of cross-entropy loss (using only the first loss term of Equation T). We also compare with GSDRo(Sagawa al. 2019), which also has the objective of
avoiding to learn known irrelevant features but is supervised through group label (see Section 6). Although the comparison is unfair toward G-DRO because MI. methods use richer supervision of per-example masks, it serves baseline that can be slightly better than ERM in some cases.
6

---

Regulaization-basel methods. Grad-Reg and CDEP, which were discussed in Section 2 We omit comparison with Shao al. (202I) because their code not publicly available and non non-trivial to implement the influence-function based regularization.
Robustness- based methods. Avg-Ex, PGD-Ex, IBP-Ex along with combined robustness and regularization methods. IBP-Ex+Grad-Reg, PGD-Ex+Grad-Res that are described in Section
5.1.2 Metrics
Since two of our datasets are highly skewed (Plant and ISIC), we report aavero-averaged over labels) accuracy denoted "Avg Acc" and worst accuracy "Wg Acc" over groups of examples with groups appropriately defined as described below. On Salient-Imagenet, we report using average accuracy and Relative Core Sentivity (RCS) following the standard practice.
Wg Ace. Worst accuracy among groups where groups are appropriately defined. Different labels define the groups for decoy-MNIST and plant dataset, which therefore have ten and two groups respectively. ISIC dataset, different groups are defined by the cross-product of label and presence or absence of the patch. We denote this metric as "Wg Acc", which is a standard metric when evaluating on datasets with shorteut features (Sagawa et al. 2019).
RCS proposed Singla et al. (2022) was used to evaluate models on Salient- Imanntt-mmgenet... The metric measures the sensitivity of model noise in spurious and core regions. High RCS implies low dependence on irrelevant regions, and therefore desired. Please see Appendix E.2 for more details.
5.1.3 Training and Implementation details
Choice of the best model. We picked the best model using the held-out validation data. We then report the performance on test data averaged over three seeds corresponding to the best hyperparameter.
Network details. We four-layer CNN followed by three-fully connected layers for binary classification on ISIC and plant dataset following the setting in Zhang et al. (2019), and three-fully connected layers for multi classification decoy-MNIST dataset. Pretrained Resnet-18 18 is used as the backbone model for Salient-lmagened
More details about network architecture, datasets, data splits, computing specs, and hyperparameters can be found in Appendix
Dataset
Deecoy-NNIS
Plant
ISIC
Dalient-Imagener
Method
Avg Acc
Wg Acc
Avg Acc
We Acc
Avg Acc
Wg Acc
Avg Acc
RCS
ERM
15.1
10.5
71.3
54.8
77.3
55.9
96.4
47.9
G-DRO
64.1
28.1
74.2
58.0
66.6
58.5
Grad-Reg
72.5
46.2
72.4
68.2
76.4
60.2
88.3
52.5
CDEP
14.5
10.0
67.9
54.2
73.4
60.9
Avg-Ex
29.5
19.5
76.3
64.5
77.1
55.2
PGD-Ex
67.6
51.4
79.8
78.5
78.7
64.4
93.8
58.7
IBP-Ex 68.1
47.6
76.6
73.8
75.1
64.2
P+G 96.9
95.8
79.4
76.7
79.6
67.5
94.6
65.0
IGG
96.9
95.0
81.7
80.1
78.4
65.2
Table I: Macro-averaged (Avg) accuracy and worst group (Wg) accuracy on (a) decoy-MNIST,
(b) plant dataset, (c) ISIC dataset along with relative core sensitivity (RCS) metric on (d) Salient- Imagenet. Statistically significant numbers are marked bold. Please see Table Appendix for standard deviations. I+G is short for IBP-Ex+ SBP-Ex+Grad-Rees and P+G for PGD-Ex+ -ee--Ex-rrrdReeg
5.2 Decoy-MNIST
-NNSS dataset is inspired from MNIST-CIFAL dataset of Shah et al. (2020) where very simple label-revealing color based feature (decoy) is juxtaposed with a more complex feature (MNIST image) as shown in Figure [] We also randomly swap the position of decoy and MNIST parts, which makes ignoring the decoy part more challenging. We then validate and test on images where decoy part is set to correspond with random other label. Note that our setting is different from decoy-mnist of CDEP (Rieger et al. 2020), which is likely why we observed discrepancy in results reported in our paper and CDEP (Rieger al. 2020). We further elaborate on these differences in Appendix G

---

We make the following observations from Decoy-MNIST results presented in Table ERM is only slightly better than a random classifier confirming the simplicity bias observed in the past (Shah et al. 2020). Grad-Reg, PGD-Ex and IBP-Ex perform comparably and better than ERM, but when combined Beee-e--Grrareeg.gPi----------e
they far exceed their individual performances.
In order to understand the surprising gains when combining regularization and robust- ness methods, we draw insights from gradient explanations on images from train split
for Grad-Reg and IBP-Ex. We looked at $1 M (n)
#f(x(")) and 52 cxem)
M mm)) ()( (x
(1
2 (x("))
where M(+ is median over all the examples
and I| is En-nomm. For effective algorithm, we expect both 82,82 be close to zero. However, the values of $1, $2 is 2.3e-3. 0.26 for the best model fitted using Grad-Reg and 6.7, 0.05 for IBP-Ex. We observe that Grad-Reg has lower 81 while IBP-Ex has lower s2, which shows that Grad-Reg is good at dampening the contribution of decoy part but also dampened contribution of non-decoy likely due over-smoothing, IBP-Ex improves the contribution of the non-decoy part but did not fully dampen the decoy part likely because high dimensional space of irrelevant features, i.e. half the image is irrelevant and every pixel of irrelevant region is indicative of the label. When combined, 1BP-Ex+Grad-Pe -eg has low $1, 82, which explains the increased performance when they are combined.
5.3 Plant Phenotyping
Plant phenotyping is a real-world task of classifying images of a plant leaf as healthy or unhealthy. About half of leaf images are infected with a Cercospora Leaf Spot (CLS), which are the black spots on leaves as shown in the first image in the second row of Figure 2 [Schramowski et al. (2020) discovered that standard models exploited unrelated features from the nutritional solution in the background in which the leaf is placed, thereby performing poorly when evaluated outside of the laboratory setting. Thus, we aim to regulate the model not focus on the background of the leaf using binary specification masks indicating where the background is located. Due to lack of out-of-distribution test set, we evaluate with in-domain test images but with background pixels replaced by a constant pixel value, which is obtained by averaging over all pixels and images in the training set. We replace with an average pixel value in order avoid any undesired confounding from shifts in pixel value distribution. In Appendix F.5] we present accuracy results when adding varying magnitude of noise to the background, which agree with observations we made in this section. More detailed analysis of the dataset can be found in SShrrmowski et al. 2020))
Table [ contrasts different algorithms on the plant dataset, All the algorithms except CDEP improve over ERM, which is unsurprising given our test data construction; any algorithm that can divert focus from the background pixels can perform well. Wg accuracy of robustness (except Avg- Ex) and combined methods far exceed any other method by 5-12% over the next best baseline and by 19-26% ac- curacy point over ERM. Surprisingly, even Avg-Ex has significantly improved the performance over ERM likely because spurious features in the background are spiky or unstable, which vanishes with simple perturbations.
ERM
PUUR
PGD-EX
BPEx ESP EE-EEERRR
Figure 3: Visual heatmap of salient fea- tures for different algorithms on three sample train images of Plant data. Im- portance score computed with Smooth- Grad (Smilkov et al. 2017).
We visualize the interpretations of models obtained using SmoothGrad (Smilkov et al. 2017) trained with five differ- ent methods for three sample images from the train split in Figure As expected, ERM has strong dependence on lon-leaf background features. Although Grad-Reg features are all on the leaf, they appear to be localized to small region on the leaf, which is likely due over-smoothing effect of its loss. IBP-Ex, IBP-Ex+Gradd-EE on the other hand draws features from a wider region and has more diverse pattern of active pixels.
5.4 ISIC: Skin Cancer Detection
ISIC a dataset of skin lesion images, which are to be classified cancerous or non-cancerous. Since half the non-cancerous images in the dataset contains colorful patch as shown in Figure 2 standard
8

---

DNN models depend on the presence of patch for classification while compromising the accuracy on non non-cancerous images without patch (Codella et al. 2019; Tschandl al., 2018). We follow the standard setup and dataset released by Rieger et al. (2020), which include masks highlighting the patch. Notably, (Rieger et al. 2020) employed a different evaluation metric, F1, and a different architecture VGG model pre-trained on ImageNet. This may explain the worse-than-expected performance of CDEP from Rieger et al. (020). More details are available in Appendix G
Traditionally, three groups are identified in the dataset: non-cancerous images without patch (NCNP) and with patch (NCP), and cancerous images (C). In Table 2 we report on per- group accuracies for different algorithms. Detailed results with error bars are shown in Ta- ble [ of Appendix F The Wg accuracy (of Table I may not match with the worst of
the average group accuracies in Table 2 because we report average of worst accuracies.
We now make the following observations. FRM performs the worst on the NPNC group confirming that predictions made by a standard model depend on the patch. The accu- racy on the PNC group is high overall perhaps because PNC group images that are consistently at lower scale (see mid- dle column of Figure 2 for example) are systematically more easier to classify even when the patch is not used for classification. Although buman-explanation for this dataset,
Method ERM
NPNC
PNC
C
55.9
96.5
79.6
Grad-Reg
67.1
99.0
63.2
CDEP
72.1
98.9
62.2
Avg-Ex
62.3
97.8
71.0
PGD-Ex
65.4
99.0
71.7
IBP-Ex
68.4
98.5
67.7
which only identifies the patch if present, do not full specify all spurious correlations, from them. Grad-Reg Begd-RRe and CDEP improved NPNC accuracy at the expense of C's accuracy while still performing rela-
I+G
66.6
99.6
68.9
still saw gains when learning
P+G
69.6
98.8
70.4
Table Per-group accuracies ISIC.
tively poor on Wg accuracy. Avg-Ex performed no better than ERM whereas PGD-Ex, IBP-Ex, IBP-Ex+Grad-RRe and PGD-EX+Grad-Red significantly improved Wg accuracy over other baselines. The reduced accuracy gap between NPNC and C when using combined methods is indicative of reduced dependence on patch. We provide detailed comparison of PGD-Ex and IBP-Ex in Appendix F3
5.5 Salient-Imagenet
Salient-lmagenet (Singla & Feizi 2021 Singla et al. 2022) is a large scale dataset based on Imagenet with pixel-level annotations of irrelevant (spurious) and relevant (core) regions. Salient-Imagenet is an exemplary effort on how explanation masks can be obtained at scale. The dataset contains 232 classes and 52,521 images (with about 226 examples per class) along with their core and spurious masks. We made a smaller evaluation subset using six classes with around 600 examples. Each of the six classes contain least one spurious feature identified by the annotator. Please refer to Appendix E.4 for more details. We then tested different methods for their effectiveness in learning model that ignores the irrelevant region. We use a ResNet-18 18 model pretrained on ImageNet as the initialization.
In Table we report RCS along with overall accuracy. Appendix F.2 contains further results including accuracy when noise is added to spurious (irrelevant) region. The results again substantiate the relative strength of coaussteess-mehhote over regularization-basse Furthermore, we highlight the considerable improvement offered by the novel combination of robustness-bast based and regularization-be baase methods.
5.6 Overall results
Among the regularization--ased methods, Grad-Reg performed the best while also being simple and intuitive.
pooustness-based methods except Avg-Ex are consistently and effortlessly better or comparable to regularization-baeed methods on all the benchmarks with improvement to Wg accuracy by 3-10% on the two real-world datasets. Combined methods are better than their constituents on all the datasets readily without much hyperparameter tuning.
In Appendix, we have additional experiment results; generality to new explanation methods (Ap- pendix F-6) and new attention map-based architecture (Appendix F.7 and sensitivity to hyperparam- eters PGD-Ex on Plant and ISIC dataset (Appendix) F.4) and other perturbation- baasd methods on MNIST dataset (Appendix F.8).

---

Related Work
Sub-population shift robustness. Sagawa et al. (2019) popularized a problem setting to avoid learning known spurious correlation by exploiting group labels, which are labels that identify the group an example. Unlike MLX, the sub-population shift problem assumes that training data contains groups of examples with varying strengths of spurious correlations. For example in the ISIC dataset, images with and without patch make different groups. Broadly, different solutions attempt to learn a hypothesis that performs equally well on all the groups via importance up- up-weighting (Shimodaira (2000); [Byrd & Lipton (2019). balancing subgroups et al. (2019); [zmailov et al. (2022); Kirichenko al. (2022)) or group-wise robust optimization (Sagawa al. (2019)). Despite the similarities, MLX and sub-population shift problem setting have some critical differences. Sub- population shift setting assumes sufficient representation of examples from groups with varying levels of spurious feature correlations. This assumption may not always be practical, e.g. in medical imaging, which is when MLX problem setting may be preferred.
Learning from human-providee explanations. Learning from explanations attempts to ensure that model predictions are "right for the right reasons" (Ross et al. 2017). Since madient-bbsed explanations employed by |Ross al. (2017) are known to be unfaithful (Murphy) 2023) (Chapter 33) (Wicker et al. 2022), subsequent works have proposed to replace the explanation method while the overall loss structure remained similar. [Shao et al. (2021) proposed to regularize using an influence function, while Rieger et al. 2020) proposed to appropriately regularize respective contributions of relevant or irrelevant features to the classification probability through explanation method proposed by /Singh et al. (2018). In a slight departure from these methods, Selvaraju et al. (2019) used a loss objective that penalises ranking inconsistencies between human and the model's salient regions demonstrated for VQA applications. Stammer et al. (202T) argued for going beyond pixel-level importance scores to concept-level scores for the ease of human intervention. On the other hand, Singla et al. (2022) studied performance when augmenting with simple perturbations of irrelevant features and with the gradient regularization of Ross et al.) (2017). This is the only work, to the best of our knowledge, that explored robustness to perturbations for learning from explanations.
Robust training. Adversarial examples were first popularized for neural networks by Szegedy al. (2013), and have been a significant issue for machine learning models for at least decade (Biggio& & Roli 2018). Local methods for computing adversarial attacks have been studied (Madry al. 2017). 7). but it is well known that adaptive attacks are stronger (i.e., more readily fool NNs) than general attack methods such as PGD (Tramer et al.] (2020). Certification approaches on the other hand are guaranteed to be worse than any possible attack (Mirman et al. (2018). and training with certification approaches such as IBP have been found to provide faat--of-the-an results in terms of provable robustness (Giowal et al. 2018), uncertainty (Wicker et al. 2021), explainability (Wicker et al. 2022). and fairness (Benussi et al. 2022).
7 Conclusions
By casting MLX robustness problem and using human explanations to specify the manifold of perturbations, we have shown that it is possible to alleviate the need for strong parameter smoothing evaluated two strong
of earlier approaches. Borrowing from the well-studied topic of robustness, approaches, one from adversarial robustness (PGD-Ex) and one from certified robustness (IBP-Ex). In our evaluation spanning seven methods and four datasets including found that PGD-Fx and IBP-Ex performed better than any previous approach, while our final proposal IBP-Ex+Grad-Re and PGD-Ex+Grad Reg of combining IBP-Ex and PGD-Ex with a light-weight interpretation based method respectively have consistently performed the best without compromising compute efficiency by much.
real-world datasets
Limitations. Detecting and specifying irrelevant regions per-example by humans is laborious and non-trivial task. Hence, it is interesting to see the effects of learning from incomplete explanations. which we leave for future work. Although robustness-baat methods are very effective as we demonstrated, they can be computationally expensive (PGD-Ex) or scale poorly to large networks (IBP-Ex).
Broader Impact. Our work is step toward reliable machine learning intended for improving MI. for all. To the best of our knowledge, we do not foresee any negative societal impacts.
10

---

8 Acknowledgements
MW acknowledges support from Accenture. MW and AW acknowledge support from EPSRC grant EP/V056883/1. AW acknowledges support from a Turing AI Fellowship under grant EP/VO25279/1, and the Leverhulme Trust via CFI. We thank Pingfan Song, Yongchao Huang, Usman Anwar and multiple anonymous reviewers for engaging in thoughtful discussions, which helped greatly improve the quality of our paper.
References
Benussi, E., Patane, A., Wicker, M., Laurenti, L., and Kwiatkowska, M. Individual fairness guarantees for neural networks. arXiv preprint arXiv:2205.0576. 2022.
Biggio, B. and Roli, Wild patterns: Ten years after the rise of adversarial machine learning. Pattern Recognition, 84:317-331, 2018.
Byrd, J. and Lipton, Z. What is the effect of importance weighting in deep learning? In International conference on machine learning, pp. 872-881. PMLR, 2019.
Codella, N., Rotemberg, V., Tschandl, P., Celebi, M. E., Dusza, S., Gutman, D., Helba, B., Kalloo, A., Liopyris, K., Marchetti, M., et al. Skin lesion analysis toward melanoma detec- tion 2018: challenge hosted by the international skin imaging collaboration (isic). arXiv preprint 1rXiv:1902.03368, 2019.
contributors, W. Linearity of differentiation. Wikipedia, 10 2022. URL https::/een http:///ewwwwiiiii.e..... wikipedia org/wiki/Linearityy of hfw..y/rrrffffeettatiitt.o.
Cui. Y.. Jia, M., Lin, T.-Y., Song, Y., and Belongie, S. Class-balanced loss based on effective number of samples. In Proceedings of the IEEE/CVF conference computer vision and pattern recognition, pp. 9268-9277 2019.
DeGrave, A. Janizek, J. D., and Lee, S.-I Ai for radiographic covid- 19 detection selects shorteuts over signal. Nature Machine Intelligence, 3(7):610-619, 2021.
D' Amour, A., Heller, K., Moldovan, D., Adlam, B., Alipanahi, B., Beutel, A., Chen, C., Deaton, J., Eisenstein, J., Hoffman, M. D., al. Underspecification presents challenges for credibility in modern machine learning. Journal of Machine Learning Research, 2020.
Geirhos, R., Jacobsen, J.-H., Michaelis, C.,, Zemel, R., Brendel, W., Bethge, M., and Wichmann, F. A. Shortcut learning in deep neural networks. Nature Machine Intelligence, 2(11):665-67) 673, 2020.
Gowal, S., Dvijotham, K., Stanforth, R., Bunel, R., Qin, C., Uesato, J., Arandjelovic, R., Mann, T., and Kohli, P. On the effectiveness of interval bound propagation for training verifiably robust models. arXiv preprint arXiv: 1810.12715, 2018.
Hennig, P., Osboree, M. A., and Kersting, H. P. Probabilistic Numeries: Computation Machine Learning. Cambridge University Press, 2022. doi: 10. 1316681411.
Izmailov, P., Kirichenko, P., Gruver, N., and Wilson, A. G. On feature learning in the presence of spurious correlations. Advances in Neural Information Processing Systems, 35:38516-385322
2022.
Kirichenko, P., Izmailov, P., and Wilson, A. G. Last layer re-training is sufficient for robustness to spurious correlations. arXiv preprint 22ti::22004.2997,.. 2022.
Liu, E. Z., Haghgoo, B., Chen, A. S., Raghunathan, A., Koh, P. W., Sagawa, S., Liang, P., and Finn, C. Just train twice: Improving group robustness without training group information. In International Conference Machine Learning, pp. 6781-6792. PMLR, 2021.
Madry, A., Makelov, A., Schmidt, L., Tsipras, D.,, and Vladu, A. Towards deep learning models resistant to adversarial attacks. arXiv preprint arXiv: 06083, 2017.
Miao, K., Gokul, A., Singh, R., Petryk, S., Gonzalez, J., Keutzer, K., and Darrell, Prior knowledge- guided attention in self-supervised vision transformers. arXiv preprint arXiv:2209.0377 2022.

---

Mirman, M., Gehr, T., and Vechev, M. Differentiable abstract interpretation for provably robust
neural networks. In International Conference on Machine Learning, pp. 3578-3586. PMLR, 2018.
Murphy, K. P. Probabilistic Machine Learning; Advanced Topics. MIT Press, 2023, URL URLhttp: //probml cerobml.github..... crithub.io/boode
Rieger, L., Singh, C., Murdoch, W., and Yu, B. Interpretations are useful: penalizing explanations to align neural networks with prior knowledge. In International conference machine learning, pp. 81 16-8126. PMLR, 2020.
Ross, A. S., Hughes, M. C.,, and Doshi- F. Right for the right reasons: Training differentiable models by constraining their explanations. arXiv preprint arXiv: 703.033177 2017.
Sagawa, S., Koh, P. W., Hashimoto, T. B., and Liang, P. Distributionally robust neural networks for
group shifts: On the importance of regularization for worst-case generalization. arXiv preprint 1911009911.087/1 2019.
Schramowski, P., Stammer, W., Teso, S., Brugger, A., Herbert, F., Shao, X., Luigs, H.-G., Mahlein, A.-K. and Kersting, K. Making deep neural networks right for the right scientific reasons by interacting with their explanations. Nature Machine Intelligence, 2(8):476 486, 2020.
Selvaraju, R. R., Lee, S., Shen, Y., Jin, H., Ghosh, S., Heck, L., Batra, D., and Parikh, D. Taking a hint: Leveraging explanations make vision and language models more grounded. In Proceedings of the IEEE/CVF international conference on computer vision, pp. 2591-2600 2019.
Shah, H., Tamuly, K., Raghunathan, A., Jain, P., and Netrapalli, P. The pitfalls of simplicity bias in neural networks. Advances in Neural Information Processing Systems, 33:9573-9585, 2020.
Shah, R., Varma, V., Kumar, R., Phuong, M., Krakovna, V., Uesato, J., and Kenton, Z. Goal misgeneralization: Why correct specifications aren't enough for correct goals. arXiv preprint arXiv:2210.01790. 2022.
Shao, X., Skryagin, A., Stammer, W., Schramowski, P., and Kersting, K. Right for better reasons: Training differentiable models by constraining their influence functions. In Proceedings of the AAAI Conference on Artificial Inielligence, volume 35, pp. 9533-9540, 2021.
Shimodaira, H. Improving predictive inference under covariate shift by weighting the log likeliidl function. Journal of statistical planning and inference. 90(2):227-22 244, 2000.
Singh, C., Murdoch, W. J., and Yu, B. Hierarchical interpretations for neural network predictions. arXiv preprint arXiv:1806.0m 05337, 2018.
Singla, S. and Feizi, S. Salient imagenet: How to discover spurious features in deep learning? arXiv preprint arXiv:2 2021.
Singla, S., Moayeri, M., and Feizi, S. Core risk minimization using salient imagenet. arXiv preprint 22Xi::22033.5566 2022.
Smilkov, D., Thorat, N., Kim, B., Viégas, F., and Wattenberg, M. Smoothgrad: removing noise by adding noise. arXiv preprint arXiv: 1:Xxv:/706.0322.. 2017.
Stammer, W. Schramowski, P., and Kersting, K. Right for the right concept: Revising neuro- symoo-yymbolic concepts by interacting with their explanations. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3619-3629, 2021.
Sundararajan, M., Taly, A., and Yan, Q. Axiomatic attribution for deep networks. In International conference on machine learning, pp. 3319-3328. PMLR, 2017.
Szegedy, C., Zaremba, W., Sutskever, I., Bruna, J., Erhan, D., Goodfellow, I., and Fergus, R. Intriguing properties of neural networks. arXiv preprint arXiv: 1312.6199, 2013.
Tramer, F., Carlini, N., Brendel, W., and Madry, A. On adaptive attacks to adversarial example
defenses. Advances in Neural Information Processing Systems, 33:1633-1645, 2020.
12

---

Tschandl, P., Rosendahl, C., and Kittler, H. The ham 10000 dataset, a large collection of multi-source dermatoscopic images of common pigmented skin lesions. Scientific data, 5(1):1-9, 2018.
Wicker, M., Laurenti, L., Patane, A., Chen, Z., Zhang, Z., and Kwiatkowska, M. Bayesian inference with certifiable adversarial robustness. In International Conference on Artificial Intelligence and Statistics, pp.
PMLR, 2021.
Wicker, M., Heo, J., Costabello, L.,, and Weller, A. Robust explanation constraints for neural networks. arXiv preprint arXiv:2212.08552 2022.
Zech, J. R., Badgeley, M. A., Liu, M.. Costa, A. B., Titano, J., and Oermann, E. K. Variable generalization performance of a deep learning model to detect pneumonia in chest radiographs: cross-sectional study. PLoS medicine, 15(11)xe1002683, 2018.
Zhang. H., Chen, H., Xiao, C., Gowal, Stanforth, R., Li, B., Boning, D., and Hsieh, C.-J. Towards stable and efficient training of verifiably robust neural networks. arXiv preprint arXiv: arXiv: /10.:/69/0063//111 16, 2019.
13

---

Supporting material for "Use Perturbations when Learning from Explanations"
A Proof of Theorem
We restate the result of Theorem 1 for clarity.
We infer a regression function f from GP prior as described above with the additional supervision of (x)0x/0 = 0, vi E [1, N.. Then the function value deviations to perturbations on irrelevant feature are lower bounded by value proportional the perturbation strength ó as shown below.
28a 01zî2 +52
f + (0.0 f(x)
B
Proof outline. We first show that the posterior mean of the function estimates marginalised over hyperparameters with Gamma prior has the following closed form where d(x,, y) = (x y) and denotes original observations augmented with observations on gradients, which is described in more detail further below.
(x: A Eg [mp]
m
;a, 3)d0 200
N
3 ((n))) y ((+
[(x)
Σ n=1
j(n)
d(ar.am (n)
1 d(za. (m)
1
We then derive the following lower bound on the function value deviations and finally use simple inequalities arrive the final result.
a+1
20a
f(x (xx+10,87)) (0,8]-) -(x)) -0(0))))(xx>
s(x) 2
Σ
B
(( (m))
1 d(ar (re)
2(52 - J22 (n) ar2 +
x22 (n)
(a + 1)8n+N
-1
In
B + dl 12, x22 (n))
Proof. We first derive the augmented set of observations (0) and K explained in the main section.
w2.922........... .2.............................. UN. NN, ( (5(x
... ( /802)
cxp(- 2
(x
(1))
when i
2
1=1
0
(o(0)
exp( Σ k=1
when N.j
(9) ((())))
exp( 2 k=1
when N,
2
21
exp( Lk=1
exp(=
(3
when i, N
0g
These results follow directly from the results on covariance between observations of and its partial derivative below (Hennig et al. 2022).
0f(E),
ak ak( .)
cov(s(x),
07 0i
cov( 0f(*) 0f j2 3)
z
OxdZ
The posterior value of the function at an arbitrary point x would then of the form p(f(x) D) ~N(((x); kz) where m and h2 are have the following closed form for Gaussian prior and

---

Gaussian likelihood in our case.
Ma X) h₂ = ((2
Since my, h2 are functions of the parameters 8,, 82. we obtain the closed form for posterior mean by imposing a Gamma prior over the two parameters. For brevity, we denote by d(x, (x -)² /2 and g(t) the z(th) component of Xxx9.
f(x) - Ea [mmr) = / m. 9(0 ;,, ar,3) 3)000) ?d0?
N
(I2 12 (m)
Σ (1,,0011))
+
k(r, x (1))
JutN
02
1 10
= =ʃʃ exp
2 - =1(1)}2
85 (x2 x2 )
ga
0 2a+2
exp exp(-3079
2
2
r(a)
g
0g
2a+2
exp
(-30g
In d0
2 d02²
[(a)
B
B
yn
8 +
(2)
-
3
(2g -
bn
2
[
122
(n)
x(xxx((1))
2a (,,08 2d0
02
B
3 (T/)(a)
(x2
Un+N
B
+
(21-2) (
(B + (#2-x)
Jot1 ((a 11)
8
a (x2
B
Un+N
8 +
(s) v
(22 - 27 12)3
3 +
1n p (22 m
Overall, we have the following result.
N
(x2 (n)
f(x)
In +
In+N
d(z
(n))
(m))
1 diagg
1
1 +
We now derive the sensitivity to perturbations on the second dimension for xx= 10,0
f(x + Ax) f(xx== =
In
+ d/ v (n)
d(22-6,
8
3 (x2 + 8 (n)
(m)
(12
22
In+N
(1 +
d(x2 4 5,
a-
(1
d(1r2 (e 3
a++1
8
15

---

Using Bernoulli inequality, (1+x)" 2 1 + rz if & O, we derive the following inequalities.
d(ag+baa (1))
diaz.an mi)) B
1
B + d( J22 (1))
dl (m)y
d(02 + 8, x22 (m)) 3
d 82 + 6, x2 (1)))
d(zza) (m)
a
B + (1, 122 (m))
d( 122 (1)) d(I2 + 2, (m))
1+ d(zgaa (n)
B + d 22 x22 (m))
Assuming
D2
12
>
vn E [N]
(8)
20(122 - x (m))
(9)
d(122,2 Imb
B + d(Ir2 (n))
1
Similarly,
(n))
(n)
(22 5 - 222
9 (22 x22
(1 + d(xc+6,22
jatl
(1 + d(as.)) )at1
220 - 2b) (1))
8
8 222 (m))
(a+1)
1 d(sraaa (m)) B
3 + d(ag, =mm
d(a a a+1
+
8
a+1
-2 n.yy ((n))
0
3 12 - 2e (m))
(a + 1)
a(2z,a (n) B
3 + d(022 (1))
1
8
28(x22 - 2(1))
(a + 1)
3 + (x2, x2 (1)) +1
(1 d(wa (m)
3
a - 1
-28 22 ) aa/2 1 - 28 28/22/////// 0/3(12222
x22
(x)
da
+
(1 + d(sz.ee (m))
jatt B
3 d(o2,22" 222
(n)
B
-28a(a 11
2( 2.10 )22 + 8 x22 (n)
(10)
3(1 + d( (1))
d(z2 m(m)
(n)
aa+1
Using inequalities 9 10 in Equation 7
we have the following.
f(x + Ax) f(x) 2 Σ
d( (n)g +
(nt (n)
B
1
28aT
28a(a + 1)8n+N
-2(x1 (a) [(22)) + 8 - (m)
+1
3 + d( (22, (n)
3 + (122)
+ d( (n)
16

---

a+1
20σ f(x+ Ax) f(x) 2
Σ
1 d(r s((1) B
1 d(ra. 2
B
22 (1s) 122 + (m)
(16)
(a + 1)b++N
-1 In
(11)
B + d( 22
Using the inequality (1 x)* 2 1 rx if S 0, we have
f(x Ax)
-(xx=
2
20a
Σ
{
y _| 2101)) (1 (1-
a +
2122 (m])
B
a +
Un+N (2xx2 (m) [x2 + 2 (1 - d( 222
2( (11))) - 1) iha
B
28a
(z
5 + 627
B Proof of Theorem 2
We restate the result of Theorem 2 for clarity.
When we use adversarial robustness algorithm to regularize the network, the fitted function has following property.
[f(x + 10,00))-
f(xx) S 3
mmes fmaz
where C = max min [x2 S22
xEX &E
δmaz and fmaz are maximum value of Aro and [x) in the input domain (2) respectively. A denotes the subset of inputs covered by the robustness method. C therefore captures the maximum gap in coverage of the robustness method.
Proof. We begin by estimating the Lipschitz constant of GP with squared exponential kemnl..
f(x) Ka
XxX
0f(x)
OK,x oK5XK*X
Xx!
Kr KXXY
Ox
Ox2
9
((01 - =x( (x2 x22 (2)
(m)
yp)}
where [KaX
exp(-
Oxz
20²
(x2 - (m))
[Kz)]
B²
0f(r)
N
(I2
K Kax]mKX
dx2
02
==1
We denote with 8maz the maximum deviation of any input from the training points, i.e. we define òmaz as maXxex minneiN] [22 (n) Also, we denote by fmaz the maximum function value in the input domain, i.e. Jmaz A maxxex (x). We can then bound the partial derivative wrt second dimension as follows.
0f(x)
f(x) 82
8max fmax g2
Oz
For any arbitrary point the maximum function deviation is upper bounded by the product of maximum slope and maximum distance from the closest point covered by the adversarial distance method. [[(121)22)²) (1x1,02)))
8man J'aa
8man. J'aa
max min ID2 T2l XEX &EX
C
Ø²
8²
17

---

Therefore,
Smox C
If(x + (0,0 f(x)
0
Marginalising 8- with the Gamma prior leads to the final form below.
ff(x + [0, f(x)) $ 2C 8maz
C Proof of Proposition I
We restate the result here for clarity.
Consider a regression task with D+ + 1-dimensional imputs x where the first D dimensions are irrelevant, and assume they Id = y. d E [1, while ID+1 ~ NNy, 1/K). The MAP estimate of linear regression parameters f'xx Dde1 D41 waza when fitted using Avg-Ex are as follows: 1/(D K), d [1, D) and WD+1 K/(K D).
Proof. Without loss of generality, we assume a, o² parameters of Avg-Ex are set to In effect, our objective is to fit parameters that predict well for inputs sampled using standard normal perturbations, i.c. x()) + me, Vn E [1, E ~ W(0, 1), = [1, (.... 1, 0) E {0, 1] The original problem therefore is equivalent fitting on transformed input such that x(") Ny,, where of 1 for all i < D and is 1/K when i = D + 1.
I Likelihood of observations for the equivalent problem is obtained as follows.
D+1
D+1
P(y 12, .
I+1)
II Ply (.)) x II
P(t: P(y)
(y 22)²
II (a y, of x exp( Σ
20
exp
- yy²
+ y( Σ
)
+
Σ
20
20
x No: Σ
Si P,P)
1
where P
Σ, 1/07
Substituting, the value a, defined as above, we have P-D+K and the MLE estimate for the linear regression parameters are as shown in the statement. The MAP estimate also remains the same since we do not impose any informative prior on the regression weights.
D Parametric Model Analysis
In this section we show that a similar result what is shown for non- con-marmetrric models also holds for parametric models. We will analyse the results for two-layer neural networks with ReLU activations. We consider a more general case D dimensional input where the first d dimensions identify the spurious features. We wish to fit function f RDD - IR such that f(x) is robust to perturbations to the spurious features. We have the following bound when training model using gradient regularization of Ross et al. (2017).
We assume that the model is parameterised as a two-layer network with ReLU activations such that f(x) = E, B WysT + by) where 3 E RF, W FxD De RF are the parameters, and Ø(2) max(2, 0) is the ReLU activation. For any function such that gradients
Proposition 2.
18

---

WTI to the first d features is exactly zero, i.e. ( = W [1,d], E [1,N]. we have the following bound on the function value deviations for input perturbations from training instance x: [Ax 0 04t1.Dl"
T I Ax
(f( f(x)] +
(12)
For two-layer network trained to regularize gradients wrt first d dimensions on training data, the function value deviation from an arbitrary point x from training point x such that x X Ax [x 0g D2+1: is bounded as follows.
(f( f(x)] =(())= ||
Proof. Recall that the function is parameterised using parameters प, 5, such that f(x) B5 Wzs2; + b,) where 3 RF, の E RPDD R are the parameters, and d(2)
max(2, is the ReLU activation.
Since we train such that af(x) 8x,
0, iE [1, we have that 0f(x) 2;
E3 wayss + bij
where d(a) max fG.0): 0).
We now bound the variation in the function value for changes in the input when moving from - x where x is an instance from the training data. We define four groups of neurons based on the sign of E; wyrr; + by and E. Wysce + bj. gi both positive, 92 is negative and positive, Ia is positive and negative, 94 is both negative, By defining groups, we can omit the ReLU activations as below.
f(x) f(x)= = Σ 3 Σ
WyyCe + b,)
Σ
3,01 Σ
WysT: + by)
=
Σ
31 Σ
Wy((o, - -x+) + Σ 3,
WjsCe + +b,) bb)) Σ
8
Wyzc + bb))
jEgI
jEg2
jEga
31
Wir(0, - x2)= + Σ 8,
Wettt + bb)
Σ
8
Wjzz; + bj)
JE91
jE9a
jEgs
Since we have that Σ
Ugs
Byusi = 0, Vi E [1, d). we have
d
d
D
D 3,( Σ
Σ
3
W +
W
Σ
B3
W +
Σ
WyHT + +b²)
Σ
Wait + by)
i=1
jEna
imd+1
jEgs
ind+1
B3
JEM1
i=l
jEgB
=Id
Esesr
P,
Σ 199
3,
M i=1
+
Σ JE92
B
Σ
Wy + b,)
Σ B
Σ wy
b,)
i=d+1
JE03
(=d+1
retaining only the terms that depend on Ar f x, the expression is further simplified as term that grows with Ax and a constant term that depends on the value of
Σ B3 Uga
Az, + constant
i=1
Ax()) 2( + |ل |))
|||||||||||]
Sauchy-Sehwartz inequality
19

---

E Further Experiment Details
E.1 Hyperparameters.
We picked the learning rate, optimizer, weight decay, and initialization for best performance with ERM baseline on validation data, which are not further tuned for other baselines unless stated otherwise. We picked the best À for Grad-Reg and CDEP from [1. 10, 100, 1000]. Additionally, we also tuned (weight decay) for Grad-Reg from [1e-4, le-2, 1, 10]. For Avg-Ex, perturbations were drawn from 0 mean and o² variance Gaussian noise, where was chosen from [0.03, 0.3, 1, 1.5, 2]. In PGD-Ex, the worst perturbation was optimized from lo norm e-ball through seven PGD iterations. where the best is picked from the range 0.03-5. We did not see much gains when increasing PGD iterations beyond 7, Appendix F contains some results when the number of iterations is varied. In IBP-Ex, we follow the standard procedure of Gowal et al. (2018) to linearly dampen the value of a from to 0.5 and linearly increase the value of from to Emaz, where Emaz is picked from 0,01 to 2. We usually just picked the maximum possible value for émaz that converges. For DBP-Ex+Grad Reg, we have the additional hyperparameter (Eqn. 4). which we found to be relatively stable and we set it to 1 for all experiments.
E.2 Metries
Relative Core Sensitivity (RCS) (Singla et al. 2022). The metric measures the relative dependence of the model on core features and is normalised such that the best value is 100. Higher value of RCS imply that the model is exploiting core features more than the spurious features.
RCS = 100 x (cc acc(s)
2 2mi a) where ā == acc(C) + (cc
2
acc
A
Σ ( (x()) + t ( -yn))
acc
A
Σ1
( (z(20 (1 -) =y(r))
N
"
where z ~ (N I) and / 0.25 for all experiments. The interpretation of (C) is the accuracy when noise is added outside of core region, and acc(5) is the accuracy when noise is added outside of spurious region.
E.3 Data splits
We randomly split available labelled data in to training, validation, and test sets in the ratio of (0.75, 0.1, 0.15) for ISIC, (0.65, 0.1, 0.25) for Plant (similar to (Schramowski et al. (2020)) and (0.6, 0.15, 0.25) for Salient- Imagenet, We use the standard train-test splits on MNIST.
E.4
Datasets
SIC dataset The ISIC dataset consists of 2,282 cancerous (C) and 19,372 non-cancerous (NC) skin cancer images of 299 by 299 size, each with ground-truth diagnostic label. We follow the standard setup and dataset released by |Rieger et al. 2020), which included masks with patch segmentations. In half of the NC images, there is spurious correlation in which colorful patches are only attached next to the lesion. This group is referred to as patch non-cancerous (PNC) and the other half is referred to as not-patched non-cancerous (NPNC) Codella al. (2019). Since trained models tend to learn casy-to-learn and useful features, they tend to take a shortcut by learning spurious features instead of understanding the desired diagnostic phenomena. Therefore, our goal is to make the model invariant to such colorful patches by providing a human specification mask indicating where they are.
decoy-MNIST dataset The MNIST dataset consists of 70,000 images of handwriting digit from 0 to Each class has about 7,000 images of 28 by 28 size. We use three-fully connected layers for multi classification with 512 hidden dimension and 3 channels.
Salient- Imagent., The six classes we considered are Rhinoceros Beetle, Dowitcher, Alaskan Tundra Wolf. Dragonffy, Gorilla, Snoek Fish.
20

---

E,5 Computational cost
Run time and memory usage Table presents the computation costs, including rum time and memory usage, for each method using GTX 1080 Ti. It is worth noting that IBP-Ex has significantly less run time and memory usage compared PGD-Ex. with a 10-fold reduction in run time and 2.5-fold reduction in memory usage. Considering that RGD-Ex and IBP-Ex have similar performance in terms of worst group accuracy, as shown T IBP-Ex+Grad- appears to be comparably effective and efficient for model modification. Additionally, the combined method IBP-Ex+Grad--Re which presents the best performance terms of averaged and worst group accuracy compared to PGD-Ex, also has a 3-fold reduction in run time and 2-fold reduction in memory usage compared to PGD-Ex.
Grad-Reg
PGD-Ex
IBP-Ex
Eop-Ex+Grad-Reg
PGD-Ex+Grad-Res
x2.3
x4.9
x2.2
X3.5
7.0
Table 3: Running time in comparison to ERM on the ISIC dataset
E.6 Network Architecture
Model architecture on the decoy-MNIST dataset
Sequential
(0): Conv2d 32, kernel_size=(3, 3), stride=(1, 1), padding= 1))
(1): ReLU()
(2) : Conv2d (32, 32, kernel_size= (4, 4), stride= 2), padding= (1, 1))
(3): : ReLU()
(4) Conv2d (32, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1. (1, 1))
(5) : ReLU()
(6) Conv2d 64, kernel_size=(4 4), stride=(2, 2), padding= (1, 1))
(7): ReLU()
(8) : Flatten Flatten(star__dim-11 end and_dim=-1)
(9): : Linear (in_aaaettteturee220001
out out__eaature=1002, bias=True)
(10): ReLU()
(11): Linear (in_features=1024, out_ out_feattues=1124, bias=True)
(12): ReLU()
(13) Linear (in fii__ffatuurs=1024, out_features=2, bias=True)
Model architecture on the ISIC dataset
Sequential(
(0): Plattensttart_digg end_ and_dim=-1)
(1): Linear (ia__aiinttfuuss22355 out out_features=512, bias=True)
(2) ReLU()
(3) : Linear (in_featuues=512, out out_features=512, bias=True)
(4) ReLU()
(5) : Linear (in_features=551,, out out_featture=-12, bias=True)
(6) ReLU()
(7): Linear (in_features=512, out fut_feetures=10, bias=True)
Model architecture on the Plant phenotyping dataset
Sequential
(0): Conv2d 32, kernel_size- (3, 3), stride= (1, 1),, padding= (1, 1))
(1): ReLU()
(2) : Conv2d (32, 32, kernel_size= (4, 4), stride (2, 2), padding (1, 1))
(3): : ReLU()
(4) Conv2d (32, 64, kernel_size= (3, 3), stride=(1, 1), padding=(( (1, 1))
(5) : ReLU()
(6) Conv2d(64, 64, kernel_sizes 4), stride= (2, 2), padding=(( (1, 1))

---

(7): ReLU()
(8): Fiatten end_dim=-1)
(9): (inearrin_ out out_ffaature=1024, bias=True)
(10): ReLU()
(11): Linear Lin_ffaature=1122, out_features=1024, bias=True)
(12): ReLU()
(13): Linear (in fin_treeture-1024, out_features=2, bias=True)
Additional Results
F. Standard deviations
We repeated all our experiments on Decoy-MNIST, Plant and ISIC dataset three times and report the mean and standard deviation in Table Similarly, report in Table 5 the standard deviations for the corresponding Table of the main paper.
Dataset->
Decoy-MNIST
Plant
ISIC
Method
Avg Acc
Wg Acc
Avg Acc
Wg Acc
Avg Acc
Wg Acc
ERM
15.1 + 1.3
10.5
71.3 1 2.5
54.8 1.3
77.3 7
55.9 2.3
G-DRO
64.1 0.1
28.1 0.1
74.2 + 5.8
58.0 - 4.6
66.6
58.5 - 10.7
Grad-Reg
1.12
72.4
68.2
7.44
60.2 7.4
CDEP
14.5 1.8
10.0
67.9 10.3
24.7
7
60.9 3.0
Avg-Ex
29.5
1.4
76.3
64.5
2.1
55.2
PGD-Ex
6.66
0.3
79.8 0.3
78.5 0.3
64.4
IBP-Ex
68.1 2.2
47.6
+ 2.0
76.6 3.5
73.8 1.7
75.1 + 1.2
64.2 + 1.2
P+G
96.9 + 0.3
95.8
0.4
79.4
76.7
2.8
79.6 + 0.5
67.5 + 1.1
I+G
96.9 + 0.2
95.0 + 0.6
81.7
+ 0.2
80.1
+ 0.3
78.4 + 0.5
65.2 + 1.8
Table Macro-averaged (Avg) accuracy and worst group (Wg) accuracy on (a) decoy-MNIST (b)
plant dataset, (c) ISIC dataset. Results are averaged over three runs and their standard deviation is
shown after +. I+G short for IBP-Ex+Grad-Res and P+G for PGD-Ex+Grad-RE Grad-Reg. See text for
more details.
Method
NPNC
PNC
C
Avg
Wg
ERM
55.9 L 2.3
96.5 2.4
79.6 1 6.6
77.3
2.4
55.9 1 2.3
G-DRO
72.4 1: 4.0
63.2 1: 14.8
64.1 5.6
66.6 i 5.4
58.5
1 10.7
Grad-Reg
67.1 4.8
99.0
1.0
15.21113
76.4
2.4
60.2 1
CDEP
1
98.9 1.9-:00 0.7
0.7
42.2.44.7
73.4
60.9 4. 3.0
Avg-Ex Ex
62.3 = 11.7
97.8 = 0.8
71.0
16.7
77.1 =21
55.2 35.2.60 6.6
PGD-Ex
5.4
0.3
6.7
78.7
0.5
14.4
IBP-Ex
68.4
98.5 1.0
67.7
75.1
1.2
64.2 + 1.2
P+G
6,.6
0.6
4.14
79.6
= 0.5
6.1
I+G
66.6 1.6-33 3.1
0.2
68.9 4.7
78.4
= 0.5
65.2 + 188
Table 5: Macro-averaged (Avg) accuracy and worst group (Wg) accuracy on ISIC dataset. Also
shown are the average precision scores for each of the three groups. All the results are averaged over three runs and their standard deviation is shown after =. Note that the worst group for each run can be different
F.2 Additional results on Salient-Imagenet
In T show average accuracy and accuracy when noise (drawn from standard normal) is added to spurious or irrelevant regions (N-Acc column). We observe that for PGD-Ex + Grad-Reg, the accuracy did not diminish by much when noise is added to the spurious region.
E3
Comparison of PGD-Ex and IBP-Ex
In T it is difficult to compare the worst group accuracy of IBP-Ex (64.2) and PGD-Ex (64.4) due to the comparably high standard deviation of PGD-Ex (4.3). Therefore, we additionally compare
22

---

Method
Accuracy
N-Acc
RCS 1
ERM
96.4
87.5
47.9
Grad-Reg
88.3
82.2
52.5
PGD-Ex
93.8
90.2
58.7
PGD-Ex + Grad-Reg
94.6
93.8
65.0
Table 6: The columns in that order the average accuracy, accuracy when noise is added to spurious (or irrelevant) regions and RCS value for our Salient- Imagenet data setup.
the accuracy drop when colorful patches are removed from images in the PNC group in Table 7 We replace the colorful patch of the image with its mean value, making looks like background skin color. Note that we evaluate the robustness to concept-level perturbations rather than pixel- level perturbations, as our focus is on avoiding spurious concept features rather than robustness to adversarial attacks. Interestingly, the accuracy drops about 17% and 37% in IBP-Ex and PGD-Ex, respectively, showing that IBP-Ex is more robust to concept perturbations. This can be explained by the effectiveness of robustness methods in covering the epsilon ball with the center of each input point defined in a low-dimensional manifold annotated in the human specification mask. IBP guarantees robustness any possible pixel combination within the epsilon ball while PGD only considers the worst case in the epsilon ball. When the inner maximization to find the PGD attack is non-convex, an inappropriate local worst case is found instead of the global one. Thus, IBP-Ex shows better robustness when spurious concepts are removed, which involves large perturbations on irrelevant parts within the defined epsilon ball. The combined method IBP-Ex+Grad-Rey where Grad-Reg compensates for the practical limitations of the training procedure of IBP-EX, shows about 1% higher worst group accuracy than IBP-Ex alone.
Method PGD-Ex
PNC
PNC (Remove patch)
99.0 + 0.3
62.2 + 17.0
IBP-Ex
98.5 1.0
81.6 16.5
IBP-Ex+Graa-RRg
99.6 + 0.2
82.5 + 9.5
Table 7: Comparison between robustness based methods. Macro-averaged accuracy and regval loss before and after removing color patch part of images in PNC group on ISIC dataset.
E4 Results of PGD-Ex with different epsilon and iteration number.
We experimented with different values of epsilon and iteration numbers on the ISIC and Plant phenotyping datasets. The epsilon values tested were 0.03, 0.3, 1, 3. and 5, and the iteration numbers were and 25. In Figure 4 the results on the ISIC dataset showed that using an iteration of 7 with different epsilon values resulted in stable results, but using iteration of 25 resulted in unstable worst group accuracy. However, in the Plant phenotyping dataset, we found that both average and worst group accuracy were similar regardless of the epsilon and iteration values used.
PGD BX on She ISIC datanet
2GD-BE Ex the Plaatt phenotyping datases
ara_ waga ang 10922
1. 00
00
0.75
C.75
Acouracy
0.55
0.50
0.25
0.25
0.00
0.00
0.
0.3
0. 13
0.3
3
Spallon
spollon
(a) PGD-Ex on the ISIC data
(b) PGD-Ex on the Plant phenotying data
Figure 4: PGD-Ex results the ISIC and Plant phenotyping dataset with different epsilon and iteration numbers in (a) and (b), respectively.
23

---

F,5 Out-of-distribution scenarios on the Plant data
In the main paper, we follow the dataset construction from Schramowski et al. (2020) to replace background with the average pixel value, which is obtained from train split. Here, in Table we evaluated on a test set obtained by adding varying magnitude of noise to the background to test methods under out-of-distribution scenarios. We observe that robustness and regularization methods when combined led to model that is far more robust to noise the background, aligning with our original results on the plant dataset.
Noise (O)
Noise (O(0, 10))
Noise (N(0,30)
Avg Acc
Wg Acc
Avg Acc
Wg Acc
Avg Ace
Wg Acc
ERM
59.8 11.9
43.5 + 2.0
7.6
38.1 4.8
55.8 1.7
22.0 3.7
Grad-Reg
71.6 + 2.0
66.1 + 1.8
68.7
53.4 + 4.3
56.1
34.8 1.8
PGD-Ex+Grad
69.8 + 1.8
2.1
6.7
60.6 4.8
4.5
50.8 2.4
Table 8: Out-of-distribution scenarios on the Plant data
F.6
Generality to new explanation methods: Integrated-grradien and CDEP
In Table 9 we introduced evaluation using Integrated gradient (Sundararajan et 2017) based regularization and also added evaluation with PGD-EX+CDEP that we did not originally include on Decoy-MNIST dataset.
Alg.
Avg Acc
Wg Ace
Integrated-Grad
26.7 +1.3
17.6 11.2
CDEP
14.5 +1.
10.0 #0.7
PGD-Ex
67.6 #1.6
51.4 #0.3
PGD- BetD--Extggggaed-GGrre Grad
80.5 +2.1
62.1 = 6.8
PGD-EX+CDEP
84.8 +0.8
64.2 1.6
Table 9: PGD-EEx-Innegratel-Gaad and PGD-EX+CDEP on the Decoy-MNIST data
Generality new model architecture: Attention map-based (ViT)
Using Visual transformer architecture (of depth 3 and width 128), evaluated regularization using local explanations obtained using an attention map regularization based on attention maps was used supervise prior knowledge in Miao al. (2022) and is called SPAN. We obtained saliency explanations on inputs using the procedure proposed in Miao et al. (2022), shown in Table TO
Alg.
Avg Acc
Wg Acc
ERM
10.0 +0.3
8.1 +0.3
SPAN
19.0 +0.3
8.1 +0.3
PGD-Ex
64.6 14.7
37.4 +33.5
SGAN
63.1 +2.6
39.4 + 2.9
Table 10: Attention Map based local explanations with ViT on the Decoy-MNIST data
F.8 Sensitivity to hyperparameters Decoy-MNIST dataset
In Table 11 we show sensitivity to hyperparameters on Decoy-MNIST dataset. In summary, results are broadly stable with the choice hyperparameters, and we did not extensively search for the best hyperparameters.
24

---

Decoy-MNIST
Lambda (Grad-Reg)
Eps (PGD-Ex)
Avg -Ace
Wg -Acc
PGD-Ex + Grad-Reg
96.9 0.3
95.8 + 0.4
0.1
3
96.8 + 0.8
94.2 + 0.2
1.5
95.6 1.0
93.0 1.0
5
3
91.6 0.9
2.3
0.0001
3
0.9
3.6
1
95.1 2.0
919
2
9.6
4
97.5
0.84
I
9
83.3
0.1
58.5 7.9
2.70
0.0001
59.5 1.1
40.1
PGD-Ex
0
3
67.6 + 1.6
51.4
0
0,0001
16.5 1.1
2.74
0
0.1
19.1 0.7
1..6
0
6
4..
0
2
5.6
528
+.5
12.6
0
5
+.1
42.6
Grad-Reg
10
0
64.1 0.1
2.11
21.5 392
20
0
49.1 2.7
33.2 4.3
100
0
50.1 + 0.4
35.2 + 2.3
500
48.0 0.9
36.2 + 1.7
1000
49.6 + 1.7
30.5 + 6.1
Table 11: Sensitivity to hyperparameters on Decoy-MNIST dataset
G Discussion on poor CDEP performance
Regarding ISIC dataset discrepancy: In Table 5 CDEP demonstrates better performance in worst group accuracy compared to ERM on the ISIC dataset. However, it fails to surpass RRR, which contradicts results from previous research in Rieger et al. (2020) where CDEP was found to perform better than RRR. This discrepancy may be attributed to the fact that Rieger et al. (2020) used different metrics (F1 and AUC) and employed a pretrained VGG model to estimate the contribution mask features, whereas in study used worst group accuracy and employed four-layer CNN followed by three fully connected layers without any pretraining. We do not use a pre-trained model for CDEP in order make fair comparison improve worst group accuracy over FRM on the Plant Phenotyping and Decoy-MNIST datasets. We further illustrate the interpretations of CDEP the Plant Phenotyping dataset using Smooth Gradient in Figure 5 In comparison to the interpretations of other methods shown in Figure in the main paper, CDEP appears to focus primarily on the spurious agar part instead of the main leaf part.
other methods. As result, CDEP also fails to
Regarding MNCOT dataset discrepancy: Note that our Decoy-MNIST setting is inspired from decoy-mnist mnist of CDEP (Rieger al. 2020), but not the same. All the methods were found to be equally good on the original decoy-mnist dataset (Rieger et al. 2020)), which is why we had to alter the dataset be more challenging. A key difference that the volume of spurious/simple features in our version of decoy-mnist dataset is much higher, making it harder to remove dependence of model on decoy/spurious features. This explains why there is the performance gap this dataset reported in our paper and CDEP (Rieger et al. 2020).
25

---

CDEP
Figure 5: Visual heatmap of salient features for CDEP on three sample images from the train split of Plant phenotyping data. Importance score from SmoothGrad Smilkov al. (2017) method is normalized between 0 to I and visualized with a threshold 0.6.
26


Reviews:
[{"review_id": "boV3jXissk", "reviewer": "Reviewer_haST", "summary": "The paper is in the domain of MLX (machine learning from explanations). In this approach, human annotated data for each input example is available denoting features that are _relevant_ and which are _irrelevant_. It is desired that the model doesn't learn from irrelevant features.\n\nIn this paper, the authors utilize robustness for this domain. To elaborate, they expect the model to be robust to perturbations along the features which are considered irrelevant. According to the authors, this is the first use of this technique in the domain of MLX. They also combine with existing regularization-based approaches.\n\nA theoretical framework is provided and approach evaluated on three datasets -- Decoy-MNIST, Plant, ISIC -- which are designed to capture the extent to which the models are learning from the irrelevant features. Combination of the robustness-based approaches with regularization-based approaches is shown to outperform prior approaches.", "questions": "Questions\n- Please address the choice of limiting the evaluation to the 3 datasets used.\n\nMinor suggestion (no response needed)\n- L63: State the domain of m^(n)\n- Consider placing figure closer to their first reference. For example, Figure 2 is referenced on Pg 2 but appears on Pg 6.\n- L70: \"while not exploiting\" -- please provide a the definition for this at this stage (addendum: if there is one at all)\n", "limitations": "Yes", "rating": 5, "confidence": 1, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "The use of robustness for MLX is well-motivated, and as per the paper, novel.\n\nThe empirical results appear to back the utility of combining robustness with existing techniques.\n\nThe paper is well-written.\n", "weaknesses": "The prior works (such as Sagawa et al., 2019; Piratla et al., 2021) have several more evaluation datasets, which the approach is not evaluated on. The generality and robustness (pardon the pun) of the approach is not completely clear.\n", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their time and comments.\n> The prior works (such as Sagawa et al., 2019; Piratla et al., 2021) have several more evaluation datasets, which the approach is not evaluated on. … Please address the choice of limiting the evaluation to the 3 datasets used. The generality and robustness … not completely clear.\n\nOur problem setting is such that we require an input mask per training instance highlighting irrelevant features. Standard sub-population shift datasets such as the ones used in Sagawa et al., 2019; Piratla et al., 2021 do not contain any input mask, which is why we cannot evaluate them. We elaborated on differences between ours and the sub-population shift problem in L335-346 of Section 6. \n\n\nStandard datasets with input masks highlighting irrelevant regions are somewhat hard to find; to the best of our knowledge, the three datasets included were the only standard datasets that were used in the past. Nevertheless, we evaluated using a recent dataset that included relevance masks, called Salient-Imagenet. Results on Salient-Imagenet can be found in the shared response, and are in agreement with other results in the paper. We hope that the results on Salient-Imagenet and other results presented in the shared response to be convincing of the generality of our proposal. \n\nThanks a lot for your suggestions on presentation, we will make sure to incorporate these in the final version.\n"}, {"review_id": "E7MbKeaBDr", "reviewer": "Reviewer_k7a7", "summary": "This paper proposes a new approach to machine learning from explanations (MLX). This new approach is still based on human-provided explanations of (ir)relevant features for each input in training, but recasts the MLX problem itself essentially into a robustness problem. The authors achieve SOTA performance when combining their method with previous ones on several benchmarks.", "questions": "Very minor: What is the bolding criteria for Table 1? Usually, the best model is in bold, but here there are 1, 2, or 3 models bolded depending on the dataset, and it's not clear what determined the number of bolded models for each.\n", "limitations": "It would be good to expand the \"Limitations\" section and provide more detail on, e.g., when the scaling breaks to large NNs, i.e. provide some guidance here about when one should expect the robustness methods to no longer be feasible to use. ", "rating": 7, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The authors show that the need for strong parameter smoothing of earlier approaches can be overcome, and they achieve SOTA performance on several benchmarks. Their method is intuitive and easy to understand. \n", "weaknesses": "Obtaining human-specified masks is at best a lot of effort and in many cases simply not available, which limits the scope of problems for which their method can be applied.\n", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their comments and time. We are glad that the reviewer found our method intuitive and our experiments convincing.  \n\n> Obtaining human-specified masks is at best a lot of effort …\n\nWe agree that manually specifying explanation masks can be impractical. However, the procedure can be automated if the nuisance/irrelevant feature occurs systematically or if it is easy to recognize, which may then be obtained automatically using a procedure similar to [1,2]. A recent effort called Salient-Imagenet used neuron activation maps to scale curation of such human-specified masks to Imagenet-scale [3, 4]. These efforts may be seen as a proof-of-concept for obtaining richer annotations beyond content labels, and towards better defined tasks.\n\n> Very minor: What is the bolding criteria for Table 1? \n\nBold numbers in Table 1 are the ones within statistical significance bounds of the best number. \n\n> provide more detail … when the scaling breaks to large NNs…\n\nWe discussed this limitation pertaining to IBP-Ex, which works by propagating axis aligned input intervals through the model. Despite its computational efficiency, IBP is known to suffer from scaling issues when the model is too big. Consequently, it is better to use IBP-Ex only when the model is small (<4 layers of CNN/feed-forward) and if computational efficiency is desired. Thanks for pointing it out, we will add this detail in the improved version of the paper.  \n\nWe also wish to emphasise that we do not anticipate any scaling issues when using PGD-Ex. Irrespective of the scale of the network or the size of the explanation region, we expect PGD-Ex+Grad-Reg to be at least as effective as Grad-Reg or ERM. \n\nReferences.  \n[1] Liu, Evan Z., et al. \"Just train twice: Improving group robustness without training group information.\" International Conference on Machine Learning. PMLR, 2021.   \n[2] Rieger, Laura, et al. \"Interpretations are useful: penalizing explanations to align neural networks with prior knowledge.\" International conference on machine learning. PMLR, 2020.   \n[3] Singla, Sahil, and Soheil Feizi. \"Salient ImageNet: How to discover spurious features in Deep Learning?.\" arXiv preprint arXiv:2110.04301 (2021).   \n[4] Singla, Sahil, Mazda Moayeri, and Soheil Feizi. \"Core risk minimization using salient imagenet.\" arXiv preprint arXiv:2203.15566 (2022).\n"}, {"review_id": "eSKyBZaKaf", "reviewer": "Reviewer_Hnkb", "summary": "This paper reinterprets Machine learning from explanations (MLX) as a robustness problem, where human explanations define a lower-dimensional manifold for perturbations. The paper points out that the previous regularization-based MLX approaches require strong model smoothing in order to be globally effective at reducing shortcut learning. The authors propose a novel approach that combines robust training methods with an earlier MLX technique, achieving state-of-the-art results on both synthetic and real-world benchmarks. The theoretical and empirical analyses explaining how the combination of robustness and regularization can reduce the need for strong model smoothing are provided.\n \n", "questions": "1. Is it possible to apply the proposed methods to other MLX approaches? To better understand the generalizability of the approach, it would be helpful to see the performance of robust training methods combined with various MLX methods.\n\n2. The authors mention in lines 277-279 that they evaluate in-domain test images with background pixels replaced by a constant pixel value. Have you tested the methods' performance under other out-of-distribution (OOD) scenarios, such as replacing background pixels with the background pixels from randomly selected images?\n\n3. A sensitivity analysis on the hyper-parameters should be provided. ", "limitations": "See weaknesses.\n\nAt this moment, the main limitations of the paper revolve around the limited novelty and generalizability of the proposed method. I tend to reject the paper primarily for this reason. However, I may reconsider my evaluation if the authors can provide solid evidence demonstrating the generalizability of their approach.", "rating": 5, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "1. The paper is well-structured and clearly organized, making it easy to follow;\n\n2. The authors offer theoretical analyses to explain how the combination of robustness and regularization can minimize the need for strong model smoothing, which adds to the rigor and solidity of the paper.\n\n", "weaknesses": "1. The contribution of the proposed method is restricted to the combination of two established robust training techniques with an existing MLX approach;\n\n2. Although the authors thoroughly review various regularization-based MLX approaches in the introduction, the paper only showcases the effectiveness of combining robust training methods with a single MLX approach, Grad-Reg. It remains unclear whether the proposed approach can be applied to other MLX methods, and further investigation is necessary to determine its generalizability.\n\nIn addition, Table 1 does not include the results of the more recent MLX approaches that are well discussed in the introduction section.\n", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their detailed review. We tried our best to address their concerns on contribution and generalizability of our method; we are more than happy to answer any further concerns. \n\n> The contribution of the proposed method is restricted to the combination of two established robust training techniques with an existing MLX approach;\n\nWe wish to highlight that our contribution also lies in (a) systematic study of robustness-based methods for learning from explanations, which is missing in the existing literature, (b) theoretical analysis and experimental validation of relative merits of regularisation and robustness-based methods. \n\nAlthough our final recommendation of combining regularization with robustness-based methods is simple, we found consistent gains with it across multiple datasets and architectures. In that regard, we only view the simplicity of our recommendation as an appealing aspect. \n\n> It remains unclear … can be applied to other MLX methods … necessary to determine generalizability. In addition, Table 1 does not include the results of the more recent MLX approaches that are well discussed in the introduction section.\n\nWe only evaluated Grad-Reg when combined with robustness methods because Grad-Reg far exceeded the performance of CDEP as reported in our results table (Table 1). “We omit comparison with Shao et al. (2021) [5] because their code is not publicly available and is non-trivial to implement the influence-function based regularization.” (L224-225). Schramowski et.al. (2020) [7] simply studied regularizing using gradient explanations (just like Grad-Reg) for supervising explanations.  \n\nNevertheless, we appreciate the concern and added in the shared response the results when combining three other MLX methods with a robustness-based method: PGD-Ex. We evaluated CDEP + PGD-Ex, Integrated-Gradient [8] and attention map using Visual Transformers (ViT) [2]. We observe from the results in the shared response that irrespective of the explanation method used, our claims remained valid: (a) using perturbations (i.e. robustness-based methods) for supervising explanations is better than using regulairzation-based methods, (b) combining robustness with regularization-based methods is at least as good or better than using robustness-based methods. \nWe hope that these results with new explanation methods and other results presented in the shared response to be convincing of the generalizability of our proposal. \n\n> Have you tested under other out-of-distribution (OOD) scenarios…plant dataset\n\nWe were just following the dataset construction from Shramowski et.al.[7] to replace background with the average pixel value (obtained using train split). We did not originally evaluate other shifts to tease out dependence on the background. Acting on your suggestion, we evaluated on a test set obtained by adding varying magnitude of noise to the background. We observe that robustness and regularization methods when combined led to a model that is far more robust to noise in the background, aligning with our original results on the plant dataset. \n\n|                              |  Noise (N(0, 1)) |                  | Noise (N(0, 10)) |                  | Noise (N(0, 30)) |                  |\n|------------------------------|:----------------:|:----------------:|:----------------:|:----------------:|:----------------:|:----------------:|\n|                              |      Avg Acc     |       Wg Acc     |      Avg Acc     |       Wg Acc     |      Avg Acc     |       Wg Acc     |\n| ERM                          | 59.8 ± 11.9    | 43.5 ± 2.0     | 57.4 ± 7.6     | 38.1 ± 4.8     | 55.8 ± 1.7     | 22.0 ± 3.7      |\n| Grad-Reg                     | 71.6 ± 2.0     | 66.1 ± 1.8     | **68.7 ± 6.2** | 53.4 ± 4.3     | 56.1 ± 3.3     | 34.8 ± 1.8     |\n| PGD-Ex+Grad-Reg | 69.8 ± 1.8     | **67.2 ± 2.1** | **69.5 ± 3.7** | **60.6 ± 4.8** | **67.5 ± 4.5** | **50.8 ± 2.4** |\n\n> A sensitivity analysis on the hyper-parameters should be provided.\n\nIn Appendix F, Figure-4 and L668-674, we presented sensitivity analysis of PGD-Ex hyperparameters (number of steps used for optimization and epsilon) on performance for plant and ISIC dataset. Below, we also show sensitivity to hyperparameters for PGD-Ex + Grad-Reg on Decoy-MNIST dataset. \nIn summary, results are broadly stable with the choice of hyperparameters, and we did not extensively search for the best hyperparameters. \n\n| Decoy-MNIST       | Lambda (Grad-Reg) | Eps (PGD-Ex) | Avg -Acc          | Wg -Acc           |\n|-------------------|-------------------|--------------|-------------------|-------------------|\n| PGD-Ex + Grad-Reg |                   |              |                   |                   |\n|                   | **1**             | **3**        | **96.9 ± 0.3** | **95.8 ± 0.4** |\n|                   | 0.1               | 3            | 96.8 ± 0.8      | 94.2 ± 0.2      |\n|                   | 5                 | 3            | 91.6 ± 0.9      | 87.6 ± 2.3      |\n|                   | 0.0001            | 3            | 75.5 ± 0.9      | 57.2 ± 3.6      |\n|                   | 1                 | 5            | 93.8 ± 1.6      | 86.3 ± 3.1      |\n|                   | 1                 | 0.1          | 58.5 ± 7.9      | 30.0 ± 2.7      |\n|                   | 1                 | 0.0001       | 59.5 ± 1.1      | 40.1 ± 2.0      |\n\nWe could not share all sensitivity analysis results here due to space constraints. We will, however, make sure to include them in the final version. \n\n> … reconsider my evaluation if … provide solid evidence … generalizability\n\nWe hope the reviewer finds our response to be convincing of the generalizability of our approach. We are happy to engage further to clarify any further concerns. We thank the reviewer once again for careful consideration of our paper. \n\nFor references, please see our shared response. "}, {"review_id": "4H0PEbgSMY", "reviewer": "Reviewer_WA6d", "summary": "This paper proposes a novel approach to Machine Learning from Explanations (MLX), the setting in which data points for learning are paired with human-annotated “explanations”, usually in the form of image masks marking regions that are known to be irrelevant with respect to the downstream prediction task. Previous methods for MLX made use of regularization-based techniques, forcing the learnt function to be locally independent of the irrelevant features for the training data points. The authors show that these methods require strong model smoothing to be effective, to the extent that downstream task performance is heavily penalized. They then propose “robustness-based” methods, which force the model to be robust to perturbation across the irrelevant features, and show theoretically and practically that these techniques work even without heavy regularization smoothing. The authors then show, using three benchmarks, that their technique works in practice, but best results are obtained by combining robustness and regularization-based methods.", "questions": "It would be interesting, to place the present work in perspective, to at least spend some time in related work to investigate how “explanations” are currently treated in the literature on transformer-based image models like ViTs. Is there any existing literature showing techniques for MLX (maybe not labeled as such) that are dependent on a particular architecture choice? If so, what makes the present work still interesting?\n\nEdit: Increased the score after author rebuttal addressing the points above - see comment below.", "limitations": "The authors briefly mention limitations related to the possibility, in future work, to learn from incomplete explanation data. It would be interesting to add a segment related to the points raised in Weaknesses and Questions.\n", "rating": 7, "confidence": 4, "soundness": 4, "presentation": 3, "contribution": 3, "strengths": "The paper improves on methods from an existing problem setting, applies robustness techniques where they had not been tried before, and convincingly shows both with theory and experiments that such techniques work.\n\nThe theorems, despite their simplified setting, intuitively demonstrate why the approach works better than the alternatives.\n\nThe experimental results convincingly show that the robustness-based techniques work better than regularization-based techniques.", "weaknesses": "While the contribution itself is nicely self-contained and perfectly valid when taken at face value (that in the MLX setting, training GPs or simple CNNs, robustness-based methods are better than regularization-based methods), one cannot help but wonder whether the entire setup is relevant or interesting, considering current state of the art architectures and how they attend to their input. Attention-based architectures like ViTs, due to intermediate-layer attention maps, lend themselves to very natural investigations commensurable to the kind of pixel-wise explanations used in this work.\n\nOne can imagine that, as these transformer models also easily incorporate masking, the whole MLX problem could be reframed to be about either manipulating attention maps, or dynamically learning attention masks. And example is [Yang et al. (2023)](https://openaccess.thecvf.com/content/CVPR2023/html/Yang_Improving_Visual_Grounding_by_Encouraging_Consistent_Gradient-Based_Explanations_CVPR_2023_paper.html). Addressing these concerns either in the introduction or related work could help contextualize the paper’s contributions.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their passionate assessment of our work. We are very glad that the reviewer is appreciative of the novelty of our contribution, intuition from our analysis and completeness of experiments. We enjoyed addressing their concerns on the future relevance of our contributions. We will make sure to include the details of our response in the final version. \n\n> It would be interesting, to place the present work in perspective, to at least spend some time in related work to investigate how “explanations” are currently treated in the literature on transformer-based image models like ViTs. Is there any existing literature showing techniques for MLX (maybe not labeled as such) that are dependent on a particular architecture choice? If so, what makes the present work still interesting?\n……..\nAttention-based architectures like ViTs, due to intermediate-layer attention maps, lend themselves to very natural investigations commensurable to the kind of pixel-wise explanations used in this work.\n\nWe thank the reviewer for drawing our attention to architecture specific regularization of explanation masks. We agree that ViTs can much more readily provide a local explanation through attention maps, which were used to encode prior knowledge in the past [1, 2] (thanks for pointing us to [2]). However, we observe that attention maps are just another local explanation method like gradients or CDEP that we studied in our paper. In that spirit, we expect attention maps based explanations to have the same trend of results as the other explanation methods we studied, i.e. attention maps regularization $\\leq$ perturbation-based methods $\\leq$ their combination. We confirm this trend through empirical validation on the Decoy-MNIST dataset.\n\nIn the shared response, we provided results when using attention maps based regularization methods proposed in [1] called SPAN and when using a Visual Transformer architecture of depth 3 and width 128. We observed that ViT architecture much more strongly latches onto the spurious correlation (when compared with Feed-Forward Network architecture that we were originally using), perhaps because of the phenomena observed in [3], making it harder to remove dependence on the decoy part. Nevertheless, we note the following trend in the results: SPAN < PGD $\\leq$ PGD-Ex + SPAN. Results from our experiment are aligned with our expectations, and because attention maps are just another local explanation method, much of our intuition and takeaways continue to hold even for Visual Transformers and attention maps. \n\nMore generally, we expect our method and takeaways to go stale when we have architectures that can faithfully attribute importance of different input regions (by design or otherwise) while also not compromising performance. To the best of our knowledge, no existing architecture can faithfully attribute per-region saliency. Perhaps, it is possible to come up with a carefully designed transformer architecture that can roll out per-region contributions more faithfully as the reviewer is alluding to, but such an architecture is not here yet to our knowledge. Attention maps of ViT aggregate information from other patches at each layer, which is why importance of a patch in the last layer (which is used for regularization as proposed in [1]) cannot attribute importance of a patch faithfully. Our work therefore stands relevant even with recently popular transformer-based networks . \n\nReferences.  \n[1] Miao, Kevin, et al. \"Prior Knowledge-Guided Attention in Self-Supervised Vision Transformers.\" arXiv preprint arXiv:2209.03745 (2022).    \n[2] Yang, Ziyan, et al. \"Improving Visual Grounding by Encouraging Consistent Gradient-based Explanations.\" Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2023.   \n[3] Sagawa, Shiori, et al. \"An investigation of why overparameterization exacerbates spurious correlations.\" International Conference on Machine Learning. PMLR, 2020.   "}]

Author rebuttal:
We thank the reviewers for detailed assessment and queries. We are glad that all the reviewers found our paper well presented and well motivated. We are also happy for an overall positive assessment of our work. 

Reviewer Hnkb, WA6d, haST raised some concerns regarding overall generality of our work and raised issues regarding applicability of our proposal to alternate explanation methods (Hnkb, WA6d) or datasets (haST). We address these concerns by (a) evaluating two new explanation methods (and their combination with a robustness method: PGD-Ex) on the Decoy-MNIST dataset, (b) presenting results on a new dataset called Salient-Imagenet. We will evaluate more extensively and include these results in the final version of the paper. 

In all, our response included results from two new explanation methods (Integrated-Gradient and attention map based regularization), two new architectures (Visual Transformers, ResNet-18) and one new dataset (Salient-Imagenet). As our results below demonstrate, irrespective of the explanation method or architecture or dataset, our claims remained valid: (a) using perturbations (i.e. robustness-based methods) for supervising explanations is better than using regularization-based methods, (b) combining robustness with regularization-based methods is at least as good or better than using robustness-based methods alone. 

We tried our best to address all the concerns, and are more than happy to engage further to resolve any further concerns.  

## Generality to new explanation methods

### Integrated-Gradient and CDEP

We introduced evaluation using Integrated-gradient [1] based regularization and also added evaluation with PGD-Ex+CDEP that we did not originally include on Decoy-MNIST dataset.

| Alg.                   | Avg Acc          | Wg Acc           |
|------------------------|------------------|------------------|
| Integrated-Grad        | 26.7 $\pm$ 1.3     | 17.6 $\pm$ 1.2     |
| CDEP                   | 14.5 $\pm$ 1.8     | 10.0 $\pm$ 0.7     |
| PGD-Ex                 | 67.6 $\pm$ 1.6     | 51.4 $\pm$ 0.3     |
| PGD-Ex+Integrated-Grad | 80.5 $\pm$ 2.1     | **62.1 $\pm$ 6.8** |
| PGD-Ex+CDEP            | **84.8 $\pm$ 0.8** | **64.2 $\pm$ 1.6** |

### Attention Map based local explanations

Using a Visual transformer architecture (of depth 3 and width 128), we evaluated regularization using local explanations obtained using an attention map – regularization based on attention maps was used to supervise prior knowledge in [2] and is called SPAN. We obtained saliency explanations on inputs using the procedure proposed in [2].  

| Decoy-MNIST | Avg Acc          | Wg Acc         |
|-------------|------------------|------------------|
| ERM         | 10.0 $\pm$ 0.3     | 8.1 $\pm$ 0.3      | 
| SPAN        | 19.0 $\pm$ 0.3     | 8.1 $\pm$ 0.3      | 
| PGD-Ex      | **64.6 $\pm$ 4.7** | **37.4 $\pm$ 3.5** |
| PGD-Ex+SPAN | **63.1 $\pm$ 2.6** | **39.4 $\pm$ 2.9** | 

## Generality to new dataset/architecture
As further evidence of generalizability of our proposal, we also evaluated on a subset of Salient-Imagenet [3, 4] using pretrained (on ImageNet) ResNet-18. Our training dataset included six classes with around 600 examples. For each example, the dataset also included a human-approved input mask highlighting spurious features. The results are as follows. 

|                 | Original accuracy | Accuracy under noise |   RCS |
|-------------|------------------|------------------|-----------------------------------------------|
| ERM             |             96.43 |                87.50 | 47.88 |
| Grad-Reg        |             89.29 |                82.14 | 52.54 |
| PGD-Ex          |             93.75 |                90.18 | 58.69 |
| PGD-Ex+Grad-Reg |             94.64 |                93.75 | **65.02** |

Since the dataset did not include any natural example grouping, we could not use Worst-group accuracy metric. Instead we report using the relative core spurious (RCS) metric proposed in [4]. RCS measures relative stability of the model to noise in the core vs spurious regions. High RCS, therefore, means low dependence on spurious features. Also shown in the table are original test accuracy and accuracy when normal noise is added to spurious regions. 

References.  
[1] Sundararajan, Mukund, Ankur Taly, and Qiqi Yan. "Axiomatic attribution for deep networks." International conference on machine learning. PMLR, 2017.   
[2] Miao, Kevin, et al. "Prior Knowledge-Guided Attention in Self-Supervised Vision Transformers." arXiv preprint arXiv:2209.03745 (2022).   
[3] Singla, Sahil, and Soheil Feizi. "Salient ImageNet: How to discover spurious features in Deep Learning?." arXiv preprint arXiv:2110.04301 (2021).   
[4] Singla, Sahil, Mazda Moayeri, and Soheil Feizi. "Core risk minimization using salient imagenet." arXiv preprint arXiv:2203.15566 (2022).
References.  
[5] Shao, X., Skryagin, A., Stammer, W., Schramowski, P., and Kersting, K. Right for better reasons: Training differentiable models by constraining their influence functions.   
[6] Stammer, W., Schramowski, P., and Kersting, K. Right for the right concept: Revising neuro-symbolic concepts by interacting with their explanations.   
[7] Schramowski, P., Stammer, W., Teso, S., Brugger, A., Herbert, F., Shao, X., Luigs, H.-G., Mahlein, A.-K., and Kersting, K. Making deep neural networks right for the right scientific reasons by interacting with their explanations.   
[8] Sundararajan, Mukund, Ankur Taly, and Qiqi Yan. "Axiomatic attribution for deep networks." International conference on machine learning. PMLR, 2017.   


Meta-review: All reviewers agree on the acceptance and the authors have been able to clarify remaining concerns. It tackles an interesting and important problem and should thus be of interest to the overall NeurIPS community
Decision: Accept (poster)