Title: Stability Guarantees for Feature Attributions with Multiplicative Smoothing
Venue: NeurIPS 2023
Accepted: yes
Keywords: Feature Attribution

Smoothing

Explainable

Interpretable

Provable Guarantees
Abstract: Explanation methods for machine learning models tend not to provide any formal guarantees and may not reflect the underlying decision-making process.
In this work, we analyze stability as a property for reliable feature attribution methods. 
We prove that relaxed variants of stability are guaranteed if the model is sufficiently Lipschitz with respect to the masking of features. 
We develop a smoothing method called Multiplicative Smoothing (MuS) to achieve such a model.
We show that MuS overcomes the theoretical limitations of standard smoothing techniques and can be integrated with any classifier and feature attribution method.
We evaluate MuS on vision and language models with various feature attribution methods, such as LIME and SHAP, and demonstrate that MuS endows feature attributions with non-trivial stability guarantees.

Stability Guarantees for Feature Attributions with Multiplicative Smoothing

Abstract

Explanation methods for machine learning models tend not to provide any formal guarantees and may not reflect the underlying decision-making process. In this work, we analyze stability as a property for reliable feature attribution methods. We prove that relaxed variants of stability are guaranteed if the model is sufficiently Lipschitz with respect to the masking of features. We develop a smoothing method called Multiplicative Smoothing (MuS) to achieve such a model. We show that MuS overcomes the theoretical limitations of standard smoothing techniques and can be integrated with any classifier and feature attribution method. We evaluate MuS on vision and language models with various feature attribution methods, such as LIME and SHAP, and demonstrate that MuS endows feature attributions with non-trivial stability guarantees.

1 Introduction

Modern machine learning models are incredibly powerful at challenging prediction tasks but notoriously black-box in their decision-making. One can therefore achieve impressive performance without fully understanding why . In settings like medical diagnosis [1, 2] and legal analysis [3, 4], where accurate and well-justified decisions are important, such power without proof is insufficient. In order to fully wield the power of such models while ensuring reliability and trust, a user needs accurate and insightful explanations of model behavior.

One popular family of explanation methods is feature attributions [5, 6, 7, 8]. Given a model and input, a feature attribution method generates a score for each input feature that denotes its importance to the overall prediction. For instance, consider Figure 1, in which the Vision Transformer [9] classifier predicts the full image (left) as 'Goldfish'. We then use a feature attribution method like SHAP [7] to score each feature and select the top-25%, for which the masked image (middle) is consistently predicted as 'Goldfish'. However, adding a single patch of features (right) alters the prediction confidence so much that it now yields 'Axolotl'. This suggests that the explanation is brittle [10], as small changes easily cause it to induce some other class. In this paper, we study how to overcome such behavior by analyzing the stability of an explanation: we consider an explanation to be stable if, once the explanatory features are included, the addition of more features does not change the prediction.

Stability implies that the selected features are enough to explain the prediction [12, 13, 14] and that this selection maintains strong explanatory power even in the presence of additional information [10, 15]. Similar properties are studied in literature and identified as useful for interpretability [16], and we emphasize that our main focus is on analyzing and achieving provable guarantees. Stability guarantees, in particular, are useful as they allow one to predict how model behavior varies with the explanation. Given a stable explanation, one can include more features, e.g., adding context, while

maintaining confidence in the consistency of the underlying explanatory power. Crucially, we observe that such guarantees only make sense when jointly considering the model and explanation method: the explanation method necessarily depends on the model to yield an explanation, and stability is then evaluated with respect to the model.

Thus far, existing works on feature attributions with formal guarantees face computational tractability and explanatory utility challenges. While some methods take an axiomatic approach [8, 17], others use metrics that appear reasonable but may not reliably reflect useful model behavior, a common and known limitation [18]. Such explanations have been criticized as a plausible guess at best, and completely misleading [19] at worst.

In this paper, we study how to construct explainable models with provable stability guarantees. We jointly consider the classification model and explanation method and present a formalization for studying such properties that we call explainable models . We focus on binary feature attributions [20] wherein each feature is either marked as explanatory (1) or not explanatory (0). We present a method to solve this problem, inspired by techniques from adversarial robustness, particularly randomized smoothing [21, 22]. Our method can take any off-the-shelf classifier and feature attribution method to efficiently yield an explainable model that satisfies provable stability guarantees. In summary, our contributions are as follows:

We formalize stability as a key property for binary feature attributions and study this in the framework of explainable models. We prove that relaxed variants of stability are guaranteed if the model is sufficiently Lipschitz with respect to the masking of features.

To achieve the sufficient Lipschitz condition, we develop a smoothing method called Multiplicative Smoothing (MuS). We show that MuS achieves strong smoothness conditions, overcomes key theoretical and practical limitations of standard smoothing techniques, and can be integrated with any classifier and feature attribution method.

We evaluate MuS on vision and language models along with different feature attribution methods. We demonstrate that MuS-smoothed explainable models achieve strong stability guarantees at a small cost to accuracy.

2 Overview

We observe that formal guarantees for explanations must take into account both the model and explanation method, and for this, we present in Section 2.1 a pairing that we call explainable models . This formulation allows us to describe the desired stability properties in Section 2.2. We show in Section 2.3 that a classifier with sufficient Lipschitz smoothness with respect to feature masking allows us to yield provable guarantees of stability. Finally, in Section 2.4, we show how to adapt existing feature attribution methods into our explainable model framework.

2.1 Explainable Models

We first present explainable models as a formalism for rigorously studying explanations. Let X = R n be the space of inputs, a classifier f : X → [0 , 1] m maps inputs x ∈ X to m class probabilities that sum to 1 , where the class of f ( x ) ∈ [0 , 1] m is taken to be the largest coordinate. Similarly, an explanation method φ : X → { 0 , 1 } n maps an input x ∈ X to an explanation φ ( x ) ∈ { 0 , 1 } n that indicates which features are considered explanatory for the prediction f ( x ) . In particular, we may pick and adapt φ from among a selection of existing feature attribution methods like LIME [6], SHAP [7], and many others [5, 8, 23, 24, 25], wherein φ may be thought of as a topk feature selector. Note that the selection of input features necessarily depends on the explanation method executing or analyzing the model, and so it makes sense to jointly study the model and explanation method: given a classifier f and explanation method φ , we call the pairing ⟨ f, φ ⟩ an explainable model . Given some x ∈ X , the explainable model ⟨ f, φ ⟩ maps x to both a prediction and explanation. We show this in Figure 2, where ⟨ f, φ ⟩ ( x ) ∈ [0 , 1] m ×{ 0 , 1 } n pairs the class probabilities and the feature attribution.

For an input x ∈ X , we will evaluate the quality of the binary feature attribution φ ( x ) through its masking on x . That is, we will study the behavior of f on the masked input x ⊙ φ ( x ) ∈ X , where ⊙ is the element-wise vector product. To do this, we define a notion of prediction equivalence : for two x, x ′ ∈ X , we write f ( x ) ∼ = f ( x ′ ) to mean that f ( x ) and f ( x ′ ) yield the same class. This allows us to formalize the intuition that an explanation φ ( x ) should recover the prediction of x under f .

Definition 2.1. The explainable model ⟨ f, φ ⟩ is consistent at x if f ( x ) ∼ = f ( x ⊙ φ ( x )) .

Evaluating f on x ⊙ φ ( x ) this way lets us apply the model as-is and therefore avoids the challenge of constructing a surrogate model that is accurate to the original [26]. Moreover, this approach is reasonable, especially in domains like vision - where one intuitively expects that a masked image retaining only the important features should induce the intended prediction. Indeed, architectures like Vision Transformer [9] can maintain high accuracy with only a fraction of the image present [27].

Particularly, we would like for ⟨ f, φ ⟩ to generate explanations that are stable and concise (i.e. sparse). The former is our central guarantee and is ensured through smoothing. The latter implies that φ ( x ) has few ones entries, and is a desirable property since a good explanation should not contain too much redundant information. However, sparsity is a more difficult property to enforce, as this is contingent on the model having high accuracy with respect to heavily masked inputs. For sparsity, we present a simple heuristic in Section 2.4 and evaluate its effectiveness in Section 4.

2.2 Stability Properties of Explainable Models

Given an explainable model ⟨ f, φ ⟩ and some x ∈ X , stability means that the prediction does not change even if one adds more explanatory features to φ ( x ) . For instance, the model-explanation pair in Figure 1 is not stable, as the inclusion of a single feature group (patch) changes the prediction. To formalize this notion of stability, we first introduce a partial ordering: for α, α ′ ∈ { 0 , 1 } n , we write α ⪰ α ′ iff α i ≥ α ′ i for all i = 1 , . . . , n . That is, α ⪰ α ′ iff α includes all the features selected by α ′ .

Definition 2.2. The explainable model ⟨ f, φ ⟩ is stable at x if f ( x ⊙ α ) ∼ = f ( x ⊙ φ ( x )) for all α ⪰ φ ( x ) .

Note that the constant explanation φ ( x ) = 1 , the vector of ones, makes ⟨ f, φ ⟩ trivially stable at every x ∈ X , though this is not a concise explanation. Additionally, stability at x implies consistency at x .

Unfortunately, stability is a difficult property to enforce in general, as it requires that f satisfy a monotone-like behavior with respect to feature inclusion - which is especially challenging for complex models like neural networks. Checking stability without additional assumptions on f is also hard: if k = ∥ φ ( x ) ∥ 1 is the number of ones in φ ( x ) , then there are 2 n -k possible α ⪰ φ ( x ) to check. This large space of possible α ⪰ φ ( x ) motivates us to examine instead relaxations of stability. We introduce lower and upper relaxations of stability below.

Definition 2.3. The explainable model ⟨ f, φ ⟩ is incrementally stable at x with radius r if f ( x ⊙ α ) ∼ = f ( x ⊙ φ ( x )) for all α ⪰ φ ( x ) where ∥ α -φ ( x ) ∥ 1 ≤ r .

Incremental stability is the lower relaxation since it considers the case where the mask α has only a few features more than φ ( x ) . For instance, if one can probably add up to r features to a masked x ⊙ φ ( x ) without altering the prediction, then ⟨ f, φ ⟩ would be incrementally stable at x with radius r . We next introduce the upper relaxation that we call decremental stability.

Definition 2.4. The explainable model ⟨ f, φ ⟩ is decrementally stable at x with radius r if f ( x ⊙ α ) ∼ = f ( x ⊙ φ ( x )) for all α ⪰ φ ( x ) where ∥ 1 -α ∥ 1 ≤ r .

Decremental stability is a subtractive form of stability, in contrast to the additive nature of incremental stability. Particularly, decremental stability considers the case where α has much more features than φ ( x ) . If one can provably remove up to r non-explanatory features from the full x without altering the prediction, then ⟨ f, φ ⟩ is decrementally stable at x with radius r . Note also that decremental stability necessarily entails consistency of ⟨ f, φ ⟩ , but for simplicity of definitions, we do not enforce this for incremental stability. Furthermore, observe that for a sufficiently large radius of r = ⌈ ( n -∥ φ ( x ) ∥ 1 ) / 2 ⌉ , incremental and decremental stability together imply stability.

Remark 2.5. Similar notions to the above have been proposed in the literature, and we refer to [16] for an extensive survey. In particular, for [16], consistency is akin to preservation , and stability is similar to continuity , except we are concerned with adding features. In this regard, incremental stability is most similar to incremental addition and decremental stability to incremental deletion .

2.3 Lipschitz Smoothness Entails Stability Guarantees

If f : X → [0 , 1] m is Lipschitz with respect to the masking of features, then we can guarantee relaxed stability properties for the explainable model ⟨ f, φ ⟩ . In particular, we require for all x ∈ X that f ( x ⊙ α ) is Lipschitz with respect to the mask α ∈ { 0 , 1 } n . This then allows us to establish our main result (Theorem 3.3), which we preview below in Remark 2.6.

Remark 2.6 (Sketch of main result) . Consider an explainable model ⟨ f, φ ⟩ where for all x ∈ X the function g ( x, α ) = f ( x ⊙ α ) is λ -Lipschitz in α ∈ { 0 , 1 } n with respect to the ℓ 1 norm. Then at any x , the radius of incremental stability r inc and radius of decremental stability r dec are respectively:

where g A -g B is called the confidence gap, with g A , g B the top-two class probabilities:

̸

Observe that Lipschitz smoothness is, in fact, a stronger assumption than necessary, as besides α ⪰ φ ( x ) , it also imposes guarantees on α ⪯ φ ( x ) . Nevertheless, Lipschitz smoothness is one of the few classes of properties that can be guaranteed and analyzed at scale on arbitrary models [22, 28].

2.4 Adapting Existing Feature Attribution Methods

Most existing feature attribution methods assign a real-valued score to feature importance, rather than a binary value. We therefore need to convert this to a binary-valued method for use with a

stable explainable model. Let ψ : X → R n be such a continuous-valued method like LIME [6] or SHAP [7], and fix some desired incremental stability radius r inc and decremental stability radius r dec . Given some x ∈ X a simple construction for binary φ ( x ) ∈ { 0 , 1 } n is described next.

Remark 2.7 (Iterative construction of φ ( x ) ) . Consider any x ∈ X and let ρ be an index ordering on ψ ( x ) from high-to-low (i.e. most probable class first). Initialize α = 0 , and for each i ∈ ρ : assign α i ← 1 then check whether ⟨ f, φ : x ↦→ α ⟩ is now consistent, incrementally stable with radius r inc , and decrementally stable with radius r dec . If so, terminate with φ ( x ) = α , and continue otherwise.

Note that the above method of constructing φ ( x ) does not impose sparsity guarantees in the way that we may guarantee stability through Lipschitz smoothness. Instead, the ordering from a continuousvalued ψ ( x ) serves as a greedy heuristic for constructing φ ( x ) . We show in Section 4 that some feature attributions (e.g., SHAP [7]) tend to yield sparser selections on average compared to others (e.g., Vanilla Gradient Saliency [5]).

3 Multiplicative Smoothing for Lipschitz Constants

In this section, we present our main technical contribution of Multiplicative Smoothing (MuS). The goal is to transform an arbitrary base classifier h : X → [0 , 1] m into a smoothed classifier f : X → [0 , 1] m that is Lipschitz with respect to the masking of features. This then allows one to couple f with an explanation method φ in order to form an explainable model ⟨ f, φ ⟩ with provable stability guarantees.

We give an overview of our MuS in Section 3.1, where we illustrate that a principal motivation for its development is because standard smoothing techniques may violate a property that we call masking equivalence . We present the Lipschitz constant of the smoothed classifier f in Section 3.2 and show how this is used to certify stability. Finally, we give an efficient computation of MuS in Section 3.3, allowing us to exactly evaluate f at a low sample complexity.

Stage 1

Stage 2

Stage 3

0.8

Sample

Apply

0.6

masks

maskings

h

0.4

Masks

D

0.2

0.0

Eraser

Goldfish

Lipstick

3.1 Technical Overview of MuS

Our key insight is that randomly dropping (i.e., zeroing) features will attain the desired smoothness. In particular, we uniformly drop features with probability 1 -λ by sampling binary masks s ∈ { 0 , 1 } n from some distribution D where each coordinate is distributed as Pr[ s i = 1] = λ . Then define f as:

where B ( λ ) is the Beronulli distribution with parameter λ ∈ [0 , 1] . We give an overview of evaluating f ( x ) in Figure 3. Importantly, our main results on smoothness (Theorem 3.2) and stability (Theorem 3.3) hold provided each coordinate of D is marginally Bernoulli with parameter λ , and so we avoid fixing a particular choice for now. However, it will be easy to intuit the exposition with D = B n ( λ ) , the coordinate-wise i.i.d. Bernoulli distribution with parameter λ .

We can equivalently parametrize f using the mapping g ( x, α ) = f ( x ⊙ α ) , where it follows that:

Note that one could have alternatively first defined g and then f due to the identity g ( x, 1 ) = f ( x ) . We require that the relationship between f and g follows an identity that we call masking equivalence :

This follows by the definition of g , and the relevance to stability is this: if masking equivalence holds, then we can rewrite stability properties involving f in terms of g 's second parameter as follows:

where incremental and decremental stability may be analogously defined. This translation is useful, as we will prove that g is λ -Lipschitz in its second parameter (Theorem 3.2), which then allows us to establish the desired stability properties (Theorem 3.3).

Since many choices are valid, we have not given an explicit construction for D . Rather, so long as each coordinate of s ∼ D obeys s i ∼ B ( λ ) then the Lipschitz properties for g follow. The implication here is that although simple distributions like B n ( λ ) suffice for D , they may not be sample efficient. We show in Section 3.3 how to exploit a structured statistical dependence in order to reduce the sample complexity of computing MuS.

Importantly, we are motivated to develop MuS because standard smoothing techniques, namely additive smoothing [21, 22], may fail to satisfy masking equivalence. Additive smoothing is by far the most popular smoothing technique and differs from our scheme (3) in how noise is applied, where let D add and D mult be any two distributions on R n :

Particularly, additive smoothing has counterexamples for masking equivalence.

Proposition 3.1. There exists h : X → [0 , 1] and distribution D , where for

we have g + ( x, α ) = g + ( x ⊙ α, 1 ) for some x ∈ X and α ∈ { 0 , 1 } n .

̸

Proof. Observe that it suffices to have h, x, α such that h ( x ⊙ ( α + s )) > h (( x ⊙ α ) ⊙ ( 1 + s )) for a non-empty set of s ∈ R n . Let D be a distribution on these s , then:

Intuitively, this occurs because additive smoothing primarily applies noise by perturbing feature values, rather than completely masking them. As such, there might be 'information leakage' when non-explanatory bits of α are changed into non-zero values. This then causes each sample of h ( x ⊙ ˜ α ) within g ( x, α ) to observe more features of x than it would have been able to otherwise.

3.2 Certifying Stability Properties with Lipschitz Classifiers

Our core technical result is in showing that f as defined in (2) is Lipschitz to the masking of features. We present MuS in terms of g , where it is parametric with respect to the distribution D : so long as D satisfies a coordinate-wise Bernoulli condition, then it is usable with MuS.

Theorem 3.2 (MuS) . Let D be any distribution on { 0 , 1 } n where each coordinates of s ∼ D is distributed as s i ∼ B ( λ ) . Consider any h : X → [0 , 1] and define g : X × { 0 , 1 } n → [0 , 1] as

Then the function g ( x, · ) : { 0 , 1 } n → [0 , 1] is λ -Lipschitz in the ℓ 1 norm for all x ∈ X .

The strength of this result is in its weak assumptions. First, the theorem applies to any model h and input x ∈ X . It further suffices that each coordinate is distributed as s i ∼ B ( λ ) , and we emphasize that statistical independence between different s i , s j is not assumed . This allows us to construct

D with structured dependence in Section 3.3, such that we may exactly and efficiently evaluate g ( x, α ) at a sample complexity of N ≪ 2 n . A low sample complexity is important for making MuS practically usable, as otherwise, one must settle for the expected value subject to probabilistic guarantees. For instance, simpler distributions like B n ( λ ) do satisfy the requirements of Theorem 3.2 -but costs 2 n samples because of coordinate-wise independence. Whatever choice of D , one can guarantee stability so long as g is Lipschitz in its second argument.

Theorem 3.3 (Stability) . Consider any h : X → [0 , 1] m with coordinates h 1 , . . . , h m . Fix λ ∈ [0 , 1] and let g 1 , . . . , g m be the respectively smoothed coordinates as in Theorem 3.2, using which we analogously define g : X ×{ 0 , 1 } n → [0 , 1] m . Also define f ( x ) = g ( x, 1 ) . Then for any explanation method φ and input x ∈ X , the explainable model ⟨ f, φ ⟩ is incrementally stable with radius r inc and decrementally stable with radius r dec :

where g A -g B is the confidence gap as in (1) .

Note that it is only in the case where the radius ≥ 1 that non-trivial stability guarantees exist. Because each g k has range in [0 , 1] , this means that a Lipschitz constant of λ ≤ 1 / 2 is necessary to attain at least one radius of stability. We present in Appendix A.2 some extensions to MuS that allow one to achieve higher coverage of features.

3.3 Exploiting Structured Dependency

We now present L qv ( λ ) , a distribution on { 0 , 1 } n that allows for efficient and exact evaluation of a MuS-smoothed classifier. Our construction is an adaption of [28] from uniform to Bernoulli noise, where the primary insight is that one can parametrize n -dimensional noise using a single dimension via structured coordinate-wise dependence. In particular, we use a seed vector v , where with an integer quantization parameter q > 1 there will only exist q distinct choices of s ∼ L qv ( λ ) . All the while, we still enforce that any such s is coordinate-wise Bernoulli with s i ∼ B ( λ ) . Thus, for a sufficiently small quantization parameter (i.e., q ≪ 2 n ), we may tractably enumerate through all q possible choices of s and thereby evaluate a MuS-smoothed model with only q samples.

Proposition 3.4. Fix integer q > 1 and consider any vector v ∈ { 0 , 1 /q, . . . , ( q -1) /q } n and scalar λ ∈ { 1 /q, . . . , q/q } . Define s ∼ L qv ( λ ) to be a random vector in { 0 , 1 } n with coordinates given by

Then there are q distinct values of s and each coordinate is distributed as s i ∼ B ( λ ) .

Proof. First, observe that each of the q distinct values of s base defines a unique value of s since we have assumed v and λ to be fixed. Next, observe that each t i has q unique values uniformly distributed as t i ∼ U (1 /q, . . . , q/q } ) -1 / (2 q ) . Because λ ∈ { 1 /q, . . . , q/q } we therefore have Pr[ t i ≤ λ ] = λ , which implies that s i ∼ B ( λ ) .

The seed vector v is the source of our structured coordinate-wise dependence, and the one-dimensional source of randomness s base is used to generate the n -dimensional s . Such s ∼ L qv ( λ ) then satisfies the conditions for use in MuS (Theorem 3.2), and this noise allows for an exact evaluation of the smoothed classifier in q samples. We have found q = 64 to be sufficient in practice and that values as low as q = 16 also yield good performance. We remark that one drawback is that one may get an unlucky seed v , but we have not yet observed this in our experiments.

4 Empirical Evaluations

We evaluate the quality of MuS on different classification models and explanation methods as they relate to stability guarantees. To that end, we perform the following experiments.

(E1) How good are the stability guarantees? A natural measure of quality for stability guarantees over a dataset exists: what radii are achieved, and at what frequency. We investigate how different combinations of models, explanation methods, and λ affect this measure.

(E2) What is the cost of smoothing? To increase the radius of a provable stability guarantee, we must decrease the Lipschitz constant λ . However, as λ decreases, more features are dropped during the smoothing process. This experiment investigates this stability-accuracy trade-off.

(E3) Which explanation method is best? We evaluate which existing feature attribution methods are amenable to strong stability guarantees. We examine LIME [6], SHAP [7], Vanilla Gradient Saliency (VGrad) [5], and Integrated Gradients (IGrad) [8], with focus on the size of the explanation.

(Experimental Setup) We use on two vision models (Vision Transformer [9] and ResNet50 [29]) and one language model (RoBERTa [30]). We use ImageNet1K [31] as our vision dataset and TweetEval [32] sentiment analysis as our language dataset. We use feature grouping from Appendix A.2.1 on ImageNet1K to reduce the 3 × 224 × 224 dimensional input into n = 64 superpixel patches. We report stability radii r in terms of the fraction of features, i.e., r/n . In all our experiments, we use the quantized noise as in Section 3.3 with q = 64 unless specified otherwise. We refer to Appendix B for training details and the comprehensive experiments.

CertifiedConsistentandIncStable

CertifiedConsistentandDecStable

0.8

入=8/8 (0.840)

0.8

入=8/8 (0.840)

Prop Hold Rate

入=4/8 (0.801)

入=4/8 (0.801)

0.6

F9°0

入=1/8 (0.515)

入=1/8 (0.515)

0.4

0.4

0.2

0.2

100

0.0

0.00

0.01

0.02

0.03

0.04

0.05

0.06

0.00

0.01

0.02

0.03

0.04

0.05

0.06

EmpiricalConsistentand IncStable

EmpiricalConsistentandDecStable

0.8

入=8/8 (0.844)

0.8

入=8/8 (0.844)

Prop Hold Rate

入=4/8 (0.816)

入=4/8 (0.816)

0.6

0.6

入=1/8 (0.496)

入=1/8 (0.496)

0.4

0.4

0.2

0.2

0.0

0.0

0.0

0.1

0.2

0.3

0.4

0.5

0.0

0.1

0.2

0.3

0.4

0.5

FractionofFeatures

Fraction ofFeatures

4.1 (E1) Quality of Stability Guarantees

We study how much radius of consistent and incremental (resp. decremental) stability is achieved, and how often. We take an explainable model ⟨ f, φ ⟩ where f is Vision Transformer and φ is SHAP with top-25% feature selection. We plot the rate at which a property holds (e.g., consistent and incrementally stable with radius r ) as a function of radius (expressed as a fraction of features r/n ).

We show our results in Figure 4, where on the left we have the certified guarantees for N cert = 2000 samples from ImageNet1K; on the right we have the empirical radii for N emp = 250 samples obtained by applying a standard box attack [33] strategy with q = 16 . We observe from the certified results that the decremental stability radii are larger than those of incremental stability. This is reasonable since the base classifier sees much more of the input when analyzing decremental stability and is thus more confident on average, i.e., achieves a larger confidence gap. Moreover, our empirical radii often cover up to half of the input, which suggests that our certified analysis is quite conservative.

4.2 (E2) Stability-Accuracy Trade-Offs

We next investigate how smoothing impacts the classifier accuracy. As λ decreases due to more smoothing, the base classifier sees increasingly zeroed-out features - which should hurt accuracy. We took N = 2000 samples for each classifier on their respective datasets and plotted the certified accuracy vs. radius of decremental stability.

VisionTransformer

ResNet50

RoBERTa

0.8

0.8

入=8/8 (0.756)

0.8

2

入=4/8 (0.791)

入=4/8 (0.722)

入=4/8 (0.748)

Accura

0.6

入=2/8 (0.752)

0.6

入=2/8(0.646)

0.6

入=2/8(0.655)

入=1/8 (0.695)

入=1/8 (0.571)

入=1/8 (0.566)

0.4

入=1/16 (0.598)

0.4

入=1/16 (0.501)

0.4

入=1/16 (0.467)

ertifi

0.2

0.2

0.2

0.0

0.0

0.0

0.00

0.05

0.10

0.15

0.00

0.05

0.10

0.15

0.0

0.1

0.2

0.3

0.4

0.5

FractionofFeatures

FractionofFeatures

FractionofFeatures

We show the results in Figure 5, where the clean accuracy (in parentheses) decreases with λ as expected. This accuracy drop is especially pronounced for ResNet50, and we suspect that the transformer architecture of Vision Transformer and RoBERTa makes them more resilient to the randomized masking of features. Nevertheless, this experiment demonstrates that large models, especially transformers, can tolerate non-trivial noise from MuS while maintaining high accuracy.

VisionTransformer

ResNet50

RoBERTa

0.80

0.70

VGrad

VGrad

VGrad

atures

IGrad

0.78

IGrad

0.8

IGrad

F

e

0.68

LIME

SHAP

0.76

LIME

SHAP

0.7

LIME

SHAP

0.66

0.74

0.6

ion

0.72

0.5

0.64

0.70

0.4

0.62

0.68

0.125

0.250

0.375

0.500

0.125

0.250

0.375

0.500

0.125

0.250

0.375

0.500

LipschitzConstant入

4.3 (E3) Which Explanation Method to Pick?

Finally, we explore which feature attribution method is best suited to stability guarantees of explainable model ⟨ f, φ ⟩ . All four methods ψ ∈ { LIME , SHAP , VGrad , IGrad } are continuous-valued, for which we samples N = 250 inputs from each model's respective dataset. For each input x , we use the feature importance ranking generated by ψ ( x ) to iteratively build φ ( x ) in a greedy manner like in Section 2.4. For some x , let k x = φ ( x ) /n be the number fraction of features needed for ⟨ f, φ ⟩ to be consistent, incrementally stable with radius 1 , and decrementally stable with radius 1 . We then plot the average k x for different methods at λ ∈ { 1 / 8 , . . . , 4 / 8 } in Figure 6, where note that SHAP tends to require fewer features to achieve the desired properties, while VGrad tends to require more. However, we do not believe these to be decisive results, as many curves are relatively close, especially for Vision Transformer and ResNet50.

5 Related Works

For extensive surveys on explainability methods see [16, 20, 34, 35, 36, 37, 38]. Notable feature attribution methods include Vanilla Gradient Saliency [5], SmoothGrad [23], Integrated Gradients [8], Grad-CAM [39], Occlusion [40], LIME [6], SHAP [7], and their variants. Of these, Shapley valuebased [17] methods [7, 24, 25] are rooted in axiomatic principles, as are Integrated Gradients [8, 41]. The work of [42] finds confidence intervals over attribution scores. A study of common feature attribution methods is done in [43]. Similar to our approach is [44], which studies binaryvalued classifiers and presents an algorithm with succinctness and probabilistic precision guarantees. Different metrics for evaluating feature attributions are studied in [16, 18, 45, 46, 47, 48, 49, 50, 51]. Whether an attribution correctly identifies relevant features is a well-known issue [52, 53]. Many methods are also susceptible to adversarial attacks [54, 55]. As a negative result, [56] shows that

feature attributions have provably poor performance on sufficiently rich model classes. Related to feature attributions are data attributions [57, 58, 59], which assigns values to training data points. Also related to formal guarantees are formal methods-based approaches towards explainability [60].

6 Conclusion

We study provable stability guarantees for binary feature attribution methods through the framework of explainable models. A selection of features is stable if the additional inclusion of other features does not alter its explanatory power. We show that if the classifier is Lipschitz with respect to the masking of features, then one can guarantee relaxed variants of stability. To achieve this Lipschitz condition, we develop a smoothing method called Multiplicative Smoothing (MuS). This method is parametric to the choice of noise distribution, allowing us to construct and exploit distributions with structured dependence for exact and efficient evaluation. We evaluate MuS on vision and language models and demonstrate that MuS yields strong stability guarantees at only a small cost to accuracy.

References

Mauricio Reyes, Raphael Meier, Sérgio Pereira, Carlos A Silva, Fried-Michael Dahlweid, Hendrik von Tengg-Kobligk, Ronald M Summers, and Roland Wiest. On the interpretability of artificial intelligence in radiology: challenges and opportunities. Radiology: artificial intelligence , 2(3):e190043, 2020.

Erico Tjoa and Cuntai Guan. A survey on explainable artificial intelligence (xai): Toward medical xai. IEEE transactions on neural networks and learning systems , 32(11):4793-4813, 2020.

Sandra Wachter, Brent Mittelstadt, and Chris Russell. Counterfactual explanations without opening the black box: Automated decisions and the gdpr. Harv. JL & Tech. , 31:841, 2017.

Adrien Bibal, Michael Lognoul, Alexandre De Streel, and Benoît Frénay. Legal requirements on explainability in machine learning. Artificial Intelligence and Law , 29:149-169, 2021.

Karen Simonyan, Andrea Vedaldi, and Andrew Zisserman. Deep inside convolutional networks: Visualising image classification models and saliency maps. arXiv preprint arXiv:1312.6034 , 2013.

Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. " why should i trust you?" explaining the predictions of any classifier. In Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining , pages 1135-1144, 2016.

Scott M Lundberg and Su-In Lee. A unified approach to interpreting model predictions. Advances in neural information processing systems , 30, 2017.

Mukund Sundararajan, Ankur Taly, and Qiqi Yan. Axiomatic attribution for deep networks. In International conference on machine learning , pages 3319-3328. PMLR, 2017.

Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929 , 2020.

Amirata Ghorbani, Abubakar Abid, and James Zou. Interpretation of neural networks is fragile. In Proceedings of the AAAI conference on artificial intelligence , volume 33, pages 3681-3688, 2019.

Joseph Paul Cohen, Joseph D. Viviano, Paul Bertin, Paul Morrison, Parsa Torabian, Matteo Guarrera, Matthew P Lungren, Akshay Chaudhari, Rupert Brooks, Mohammad Hashir, and Hadrien Bertrand. TorchXRayVision: A library of chest X-ray datasets and models. In Medical Imaging with Deep Learning , 2022.

Gavin Brown. A new perspective for information theoretic feature selection. In Artificial intelligence and statistics , pages 49-56. PMLR, 2009.

Jianbo Chen, Le Song, Martin Wainwright, and Michael Jordan. Learning to explain: An information-theoretic perspective on model interpretation. In International Conference on Machine Learning , pages 883-892. PMLR, 2018.

Xiaoping Li, Yadi Wang, and Rubén Ruiz. A survey on sparse learning models for feature selection. IEEE transactions on cybernetics , 52(3):1642-1660, 2020.

Akhilan Boopathy, Sijia Liu, Gaoyuan Zhang, Cynthia Liu, Pin-Yu Chen, Shiyu Chang, and Luca Daniel. Proper network interpretability helps adversarial robustness in classification. In International Conference on Machine Learning , pages 1014-1023. PMLR, 2020.

Meike Nauta, Jan Trienes, Shreyasi Pathak, Elisa Nguyen, Michelle Peters, Yasmin Schmitt, Jörg Schlötterer, Maurice van Keulen, and Christin Seifert. From anecdotal evidence to quantitative evaluation methods: A systematic review on evaluating explainable ai. arXiv preprint arXiv:2201.08164 , 2022.

L.S. Shapley. A value for n-person games. Contributions to the Theory of Games , pages 307-317, 1953.

Yilun Zhou, Serena Booth, Marco Tulio Ribeiro, and Julie Shah. Do feature attribution methods correctly attribute features? In Proceedings of the AAAI Conference on Artificial Intelligence , volume 36, pages 9623-9633, 2022.

Alon Jacovi and Yoav Goldberg. Towards faithfully interpretable nlp systems: How should we define and evaluate faithfulness? In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics , pages 4198-4205, 2020.

Jundong Li, Kewei Cheng, Suhang Wang, Fred Morstatter, Robert P Trevino, Jiliang Tang, and Huan Liu. Feature selection: A data perspective. ACM computing surveys (CSUR) , 50(6):1-45, 2017.

Jeremy Cohen, Elan Rosenfeld, and Zico Kolter. Certified adversarial robustness via randomized smoothing. In International Conference on Machine Learning , pages 1310-1320. PMLR, 2019.

Greg Yang, Tony Duan, J Edward Hu, Hadi Salman, Ilya Razenshteyn, and Jerry Li. Randomized smoothing of all shapes and sizes. In International Conference on Machine Learning , pages 10693-10705. PMLR, 2020.

Daniel Smilkov, Nikhil Thorat, Been Kim, Fernanda Viégas, and Martin Wattenberg. Smoothgrad: removing noise by adding noise. arXiv preprint arXiv:1706.03825 , 2017.

Mukund Sundararajan and Amir Najmi. The many shapley values for model explanation. In International conference on machine learning , pages 9269-9278. PMLR, 2020.

Yongchan Kwon and James Zou. Weightedshap: analyzing and improving shapley based feature attributions. arXiv preprint arXiv:2209.13429 , 2022.

Reza Alizadeh, Janet K Allen, and Farrokh Mistree. Managing computational complexity using surrogate models: a critical review. Research in Engineering Design , 31:275-298, 2020.

Hadi Salman, Saachi Jain, Eric Wong, and Aleksander Madry. Certified patch robustness via smoothed vision transformers. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 15137-15147, 2022.

Alexander J Levine and Soheil Feizi. Improved, deterministic smoothing for l_1 certified robustness. In International Conference on Machine Learning , pages 6254-6264. PMLR, 2021.

Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition , pages 770-778, 2016.

Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692 , 2019.

Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, et al. Imagenet large scale visual recognition challenge. International journal of computer vision , 115:211-252, 2015.

Francesco Barbieri, Jose Camacho-Collados, Leonardo Neves, and Luis Espinosa-Anke. Tweeteval: Unified benchmark and comparative evaluation for tweet classification. arXiv preprint arXiv:2010.12421 , 2020.

Pin-Yu Chen, Huan Zhang, Yash Sharma, Jinfeng Yi, and Cho-Jui Hsieh. Zoo: Zeroth order optimization based black-box attacks to deep neural networks without training substitute models. In Proceedings of the 10th ACM workshop on artificial intelligence and security , pages 15-26, 2017.

Nadia Burkart and Marco F Huber. A survey on the explainability of supervised machine learning. Journal of Artificial Intelligence Research , 70:245-317, 2021.

Jianlong Zhou, Amir H Gandomi, Fang Chen, and Andreas Holzinger. Evaluating the quality of machine learning explanations: A survey on methods and metrics. Electronics , 10(5):593, 2021.

Qing Lyu, Marianna Apidianaki, and Chris Callison-Burch. Towards faithful model explanation in nlp: A survey. arXiv preprint arXiv:2209.11326 , 2022.

Xuhong Li, Haoyi Xiong, Xingjian Li, Xuanyu Wu, Xiao Zhang, Ji Liu, Jiang Bian, and Dejing Dou. Interpretable deep learning: Interpretation, interpretability, trustworthiness, and beyond. Knowledge and Information Systems , 64(12):3197-3234, 2022.

Rishi Bommasani, Drew A Hudson, Ehsan Adeli, Russ Altman, Simran Arora, Sydney von Arx, Michael S Bernstein, Jeannette Bohg, Antoine Bosselut, Emma Brunskill, et al. On the opportunities and risks of foundation models. arXiv preprint arXiv:2108.07258 , 2021.

Ramprasaath R Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedantam, Devi Parikh, and Dhruv Batra. Grad-cam: Visual explanations from deep networks via gradient-based localization. In Proceedings of the IEEE international conference on computer vision , pages 618-626, 2017.

Matthew D Zeiler and Rob Fergus. Visualizing and understanding convolutional networks. In Computer Vision-ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part I 13 , pages 818-833. Springer, 2014.

Daniel D Lundstrom, Tianjian Huang, and Meisam Razaviyayn. A rigorous study of integrated gradients method and extensions to internal neuron attributions. In International Conference on Machine Learning , pages 14485-14508. PMLR, 2022.

Dylan Slack, Anna Hilgard, Sameer Singh, and Himabindu Lakkaraju. Reliable post hoc explanations: Modeling uncertainty in explainability. Advances in neural information processing systems , 34:9391-9404, 2021.

Tessa Han, Suraj Srinivas, and Himabindu Lakkaraju. Which explanation should i choose? a function approximation perspective to characterizing post hoc explanations. arXiv preprint arXiv:2206.01254 , 2022.

Guy Blanc, Jane Lange, and Li-Yang Tan. Provably efficient, succinct, and precise explanations. Advances in Neural Information Processing Systems , 34:6129-6141, 2021.

Julius Adebayo, Justin Gilmer, Michael Muelly, Ian Goodfellow, Moritz Hardt, and Been Kim. Sanity checks for saliency maps. Advances in neural information processing systems , 31, 2018.

Sara Hooker, Dumitru Erhan, Pieter-Jan Kindermans, and Been Kim. A benchmark for interpretability methods in deep neural networks. Advances in neural information processing systems , 32, 2019.

Jay DeYoung, Sarthak Jain, Nazneen Fatema Rajani, Eric Lehman, Caiming Xiong, Richard Socher, and Byron C Wallace. Eraser: A benchmark to evaluate rationalized nlp models. arXiv preprint arXiv:1911.03429 , 2019.

Jasmijn Bastings, Sebastian Ebert, Polina Zablotskaia, Anders Sandholm, and Katja Filippova. " will you find these shortcuts?" a protocol for evaluating the faithfulness of input salience methods for text classification. arXiv preprint arXiv:2111.07367 , 2021.

Yao Rong, Tobias Leemann, Vadim Borisov, Gjergji Kasneci, and Enkelejda Kasneci. A consistent and efficient evaluation strategy for attribution methods. arXiv preprint arXiv:2202.00449 , 2022.

Yilun Zhou and Julie Shah. The solvability of interpretability evaluation metrics. arXiv preprint arXiv:2205.08696 , 2022.

Julius Adebayo, Michael Muelly, Harold Abelson, and Been Kim. Post hoc explanations may be ineffective for detecting unknown spurious correlation. In International Conference on Learning Representations , 2022.

Mengjiao Yang and Been Kim. Benchmarking attribution methods with relative feature importance. arXiv preprint arXiv:1907.09701 , 2019.

Pieter-Jan Kindermans, Sara Hooker, Julius Adebayo, Maximilian Alber, Kristof T Schütt, Sven Dähne, Dumitru Erhan, and Been Kim. The (un) reliability of saliency methods. In Explainable AI: Interpreting, Explaining and Visualizing Deep Learning , pages 267-280. Springer, 2019.

Dylan Slack, Anna Hilgard, Himabindu Lakkaraju, and Sameer Singh. Counterfactual explanations can be manipulated. Advances in neural information processing systems , 34:62-75, 2021.

Adam Ivankay, Ivan Girardi, Chiara Marchiori, and Pascal Frossard. Fooling explanations in text classifiers. arXiv preprint arXiv:2206.03178 , 2022.

Blair Bilodeau, Natasha Jaques, Pang Wei Koh, and Been Kim. Impossibility theorems for feature attribution. arXiv preprint arXiv:2212.11870 , 2022.

Pang Wei Koh and Percy Liang. Understanding black-box predictions via influence functions. In International conference on machine learning , pages 1885-1894. PMLR, 2017.

Andrew Ilyas, Sung Min Park, Logan Engstrom, Guillaume Leclerc, and Aleksander Madry. Datamodels: Predicting predictions from training data. arXiv preprint arXiv:2202.00622 , 2022.

Sung Min Park, Kristian Georgiev, Andrew Ilyas, Guillaume Leclerc, and Aleksander Madry. Trak: Attributing model behavior at scale. arXiv preprint arXiv:2303.14186 , 2023.

Shahaf Bassan and Guy Katz. Towards formal xai: Formally approximate minimal explanations of neural networks. In International Conference on Tools and Algorithms for the Construction and Analysis of Systems , pages 187-207. Springer, 2023.

Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 , 2014.

Alexey Kurakin, Ian Goodfellow, and Samy Bengio. Adversarial machine learning at scale. arXiv preprint arXiv:1611.01236 , 2016.

Logan Engstrom, Andrew Ilyas, Hadi Salman, Shibani Santurkar, and Dimitris Tsipras. Robustness (python library), 2019.

A Proofs and Theoretical Discussions

In this Section, we present the proofs of our main results and some extensions to MuS.

A.1 Proofs of Main Results

A.1.1 Proof of Theorem 3.2

Proof. By linearity, we have:

̸

so it suffices to analyze an arbitrary term by fixing some s ∼ D . Consider any x ∈ X , let α, α ′ ∈ { 0 , 1 } n , and define δ = α -α ′ . Observe that ˜ α i = ˜ α ′ i exactly when | δ i | = 1 and s i = 1 . Since s i ∼ B ( λ ) , we thus have Pr[˜ α i = ˜ α ′ i ] = λ | δ i | , and applying the union bound:

̸

̸

̸

and so:

̸

Note that E [ h ( x ⊙ ˜ α ) -h ( x ⊙ ˜ α ′ ) | ˜ α = ˜ α ′ ] = 0 , and so

̸

̸

Thus, g ( x, · ) is λ -Lipschitz in the ℓ 1 norm.

A.1.2 Proof of Theorem 3.3

Proof. We first show incremental stability. Consider any x ∈ X , then by masking equivalence:

and let g A , g B be the top-two class probabilities of g as defined in (1). By Theorem 3.2, both g A , g B are Lipschitz in their second parameter, and so for all α ∈ { 0 , 1 } n :

Observe that if α is sufficiently close to φ ( x ) , i.e.:

then the top-class index of g ( x, φ ( x ) and g ( x, α ) are the same. This means that g ( x, φ ( x )) ∼ = g ( x, α ) and thus f ( x ⊙ φ ( x )) ∼ = f ( x ⊙ α ) , thus proving incremental stability with radius d ( x, φ ( x )) / (2 λ ) .

The decremental stability case is similar, except we replace φ ( x ) with 1 .

A.2 Some Basic Extensions

Below, we present some extensions to MuS that help increase the fraction of the input to which we can guarantee stability.

̸

̸

A.2.1 Feature Grouping

We have so far assumed that X = R n , but sometimes it may be desirable to group features together, e.g., color channels of the same pixel. Our results also hold for more general X = R d 1 ×··· × R d n , where for such x ∈ X and α ∈ R n we lift ⊙ as

All of our proofs are identical under this construction, with the exception of the dimensionalities of terms like ( x ⊙ α ) . Figure 1 gives an example of feature grouping.

B All Experiments

Models, Datasets, and Explanation Methods We evaluate on two vision models (Vision Transformer [9] and ResNet50 [29]) and one language model (RoBERTa [30]). For the vision dataset, we use ImageNet1K [31]; for the language dataset, we use TweetEval [32] sentiment analysis. We use four explanation methods in SHAP [7], LIME [6], Integrated Gradients (IGrad) [8], and Vanilla Gradient Saliency (VGrad) [5]; where we take φ ( x ) as the topk weighted features.

Training Details We used Adam [61] as our optimizer with default parameters and a learning rate of 10 -6 for 5 epochs. Because we consider λ ∈ { 1 / 8 , 2 / 8 , 3 / 8 , 4 / 8 , 8 / 8 } and h ∈ { Vision Transformer , ResNet50 , RoBERTa } , there are a total of 15 different models for most experiments. To train with a particular λ : for each training input x , we generate two random maskings one where λ of the features are zeroed and one where λ/ 2 of the features are zeroed. This additional λ/ 2 zeroing is to account for the fact that inputs to a smoothed model will be subject to masking by λ as well as φ ( x ) , where the scaling factor of 1 / 2 is informed by our prior experience about the size of a stable explanation.

Miscellaneous Preprocessing For images in ImageNet1K we use feature grouping (Section A.2.1) to group the 3 × 224 × 224 dimensional image into patches of size 3 × 28 × 28 , such that there remains n = 64 feature groups. Each feature of a feature group then receives the same value of noise during smoothing. We report radii of stability as a fraction of the feature groups covered. For example, if at some input from ImageNet1K, we get an incremental stability radius of r , then we report r/ 64 as the fraction of features up to which we are guaranteed to be stable. This is especially amenable to evaluating RoBERTa on TweetEval where inputs do not have uniform token lengths, i.e., do not have uniform feature dimensions. In all of our experiments, we use the quantized noise as in Section 3.3 with a quantization parameter of q = 64 , with the exception of Appendix B.2 and Appendix B.3. Our experiments are organized as follows:

(Section B.1) What is the quality of stability guarantees?

(Section B.2) What is the theoretical vs empirical stability that can be guaranteed?

-We q = 64 for theoretical guarantees, q = 16 for empirical guarantees.

(Section B.3) What are the stability-accuracy trade-offs?

-We use q ∈ { 16 , 32 , 64 , 128 } to study the effect of q on MuS-smoothed performance.

(Section B.4) Which explanation method is best?

(Section B.5) Does additive smoothing improve empirical stability?

(Section B.6) Does adversarial training improve empirical stability?

B.1 Quality of Stability Guarantees

Here we study what radii of stability are certifiable, and how often these can be achieved with different models and explanation methods. We therefore consider explainable models ⟨ f, φ ⟩ constructed from base models h ∈ { Vision Transformer , ResNet50 , RoBERTa } and explanation methods φ ∈ { SHAP , LIME , IGrad , VGrad } with topk ∈ { 1 / 8 , 2 / 8 , 3 / 8 } feature selection. We take N = 2000 samples from each model's respective datasets and compute the following value for each radius:

Plots of incremental stability are on the left; plots of decremental stability are on the right.

Vision Transformer, SHAP

0.8

0.8

Cons+IncStab (SHAP-1/8)

Cons+DecStab (SHAP-1/8)

Hold Rate

入=8/8(0.628)

0.6

==-入=4/8 (0.577)

0.6

==- 入=4/8 (0.577)

......

....

0.4

-入=1/8 (0.075)

0.4

-入=1/8 (0.075)

入=1/16 (0.001)

入=1/16 (0.001)

Prop

0.2

0.2

0.0

0.0

0.00

0.01

0.02

0.03

0.04

0.05

0.06

0.00

0.01

0.02

0.03

0.04

0.05

0.06

0.8

Cons+IncStab (SHAP-2/8)

0.8-

Cons+DecStab(SHAP-2/8)

Rate

入=8/8 (0.840)

入=8/8 (0.840)

0.6

入=4/8 (0.801)

0.6-

入=4/8 (0.801)

Prop Hold

入=1/8 (0.515)

入=1/8 (0.515)

0.4

入=1/16 (0.052)

0.4

入=1/16 (0.052)

0.2

0.2

0.0

10.0

0.00

0.01

0.02

0.03

0.04

0.05

0.06

0.00

0.01

0.02

0.03

0.04

0.05

0.06

Cons+IncStab (SHAP-3/8)

Cons+DecStab(SHAP-3/8)

0.8

0.8

Rate

入=8/8 (0.896)

入=8/8 (0.896)

入=4/8 (0.867)

入=4/8 (0.867)

R

0.6

0.6

PIOH

入=1/8 (0.679)

入=1/8 (0.679)

0.4

入=1/16 (0.326)

0.4

入=1/16 (0.326)

Prop

0.2

0.2

0.0

0.0

0.00

0.01

0.02

0.03

0.04

0.05

0.06

0.00

0.01

0.02

0.03

0.04

0.05

0.06

Fraction ofFeatures

Fraction of Features

Vision Transformer, LIME

0.8

0.8

Cons+IncStab (LIME-1/8)

Cons+DecStab (LIME-1/8)

Rate

入=8/8 (0.538)

入=8/8 (0.538)

0.6

入=4/8 (0.512)

0.6

入=4/8 (0.512)

PIOH

0.4

入=1/8 (0.074)

0.4

入=1/8 (0.074)

入=1/16 (0.001)

入=1/16 (0.001)

Prop

0.2

0.2

0.0

0.0

0.00

0.01

0.02

0.03

0.04

0.05

0.06

0.00

0.01

0.02

0.03

0.04

0.05

0.06

0.8

0.8

Cons+IncStab (LIME-2/8)

Cons+DecStab (LIME-2/8)

Rat

0.6

入=4/8 (0.731)

入=8/8 (0.727)

9°0

入=4/8 (0.731)

入=8/8 (0.727)

IOH

=

0.4

.....

-入=1/8 (0.494)

0.4

入=1/8 (0.494)

入=1/16 (0.053)

入=1/16 (0.053)

Prop

0.2

0.2

00

0.0

0.00

0.01

0.02

0.03

0.04

0.05

0.06

0.00

0.01

0.02

0.03

0.04

0.05

0.06

0.8

Cons+IncStab (LIME-3/8)

0.8

Cons+DecStab (LIME-3/8)

Rate

入=8/8 (0.815)

入=8/8 (0.815)

0.6

入=4/8 (0.805)

0.6

入=4/8 (0.805)

入=2/8 (0.762)

入=2/8 (0.762)

Hold

0.4

入=1/8 (0.666)

0.4

入=1/8 (0.666)

入=1/16 (0.325)

入=1/16 (0.325)

Prop

0.2

0.2

100

0.0

0.00

0.01

0.02

0.03

0.04

0.05

0.06

0.00

0.01

0.02

0.03

0.04

0.05

0.06

Fraction ofFeatures

Fraction ofFeatures

Vision Transformer, IGrad

0.8

0.8

Cons+IncStab (IGrad-1/8)

Cons+DecStab (IGrad-1/8)

Hold Rate

入=8/8(0.534)

0.6

==- 入=4/8 (0.500)

0.6

......

....

0.4

-入=1/8 (0.087)

0.4

.-入=1/8 (0.087)

入=1/16 (0.001)

入=1/16 (0.001)

Prop

0.2

0.2

0.0

0.0

0.00

0.01

0.02

0.03

0.04

0.05

0.06

0.00

0.01

0.02

0.03

0.04

0.05

0.06

0.8

0.8

Cons+IncStab (IGrad-2/8)

Cons+DecStab (IGrad-2/8)

Rate

入=8/8 (0.764)

入=8/8 (0.764)

0.6

入=4/8 (0.724)

0.6

入=4/8 (0.724)

Hold

0.4

入=1/8 (0.486)

0.4

入=1/8 (0.486)

入=1/16 (0.064)

入=1/16 (0.064)

Prop

0.2

0.2

00

10.0

0.00

0.01

0.02

0.03

0.04

0.05

0.06

0.00

0.01

0.02

0.03

0.04

0.05

0.06

0.8

Cons+IncStab (IGrad-3/8)

0.8

Cons+DecStab (IGrad-3/8)

Rate

入=8/8 (0.852)

入=8/8 (0.852)

0.6

入=4/8 (0.825)

0.6

入=4/8 (0.825)

PIOH

入=1/8 (0.667)

入=1/8 (0.667)

0.4

入=1/16 (0.339)

0.4

入=1/16 (0.339)

Prop

0.2

0.2

0.0

0.0

0.00

0.01

0.02

0.03

0.04

0.05

0.06

0.00

0.01

0.02

0.03

0.04

0.05

0.06

Fraction ofFeatures

Fraction ofFeatures

Vision Transformer, VGrad

0.8

0.8

Cons+IncStab (VGrad-1/8)

Cons+DecStab (VGrad-1/8)

Rate

入=8/8 (0.353)

入=8/8 (0.353)

0.6

入=4/8 (0.342)

0.6

入=4/8 (0.342)

PIOH

0.4

入=1/8 (0.049)

0.4

入=1/8 (0.049)

入=1/16 (0.001)

入=1/16 (0.001)

Prop

0.2

0.2

0.0

0.0

0.00

0.01

0.02

0.03

0.04

0.05

0.06

0.00

0.01

0.02

0.03

0.04

0.05

0.06

0.8

0.8

Cons+IncStab (VGrad-2/8)

Cons+DecStab (VGrad-2/8)

Rate

入=8/8 (0.634)

入=8/8(0.634)

0.6

0.6

=-- 入=4/8 (0.618)

Hold

.....

......

0.4

入=1/8 (0.371)

0.4

-入=1/8 (0.371)

入=1/16 (0.035)

入=1/16 (0.035)

Prop

0.2

0.2

0.0

0:0

0.00

0.01

0.02

0'0

0.04

0.05

0.06

0.00

0.01

0.02

0.03

0.04

0.05

0.06

0.8

F80

Cons+IncStab (VGrad-3/8)

Cons+DecStab (VGrad-3/8)

Rate

入=8/8 (0.782)

入=8/8 (0.782)

0.6

入=4/8 (0.768)

0.6

入=4/8 (0.768)

...

0.4

入=1/8 (0.578)

0.4

入=1/8 (0.578)

入=1/16 (0.266)

入=1/16 (0.266)

Prop

0.2

0.2

100

0.0

0.00

0.01

0.02

0.03

0.04

0.05

0.06

0.00

0.01

0.02

0.03

0.04

0.05

0.06

Fraction ofFeatures

Fraction ofFeatures

ResNet50, SHAP

0.8

0.8

Cons+IncStab (SHAP-1/8)

Cons+DecStab (SHAP-1/8)

Hold Rate

入=8/8(0.652)

入=8/8 (0.652)

0.6

==-入=4/8 (0.602)

0.6

......

....

0.4

-- 入=1/8 (0.131)

0.4

- 入=1/8 (0.131)

入=1/16 (0.001)

入=1/16 (0.001)

Prop

0.2

0.2

0.0

0.0

0.00

0.01

0.02

0.03

0.04

T

0.05

0.06

0.00

0.01

0.02

0.03

0.04

0.05

0.06

0.8

Cons+IncStab (SHAP-2/8)

0.8

Cons+DecStab(SHAP-2/8)

Rate

入=8/8 (0.806)

入=8/8 (0.806)

0.6

入=4/8 (0.764)

0.6

入=4/8 (0.764)

Prop Hc

0.4

入=1/8 (0.549)

入=1/16 (0.075)

0.4

入=1/8 (0.549)

入=1/16 (0.075)

0.2

0.2

00

0.0

0.00

0.01

0.02

0.03

0.04

0.05

0.06

0.00

0.01

0.02

0.03

0.04

0.05

0.06

0.8

Cons+IncStab (SHAP-3/8)

0.8

Cons+DecStab(SHAP-3/8)

Rate

入=4/8 (0.825)

入=4/8 (0.825)

0.6

0.6

PIOH

入=1/8 (0.673)

入=1/8 (0.673)

0.4

入=1/16 (0.386)

0.4

入=1/16 (0.386)

Prop

0.2

0.2

0.0

0.0

0.00

0.01

0.02

0.03

0.04

0.05

0.06

0.00

0.01

0.02

0.03

0.04

0.05

0.06

FractionofFeatures

Fraction of Features

ResNet50, LIME

0.8

0.8

Cons+IncStab (LIME-1/8)

Cons+DecStab (LIME-1/8)

Rate

入=8/8 (0.386)

入=8/8 (0.386)

0.6

入=4/8 (0.395)

0.6

入=4/8 (0.395)

PIOH

0.4

入=1/8 (0.067)

0.4

入=1/8 (0.067)

入=1/16 (0.001)

入=1/16 (0.001)

Prop

0.2

0.2

0.0-

0.0

0.00

0.01

0.02

0.03

0.04

0.05

0.06

0.00

0.01

0.02

0.03

0.04

0.05

0.06

0.8

0.8

Cons+IncStab (LIME-2/8)

Cons+DecStab (LIME-2/8)

Rate

入=8/8 (0.593)

入=8/8 (0.593)

0.6

入=4/8 (0.597)

90

入=4/8 (0.597)

OH

0.4

.....

入=1/8 (0.408)

0.4

入=1/8 (0.408)

入=1/16 (0.051)

入=1/16 (0.051)

Prop

0.2

0.2

0.0

0.0

0.00

0.01

0.02

0.03

0.04

0.05

0.06

0.00

0.01

0.02

0.03

0.04

0.05

0.06

0.8

0.8

Cons+IncStab (LIME-3/8)

Cons+DecStab (LIME-3/8)

Rate

入=8/8 (0.715)

入=8/8 (0.715)

0.6

入=4/8 (0.712)

0.6

入=4/8 (0.712)

IOH

0.4

入=1/8 (0.566)

0.4

入=1/8 (0.566)

入=1/16 (0.283)

入=1/16 (0.283)

Prop

0.2

0.2

0.0

0.0

0.00

0.01

0.02

0.03

0.04

0.05

0.06

0.00

0.01

0.02

0.03

0.04

0.05

0.06

FractionofFeatures

FractionofFeatures

ResNet50, IGrad

0.8

0.8

Cons+IncStab (IGrad-1/8)

Cons+DecStab (IGrad-1/8)

Hold Rate

入=8/8 (0.505)

入=8/8 (0.505)

0.6

=-- 入=4/8 (0.485)

0.6

=-- 入=4/8 (0.485)

......

....

0.4

"-入=1/8 (0.084)

0.4

.-入=1/8 (0.084)

入=1/16 (0.001)

入=1/16 (0.001)

Prop

0.2

0.2

00

0.0

0.00

0.01

0.02

0.03

0.04

0.05

0.06

0.00

0.01

0.02

0.03

0.04

0.05

0.06

0.8

0.8

Cons+IncStab (IGrad-2/8)

Cons+DecStab (IGrad-2/8)

Rate

0.6

入=4/8 (0.690)

0.6

入=4/8 (0.690)

Hold

0.4

入=1/8 (0.461)

0.4

入=1/8 (0.461)

入=1/16 (0.072)

入=1/16 (0.072)

Prop

0.2

0.2

0.0

00

0.00

0.01

0.02

0.03

0.04

0.05

0.06

0.00

0.01

0.02

0.03

0.04

0.05

0.06

0.8

80

Cons+IncStab (IGrad-3/8)

Cons+DecStab(IGrad-3/8)

Rate

入=8/8 (0.788)

入=8/8 (0.788)

0.6

入=4/8 (0.775)

0.6

入=4/8 (0.775)

PIOH

0.4

入=1/8 (0.616)

0.4

入=1/8 (0.616)

入=1/16 (0.328)

入=1/16 (0.328)

Prop

0.2

0.2

0.0

0.0

0.00

0.01

0.02

0.03

0.04

0.05

0.06

0.00

0.01

0.02

0.03

0.04

0.05

0.06

Fraction of Features

Fraction of Features

ResNet50,VGrad

0.8

0.8

Cons+IncStab (VGrad-1/8)

Cons+DecStab (VGrad-1/8)

Rate

入=8/8 (0.415)

入=8/8 (0.415)

0.6

入=4/8 (0.412)

0.6

入=4/8 (0.412)

PIOH

0.4

入=1/8 (0.079)

0.4

入=1/8 (0.079)

入=1/16 (0.001)

入=1/16 (0.001)

Prop

0.2

0.2

0.0

0.0

0.00

0.01

0.02

0.03

0.04

0.05

0.06

0.00

0.01

0.02

0.03

0.04

0.05

0.06

0.8

0.8

Cons+IncStab (VGrad-2/8)

Cons+DecStab (VGrad-2/8)

Rate

入=8/8(0.641)

0.6

F9:0

=--入=4/8 (0.630)

.....

......

OH

0.4

入=1/8 (0.416)

0.4

入=1/8 (0.416)

入=1/16 (0.059)

入=1/16 (0.059)

Prop

0.2

0.2

0.0

0.0

0.00

0.01

0.02

0.03

0.04

0.05

0.06

0.00

0.01

0.02

0.03

0.04

0.05

0.06

0.8

0.8

Cons+IncStab (VGrad-3/8)

Cons+DecStab (VGrad-3/8)

Rate

入=8/8 (0.750)

入=8/8 (0.750)

0.6

入=4/8 (0.729)

0.6

入=4/8 (0.729)

....

IOH

0.4

入=1/8 (0.575)

0.4

入=1/8 (0.575)

入=1/16 (0.294)

入=1/16 (0.294)

Prop

0.2

0.2

0.0

0.0

0.00

0.01

0.02

0.03

0.04

0.05

0.06

0.00

0.01

0.02

0.03

0.04

0.05

0.06

FractionofFeatures

FractionofFeatures

RoBERTA,SHAP

Cons+IncStab (SHAP-1/8)

Cons+DecStab (SHAP-1/8)

Rate

0.8

入=8/8(0.740)

0.8

=-入=4/8 (0.787)

pIOH

0.6

0.6

....

-入=1/8 (0.788)

-入=1/8 (0.788)

0.4

入=1/16 (0.958)

0.4

入=1/16 (0.958)

Prop

0.2

0.2

0.0-

0.0

0.0

0.1

0.2

0.3

0.4

0.5

0.0

0.1

0.2

0.3

0.4

0.5

Cons+IncStab(SHAP-2/8)

Cons+DecStab (SHAP-2/8)

Rate

0.8

入=8/8 (0.826)

0.8

入=8/8 (0.826)

入=4/8 (0.834)

入=4/8 (0.834)

ploH

0.6

0.6

入=1/8 (0.839)

入=1/8 (0.839)

工

0.4

入=1/16 (0.958)

0.4

入=1/16 (0.958)

Prop

0.2

0.2

0.0

0.0

0.0

0.1

0.2

0.3

0.4

0.5

0.0

0.1

0.2

0.3

0.4

0.5

Cons+IncStab (SHAP-3/8)

Cons+DecStab (SHAP-3/8)

IRate

0.8

入=8/8 (0.864)

0.8

N

入=8/8 (0.864)

入=4/8 (0.848)

入=4/8 (0.848)

0.6

入=1/8 (0.866)

0.6

入=1/8 (0.866)

0.4

入=1/16 (0.960)

0.4

入=1/16 (0.960)

Prop

0.2

0.2

0.0-

0.0

0.0

0.1

0.2

0.3

0.4

0.5

0.0

0.1

0.2

0.3

0.4

0.5

Fraction ofFeatures

Fraction ofFeatures

RoBERTA,LIME

Cons+IncStab (LIME-1/8)

Cons+DecStab (LIME-1/8)

Rate

0.8

入=8/8 (0.659)

0.8

入=8/8 (0.659)

入=4/8 (0.700)

入=4/8 (0.700)

ploH

R

0.6

入=1/8 (0.783)

0.6

入=1/8 (0.783)

0.4

入=1/16 (0.958)

0.4

入=1/16 (0.958)

Prop

0.2

0.2

0.0-

00

0.0

0.1

0.2

0.3

0.4

0.5

0.0

0.1

0.2

0.3

0.4

0.5

Cons+IncStab (LIME-2/8)

Cons+DecStab (LIME-2/8)

Rate

0.8

入=8/8 (0.734)

0.8

入=8/8 (0.734)

---入=4/8 (0.778)

入=4/8 (0.778)

pIOH

0.6

0.6

入=1/8 (0.804)

入=1/8 (0.804)

0.4

入=1/16 (0.958)

0.4

入=1/16 (0.958)

Prop

0.2

0.2

00

00

0.0

0.1

0.2

E'0

0.4

0.5

0.0

0.1

0.2

0.3

0.4

0.5

Cons+IncStab (LIME-3/8)

Cons+DecStab (LIME-3/8)

Rate

0.8

入=8/8 (0.788)

0.8-

入=8/8 (0.788)

入=4/8 (0.819)

入=4/8 (0.819)

PIOH

0.6

0.6

入=1/8 (0.839)

入=1/8 (0.839)

0.4

入=1/16 (0.958)

0.4

入=1/16 (0.958)

Prop

0.2

0.2

0.0

0.0

0.0

0.1

0.2

0.3

0.4

0.5

0.0

0.1

0.2

0.3

0.4

0.5

Fraction ofFeatures

Fraction of Features

RoBERTA, IGrad

Cons+IncStab (IGrad-1/8)

Cons+DecStab (IGrad-1/8)

Rate

0.8

入=8/8(0.663)

0.8

入=8/8 (0.663)

==-入=4/8 (0.768)

PloH

0.6

0.6

....

…-入=1/8(0.788)

-入=1/8 (0.788)

二

0.4

入=1/16 (0.958)

0.4

入=1/16 (0.958)

Prop

0.2

0.2

0.0-

0.0

0.0

0.1

0.2

0.3

0.4

0.5

0.0

0.1

0.2

0.3

0.4

0.5

Cons+IncStab (IGrad-2/8)

Cons+DecStab (IGrad-2/8)

Rate

0.8

入=8/8 (0.731)

0.8

入=8/8 (0.731)

入=4/8 (0.808)

入=4/8 (0.808)

ploH

0.6

0.6

入=1/8 (0.836)

入=1/8 (0.836)

0.4

入=1/16 (0.957)

0.4

入=1/16 (0.957)

Prop

0.2

0.2

0.0

0.0

0.0

0.1

0.2

0.3

0.4

0.5

0.0

0.1

0.2

0.3

0.4

0.5

Cons+IncStab (IGrad-3/8)

Cons+DecStab(IGrad-3/8)

Rate

0.8

入=8/8 (0.808)

0.8

入=8/8 (0.808)

入=4/8 (0.847)

入=4/8 (0.847)

0.6

入=1/8 (0.877)

0.6

入=1/8 (0.877)

0.4-

入=1/16 (0.961)

0.4

入=1/16 (0.961)

Prop

0.2

0.2

0.0-

0.0

0.0

0.1

0.2

0.3

0.4

0.5

0.0

0.1

0.2

0.3

0.4

0.5

Fraction of Features

Fraction of Features

RoBERTA,VGrad

Cons+IncStab (VGrad-1/8)

Cons+DecStab (VGrad-1/8)

Rate

0.8

入=8/8 (0.596)

0.8

入=8/8 (0.596)

入=4/8 (0.611)

入=4/8 (0.611)

PloH

R

0.6

0.6

入=1/8 (0.783)

入=1/8 (0.783)

0.4

入=1/16 (0.958)

0.4

入=1/16 (0.958)

Prop

0.2

0.2

0.0

00

0.0

0.1

0.2

0.3

0.4

0.5

0.0

0.1

0.2

0.3

0.4

0.5

Cons+IncStab (VGrad-2/8)

Cons+DecStab (VGrad-2/8)

Rate

0.8

入=8/8(0.682)

0.8

入=8/8(0.682)

=-- 入=4/8 (0.709)

=-- 入=4/8 (0.709)

pIOH

0.6

0.6

......

入=1/8 (0.795)

入=1/8 (0.795)

0.4

入=1/16 (0.958)

0.4

入=1/16 (0.958)

Prop

0.2

0.2

00

00

0.0

0.1

0.2

0.3

0.4

0.5

0.0

0.1

0.2

0.3

0.4

0.5

Cons+IncStab (VGrad-3/8)

Cons+DecStab (VGrad-3/8)

Rate

0.8

入=8/8 (0.753)

0.8-

入=8/8 (0.753)

入=4/8 (0.776)

入=4/8 (0.776)

PIOH

0.6

入=2/8 (0.780)

0.6

....

入=2/8 (0.780)

入=1/8 (0.819)

入=1/8 (0.819)

0.4

入=1/16 (0.958)

0.4

入=1/16 (0.958)

Prop

0.2

0.2

0.0

0.0

0.0

0.1

0.2

0.3

0.4

0.5

0.0

0.1

0.2

0.3

0.4

0.5

FractionofFeatures

Fraction ofFeatures

B.2 Theoretical vs Empirical

We compare the certifiable theoretical stability guarantees with what is empirically attained via a standard box attack search [33]. This is an extension of Section 4.2, where we now show all models as evaluated with SHAP-top25%. We take q = 64 with N cert = 2000 for the certified plots, and q = 64 with N emp = 250 for the empirical plots. This is because box attack is time-intensive.

Vision Transformer, SHAP top-25%

0.8

Cert Cons+IncStab

0.8

Cert Cons+DecStab

Rate

入=8/8(0.840)

入=8/8 (0.840)

0.6

=-入=4/8 (0.801)

0.6

入=4/8 (0.801)

Hold

0.4

-入=1/8 (0.515)

0.4

-入=1/8 (0.515)

Prop

0.2

0.2

0.0

0.0

0.00

0.01

0.02

0.03

0.04

0.05

0.06

0.00

0.01

0.02

0.03

0.04

0.05

0.06

0.8

Emp Cons+IncStab

0.8

Emp Cons+DecStab

Rate

入=8/8 (0.844)

入=8/8 (0.844)

0.6

--入=4/8 (0.816)

0.6

入=4/8 (0.816)

Hold

0.4

入=1/8 (0.496)

0.4

入=1/8(0.496)

Prop

0.2

0.2

0.0

0.0

0.0

0.1

0.2

0.3

0.4

0.5

0.0

0.1

0.2

0.3

0.4

0.5

FractionofFeatures

Fraction ofFeatures

ResNet50,SHAP top-25%

0.8

CertCons+IncStab

80

Cert Cons+DecStab

Prop Hold Rate

入=8/8(0.806)

入=8/8(0.806)

0.6

入=4/8 (0.764)

0.6

入=4/8 (0.764)

0.4

-入=1/8 (0.549)

0.4

-入=1/8 (0.549)

0.2

0.2

00

00

0.00

0.01

0.02

0.03

0.04

0.05

0.06

0.00

0.01

0.02

0.03

0.04

0.05

0.06

0.8

Emp Cons+IncStab

80

Emp Cons+DecStab

Hold Rate

入=8/8 (0.804)

入=8/8 (0.804)

0.6

入=4/8 (0.776)

0.6

入=4/8 (0.776)

0.4

入=1/8(0.556)

0.4

入=1/8(0.556)

Prop

0.2

0.2

0.0

0.0

0.0

0.1

0.2

0.3

0.4

0.5

0.0

0.1

0.2

0.3

0.4

0.5

Fraction ofFeatures

Fraction ofFeatures

RoBERTa,SHAP top-25%

0.8

CertCons+lncStab

0.8

CertCons+DecStab

Rate

入=8/8 (0.826)

入=8/8 (0.826)

0.6

入=4/8 (0.834)

0.6

入=4/8 (0.834)

PIOH

0.4

-入=1/8 (0.839)

0.4

入=1/8 (0.839)

Prop

0.2

0.2

00

0.0

0.0

0.1

0.2

E'0

0.4

0.0

0.1

0.2

0.3

0.4

0.8

0.8

Rate

0.6

0.6

PIOH

Emp Cons+IncStab

Emp Cons+DecStab

0.4

0.4

Prop

入=4/8 (0.828)

入=4/8 (0.828)

0.2

.....

0.2

.....

入=1/8 (0.852)

入=1/8 (0.852)

0.0

0.0

0.0

0.1

0.2

0.3

0.4

0.0

0.1

0.2

0.3

0.4

FractionofFeatures

FractionofFeatures

B.3 Stability-Accuracy Trade-Offs

Westudy how the accuracy degrades with λ . Weconsider a smoothed model f constructed from a base classifier h ∈ { Vision Transformer , ResNet50 , RoBERTa } and vary λ ∈ { 1 / 16 , 1 / 8 , 2 / 8 , 4 / 8 , 8 / 8 } . We then take N = 2000 samples from each respective dataset and measure the accuracy of f at different radii. We use f ( x ) ∼ = true_label to mean that f attained the correct prediction at x ∈ X , and we plot the following value at each radius r :

Below, we show the plots for different quantization parameters of q ∈ { 16 , 32 , 64 , 128 } . The q = 64 plots are identical to that of Figure 5. We see that increasing q generally improves the performance of MuS, but at a computational cost since this requires q evaluations of h .

Vision Transformer (q=16)

ResNet50 (q=16)

RoBERTa (q=16)

0.8

0.8

入=8/8 (0.756)

0.8

入=4/8 (0.793)

0.6

入=4/8 (0.722)

入=4/8 (0.740)

0.6

0.6

入=1/8 (0.688)

入=1/8 (0.554)

入=1/8 (0.568)

Certified

0.4

入=1/16 (0.592)

0.4

入=1/16 (0.480)

0.4

入=1/16 (0.466)

0.2

0.2

0.2

0.0

0.0

0.0

0.00

0.05

0.10

0.15

0.00

0.05

0.10

0.15

0.0

0.1

0.2

0.3

0.4

0.5

Fraction of Features

Fraction of Features

Fraction of Features

Vision Transformer(q=32)

ResNet50 (q=32)

RoBERTa (q=32)

0.8

0.8

入=8/8 (0.756)

0.8

入=4/8 (0.790)

入=4/8 (0.727)

入=4/8 (0.751)

0.6

0.6

入=2/8 (0.648)

0.6

入=2/8 (0.646)

入=1/8 (0.693)

入=1/8 (0.557)

入=1/8 (0.553)

Certified

0.4

入=1/16 (0.617)

0.4

入=1/16 (0.492)

0.4

入=1/16 (0.465)

0.2

0.2

0.2

0.0

0.0

0.0

0.00

0.05

0.10

0.15

0.00

0.05

0.10

0.15

0.0

0.1

0.2

0.3

0.4

0.5

FractionofFeatures

Fraction of Features

Fraction ofFeatures

Vision Transformer(q=64)

ResNet50 (q=64)

RoBERTa (q=64)

0.8

0.8

入=8/8 (0.756)

0.8

2

入=4/8 (0.791)

入=4/8 (0.722)

入=4/8 (0.748)

Accurac

0.6

入=2/8 (0.752)

0.6

入=2/8 (0.646)

0.6

入=1/8 (0.695)

入=1/8 (0.571)

入=1/8 (0.566)

Certified

0.4

入=1/16 (0.598)

0.4

入=1/16 (0.501)

0.4

入=1/16 (0.467)

0.2

0.2

0.2

0.0

0.0

0.0

0.00

0.05

0.10

0.15

0.00

0.05

0.10

0.15

0.0

0.1

0.2

0.3

0.4

0.5

Fraction of Features

Fraction of Features

Fraction of Features

Vision Transformer(q=128)

ResNet50 (q=128)

RoBERTa (q=128)

0.8

0.8

入=8/8 (0.756)

0.8

入=4/8 (0.795)

入=4/8 (0.733)

入=4/8 (0.757)

0.6

0.6

入=2/8 (0.646)

0.6

入=1/8 (0.691)

入=1/8 (0.572)

入=1/8 (0.561)

Certified

0.4

入=1/16 (0.624)

0.4

入=1/16 (0.513)

0.4

入=1/16 (0.464)

0.2

0.2

0.2

0.0

0.0

0.0

0.00

0.05

0.10

0.15

0.00

0.05

0.10

0.15

0.0

0.1

0.2

0.3

0.4

0.5

Fraction of Features

Fraction of Features

Fraction of Features

B.4 Which Explanation Method is the Best?

We first investigate how many features are needed to yield consistent and non-trivially stable explanations, as done by the greedy selection algorithm in Section 2.4. For some x ∈ X , let k x denote the fraction of features that ⟨ f, φ ⟩ needs to be consistent, incrementally stable, and decrementally stable with radius 1 . We vary λ ∈ { 1 / 8 , . . . , 4 / 8 } , where recall λ ≤ 4 / 8 is needed for non-trivial stability, and use N = 250 samples to plot the average k x . This part is identical to Section 4.3.

VisionTransformer

ResNet50

RoBERTa

0.80

0.70

VGrad

VGrad

VGrad

atures

IGrad

0.78

IGrad

0.8

IGrad

F

e

0.68

LIME

SHAP

0.76

LIME

SHAP

0.7

SHAP

LIME

0.66

0.74

0.6

uo!

0.72

0.5

0.64

0.70

0.4

0.62

0.68

0.125

0.250

0.375

0.500

0.125

0.250

0.375

0.500

0.125

0.250

0.375

0.500

We next investigate the ability of each method to predict features that lead to high accuracy. Let f ( x ⊙ φ ( x )) ∼ = true_label , mean that the masked input x ⊙ φ ( x ) yields the correct prediction. We then plot this accuracy as we vary the topk ∈ { 1 / 8 , 2 / 8 , 3 / 8 } for different methods φ , and λ ∈ { 1 / 8 , . . . , 8 / 8 } , using N = 2000 samples.

Vision Transformer

0.7

0.75

0.5

0.70

0.4

0.6

0.65

p(x)-Masked /

0.3

Top-1/8

0.5

Top-2/8

0.60

8/-d01

VGrad

VGrad

VGrad

0.2

IGrad

0.4

IGrad

0.55

IGrad

0.1

.......

LIME

SHAP

LIME

SHAP

0.50

LIME

SHAP

0.3+

0.2

0.3

0.4

0.5

0.2

0.3

0.4

0.5

0.2

0.3

0.4

0.5

ResNet50

0.70

0.6

0.65

0.4

0.60

p(x)-Masked A

0.5

0.3

Top-1/8

Top-2/8

0.55

Top-3/8

VGrad

VGrad

VGrad

0.2

IGrad

0.4

IGrad

0.50

IGrad

LIME

LIME

0.45

LIME

0.1

SHAP

SHAP

SHAP

0.3

0.40

0.2

0.3

0.4

0.5

0.2

0.3

0.4

0.5

0.2

0.3

0.4

0.5

RoBERTa

0.70

0.75

0.75

0.65

0.70

0.70

0.65

0.60

0.65

sked

0.55

Top-1/8

0.60-

Top-2/8

0.60

Top-3/8

VGrad

0.55

VGrad

VGrad

0.50

IGrad

IGrad

0.55

IGrad

........

LIME

0.50

........

LIME

LIME

0.50

SHAP

0.45

SHAP

SHAP

0.2

'0

0.4

0.5

0.2

0.3

0.4

0.5

0.2

0.3

0.4

0.5

B.5 Empirical Stability of Additive Smoothing

We now study how well additive smoothing impacts empirical robustness compared to multiplicative smoothing. Although additive smoothing cannot yield theoretical stability guarantees (c.f. Proposition 3.1), we nevertheless investigate its empirical performance. We use explanations generated by SHAP top-25% and use Vision Transformer as our base model h . We fine-tuned different variants of Vision Transformer at λ ∈ { 8 / 8 , 4 / 8 , 2 / 8 , 1 / 8 } and used these in two general classes of models: Vision Transformer with multiplicative smoothing, and Vision Transformer with additive smoothing. We call these two f mus and f add respectively, and define them as follows:

where for multiplicative smoothing, D is as in Theorem 3.2. Over N = 250 samples from ImageNet1K, we check how often incremental stability of radius ≥ 1 is obtained, and plot our results in Figure 17 (left). We see that multiplicative smoothing yields better empirical performance than additive smoothing.

B.6 Empirical Stability of Adversarial Training

Similar to Section B.5, we also check how adversarial training [62] affects empirical stability. We consider ResNet50 with different adversarial training setups from the Robustness Python library [63] and compare them to their respective MuS-wrapped variants. As with Section B.5 we take N = 250 samples from ImageNet1K, and plot our results in Figure 17 (right).

IncrementallyStablewithRadius≥1

IncrementallyStablewithRadius≥1

1.0

0.6

AdditiveSmoothing

AdvRobustTrained

MultiplicativeSmoothing

MuS-Wrapped

0.5

0.8

0.448

Samples

0.716

Samples

0.432

0.66

0.4

0.356

0.6

0.324

0.512

Fraction of

Fraction of

0.3

0.4

0.32

0.36

0.364

0.2

0.208

0.176

0.18

0.232

0.2

0.184

0.156

0.1

0.0

0.0

入=8/8

入=4/8

入=2/8

入=1/8

PretrainOnlyL2(ε=3.0)

L∞ (ε=4)

L∞ (ε=8)

B.7 Discussion

Effect of Smoothing We observe that smoothing can yield non-trivial stability guarantees, especially for Vision Transformer and RoBERTa, as evidenced in Appendix B.1. We see that smoothing is least detrimental on these two transformer-based architectures, and most negatively impacts the performance of ResNet50. We conjecture that although different training set-ups may improve performance across every category, this still serves to illustrate the general trend.

Theoretical vs Empirical It is expected that the certifiable radii of stability is more conservative than what is empirically observed. As mentioned in Section 3.2, for each λ , there is a maximum radius to which stability can be guaranteed, which is an inherent limitation of using confidence gaps and Lipschitz constants as the main theoretical technique. We emphasize that the notion of stability need not be tied to smoothing, though we are currently not aware of other viable approaches.

Why these Explanation Methods? We chose SHAP, LIME, IGrad, and VGrad from among the large variety of methods available primarily due to their popularity, and because we believe that they are collectively representative of many techniques. In particular, we believe that LIME remains a representative baseline for surrogate model-based explanation methods. SHAP and IGrad are, to our knowledge, the two most well-known families of axiomatic feature attribution methods. Finally, we believe that VGrad is representative of a traditional gradient saliency-based approach.

Which Explanation Method is the Best? Based on our experiments in Appendix B.4 we see that SHAP generally achieves higher accuracy using the same amount of topk features as other methods. On the other hand, VGrad tends to perform poorly. We remark that there are well-known critiques against the usefulness of saliency-based explanation methods [53].

C Miscellaneous

Relevance to Other Explanation Methods Our key theoretical contribution of MuS in Theorem 3.2 is a general-purpose smoothing method that is distinct from standard smoothing techniques, namely additive smoothing. Therefore, MuS is applicable to other problem domains beyond what is studied in this paper and would be useful where small Lipschitz constants with respect to maskings are desirable.

Broader Impacts Reliable explanations are necessary for making well-informed decisions and are increasingly important as machine learning models are integrated with fields like medicine, law, and business - where the primary users may not be well-versed in the technical limitations of different methods. Formal guarantees are thus important for ensuring the predictability and reliability of complex systems, which then allows users to construct accurate mental models of interaction and behavior. In this work, we study a particular kind of guarantee known as stability, which is key to feature attribution-based explanation methods.

Peer reviews:
comments: SUMMARY:
The paper is about the stability of explanations in the feature attribution setting for image classification tasks. They illustrate that in certain settings, swapping/removing a pixel of an explanation completely changes the classifiers prediction, which is undesired.

They define what a "stable" explanation is formally and introduce Multiplicative Smoothing as a method to obtain these stable explanations. In principle it's multiplying the image with feature attribution masks such that the classifier becomes smooth in the sense that it doesn't suddenly (non-smoothly) change it's prediction from for example cat to goldfish. 

Experiments demonstrate the efficacy of MuS.

STRENGTHS:
1. It's clear what the paper is trying to achieve
2. Solution proposal is simple and thoroughly explained with good theoretical justification.

WEAKNESSES:
1. It's a bit unclear what exactly smoothness refers to, currently it seems to refer to no discontinuous jumps when removing a feature attribution pixel in classification output? 
2. The experiments are a bit unintuitive, needs clarification what's being tested here and why.

QUESTIONS:
1. What's smoothness exactly, never got a good intuition what this exactly means.
2. Don't understand what the experiments do, please explain how and why these experiments demonstrate that MuS is working as intended. 
3. Is it correct to understand that the paper is in some sense looking for the "minimal" set up features that explains a class that won't change with some perturbation ? 

-----

Post rebuttal: Authors did a good job explaining what's going on, left some feedback for exposition, raised to 6, it's an interesting paper.

LIMITATIONS:
N/A
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2023-07-01

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper aims to make the classifier robust to feature removal and addition. The authors find that adding a patch to the mask obtained from explanation method may cause the classifier to make substantially different prediction. They introduce the notion of incremental\nstability and decremental stability to measure classifier's stability w.r.t mask change. They introduce MuS to any base classifier and show that MuS provides stability guarantee for this classifier.\n\nSTRENGTHS:\n1. The notion of incremental stability and decremental stability is novel. \n2. They motivate multiplicative masking instead of the widely-adopted additive masking by pointing out the inconsistency involved in additive masking. \n3. Certified stability guarantee is obtained. \n4. A sample-efficient algorithm is introduced to reduce the sample complexity to sample from Bernoulli distribution.\n\nWEAKNESSES:\n1. Although masks can be sampled efficiently, to get the final prediction, a lot of inference steps are involved. This might be time-consuming. The authors do not talk about how this could be tackled. \n2. MuS seems to be similar with Randomized Smoothing with Bernoulli distribution. By averaging predictions under several masks, the classifier use information from the whole image to make prediction. It is not too surprising that the classifier is more robust to mask change.\n\nQUESTIONS:\n1. Are robust models also sensitive to mask removal or addition?\n2. There are many works discussing robust attributions (e.g., [1], [2]) and establishing the connection between the robustness of explanations and the robustness of black-box models (e.g., [3], [4]). What's the relationship between these works and this paper?\n\n[1] Dombrowski et al., Explanations can be manipulated and geometry is to blame, NeurIPS 2019\n\n[2] Wang et al., Smoothed Geometry for Robust Attribution, NeurIPS 2020\n\n[3] Tan et al., Robust Explanation for Free or At the Cost of Faithfulness, ICML 2023\n\n[4] Agarwal et al., Rethinking Stability for Attribution-based Explanations, ICLR 2022\n\nLIMITATIONS:\nListed is Weakness and Questions sections.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-02"}

comments: SUMMARY:
In this paper the authors have introduced a framework to measure the stability of feature attribution methods. They do so by introducing two relaxed notions of stability called incremental stability and descremental stability which check for stability in a neighbourhood of the original feature set. They show that size of these stable neighbourhoods can be measured for lipschitz smooth classification functions and then present a methdology to convert any classified into a Lipschitz smooth classifier using random sampling. Finally they conclude the paper with numerical experiments to illustrate the benefits of their approach.

STRENGTHS:
I feel that this paper has several strengths. 
Understanding the stability of feature attribution methods is a key requirement to evaluate them. This paper presents a novel and efficient approach to understanding these stability metrics which can then be used to compare quality of the features identified by the different methods. The stabiilty metric makes intuitive sense and the theoretical results justify the measurement techniques for them. They also present a way of how this stability metric can be extended to any classifier. 
Overall, I believe this paper makes a key contribution in this area.

WEAKNESSES:
The metric seems more descriptive than prescriptive. It would be difficult to some how incorporate he metric into the optimization problem so as to guide the network and method towards more stable models and feature attributers.

QUESTIONS:
I think it is important to provide a discussion about what the impact of Mulitplicative smoothening is on the original classifier. 
How is the stability metric of the smoothened classifier associated to the original classifier.

LIMITATIONS:
Adequately addressed.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2023-07-11

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper presents a technique for extracting feature attributions that are certifiably stable in the sense that the model's predictions are consistent on supersets of attributed features. As the title suggests, the approach is based on multiplicative smoothing, a novel type of Bernoulli smoothing based on masking, rather than perturbing, input features. Notably, the paper shows how to construct a distribution with coordinate-wise Bernoulli features that requires significantly fewer samples to obtain a certificate.\n\nSTRENGTHS:\nThis work introduces a new type of attribution stability that is not addressed by prior work on robust feature attribution, and presents rigorous yet practical techniques for obtaining stable explanations. \n\nAs the core technique is based on randomized smoothing, it is applicable broadly to different types of models and even different feature attribution methods. Thus, it complements a large body of existing work, and gives concrete, practical ways to improve it with respect to stability.\n\nWhile smoothing-based certification methods are typically very expensive, the structured dependency sampling described in section 3.3 gives a clever way to avoid this, making the approach reasonably inexpensive.\n\nThe empirical results give a sense of what the guarantees look like on real data, and compare several widely-used attribution methods under the certification regime.\n\nThe writing is polished and clear on most of the important points. Overall the paper is an enjoyable read.\n\nWEAKNESSES:\nThe accuracy penalty imposed by multiplicative smoothing is large (a related small point: the horizontal axis between figures 4 and 5 isn't the same, making these results a little harder to line up). It doesn't seem that the models were trained with augmentations that match smoothing noise, or that denoising was used on smoothing samples, so there may be opportunities to improve on this. Without improvements, the results in Figure 5 pull somewhat against the practical significance of the work.\n\nThe writing is a bit unclear around the sufficiency of Lipschitzness and masking equivalence. A less-careful read of the paper that focuses more on early sections might come away with the impression that most techniques for obtaining Lipschitz models will suffice for the stability guarantees sketched in remark 2.6. The authors might consider moving some of the intuitions from proposition 3.1 into this discussion.\n\nThe notion of stability at the center of this work is different than what I would have expected, being familiar with some prior work on this topic and reading the abstract. I agree that the model needs to play a central role in characterizing the stability of explanations, but it's less clear that being stable to added features is always an important property. The paper might have wider appeal if the authors are able to justify this, using examples or otherwise (Figure 1 is indeed an example, but doesn't address the question of why the classifier's behavior over the right two images is harmful or undesirable).\n\nQUESTIONS:\n1. Were the models used in figures 4 and 5 trained on appropriately augmented data?\n2. Have you attempted to use denoising methods as an intermediate step after sampling from the smoothing distribution?\n3. Are there other existing methods for obtaining Lipschitz models, perhaps without smoothing, that could also work with your certificates?\n\nLIMITATIONS:\nThe paper does not have an explicit limitations section, but these points are discussed where it is appropriate.", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 4, "DATE": "2023-07-12"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe draft addresses an important problem of feature attribution method selection and formalizes a notion of attribution stability to do this. The approach is based on modifying classifiers to make them Lipschitz wrt to feature masking. Once done, the new classifier has provable radii of stability defined as L1 distances within which the found sparse explanation is sufficient and adding new features will not change the prediction.\n\nThe approach is tested on vision and NLP tasks with four different explanation methods.\n\nDespite some weaknesses the paper is a good contribution to the field of feature attribution explanations.\n\nSTRENGTHS:\n- new measurable definition of explanation stability\n- a method to convert any classier into a one that permits the stability calculation\n- theoretical analysis; efficient computation\n- comprehensive empirical evaluation\n\nWEAKNESSES:\n- The need to modify the classifier, before guarantees can be calculated, somewhat diminishes the practical value. In practice, we would be interested in the classifier that actually does the prediction, not a smoothed version of it; as the authors in Sec. 4.1 measured the drop of accuracy from the smoothing to be tens of %s, it could be a no-go for critical classifiers.\n- The proposed stability is operationalized only for binarized explanations and features as image patches, which throws the classifier into the out-of-distribution regime and, on top of smoothing, further complicates drawing conclusions from the approach. In fact all feature attribution methods compared with proposed stability are continuous and require binarization before they can be used.\n\nQUESTIONS:\n- typo in the last word of line 44\n- nitpick: logits are usually defined as pre-softmax values, not probabilities (line 71)\n- Figures are not readable in black and white\n\nLIMITATIONS:\nNo dedicated Limitations section but limitations mentioned in the text", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 2, "SOUNDNESS_CORRECTNESS": 4, "DATE": "2023-07-23"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper studies the stability of binary attributions. The attribution is defined to be stable if the prediction does not change when adding additional features. The multiplicative smoothing is proposed to achieve the Lipschitz condition, which is proved to infer the relaxation of stability. Experiments verify that the smoothing technique is useful for stability.\n\nSTRENGTHS:\n1. The paper is well-written and easy to understand. The problem of attribution stability is novel. \n2. The proposed multiplicative smoothing is effective under the defined stability setting and can be applied to any models.\n\nWEAKNESSES:\n1. For the smoothing technique, my main concern is its application scope, which currently focuses only on binary attribution methods. Although a method is proposed to convert non-binary attributions into binary ones, this conversion process disregards the continuous information in the attribution maps. The potential loss of information during this conversion is significant, especially when dealing with features grouped into 64 patches.\n2. The motivation behind this work can be emphasized further. While it is mentioned that explanations can be fragile, more details are necessary to inform readers about the specific circumstances under which explanations may fail. In order for the stabilty of explanations to be meaningful, classifiers must rely solely on attributions, or more features added on attributions, to make correct decisions. However, it remains unclear why practitioners would opt to use attributions for decision-making when they already have access to the original input data.\n3. The accuracy drop can also be a weakness of the proposed method. While the decrease is expected, a significant decrease in accuracy may suggest a limited usefulness of the method.\n\nQUESTIONS:\n1. I'm interested in the performance comparison of additive smoothing and multiplicative smoothing on the stability setting.\n\nLIMITATIONS:\nSee weakness", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-25"}