O N
E XPLAINING
N EURAL N ETWORK R OBUSTNESS WITH A CTIVATION P ATH
Anonymous authors
Paper under double-blind review
A BSTRACT
Despite their verified performance, neural networks are prone to be misled by maliciously designed adversarial examples.
This work investigates the robustness of neural networks from the activation pattern perspective.
We find that despite the complex structure of the deep neural network, most of the neurons provide locally stable contributions to the output, while the minority, which we refer to as float neurons, can greatly affect the prediction.
We decompose the computational graph of the neural network into the fixed path and float path and investigate their role in generating adversarial examples.
Based on our analysis, we categorize the vulnerable examples into Lipschitz vulnerability and float neuron vulnerability.
We show that the boost of robust accuracy from randomized smoothing is the result of correcting the latter.
We then propose an SC-RFP (smoothed classifier with repressed float path) to further reduce the instability of the float neurons and show that our result can provide a higher certified radius as well as accuracy.
I NTRODUCTION
Despite their verified performance, neural networks are prone to be misled by maliciously designed adversarial examples.
In response to this issue, many studies focus on defensive algorithms that aim to increase the robustness of deep neural networks.
One of the emerging topics in this field is certifiable methods that aim to construct a guaranteed region, within which classifiers are able to provide stable results regardless of the perturbation.
The certifiable methods can appear in two different forms: verifiable training and randomized smoothing.
This work introduces an SC-RFP (smoothed classifier with repressed float path) which builds on randomized smoothing algorithms and is able to further improve their robustness accuracy.
We decompose the local mapping function into fixed paths and float paths according to the stability of neurons alone on the path.
The fixed paths have a stable mapping relationship between input and output, while the float paths can result in a sudden change of the mapping function and alter the result.
We categorize the adversarial examples into Lipschitz vulnerable and float neuron vulnerable.
With a discussion of the ability of randomized classifiers in correcting misclassified data, we conclude that the essence of the smoothed classifier is to average the contribution of the float path and achieve a locally stable result.
Based on this, we further repress the float paths of the network and show that such a classifier can achieve better performance.
The theoretical basis of this work is developed from the analysis of the activation region that was initially proposed for explaining the performance of neural network with a piecewise linear activation function.
The input domain of such a neural network N is separated into many regions, within which the mapping of N is piecewise linear.
Previous investigation of this field includes the expressivity, sensitivity, and potential issues of the network.
However, due to the complexity of neural network, the theoretical investigation only provides insights into the neural network but has yet to be deployed downstream.
In this work, we use the theory to explain the model robustness and show introduce a novel way to apply the complex theory to practical.
The contributions of this work are: (1) we introduce a complete framework to describe and decompose the neural network according to the activation status of each neuron; (2) we provide an explanation of adversarial examples and discuss the role of smoothed classifiers as well as their contribution in correcting misclassified example; (3) we introduce SC-RFP that achieves better performance in certifying the network.
R ELATED W ORKS
The adversarial examples are maliciously inputs that are formed by applying an imperceptible perturbation to the original inputs but result in misclassification of a well-trained network (Biggio et al.
; Szegedy et al.
To explain the existence of adversarial example, previous works presented several hypotheses, such as linearity hypothesis (Szegedy et al.
; Luo et al. (2015)) and Evolutionary stalling (Rozsa et al.
Early works on increasing the robustness of neuron networks focused on adversarial training methods (Goodfellow et al.
; Wong et al.
Tram`er et al.
; Dong et al.
; Kurakin et al.
(2016)), while recent investigation shows adversarial training methods can always be broken by more advanced attacks.
To address the issue, certifiable training and randomized smoothing methods aim to provide a certified region, within which the input data are free from attack.
By viewing the training as a convex optimization problem, dual relaxation approaches apply duality to provide a sound bound for training as well as verify the network (Wong & Kolter (2018); Wong et al.
An alternative is to estimate the Lipschitz boundary of the network and introduce constraints on either objective loss (Tsuzuku et al.
(2018)) or forward propagation (Lee et al.
; Weng et al.
; Zhang et al.
; Huang et al.
As verifiable training methods often come with a compromise of performance, recent works focus on bridging the gap between adversarial and verifiable training to address the scalability and accuracy issue (Xiao et al. ; Balunovi´c & Vechev (2020); De Palma et al.
On the other hand, randomized smoothing introduces a smoothed classifier to the base classifier, therefore has a limited effect on the performance of standard models.
Cao & Gong (2017) first propose to ensemble the information around input data to smooth the prediction, but fails to provide a theoretical guarantee on the result.
Lecuyer et al.
(2019) certify the result of the smoothed classifier with differential privacy.
Cohen et al.
(2019) provides a theoretical analysis of the certifiable with
Monte Carlo, followed by Levine et al.
; Li et al.
Jeong & Shin (2020) introduces a regularized to improve the prediction constantly over noise, Jeong et al.
(2021) trains the model on a convex combination of samples and Salman et al.
(2019) employs PGD attack with randomized smoothing to further increases the robustness accuracy.
Another related topic is the explainability of neuron networks.
Lin et al.
Hornik et al.
and Park et al.
(2020) investigate how deep models are able to approximate an objective function.
An inspiring observation is that the network is a piecewise function when the activation function is piecewise linear Pascanu et al.
The number of linear regions is then adopted as a proxy of network complexity (Montufar et al.
Hanin & Rolnick (2019a;b)).
Novak et al.
(2018) studies the network sensitivity by countering the transition density of trajectory in the input space.
Jiang et al.
(2022) compares the similarity of activation patterns globally to study the limitation of deep neural network.
Inspired by the theoretical investigation, Jordan et al. (2019) introduces an algorithm named GeoCert that computes the l p bound of the network with a piecewise linear activation function. Zhang et al.
(2022) proposes an algorithm that systematically searches the adversarial example based on the activation space of ReLU network  P RELIMINARIES 3. N OTATIONS
Let N be a d block feedforward neural network for classification task with measure zero parameter set θ with respect to Lebesgue measure. Each of the block h i consists of a linear affine ϕ
i , an optional batch-normalization layer ψ i , and piecewise linear activation function σ i , while the last block h d omits the activation function.
Consider D as the distribution of a classification problem with c classes from R n 0 to Y = { 1 , 2 , . . . , c } , network N computes a function f : R n 0 → R c , where f is a composition of d blocks f = h n ◦ h d − 1 · · · ◦ h 1 . For every ( x, y ) ∼ D , the network computes a probability for each class f ( x ) ∈ R c and predicts the label of x as the class with the highest probability: ˆ y = arg max m ∈ Y f m ( x ) , where f m ( x ) is the m -th element of the network output vector.
We use x i ( x ; θ ) , y i ( x ; θ ) and z i ( x ; θ ) to denote the input, output, and pre-activation value of block i for data x .
We use a tuple pair ( i, j ) to denote the j -th neuron of layer i and a set I =
∪ d − 1 i =1 { ( i, j ) | j ∈ { 1 , . . . , n i }} to denote the index collection of neurons in intermediate layers, where n i is the output size of layer i .
3. R ANDOMIZED C ERTIFIABLE C LASSIFIER
The research on certifiable training aims to provide a guaranteed region for its input x , within which a classifier always provide stationary result.
To be specific, a classifier is regarded as robust for an input x for perturbation of size r if:
[Equation]
where B p ( x, r ) := { x ′ : ∥ x ′ − x ∥ p ≤ r } is the sphere with radius r measured by the metric induced by p -norm.
By taking the standard performance of the classifier into account, the robust accuracy of f with radius r is then defined as:
[Equation]
However, this robustness of certificated models are achieved with an impairment of expressivity and staleness.
As a concession for that, randomized algorithms are proposed to verify the network with a sound theoretical bound at the cost of slight additional computation other than model performance.
Let g be a randomized algorithm constructed based on classifier f .
Given a data ( x, y ) ∼ D , g employs a certain degree of randomness during the induction of f .
For instance, smoothed classifier g computes the probability of f ( x + ϵ ) belongs to class i given ϵ ∈ N (0 , σ 2 I ) Cohen et al.
[Equation]
With certain confidence level α , the lower bound of p A of random variable g y ( x ) and the upper bound p B of the probability of second possible class max m ∈ Y g y ( x ) can be computed.
This induces a certified radius of classifier g ( x ) : [Equation] For every x ′ ∈ B 2 ( x, r ) , if arg max m ∈ Y g m ( x ) = Y , then arg max m ∈ Y g m ( x ′ )
= y .
In other words, with confidence level α , every x ′ within the radius can be correctly classifier by the smoothed classifier. The smoothed classifier can also appears in other forms depends on the randomness and training algorithm (Levine et al. Salman et al. ; Jeong & Shin (2020)).
3. A CTIVATION P ATTERN AND C OMPUTATIONAL P ATH
Intuitively, the input space is partitioned into small linear regions by a neural network with piecewise linear activation function.
Each of the regions is initially referred to as an activation region.
Previous works mostly focused on the general properties of linear regions in investigating the expressivity and limitation of neural network.
As our objective is to study the unit-wise reaction towards perturbation, introducing additional notations to describe the neurons is necessary for our analysis.
Definition 1 (Generalized Activation Pattern / Region) .
Let N be a network defined as Section 3.1.
Denote Γ = { γ 1 , γ 2 , . . , γ q } as a set of breakpoints that separates the domain of activation function into q + 1 intervals U =
{ U 0 , U 1 , . . .
, U q } .
An activation pattern of N is defined as an indexed family that assigns each neuron a sign to represent the status of its pre-activation in U :
[Equation]
Given an activation pattern A , the activation region of A is defined as:
[Equation]
Conversely, given any x ∈ R in
[Equation]
we denote the activation pattern of neuron ( i, j ) at x as:
[Equation]
The activation pattern of x is the collection of the all the pattern of neurons: [Equation]
Activation Region Operator and Its Inverse.
The generalized activation pattern describes the activation status of each unit at the intermediate layers.
Given an activation pattern, R ( · ) finds the region, such that, for every x ∈ R ( A ; θ, σ, Γ) , the pre-activation value of each neuron z ij ( x ; θ ) locates within the interval U a ij that determined by its pattern a ij ∈ { 0 , 1 , . . . , q } .
On the basis of previous works (Hanin & Rolnick (2019b)Raghu et al. , we further introduce an inverse operator ˆ A ( x ; θ, σ, Γ) that computes the activation pattern of an input x ∈ R n 0 in Definition 1. This allows us to study the robustness of network by investigating the stability of activation pattern in the sphere B ( x, r ) . With quantitatively analysis of the pattern, we decompose the computational graph of single input x in Lemma 2 and the prediction difference between inputs x, x ′ ∼ D in Lemma 1.
When dependencies are fixed, we abbreviate Equation 5 and 7 as R ( A ) and ˆ A ( x ) .
From Single Region to Its Neighbor.
With the scale of network grows, the average volume of single activation region decreases exponentially (Hanin & Rolnick (2019a)).
Therefore, it is necessary to generalize our investigation from single region to its neighbor.
Equation 5 suggests that the essence of operator R ( · ) is to find all the x that satisfies a certain constraint determined by the activation pattern.
It can be expressed as the intersection of a sequence of subspaces:
[Equation]
Given a subset of indexes I c ⊂ I , releasing the constraints of neuron set I\I c then equals to remove related intersection.
This can be viewed as merging several close activation regions into one. From a geometric point of view, the input space is partitioned into small activation regions by a set of hyperplanes { H ijk ( θ ) } , where: H ijk ( θ ) : = { x ∈ U | z ij ( x ; θ ) = γ k } is determined by neuron ( i, j ) and breakpoint γ k .
This means that two adjacent activation regions R 1 , R 2 can be merged into one by removing the bent-hyperplanes that separate them.
Combining with Equation 8, the every data x in the merged region has same activation pattern expect for the neuron ( i, j ) based on which the removed hyperplane is defined on.
We use incomplete activation pattern A c to denote the stable neurons, and describe the merged region as:
Definition 2.
Let N be a network defined as Section 3.1.
Given an activation pattern A and a subset of the index set I c ∈
I , we denote A I c ⊂
A is an incomplete activation pattern of A :
A I c := { a ij | a ij ∈ A , ( i, j ) ∈ I C } .
The merged activation region of A I c is denoted as: R ( A c ) =
(cid:84) ( i,j
) ∈I c { x ∈ R in | z ij ( x ) ∈ U a ij } .
Definition 2 proposes a way to describe merged activation regions with incomplete activation patterns.
However, the merged regions are irregular spaces determined by bent-hyperplanes that defined by complex function.
Analyzing the properties of such space is not feasible. Therefore, we use following definition to describe the neurons in a subset R ⊂ R n 0 . Definition 3 (Float and Fixed Neuron) .
Let N be a network defined as Section 3.1. For any space R ⊂ R n 0 , if a neuron z has the same pattern for any x ∈ R , we refer it as a fixed neuron of R , otherwise it is a float neuron. We denote the collection of fixed neurons and float neurons as I X ( R ) and I T ( R ) , respectively:
[Equation] To build a connection between Definition 2 and 3, we propose the following lemma to show: (1) the fixed neurons and float neurons are complementary sets in I and (2) any subset R ∈ R n 0 can be covered by a merged region defined by the fixed neurons of R .
Lemma 1.
Let N be a neural network defined as Section 3.1. Given R ⊂ R n [-
-]0 , denote I X and I T are the set of float neuron and fixed neuron in R .
Then: [Equation] Now we are able to identify the stability of paths in the computational graph of network N with the following two Definitions:
Definition 4.
Let N be a network defined as Section 3.1.
A path of N is a set of neuron indexes:
[Equation]
where [ n i ] = { 1 , 2 , . . n i } is neuron index for layer i .
Definition 5 (Float Path and Fixed Path) . Let N be a neural network defined as Section 3.1. Given a subspace R ∈ R n 0 , a path ζ of N is a float path in R if there exist a neuron ( i, ζ i ) ∈ ζ is a float neuron, otherwise it is a fixed path. The collection of float path and float path are denoted as:
[Equation] To understand the motivation of above definition, consider the simple case that there is only 1 neuron ( i, j ) is float neuron in region R ⊂ R n 0 . The only non-linearity in R is provided by neuron ( i, j ) . The computational graph of N can be decomposed into float paths and fixed paths according to whether ( i, j ) it on the path.
This is formally described as the first statement in Theorem 1. Moreover, given x, x ′ ∈ R , the difference between f ( x ) and f ( x ′ ) can be viewed the combination of a linear function and an unstable function with high non-linearity as the second statement in Theorem 1 suggests. The remaining problem is how to decompose the computational graph of network N , which we discuss in the following section. D ECOMPOSING
THE C OMPUTATIONAL G RAPH
Now that we introduced the framework for describing neuron status, we aim to build a connection between activation pattern and mapping function f .
Lemma 2 (Decomposition of Neural Network) .
Let N be a neural network defined as Section 3.1.
Given activation pattern A , for any x ∈ R ( A ; θ, σ, Γ) .
The k -th component of output vector of layer i can be represented as:
[Equation]
where x j,ζ j is the ζ j -th element of input in layer j , W ′ ( i ) and β ( i ) are the equivalent matrix and bias of linear transformation ψ i ◦ ϕ
i , A m,ζ m is the activation pattern of ζ m -th component of layer m and d A m,ζm is the slope of pattern A m,ζ m : d A m,ζm :=
σ ′ ( t ) , t ∈ U
A m,ζm .
Above Lemma decomposes f in the manner of input and activation pattern. Equation 11 shows that y ij ( x ) can be decomposed into summation of paths, while the value of each path depends on either input x (former part) or bias β at each layer (latter part), as well as the pattern of neurons on the path.
We use ζ ( i,j ) ( v, A ) to denote the value of a path ζ given v as the input of that path:
[Equation]
In a single activation region R , ∀ x, x ′ ∈ R have same activation pattern A .
The paths from bias ζ ( i,j ) ( β ( j ) ζ j , A ) are constant, while that from input ζ ( i,j ) ( x j,ζ j , A ) are linear to x j,ζ j .
On the other hand, for x, x ′ with different activation patterns A , A ′ , each of the paths can be categorized as fixed or float.
Since neurons on fixed paths has same pattern at x and x ′ , the above linearity remains unchanged, which means ζ ( i,j ) ( v, A )
=
ζ ( i,j ) ( v, A ′ )
if ζ is a fixed path.
In other words, given region R ⊂ R n 0 , the float paths represent the linear part of f in R , while the fixed paths are the aggregation of all the non-linearity from the float neurons, as described bellow:
Theorem 1.
Let N be a neural network defined as 3.
Given R ⊂ X , for any x, x ′ ∈ R with activation pattern A and A ′ , we have :
[Equation]
where Z I ( x, A ; R ) = (cid:80) ζ ∈Z I ( R ) ζ ( x, A ) , Z T ( x, A ; R ) =
(cid:80) ζ ∈Z T ( R ) ζ ( x, A ) are the sum of fixed path and float path given the region R , J ( x ) is the Jacobian matrix of f at x .
The second statement of above theorem suggests that the difference between f ( x ) and f ( x ′ ) of network N can be decomposed into: (1) a linear part that depends on distance between x and x ′ as well as the slope at x , and (2) a non-linear part that resulted from instability of float neuron.
According to the prime contribution of prediction difference, we can categorize the unsecured data into Lipschitz vulnerable and float neuron vulnerable . In the following section, we discuss the effect of smoothed classifier on the two types of data and introduce the SC-RFP based on our discussion.
4. F LOAT P ATH AND N ETWORK R OBUSTNESS
Consider ( x, y ) ∼ D has extreme large Lipschitz constant in B ( x, r ) , then there exists x ′ ∈ B ( x, r ) such that f ( x ′ )
− f ( x ) ≫ f y ( x ) − f m ̸ = y ( x ) . We refer such an x as a Lipschitz vulnerable data.
For example, the Lipschitz constant of a large scale network without any defenses up to hundreds, which means small perturbation ϵ can alter the prediction of f ( x + ϵ ) . By applying the randomized smoothing on Theorem 1, we show that the smoothed classifier fails to correct such data.
Theorem 2.
Let f be the base classifier.
Given ( x, y ) ∼ D with f ( x ) =
y , denote M ( f ( x ) , y ) :
=
min
y ′ ̸
= y | f ( x ) y − f ( x ) ′
y
| as the margin operator of prediction vector.
If ∥ J ( x ) ∥ > M ( f ( x ) , y ) + E [ M ( Z T ( x, A ; R ) , y )] r , then for any smoothed classifier g defined as above, there exist x ′ ∈ B ( x, r ) such that g ( x ′ ) ̸
= y .
On the other hand, consider x ′ ∈ B ( x, r ) as an adversarial example that misclassified by f but corrected by g , then given ϵ ∼ D noise , g y ( x ′ + ϵ ) > arg max m ∈ Y g m ( x ′ + ϵ ) . This means that the adversarial example x ′ appears by change , while the majority of its neighbor are still voting for the correct label.
In other word, the instability of float path causes the misclassification, we refer such data as float neuron vulnerable .
[Figure]
The above discussion suggests that smoothed classifier fails to boost the performance of Lipschitz vulnerable example, but is able to correct float neuron vulnerable data by smoothing the sudden change of float path in a region.
Based on this, we introduce the SC-RFP in Algorithm 1.
Figure 1 illustrates the SC-RFP by showing the prediction, fixed path and float path.
We first present the prediction and the fixed path value in Figure 1(a).
The float path value is then computed as Z T ( x, A ; R ) =
f ( x ) − Z I ( x, A ; R ) in Figure 1(b).
At last, by repressing the float value, we achieve a locally stable prediction from SC-RFP.
At each block, we first compute the pre-activation value of x and x + ϵ .
Prior to passes to the activation function, we compare the activation pattern between z i ( x ) and z i ( x + ϵ ) according to the separation Γ , and use a repression factor η to reduce the value of float path.
Algorithm
Inputs: Network f with parameter θ , randomized algorithm g , input x , repress ratio η .
Outputs: Predicted label ˆ y while g sample noise ϵ do for Block i in Block 1 , 2 , . . .
, d − 1 do
I Ti ← A ( z i ( x )) ̸
=
A ( z i ( x + ϵ ))
z i ( x + ϵ ) ← z
i ( x + ϵ ) − z i ( x + ϵ ) ×
I Ti x i +
( x + ϵ )
=
σ ( z i ( x + ϵ )) counts ← ϕ d ( x d ( x + ϵ ))
ˆ c A , ˆ c B ← top two indices in counts ˆ n A , ˆ n B ← counts [ˆ c
A ] , counts [ˆ c B ] if BinomPV alue ( n A , n A + n B , 0 . 5) ≤ α then return ˆ c
A else return Abstain
Intuitively, manipulating the computational graph can greatly change the prediction result, while we show that there are only a small proportion of path are affected by our method.
Figure 2 present the proportion of fixed neuron between x and x + ϵ of VGG16 network trained on CIFAR10.
It shows that models trained with noised sample have relatively more stable activation pattern, as the ratio of fixed neurons are lower.
In particular, the model trained with clean data has average fixed ratio around 72%, while after adding a minimum scale of noise σ = 0 .
05 , it increases to around 90%.
[Equation]
At the end of this section, we present Theorem 3 to link the certifiable boundary of proposed algorithm with previous works.
Theorem 3.
Let N be a network defined as Section 3.1.
Let g be a smoothed classifier that samples noise from distribution D noise and g ′ is the SC-RFP built on g .
Assume that the direction of ϵ is uniformly distributed. [Equation] If arg max m ∈ Y f y ( x ) = y , then p
[Equation]
where p ′ A , p A are the lower bound of g ′ y ( x ) and g y ( x ) , p ′ B , p B are the upper bound of g ′ m
̸ = y ( x ) , g ′ m
̸ = y ( x ) . Moreover, arg max m ∈ Y g ′ ( x ) = y for all ∥ ϵ ∥ ≤ R , [Equation]
Notice that in the above theorem, we do not specify the randomized classifier g but introduce a constraint of the randomness of g .
This means that the theorem holds for not only the naive smoothed classifier, but also its variants.
However, it does not hold for the classifiers that sample directed noise.
The proof of Theorem 3 can be divided into three steps.
First, we show that given ϵ ∼ D noise , the expectation of fixed paths for the any E ( Z X ( x + ϵ ))
= Z X ( x ) . This means that the fixed part of g as well as g ′ equals to that of f ( x ) . Next, we discuss how the float paths affects the prediction f ( x + ϵ ) and prove that repressing the float path can increase the expectation of g ′ y ( x ) and its variance.
With
Chebyshev’s Inequality, we conclude the lower bound p ′
A is larger than that of p A , and similar to p ′ B .
Since both g and g ′ are random function that sample noise from same distribution, the certifiable radius holds for both g and g ′ .
E XPERIMENTS
[Figure]
As SC-RFP is built on a base smoothed classifier, we measure the method on different model structures and randomized smoothing algorithms.
The experiments are performed on CIFAR10 and ImageNet datasets.
For the CIFAR10 dataset, we compare our method with the benchmark classifier proposed by Cohen et al.
(2019) with VGG16 network to show the effectiveness of our method.
For the ImageNet dataset, we choose ResNet50 as model architecture and add the adversarial smoothed classifier Salman et al.
as the base model.
We also compare our model with recent work Jeong & Shin (2020); Jeong et al.
(2021) to obtain a general evaluation.
In line with previous works, we use certifiable accuracy at different radius computed by Cohen et al.
as the metric.
CIFAR10.
We train the VGG16 network with different scales of noisy samples.
Each of the models is trained for 200 epochs with SGD optimizer and an initial learning rate of 0.1, which decays after 60, 120, and 160 epochs with a rate of 0.2.
Table 1 compares the robust accuracy of SC-RFP and benchmark.
Generally, our SC-RFP increases have an increasingly robust accuracy between 6% to 10% compared with the benchmark.
We notice that by repressing the float path, SC-RFP is able to improve the accuracy of clean data, which supports our analysis that the float neuron vulnerability is one of the causes of misclassification.
Figure 3 present the robust accuracy at different noise scale.
We find that the performance boost from SC-RFP increases as the radius increases for all the models.
[Table]
Experiments on ImageNet.
We train ResNet50 using the benchmark model (Cohen et al.
and SmoothAdv (Salman et al.
on ImageNet-2012 for 80 epochs with same training sched- [Figure] uler and optimizers as the original work.
Each of the model are certified with same level of noise as they were trained.
Table 2 compares the certified robust accuracy of different models at different radius on ImageNet.
It shows that when deployed on the PGD+noise training (Salman et al.
, the SC-RFP has the highest robust accuracy at all the radius.
Moreover, an increasing boost in the robust accuracy is observed as the radius increase, which is consistent with our previous analysis of CIFAR10 result.
Figure 4 provides more details on our experiments.
For a noise level of σ = 0 .
25 , SmoothAdv and
SmoothAdv + SC-RFP has neglectable difference, while in 4(c), the SC-RFP ( σ = 0 .
25 ) increases the robust accuracy by around 5%.
By introducing SC-RFP ( σ = 0 . 1 ) to the benchmark classifier, it can achieve comparable results with other state-of-art models when l 2 radius is around 1.5. l 2 radius 0.
Clean
Jeong & Shin (2020) 59.
Jeong et al.
Cohen et al.
+ SC-RFP (ours) 59.
Salman et al.
[Table]
+ SC-RFP (ours)
[Table]
We find two insightful observations from the above results.
First, introducing the repression factor to the float path has a minimum negative effect on the standard accuracy, while can greatly boost the robust accuracy.
This supports our discussion that instability vulnerable examples are caused by the float path, and can be cured by reducing their contribution.
Second, we find that as the radius grows, the boost from our method increases.
This is consistent with the theoretical basis of SC-RFP as well as the result from CIFAR10 dataset.
When size of the perturbation grows, the uncertainty of the randomized algorithm increases along with the number of float paths as Figure 2 suggests.
This results in a higher abstain rate and a higher possibility of misclassification.
By repressing the value of the float path, the deviation of g ( x + ϵ ) from f ( x ) is reduced, therefore a more stable result can be achieved.
C ONCLUSION
In this work, we introduce SC-RFP algorithm that can improve the performance of a randomized smoothing classifier.
We first introduce a framework for describing the local activation status of neurons and show that most of the neurons are locally stable, while the others can greatly affect the model prediction.
By decomposing the computational graph of the network, we find that the boost of robust accuracy provided by the smoothed classifier is averaging the deviation of float paths.
Based on this, we suggest further repressing the value of float paths with SC-RFP method.
The experiments show that our method can improve the performance of smoothing models.