C URVATURE - BASED R OBUSTNESS C ERTIFICATES AGAINST A DVERSARIAL E XAMPLES
Anonymous authors
Paper under double-blind review
A BSTRACT
A robustness certiﬁcate against adversarial examples is the minimum distance of a given input to the decision boundary of the classiﬁer (or its lower bound).
For any perturbation of the input with a magnitude smaller than the certiﬁcate value, the classiﬁcation output will provably remain unchanged.
Computing exact robustness certiﬁcates for deep classiﬁers is difﬁcult in general since it requires solving a nonconvex optimization.
In this paper, we provide computationally-efﬁcient robustness certiﬁcates for deep classiﬁers with differentiable activation functions in two steps.
First, we show that if the eigenvalues of the Hessian of the network (curvatures of the network) are bounded, we can compute a robustness certiﬁcate in the lnorm efﬁciently using convex optimization.
Second, we derive a computationallyefﬁcient differentiable upper bound on the curvature of a deep network.
We also use the curvature bound as a regularization term during the training of the network to boost its certiﬁed robustness against adversarial examples.
Putting these results together leads to our proposed C urvature-based R obustness C ertiﬁcate (CRC) and
C urvature-based R obust T raining (CRT).
Our numerical results show that CRC outperforms CROWN’s certiﬁcate when trained with our regularizer while CRT leads to higher certiﬁed accuracy compared to standard adversarial training.
I NTRODUCTION
Modern neural networks achieve high accuracy on tasks such as image classiﬁcation and speech recognition, but are known to be brittle to small, adversarially chosen perturbations of their inputs (Szegedy et al., 2014).
A classiﬁer which correctly classiﬁes an image x , can be fooled by an adversary to misclassify an adversarial example x + δ , such that x + δ is indistinguishable from x to a human.
Adversarial examples can also fool systems when they are printed out on a paper and photographed with a smart phone (Kurakin et al., 2016a).
Even in a black box threat model, where the adversary has no access to the model parameters, attackers could target autonomous vehicles by using stickers or paint to create an adversarial stop sign that the vehicle would interpret as a yield or another sign (Papernot et al., 2016).
This trend is worrisome and suggests that these vulnerabilities need to be appropriately addressed before neural networks can be deployed in security critical applications.
In the last couple of years, several empirical defenses have been proposed for training classiﬁers to be robust against adversarial perturbations (Madry et al., 2018;
Samangouei et al., 2018; Zhang et al., 2019; Papernot et al., 2016; Kurakin et al., 2016b;
Miyato et al., 2017; Zheng et al., 2016)
Although these defenses robustify classiﬁers to particular types of attacks, they can be still vulnerable against stronger attacks (Athalye et al., 2018; Carlini & Wagner, 2017; Uesato et al., 2018; Athalye & Carlini, 2018).
For example, (Athalye et al., 2018) showed most of the empirical defenses proposed in ICLR 2018 can be broken by developing tailored attacks for each of them.
To end the cycle between defenses and attacks, a line of work on certiﬁed defenses has gained attention where the goal is to train classiﬁers whose predictions are provably robust within some given region (Huang et al., 2016; Katz et al., 2017; Ehlers, 2017; Carlini et al., 2017; Cheng et al., 2017;
Lomuscio & Maganti, 2017; Dutta et al., 2018; Fischetti & Jo, 2018; Bunel et al., 2017; Wang et al., 2018a; Wong & Kolter, 2017; Wang et al., 2018b; Wong et al., 2018; Raghunathan et al., 2018b;a;
Dvijotham et al., 2018a;b; Croce et al., 2018; Singh et al., 2018; Gowal et al., 2018; Gehr et al., 2018;
Mirman et al., 2018; Zhang et al., 2018b; Weng et al., 2018).
These methods, however, do not scale to large and practical networks used in solving modern machine learning problems.
Another line of defense work focuses on randomized smoothing where the prediction is robust within some region around the input with a user-chosen probability (Liu et al., 2017; Cao & Gong, 2017; L´ecuyer et al., 2018;
Li et al., 2018; Cohen et al., 2019;
Salman et al., 2019).
Although these methods can scale to large networks, certifying robustness with probability close to 1 often requires generating a large number of noisy samples around the input which leads to high test-time computational complexity.
If the classiﬁer f ( . ) was linear, the distance of an input point x to its decision boundary (i.e. the robustness certiﬁcate) can be computed efﬁciently using a convex optimization.
For example, the lrobustness certiﬁcate in that case would be equal to ∣ f ( x )∣/∥∇ x f ( x )∥ .
However, modern classiﬁers based on neural networks are not linear and and can have non-zero curvatures in different parts of the input domain.
The deviation of the classiﬁer from the linear model makes the robustness certiﬁcation problem to be a non-convex optimization which is often difﬁcult to solve exactly.
However, if we could compute global bounds on the maximum curvature values of the classiﬁcation network, one may be able to compute computationally-efﬁcient lower bounds on the robustness certiﬁcate even for non-linear deep classiﬁers.
This is the key intuition of our results in this paper.
In this work, we derive a global bound on the Lipschitz constant of the gradient of deep neural networks with differentiable activation functions (such as sigmoid, tanh, softplus, etc.).
This provides an upper bound on the magnitude of the eigenvalues of the Hessian or the curvature values of the classiﬁcation network.
Using this global curvature bound and for the l 2 metric, we tackle both the certiﬁcation and attack problems.
In the certiﬁcation problem and for a given pre-trained classiﬁer, we provide a computationally-efﬁcient lower bound on the distance of a point to the classiﬁcation decision boundary.
In the related attack problem , for a given input and a region around it, our goal is to ﬁnd a perturbed input (an adversarial example) that maximizes the loss inside the given region.
The outcome of the attack problem is then used in the adversarial training procedure (Madry et al., 2018) to further robustify the network.
Furthermore, our global curvature bound is differentiable and we show that adding it to the loss function as a regularizer boosts certiﬁed robustness measures.
We note that other recent works (e.g. Moosavi Dezfooli et al.
; Qin et al.
(2019)) empirically show that using an estimate of curvature at inputs as a regularizer leads to empirical robustness on par with the adversarial training.
In this work, however, we use a provable global upper bound on the curvature (and not an estimate) as a regularizer and show that it results in high certiﬁed robustness.
Moreover, previous works have tried to certify robustness by bounding the Lipschitz constant of the neural network (Szegedy et al., 2014;
Peck et al., 2017; Zhang et al., 2018c; Anil et al., 2018; Hein &
Andriushchenko, 2017).
Our approach, however, is based on bounding the Lipschitz constant of the gradient of deep neural networks.
We discuss existing works in more details in Appendix A.
Below, we state the key theoretical results of this paper informally while detailed statements of these results are presented in Section 4.
Theorem (informal) 1.
Let z ( L ) i denotes the i th logit of an L layer fully-connected neural network with differentiable activation functions.
Then, the curvature of the neural network function is globally bounded as follows:
[Equation]
where m and M can be computed efﬁciently using parameters of the network.
This result along with the min-max theorem leads to the following curvature robustness certiﬁcate:
Theorem (informal) 2. Consider a network whose curvature values are bounded.
For a given input x ( 0 ) with the true label y and the attack target t ( t ≠ y ) , let p ∗ cert denote the exact robustness certiﬁcate, i.e. the distance of x ( 0 ) to the decision boundary.
We can efﬁciently compute d ∗ cert such that d ∗ cert ≥ p ∗ cert .
Moreover, if the solution x ( cert ) for d ∗ cert satisﬁes z ( L ) y =
z ( L ) t , then d ∗ cert = p ∗ cert .
We have similar results for the attack problem.
For simplicity, we summarize deﬁnitions of p ∗ cert ,d ∗ cert ,p ∗ attack ,d ∗ attack in Table 1.
In summary, in this paper, we make the following contributions:
• We provide global bounds on the eigenvalues of the Hessian of a deep neural network with differentiable activation functions (Theorem 3 and Theorem 4).
In addition to the adversarial robustness problem, these bounds may be of an independent interest for readers.
[Table]
problem [Equation] [Table] Attack problem
[Equation]
primal problem
[Equation]
dual function
[Equation]
When is dual solvable?
[Equation]
dual problem, d
[Equation]
−)
[Equation]
When primal = dual?
[Equation]
[Table]
[Table]
Katz et al.
[Table]
) (cid:51) (cid:51) [Table] Hein & Andriushchenko (2017) (cid:51)
[Table]
Raghunathan et al. (2018a) (cid:51) (cid:55)
[Table] Wong & Kolter (2017) (cid:51) (cid:51) [Table] Weng et al.
(2018) (cid:51) (cid:51) [Table] l 1 ,l 2 ,l ∞ Zhang et al.
(2018b) [Table]
• Using the global curvature bounds, we develop computationally efﬁcient methods for both the robustness certiﬁcation as well as the adversarial attack problems (Theorems 1 and 2).
• We show that using our proposed curvature bounds as a regularizer during training leads to improved certiﬁed accuracy on 2,3 and 4 layer networks (on the MNIST dataset) compared to standard adversarial training with PGD (Madry et al., 2018) as well as TRADES (Zhang et al., 2019).
Moreover, our robustness certiﬁcate (CRC) outperforms CROWN’s certiﬁcate (Zhang et al., 2018b) signiﬁcantly while taking less time to compute.
N OTATION AND P ROBLEM S ETUP
Consider a fully connected neural network with L layers and N I neurons in the I th layer (
L ≥and I ∈
[ L ] ) for a multi-label classiﬁcation problem with C classes ( N L = C ).
The corresponding function of the neural network is z ( L ) ∶ R D
→ R C where D is the dimension of the input.
For an input x , we use z ( I ) ( x ) ∈ R N I and a ( I ) ( x ) ∈ R N I to denote the input ( before applying the activation function) and output ( after applying the activation function) of neurons in the I th hidden layer of the network, respectively.
To simplify notation and when no confusion arises, we make the dependency of z ( I ) and a ( I ) to x implicit.
We deﬁne a ( 0 ) ( x ) = x and N 0 = D .
With a fully connected architecture, each z ( I ) and a ( I ) is computed using a transformation matrix
W ( I ) ∈
R N I × N I − 1 , the bias vector b ( I ) ∈
R N I and an activation function σ ( . ) as follows:
[Equation]
We use ( z ( L ) i − z
( L ) j )( x ) as a shorthand for z ( L ) i ( x ) −
z ( L ) j
( x ) .
We use [ p ] to denote the set { 1 ,...,p } and [ p,q ] , p ≤ q to denote the set { p,p + 1 ,...,q } .
We use small letters i,j,k etc to denote the index over a vector or rows of a matrix and capital letters I,J to denote the index over layers of network.
The element in the i th position of a vector v is given by v i , the vector in the i th row of a matrix
A is A i while the element in the i th row and j th column of
A is A i,j .
We use ∥ v ∥ and ∥ A ∥ to denote the 2-norm and the operator 2-norm of the vector v and the matrix A , respectively.
We use ∣ v ∣ and ∣ A ∣ to denote the vector and matrix constructed by taking the elementwise absolute values.
We use λ max ( A ) and λ min ( A ) to denote the largest and smallest eigenvalues of a symmetric matrix A .
We use diag ( v ) to denote the diagonal matrix constructed by placing each element of v along the diagonal.
We use ⊙ to denote the Hadamard Product, I to denote the identity matrix.
We use ≼ and ≽ to denote Linear Matrix Inequalities (LMIs) such that given two symmetric matrices A and B where A ≽ B means A − B Positive Semi-Deﬁnite (PSD).
U SING DUALITY TO SOLVE THE ATTACK AND CERTIFICATE PROBLEMS
Consider an input x ( 0 ) with true label y and the attack target t .
In the certiﬁcate problem, our goal is to ﬁnd a lower bound of the minimum l 2 distance between x ( 0 ) and the decision boundary, z ( L ) y
=
z ( L ) t .
The problem for solving the exact distance ( primal ) can be written as:
[Equation]
However, solving the above problem can be hard in general.
Using the minimax theorem (primal ≥ dual), we can write the dual of the above problem as follows:
[Equation]
From the theory of duality, we know that d cert ( η ) for each value of η gives a lower bound on the exact certiﬁcation value (the primal solution) p ∗ cert .
However, since z ( L ) y − z ( L ) t is non-convex, solving d cert ( η ) for every η can be difﬁcult.
In the next section, we will prove that the curvature of the function z ( L ) y
[Equation]
is bounded globally
[Equation]
In this case, we have the following theorem:
Theorem 1.
d cert ( η ) is a convex optimization problem for − 1 / M ≤ η ≤ − 1 / m .
Moreover, If x ( cert ) is the solution to d ∗ cert such that z ( L ) y
( x ( cert ) )
=
z ( L ) t ( x ( cert ) ) , then p ∗ cert = d ∗ cert .
Below, we brieﬂy outline the proof while the full proof is presented in Appendix D.1.
The Hessian of the objective function of the dual d cert ( η ) , i.e the function inside the min x is given by:
[Equation]
From equation (3), we know that the eigenvalues of I + η ∇ 2 x ( z ( L ) y − z
( L ) t ) are bounded between ( 1 + ηm, 1 + ηM ) if η ≥ 0 , and in ( 1 + ηM, 1 + ηm ) if η ≤ 0 .
In both cases, we can see that for − 1 / M ≤ η ≤ − 1 / m , all eigenvalues will be non-negative, making the objective function convex.
When x ( cert ) satisﬁes z ( L ) y
=
z ( L ) t , d ∗ cert = 1 / 2 ∥ x ( cert ) − x ( 0 ) ∥ 2 , using the duality theorem and deﬁnition of p ∗ cert , we get p ∗ cert = d ∗ cert .
Next, we consider the attack problem.
The goal here is to ﬁnd an adversarial example inside an l 2 ball of radius ρ such that z ( L )
y − z ( L ) t is minimized.
Using similar arguments, we can get the following theorem for the attack problem ( p ∗ attack , d ∗ attack and d attack are deﬁned in Table 1):
Theorem 2.
d attack ( η ) is a convex optimization problem for − m ≤ η .
Moreover, if x ( attack ) is the solution to d ∗ attack such that ∥ x ( attack ) − x ( 0 ) ∥
= ρ , p ∗ attack =
d ∗ attack .
The proof is presented in Appendix D.2.
Both Theorems 1, 2 hold for any non-convex function with continuous gradients.
They can also be of interest in problems such as optimization of neural nets.
Using Theorems 1 and 2, we have the following deﬁnitions for certiﬁcation and attack optimizations:
Deﬁnition 1.
( Curvature-based Certiﬁcate Optimization )
Given an input x ( 0 ) with true label y , the false target t , we deﬁne ( η ( cert ) , x ( cert ) ) as the solution of the following max-min optimization:
[Equation]
We refer to ∥ x ( cert ) − x ( 0 ) ∥ as the C urvature-based R obustness C ertiﬁcate (CRC).
Deﬁnition 2.
( Curvature-based Attack Optimization )
Given input x ( 0 ) with label y , false target t , and the l 2 ball radius ρ , we deﬁne ( η ( attack ) , x ( attack ) ) as the solution of the following optimization:
[Equation]
When x ( attack ) is used for training in an adversarial training framework, we call the method the
C urvature-based R obust T raining (CRT).
Since both curvature-based certiﬁcate and attack optimizations are convex optimization problems, any convex optimization solver can be used to solve them.
In our implementation, we use majorizationminimization to solve the dual function for a given η and bisection method to maximize over η .
More details are given in Appendix C.4 and C.5.
C URVATURE B OUNDS FOR DEEP NETWORKS
In this section, we provide a computationally efﬁcient approach to compute the curvature bounds for neural networks with differentiable activation functions.
To the best of our knowledge, there is no prior work on ﬁnding provable bounds on the curvature values of deep neural networks.
Our results rely on a closed form expression for the Hessian of the i th logit as a sum of matrix products (Section 4.1).
After establishing this result, we ﬁrst derive curvature bounds for a two-layer network in Section 4.2 and then extend the bounds to deeper networks in Section 4.3.
4. C LOSED FORM EXPRESSION FOR THE H ESSIAN
Using the chain rule of second derivatives, we can derive ∇ 2 x z ( L ) i as a sum of matrix products:
Lemma 1.
Given an L layer neural network, the Hessian of the i th hidden unit with respect to the input x , i.e ∇ 2 x z ( L ) i is given by the following formula:
[Equation]
where B ( I ) is the Jacobian of z ( I ) with respect to x (dimensions N I × D ), and F ( L,I ) is the Jacobian of z ( L ) with respect to a ( I ) (dimensions N L × N I ).
The proof is presented in Appendix D.3.
Using the chain rule of gradient, we can compute B ( I ) ,
F ( L,I ) matrices in Lemma 1 recursively as follows:
[Equation]
This leads to a fast back-propagation like method that can be used to compute the Hessian.
Note that
Lemma 1 only assumes a matrix multiplication operation from a ( I − 1 ) to z ( I ) .
Since a convolution operation can also be expressed as a matrix multiplication, we can directly extend this lemma to deep convolutional networks.
Furthermore, Lemma 1 can also be of independent interest in other related problems such as higher-order interpretation methods for deep learning (e.g. Singla et al.
4. C URVATURE
For a two-layer network and using Lemma 1, ∇ 2 x ( z ( 2 ) y − z ( 2 ) t ) is given by:
[Equation]
Note that only σ ′′ ( z ( 1 ) ) depends on x .
We can maximize and minimize each element in the diag term, ( W ( 2 ) y,i − W ( 2 ) t,i ) σ ′′ ( z ( 1 ) i ) independently subject to the constraint that σ ′′ ( . ) is bounded.
Using this procedure, we construct matrices P and N that satisfy properties given in the following theorem:
Theorem 3.
Given a two layer network whose activation function has bounded second derivative:
[Equation]
(a) We have the following linear matrix inequalities (LMIs):
[Equation]
(b) If h U ≥ 0 and h L ≤ 0 , P is a PSD matrix, N is a NSD matrix.
(c) This gives the following global bounds on the eigenvalues of the Hessian:
[Equation]
where M = λ max ( P ) , m = λ min ( N )
P and N are independent of x and deﬁned in equations (55) and (56) in Appendix D.4.
The proof is presented in Appendix D.4.
Because power iteration ﬁnds the eigenvalue with largest magnitude, we can use it to ﬁnd m and M only when P is PSD and N is NSD.
We solve for h U and h L for sigmoid, tanh, softplus activation functions in Appendix E and show that this is in fact the case for them.
Note that this result does not hold for ReLU networks since the ReLU function is not differentiable.
However, in Appendix F , we devise a method to compute the certiﬁcate for a two layer ReLU network by ﬁnding a quadratic lower bound for z ( 2 ) y − z ( 2 ) t [Equation]
4. C URVATURE BOUNDS FOR D EEP NETWORKS
Using Lemma 1, we know that ∇ 2 x z ( L ) i is a sum product of matrices B ( I ) and F ( L,I ) i .
Thus, if we can ﬁnd upper bounds for ∥ B ( I ) ∥ and
∥ F ( L,I ) i ∥
, we can get upper bounds for ∥∇ 2 x z ( L ) i ∥ .
Using this intuition (details are presented in Appendix D.5), we have the following result:
Theorem 4.
Given an L layer neural network whose activation function satiﬁes:
[Equation]
the absolute value of eigenvalues of ∇ 2 x z ( L ) i is globally bounded by the following quantity:
[Equation]
where r ( I ) and S ( L,I ) are independent of x and deﬁned recursively as:
[Equation]
The above expressions allows for an efﬁcient computation of the curvature bounds for deep networks.
We consider simpliﬁcation of this result for sigmoid, tanh, softplus activations in Appendix E.
Note that bounds for z ( L )
y − z
( L ) t can be computed by replacing W ( L ) i with W ( L ) y − W ( L ) t in Theorem 4.
The resulting bound is independent of x , and only depends on network weights W ( I ) , the true label y , and the target t .
We denote it with K ( W ,y,t ) .
To simplify notation, when no confusion arises we denote it with K .
In our experiments, for two layer networks, we use M , m from Theorem 3 (since it provides tighter curvature bounds).
For deeper networks ( L ≥ 3 ), we use M = K, m = − K .
A DVERSARIAL TRAINING WITH CURVATURE REGULARIZATION
Using Theorem 2 (b), we know that if we solve the curvature-based attack optimization and obtain
ρ = ∥ x ( attack ) − x ( 0 ) ∥ , x ( attack ) is provably the closest adversarial example to x ( 0 ) .
However, when
When γ =
(no curvature regularization), networks adversarially trained with CRT or
PGD both have high curvatures.
However,
CRT even with a small γ leads to a signiﬁcant decrease in curvature bounds (note the log -scale of y -axis).
Similar results hold for networks with Tanh activations (Appendix
Figure 2) we performed adversarial training (with ρ = 0 . 5 ), we found that the curvature bound is loose and almost none of training inputs lead to zero primal-dual gap with ρ = ∥ x ( attack ) − x ( 0 ) ∥ . To ﬁx this issue, we use a regularizer that penalizes the curvature bound, K . Using equations (6) and (7), we can compute K using absolute value, matrix multiplications, and operator norm (∥ W ( I ) ∥ ,I ∈ [ L ]) .
Since the gradient of operator norm does not exist in standard libraries, we created a new layer where the gradient of ∥ W ( I ) ∥ , i.e ∇ W ( I )
∥ W ( I ) ∥ is given by:
[Equation] This approach to compute the gradient of the largest singular value of a matrix has also been used in previous ICLR work (Miyato et al., 2018).
Implementation details are in Appendix C.1.
Thus, the per-sample loss for training with curvature regularization is:
[Equation]
where y is the true label of the input x ( 0 ) , t is the target label and γ is the regularizer for penalizing large curvatures.
Similar to the adversarial training, in CRT, we use x ( attack ) instead of x ( 0 ) .
E XPERIMENTS
The empirical robust accuracy means the fraction of test samples that were correctly classiﬁed after running an l 2 bounded PGD attack (Madry et al., 2018), the certiﬁed robust accuracy means the fraction of correctly classiﬁed test samples whose robustness certiﬁcates are greater than a prespeciﬁed radius ρ .
Unless otherwise speciﬁed, we use the class with the second largest logit as the attack target (i.e. the class t ) and ρ = 0 . 5 .
All experiments were run on the MNIST dataset.
The notation ( L × [ 1024 ] , activation) denotes a neural network with L layers with the speciﬁed activation, ( γ = c ) denotes standard training with γ set to c , while (CRT, c ) denotes CRT training with γ = c .
Certiﬁcates are computed over 150 randomly chosen correctly classiﬁed images.
Comparison with existing certiﬁcates : In Table 3, we compare CRC with CROWN-general (Zhang et al., 2018a).
For 2-layer networks, CRC outperforms CROWN signiﬁcantly.
For deeper networks,
CRC works better only when the network is trained with curvature regularization.
However, even with small γ = 0 . 005 , we see a signiﬁcant increase in CRC but a very small drop in the test accuracy (without any adversarial training).
We can see that with γ = 0 .
01 , non-trivial certiﬁed accuracies of 83 .
61% can be achieved on 2 , 3 , 4 layer sigmoid networks, respectively, without any adversarial training.
Adversarial training using CRT further boosts certiﬁed accuracy to 95 .
99% and 93 .
41% , respectively.
We observe similar results with Tanh networks in Appendix Section G.2.
We show some results on CIFAR-10 dataset in Table 6.
We again observe improvements in the robustness certiﬁcate and certiﬁed robust accuracy using CRC and CRT.
In Figure 1, we plot the effect of γ on the curvature upper bound K ub and a lower bound K lb of a 4-layer network with Sigmoid activations.
K lb is computed by taking the maximum of the largest eigenvalue of the Hessian across all test images with label y and the second largest logit t , then averaging across different ( y,t ).
Similarly, K ub is the mean of K over all pairs ( y,t ) (details in
Appendix G.5).
We observe that without any curvature regularization (when γ = 0 ), both standard adversarial training with PGD as well as the CRT lead to networks with high curvatures.
However,
CRT with even a small γ leads to a signiﬁcant decrease in curvature bounds.
Similar trends can be observed for networks with Tanh activations (Appendix Figure 2).
Curvature bounds are higher for the Tanh networks compared to the Sigmoid ones due to having larger g and h parameters for Tanh in
Theorem 4.
Moreover, we report curvature bounds for networks with different depth in Appendix
Table 12.
We observe that increasing depth increases curvature bounds.
Comparison with existing adversarial training methods : We compare CRT with adversarial training methods namely PGD (Madry et al., 2018) and TRADES (Zhang et al., 2019) in Table 4.
We observe that none of the other methods give higher certiﬁed accuracy or robustness certiﬁcates than our proposed methods.
We observe similar results with Tanh networks (Appendix Table 9).
Moreover, in Appendix Table 10, we observe that CRT outperforms Randomized Smoothing (Cohen et al., 2019) for 2 and 3 layer networks.
Since TRADES and Randomized Smoothing were designed for untargeted attacks while CRT is for targeted attacks, to have a fair comparison, we modify the multi-class version of the cross entropy loss with its binary version (details in Appendix Section G.3).
However, we emphasize that there is a fundamental difference between our certiﬁcate (and CROWN) with the smoothing based method since our certiﬁcate is deterministic while the smoothing-based certiﬁcate is probabilistic (with high probability).
We also compare against robust training technique proposed in Wong et al.
(2018) in Table 5 with lradius ρ = 1 . We note that the activation functions used in this comparison are different because the method proposed in Wong et al. (2018) uses ReLU while our proposed CRT can only work with fully differentiable activation functions such as softplus, sigmoid, tanh. For CRT, we use a network with softplus activation function with curvature regularization.
We observe that for two and three layer networks, CRT gives higher certiﬁed robust accuracy compared to that of Wong et al.
(2018) while the difference between certiﬁed accuracy of the two approaches for a 4 layer network is small (around 0 .
We emphasize that to get good certiﬁed accuracy using our method, we need to add curvature regularization during the training.
Otherwise, the curvature bounds and therefore the robustness certiﬁcates are loose.
[Table]
[Table]
E XTENSIONS AND D ISCUSSION 7. U SING LOCAL INSTEAD OF GLOBAL CURVATURE BOUNDS
From Theorems 1 and
[Table]
we can observe that if the curvature is locally bounded within a convex region around the input
[Table]
then the corresponding dual problems ( d ∗ cert , d ∗ attack ) are convex optimization problems inside this region.
This leads to the following result:
[Table]
[Table]
[Table]
Corollary 1.
Both Curvature-based Certiﬁcate and Attack Optimizations are convex optimization problems within an l 2 ball of radius ρ around the input sample where curvature values are bounded.
To use local curvature bounds, we need to ensure that the gradient descent trajectory does not escape the local region where curvature values are bounded.
To do this, we reduce the step size by a factor of two until the trajectory lies inside the l 2 ball of radius ρ around the input point.
It is straightforward to see that local bounds on curvature values can be tighter than global ones which can lead to better robustness certiﬁcations.
In Table 7, we show signiﬁcant improvements for the CRC certiﬁcate and some improvements on the certiﬁed accuracy for two-layer sigmoid and tanh networks on the MNIST dataset.
In this case, to compute local curvature bounds, we need to have a bound on h L and h U (Theorem 3) within the ball of radius ρ around the input sample.
These bounds can be signiﬁcantly tighter than the global bounds.
For example, if a sigmoid neuron’s output is bounded by -0.4 and 0.6 (those bounds can be obtained efﬁciently using CROWN), the second derivative of sigmoid is bounded between -0.048 and +0.048, much better than the worst case bound -0.09623 and +0.09623 used in current global curvature bound.
[Table]
For deeper networks, however, computing local curvature bounds is more challenging.
We leave exploring this direction for the future work.
7. E XTENSION TO CONVOLUTIONAL NEURAL NETWORKS
The formula derived in Lemma 1 is valid even for convolutional neural networks.
However, to use our methods, we need to bound the singular values of the Jacobian of the convolution operation.
In order to do this, one can use spectral bounds for convolution layers derived in Sedghi et al.
Furthermore, other heuristic techniques such as the one proposed in Miyato et al.
(2018) can be used for the curvature regularization as well.
We present some prelimiary results using these techniques for 2 layer convolutional networks on MNIST and CIFAR-10 in Appendix Section G.1.
C ONCLUSION
In this paper, we develop computationally-efﬁcient convex relaxations for robustness certiﬁcation and adversarial attack problems given the classiﬁer has a bounded curvature.
We also show that this convex relaxation is tight under some general conditions.
To be able to use proposed certiﬁcation and attack convex optimizations, we derive global curvature bounds for deep networks with differentiable activation functions.
This result is a consequence of a closed-form expression that we derived for the Hessian of a deep network.
Our empirical results indicate that our proposed curvature-based robustness certiﬁcate outperforms the CROWN certiﬁcate by an order of magnitude while being faster to compute as well.
Furthermore, adversarial training using our attack method coupled with curvature regularization results in a signiﬁcantly higher certiﬁed robust accuracy than the existing adversarial training methods.
Scaling up our proposed curvature-based robustness certiﬁcation and training methods as well as further tightening the derived curvature bounds are among interesting directions for the future work.
We thank the reviewers for their feedback and comments specially on extensions of our results to characterize local curvature bounds.
Appendix A R ELATED WORK
Many defenses have been proposed to make neural networks robust against adversarial examples.
These methods can be classiﬁed into empirical defenses which empirically seem to be robust against known adversarial attacks, and certiﬁed defenses, which are provably robust against such attacks.
Empirical defenses The best known empirical defense is adversarial training (Kurakin et al., 2016b;
Madry et al., 2018; Zhang et al., 2019).
In this method, a neural network is trained to minimize the worst-case loss over a region around the input.
Although such defenses seem to work on existing attacks, there is no guarantee that a more powerful attack would not break them.
In fact, most such defenses proposed in the literature were later broken by stronger attacks (Athalye et al., 2018; Carlini & Wagner, 2017; Uesato et al., 2018; Athalye & Carlini, 2018).
To end this arms race between defenses and attacks, a number of works have tried to focus on certiﬁed defenses that have formal robustness guarantees.
Certiﬁed defenses A classiﬁer is said to be certiﬁably robust if one can easily obtain a guarantee that a classiﬁer’s prediction remains constant within some region around the input.
Such defenses typically rely on certiﬁcation methods which are either exact or conservative.
Exact methods report whether or not there exists a adversarial perturbation inside some l p norm ball.
In contrast, conservative methods either certify that no adversarial perturbation exists or decline to make a certiﬁcation; they may decline even when no such perturbation exists.
Exact methods are usually based on Satisﬁability
Modulo Theories (Huang et al., 2016; Katz et al., 2017; Ehlers, 2017; Carlini et al., 2017) and Mixed
Integer linear programming (Cheng et al., 2017; Lomuscio & Maganti, 2017; Dutta et al., 2018;
Fischetti & Jo, 2018; Bunel et al., 2017).
Unfortunately, they are computationally inefﬁcient and difﬁcult to scale up to even moderately sized neural networks.
In contrast, conservative methods are more scalable and efﬁcient which makes them useful for building certiﬁed defenses (Wang et al., 2018a; Wong & Kolter, 2017; Wang et al., 2018b; Wong et al., 2018; Raghunathan et al., 2018b;a;
Dvijotham et al., 2018a;b; Croce et al., 2018; Singh et al., 2018; Gowal et al., 2018; Gehr et al., 2018;
Mirman et al., 2018; Zhang et al., 2018b; Weng et al., 2018).
However, even these methods have not been shown to scale to practical networks that are large and expressive enough to perform well on
ImageNet, for example.
To scale to such large networks, randomized smoothing has been proposed as a probabilistically certiﬁed defense.
Randomized smoothing Randomized smoothing was previously proposed by several works (Liu et al., 2017; Cao & Gong, 2017) as a empirical defense without any formal guarantees.
L ´ ecuyer et al.
ﬁrst proved robustness guarantees for randomized smoothing classiﬁer using inequalities from differential privacy.
Li et al.
(2018) improved upon the same using tools from information theory.
Recently, Cohen et al. (2019) provided a even tighter robustness guarantee for randomized smoothing.
Salman et al.
(2019) proposed a method of adversarial training for the randomized smoothing classiﬁer giving state of the art results in the l 2 norm metric.
B T HE A TTACK PROBLEM
For a given input x ( 0 ) with true label y and attack target t , consider the attack problem.
We are given that the eigenvalues of the Hessian ∇ 2 x ( z ( L ) y − z
( L ) t ) are bounded below i.e:
[Equation]
Here m < 0 (since z ( L ) y − z ( L ) t is not convex in general).
The goal here is to ﬁnd an adversarial example inside a l 2 ball of radius ρ such that ( z ( L ) y − z
[Equation]
L ) t )
[Equation]
) is minimized.
That is, we want to solve the following optimization:
[Equation]
This optimization can be hard in general.
Using the max-min inequality (primal ≥ dual), we have:
[Equation]
We know that for every η ≥ 0 , d attack ( η ) gives a lower bound to the primal solution p ∗ attack .
But solving d attack ( η ) for any η ≥ 0 can be hard unless the objective is convex.
We prove that if the eigenvalues of the Hessian are bounded below i.e:
[Equation]
In general m < 0 , since ( z ( L ) y − z ( L ) t ) is non-convex.
d attack ( η ) is a convex optimization problem for − m ≤ η .
Equivalently the objective function, i.e the function inside the min x :
[Equation]
is a convex function in x for − m ≤ η
The Hessian of the above function is given by:
[Equation]
Since we know that eigenvalues of ∇ 2 x ( z ( L ) y −
[Equation]
≽ m I , we know that eigenvalues of the above
Hessian are ≥ η + m .
For η ≥ − m , the eigenvalues are positive implying that the objective function is convex.
Since d attack ( η ) gives a lower bound to p ∗ attack for every η ≥ 0 , we get the following result:
[Equation]
Note that if x ( attack ) is the solution to d ∗ attack such that: ∥ x ( attack ) − x ( 0 ) ∥
= ρ , by the deﬁnition of d ∗ attack :
[Equation]
But then by the deﬁnition of p ∗ attack , p ∗ attack ≤ d ∗ attack , implying that the duality gap is zero, i.e p ∗ attack = d ∗ attack .
This procedure leads to the theorem 2.
C I MPLEMENTATION D ETAILS C. C OMPUTING THE DERIVATIVE OF LARGEST SINGULAR VALUE
Our objective is to compute derivative of the largest singular value, i.e ∥ W ( I ) ∥ with respect to W ( I ) .
u ( I ) , v ( I ) are the singular vectors such that:
[Equation]
v ( I ) , ∥ W ( I ) ∥ 2 can be computed by running power iteration on ( W ( I ) )
T W ( I ) .
u ( I ) can be computed using the identity:
[Equation]
We use 25 iterations of the power method to compute the above quantities.
C. U PDATE EQUATION FOR THE CERTIFICATE PROBLEM
Our goal is to minimize ∥ x − x ( 0 ) ∥ such that ( z ( L ) y − z
( L ) t )( x )
= 0 .
We know that the Hessian satisﬁes the following LMIs:
[Equation]
K is given by Theorem 4 for neural network of any depth ( L ≥ 2 ).
For 2 layer networks, M and m are given by Theorem 3.
But for deeper networks ( L ≥ 3 ), M = K , m = − K .
In either case,
K ≥ max (∣ m ∣ , ∣ M ∣) .
Thus, we also have:
[Equation]
We will solve the dual ( d ∗ cert ) of the attack problem ( p ∗ cert ).
The primal problem ( p ∗ cert ) is given by:
[Equation]
Using inequality (11) and Theorem 1 part (a), we know that the dual of the above problem is convex when − 1 / M ≤ η ≤ − 1 / m .
The corresponding dual problem ( d ∗ cert ) is given by:
[Equation]
For a given η , we have the following optimization:
[Equation]
We will use majorization-minimization to solve this optimization.
At a point x ( k ) , we aim to solve for the point x
( k + 1 ) that decreases the objective function.
Using the
Taylor’s theorem at point x ( k ) , we have:
[Equation]
where g ( k ) is the gradient of ( z ( L )
y − z
( L ) t ) at x ( k ) and H ( ξ ) is the Hessian at a point ξ on the line connecting x and x ( k ) .
Multiplying both sides by η , we get the following equation:
[Equation]
Using inequality (12), we know that − K
I ≼ H ( ξ )
≼ K
I ∀ ξ ∈ R D ,
[Equation]
Using equation (13) and inequality (14):
[Equation]
Adding 1 / 2 ∥ x − x ( 0 ) ∥ 2 to both sides, we get the following inequality:
[Equation]
LHS is the objective function of d cert ( η ) and RHS is an upper bound.
In majorization-minimization, we minimize an upper bound on the objective function.
Thus we set the gradient of RHS with respect to x to zero and solve for x :
[Equation]
[Equation]
This gives the following iterative equation:
[Equation]
C. U PDATE EQUATION FOR THE ATTACK PROBLEM
Our goal is to minimize z ( L )
y − z
( L ) t within an l 2 ball of radius of ρ .
We know that the Hessian satisﬁes the following LMIs:
[Equation]
K is given by Theorem 4 for neural network of any depth ( L ≥ 2 ).
For 2 layer networks, M and m are given by Theorem 3.
But for deeper networks ( L ≥ 3 ), M = K , m = − K .
In either case,
K ≥ max (∣ m ∣ , ∣ M ∣) .
Thus, we also have:
[Equation]
We solve the dual ( d ∗ attack ) of the attack problem ( p ∗ attack ) for the given radius ρ .
The primal problem ( p ∗ attack ) is given by:
[Equation]
Using inequality (16) and Theorem 2 part (a), we know that the dual of the above problem is convex when − m ≤ η .
The corresponding dual problem ( d ∗ cert ) is given by:
[Equation]
For a given η , we have the following optimization:
[Equation]
We will use majorization-minimization to solve this optimization.
At a point x ( k ) , we have to solve for the point x
( k + 1 ) that decreases the objective function.
Using the
Taylor’s theorem at point x ( k ) , we have:
[Equation]
where g ( k ) is the gradient of ( z ( L )
y − z
( L ) t ) at x ( k ) and H ( ξ ) is the Hessian at a point ξ on the line connecting x and x ( k ) .
Using inequality 17, we know that − K
I ≼ H ( ξ )
≼ K
I ∀ ξ ∈ R D ,
[Equation]
Using equation (18) and inequality (19):
[Equation]
Adding η / 2 (∥ x − x ( 0 ) ∥ 2 − ρ 2 ) to both sides, we get the following inequality:
[Equation]
LHS is the objective function of d attack ( η ) and RHS is an upper bound.
In majorization-minimization, we minimize an upper bound on the objective function.
Thus we set the gradient of RHS with respect to x to zero and solve for x :
[Equation]
This gives the following iterative equation:
[Equation]
A LGORITHM TO COMPUTE THE CERTIFICATE
We start with the following initial values of x , η, η min , η max :
[Equation]
To solve the dual for a given value of η
[Equation]
we run 20 iterations of the following update (derived in
Appendix C.2):
[Equation]
To maximize the dual d cert ( η ) over η in the range [− 1 / M, − 1 / m ] , we use a bisection method: If the solution x for a given value of η , ( z ( L ) y − z
( L ) t )( x ) > 0 , set η min =
η , else set η max = η .
Set the new η = ( η min + η max )/ 2 and repeat.
The maximum number of updates to η are set to 30.
The routine to compute the certiﬁcate example is given in Algorithm 1.
C. A LGORITHM TO COMPUTE THE ATTACK
We start with the following initial values of x , η, η min , η max :
[Equation]
To solve the dual for a given value of η , we run 20 iterations of the following update (derived in
Appendix C.3):
[Equation]
To maximize the dual d cert ( η ) over η in the range [− m, 20 ( 1 − m )] , we use a bisection method:
If the solution x for a given value of η , ∥ x − x ( 0 ) ∥ ≤ ρ , set η max =
η , else set η min = η .
Set new η = ( η min + η max )/ 2 and repeat.
The maximum number of updates to η are set to 30.
The routine to compute the attack example is given in Algorithm 2.
Algorithm
Require: input x ( 0 ) , label y , target t m,M,K ← compute bounds ( z ( L )
y − z
( L ) t )
η min ← − 1 / M η max ← − 1 / m η ← 1 / 2 ( η min + η max ) x ← x ( 0 ) for i in [ 1 ,..., 30 ] do for j in [ 1 ,..., 20 ] do g ← compute gradient ( z ( L )
y − z
( L ) t , x ) if ∥ η g
+ ( x − x ( 0 ) )
∥ < 10 − 5 then break end if x ← −( 1 + ∣ ηK ∣) − 1 ( η g − ∣ ηK ∣ x − x ( 0 ) ) end for if ( z ( L )
y − z
( L ) t )( x ) > 0 then η min ← η else η max ← η end if η ← ( η min + η max )/ 2 end for return x
Algorithm
Require: input x ( 0 ) , label y , target t , radius ρ m,M,K ← compute bounds ( z ( L )
y − z
( L ) t )
η min ← − m η max ← 20 ( 1 − m )
η ← 1 / 2 ( η min + η max ) x ← x ( 0 ) for i in [ 1 ,..., 30 ] do for j in [ 1 ,..., 20 ] do g ← compute gradient ( z ( L )
y − z
( L ) t , x ) if ∥ g + η ( x − x ( 0 ) )
∥ < 10 − 5 then break end if x ← −( K + η ) − 1 ( g − K x − η x ( 0 ) ) end for if ∥ x − x ( 0 ) ∥ <
ρ then η max ← η else η min ← η end if η ← ( η min + η max )/ 2 end for return x
D P ROOFS
P ROOF OF T HEOREM
(a)
[Equation]
We are given that the Hessian ∇ 2 x ( z ( L ) y − z
( L ) t ) satisﬁes the following LMIs:
[Equation]
The eigenvalues of I + η ∇ 2 x
( z ( L )
y − z
( L ) t ) are bounded between:
[Equation]
We are given that η satisﬁes the following inequalities where m < 0 ,M > 0 since ( z ( L ) y − z
( L ) t ) is neither convex, nor concave as a function of x :
[Equation]
We have the following inequalities:
[Equation]
Thus, I + η ∇ 2 x ( z ( L ) y − z
( L ) t ) is a PSD matrix for all x ∈ R D when − 1 / M ≤ η ≤ − 1 / m .
Thus 1 / 2 ∥ x − x ( 0 ) ∥ 2 + η ( z ( L )
y − z
( L ) t )( x ) is a convex function in x and d cert ( η ) is a convex optimization problem.
(b) For every value of η , d cert ( η ) is a lower bound for p ∗ cert .
Thus d ∗ cert = max
[Equation]
/ M
[Equation]
≤− 1 / m d cert ( η ) is a lower bound for p ∗ cert , i.e:
[Equation]
[Equation]
Let η ( cert ) , x ( cert ) be the solution of the above dual optimization ( d ∗ cert ) such that
[Equation]
d ∗ cert is given by the following:
[Equation]
=
Since we are given that z ( L ) y
( x ( cert ) )
=
z ( L ) t ( x ( cert ) ) , we get the following equation for d ∗ cert :
[Equation]
Since p ∗ cert is given by the following equation:
[Equation]
Using equations (22) and (24), p ∗ cert is the minimum value of 1 / 2 ∥ x − x ( 0 )
[Equation] [Equation]
z ( L ) y
( x [Equation]
= z ( L ) t ( x ) :
[Equation]
From equation (23), we know that d ∗ cert = 1 / 2 ∥ x ( cert ) − x ( 0 )
∥ 2 .
Thus, we get:
[Equation]
Using equation (21) we have d ∗ cert ≤ p ∗ cert and using (26), p ∗ cert ≤ d ∗ cert p ∗ cert =
d ∗ cert
D. P ROOF OF T HEOREM
[Equation]
Since the Hessian ∇ 2 x ( z ( L ) y − z ( L ) t ) is bounded below:
[Equation]
R
[Equation]
The eigenvalues of ∇ 2 x ( z ( L ) y − z
( L ) t )
+ η I are bounded below:
[Equation]
Since η ≥ − m .
η + m ≥
Thus ∇ 2 x ( z ( L )
y − z
( L ) t )
+ η I is a PSD matrix for all x ∈ R D when η ≥ − m .
Thus ( z ( L )
y − z
( L ) t )( x ) + η / 2 (∥ x − x ( 0 ) ∥ 2 − ρ 2 ) is a convex function in x and d attack ( η ) is a convex optimization problem.
(b) For every value of
η , d attack ( η
is
a lower
bound
for p
∗ attack
Thus d ∗ attack = max − m ≤ η d attack ( η ) is a lower bound for p ∗ attack :
[Equation]
Let η ( attack ) , x ( attack ) be the solution of the above dual optimization ( d ∗ attack ) such that
[Equation]
28) d ∗ attack is given by the following:
[Equation]
·„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„‚„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„¶ =
Since we are given that ∥ x ( attack ) − x ( 0 ) ∥
= ρ , we get the following equation for d ∗ attack :
[Equation]
Since p ∗ attack is given by the following equation:
[Equation]
Using equations (28) and (30), p ∗ attack is the minimum value of ( z ( L ) y −
[Equation]
From equation (29), we know that d ∗ attack =
( z ( L )
y − z
( L ) t )( x ( attack ) ) .
Thus, we get:
[Equation]
Using equation (27) we have d ∗ attack ≤ p ∗ attack and using (32), p ∗ attack ≤ d ∗ attack p ∗ attack =
d ∗ attack
D. P ROOF OF L EMMA
We have to prove that for an L layer neural network, the hessian of the i th hidden unit in the L th layer with respect to the input x , i.e ∇ 2 x z ( L ) i is given by the following formula:
[Equation]
) where B ( I ) , I ∈ [ L ] is a matrix of size N I × D deﬁned as follows:
[Equation]
and F ( L,I ) , I ∈
[ L − 1 ] is a matrix of size N L × N I deﬁned as follows:
[Equation]
∇ 2 x z ( L ) i can be written in terms of the activations of the previous layer using the following formula:
[Equation]
Using the chain rule of the Hessian and a ( I )
=
σ ( z ( I ) ) , we can write ∇ 2 x a ( L − 1 ) j in terms of ∇ x z ( L − 1 ) j and ∇ 2 x z ( L − 1 )
j as the following:
[Equation]
Replacing ∇ 2 x a ( L
[Equation]
j using equation (37) into equation (36), we get:
[Equation]
For each I ∈ [ 2 ,L ] , i ∈ N I , we deﬁne the matrix
A ( I ) i as the following:
[Equation]
Substituting A ( L ) i using equation (40) into equation (38), we get:
[Equation]
We ﬁrst simplify the expression for A ( L ) i .
Note that A ( L ) i is a sum of symmetric rank one matrices (∇ x z ( L − 1 ) j )(∇ x z ( L − 1 ) [Equation]
T with the coefﬁcient W ( L ) i,j σ ′′ ( z ( L − 1 ) j ) for each j .
We create a diagonal matrix for the coefﬁcients and another matrix B ( L − 1 ) such that each j th row of B ( L − 1 ) is the vector ∇ x z ( L − 1 ) j .
This leads to the following equation:
[Equation]
B ( I ) where I ∈
[ L ] is a matrix of size N I × D deﬁned as follows:
[Equation]
Thus B ( I ) is the jacobian of z ( I ) with respect to the input x .
Using the chain rule of the gradient, we have the following properties of B ( I ) :
[Equation]
Similarly, F ( I,J ) where I ∈
[ L ] , J ∈
[ I − 1 ] is a matrix of size N I × N J deﬁned as follows:
[Equation]
Thus F ( I,J ) is the jacobian of z ( I ) with respect to the activations a ( J ) .
Using the chain rule of the gradient, we have the following properties for F ( L,I ) :
[Equation]
[Equation]
Recall that in our notation: For a matrix E , E i denotes the column vector constructed by taking the transpose of the i th row of the matrix E .
Thus i th row of W ( L ) is ( W ( L ) i )
T and F ( L,I ) is ( F ( L,I ) i )
T .
Equating the i th rows in equation (46), we get:
[Equation]
Taking the transpose of both the sides and expressing the RHS as a summation, we get:
[Equation]
Substituting W ( L ) using equation (45) into equation (42), we get:
[Equation]
Substituting A ( L ) i using equation (48) into (41), we get:
[Equation]
Thus, equation (49) allows us to write the hessian of i th unit at layer L , i.e (∇ 2 x z ( L ) i ) in terms of the hessian of j th unit at layer L − 1 , i.e (∇ 2 x z ( L − 1 ) j ) .
We will prove the following using induction:
[Equation]
Note that for L = 2 , ∇ 2 x z ( L − 1 ) j = 0 , ∀ j ∈ N 1 .
Thus using (49) we have:
[Equation]
Hence the induction hypothesis (50) is true for L = 2 .
Now we will assume (50) is true for L − 1 .
Thus we have:
[Equation]
We will prove the same for L .
Using equation (49), we have:
[Equation]
In the next set of steps
[Equation]
we will be working with the second term of the above equation
[Equation]
i.e ∑ N
[Equation]
Substituting ∇ 2 x z ( L − 1 )
j using equation (51) we get:
[Equation]
Combining the two summations in the second term, we get:
[Equation]
Exchanging the summation over I and summation over j :
[Equation]
Since B ( I ) is independent of j , we take it out of the summation over j :
[Equation]
Using the property, α ( diag ( u ))
+ β ( diag ( v ))
=
diag ( α u + β v ) ∀ α,β ∈ R , u , v ∈ R n ; we can move the summation inside the diagonal:
[Equation]
Since σ ′′ ( z ( I ) ) is independent of j
[Equation]
we can take it out of the summation over j :
[Equation]
Using equation (47), we can replace ∑ N L
[Equation] [Equation]
1 W ( L ) i
[Equation]
with F
[Equation] [Equation]
[Equation]
D. P ROOF OF T HEOREM
Using Lemma 1, we have the following formula for ∇ 2 x ( z ( 2 ) y − z ( 2 ) t ) :
[Equation]
We are also given that the activation function σ satisﬁes the following property:
[Equation]
(a) We have to prove the following linear matrix inequalities (LMIs):
[Equation] [Equation]
where P and N are given as following:
[Equation] [Equation]
We ﬁrst prove: N ≼ ∇ 2 x
( z ( 2 ) y − z ( 2 ) t ) ∀ x ∈ R D :
We substitute ∇ 2 x ( z ( 2 ) y − z ( 2 ) t ) and N from equations (52) and (56) respectively in ∇ 2 x ( z ( 2 ) y − z ( 2 ) t ) − N :
[Equation]
Thus ∇ 2 x ( z ( 2 ) y − z ( 2 ) t ) − N is a weighted sum of symmetric rank one matrices i.e,
W ( 1 ) i ( W ( 1 ) i ) T and it is PSD if and only if coefﬁcient of each rank one matrix i.e, ( W ( 2 ) y,
i − W ( 2 ) t,i )( σ ′′ ( z ( 1 ) i ) − n i ) is positive.
Using equations (53) and (57), we have the following:
[Equation]
Thus ∇ 2 x ( z ( 2 ) y − z ( 2 ) t ) − N is a PSD matrix i.e: ∇
[Equation]
z
[Equation] [Equation]
y − z
[Equation]
N
[Equation]
∑ i =
[Equation]
·„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„‚„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„¶ always
[Equation]
Now we prove that ∇ 2 x ( z
[Equation] [Equation]
y − z
[Equation] [Equation]
≼ P ∀ x ∈ R D :
We substitute ∇ 2 x ( z ( 2 ) y − z ( 2 ) t ) and P from equations (52) and (56) respectively in P − ∇ 2 x
( z ( 2 ) y − z ( 2 ) t ) :
[Equation]
Thus P − ∇ 2 x ( z ( 2 ) y − z ( 2 ) t ) is a weighted sum of symmetric rank one matrices i.e,
W ( 1 ) i ( W ( 1 ) i ) T and it is PSD if and only if coefﬁcient of each rank one matrix i.e, ( W
[Equation]
2 ) y,i − W
[Equation]
2 ) t,i )( p i − σ ′′
( z ( 1 ) i )) is positive.
Using equations (53) and (57), we have the following:
[Equation]
Thus P − ∇ 2 x ( z ( 2 ) y − z ( 2 ) t ) is PSD matrix i.e:
[Equation] [Equation]
[Equation]
Thus by proving the LMIs (59) and (61), we prove (54).
(b
We have
to
prove
that
if
h U ≥ 0 and h L ≤
P
is a PSD matrix, N is a NSD matrix.
We are given h U ≥ 0 , h L ≤ 0 .
Using equation (57), we have the following:
[Equation]
Thus P is a weighted sum of symmetric rank one matrices i.e, W ( 1 ) i ( W ( 1 ) i ) T and each coefﬁcient p
i ( W ( 2 ) y,i − W ( 2 ) t,i ) is positive.
[Equation]
Using equation (57), we have the following:
[Equation]
·„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„‚„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„„¶ always positive
[Equation]
Thus P is a PSD and N is a NSD matrix if h U ≥ 0 and h L ≤ 0 .
(c) We have to prove the following global bounds on the eigenvalues of ∇ 2 x ( z ( 2 ) y − z ( 2 ) t ) :
[Equation]
where
[Equation]
Since ∇ 2 x ( z
[Equation] [Equation]
Let v ∗ , x ∗ be vectors such that:
[Equation]
Thus using inequality (64):
[Equation]
Since N ≼ ∇ 2 x ( z ( 2 ) y − z ( 2 ) t ) ∀ x ∈ R D : v T Nv ≤ v T [∇ 2 x
( z ( 2 ) y − z ( 2 ) t )] v ∀ v ∈ R D , ∀ x ∈ R D (66)
Let v ∗ , x ∗ be vectors such that:
[Equation]
[Equation]
min x min
[Equation]
y −
[Equation]
] v
Thus using inequality (66):
[Equation]
Using the inequalities (65) and (67), we get:
[Equation]
, where M = max ∥ v ∥= 1 v T Pv , m = min ∥ v ∥= 1 v T Nv
D. P ROOF OF T HEOREM
We are given that the activation function σ is such that σ ′ , σ ′′ are bounded, i.e:
[Equation]
We have to prove the following:
[Equation]
where S ( L,I ) is a matrix of size N L × N I deﬁned as follows:
[Equation]
and r ( I ) is a scalar deﬁned as follows:
[Equation]
We will prove the same in 3 steps.
In step (a), we will prove:
[Equation]
In step (b), we will prove:
[Equation]
In step (c), we will use (a) and (b) to prove:
[Equation]
Note that B ( I ) and F ( L,I ) are deﬁned using (34) and (35) respectively.
(a) We have to prove that
for
L
≥ 2 ,
I
∈
L
−
] , i
∈
N L , j ∈ N I :
[Equation]
∈ R D where S ( L,I ) is a matrix of size
N I × N J deﬁned as follows:
[Equation]
We ﬁrst prove the case when I =
L − 1 .
Using equation (45), F ( L,L − 1 ) i,j = W ( L ) i,j .
Since S ( L,L − 1 ) i,j = ∣ W ( L ) i,j ∣ : [Equation] F
[Equation]
L,
[Equation] [Equation]
S
[Equation]
L,
[Equation]
,j
Hence for L ≥ 2 , I =
L − 1 , we have equality in (71).
Hence proved.
Now, we will use proof by induction.
To prove the base case L = 2 , note that I
=
L − 1 = 1 is the only possible value for I .
Thus, using the result for I =
L − 1 , the theorem holds for L = 2 .
This proves the base case.
Now we assume the induction hypothesis is true for depth =
L − 1 , I ∈
[ L − 2 ] .
and prove for depth
=
L, I ∈
[ L − 1 ] .
Since for I =
L − 1 , we have proven already, we prove for I ≤ L − 2 .
Using equation (47), we have the following formula for F ( L,I ) i :
[Equation]
Taking the j th element of the vectors on both sides:
[Equation]
By induction hypothesis, we know that:
[Equation]
Using the absolute value properties for equation (74), we have:
[Equation]
Using ∣ σ ′ ( x )∣ ≤ g ∀ x ∈ R (inequality (68)) :
[Equation]
F
[Equation]
Using the induction hypothesis (inequality (75)):
[Equation]
Using equation (69) for deﬁnition of S ( L,I ) i,j
[Equation]
Hence we prove (71) for all L ≥ 2 and I ≤ L − 1 using induction.
(b) We have to prove that for 1 ≤ I ≤ M − 1 :
[Equation]
where r ( I ) is a scalar given as follows:
[Equation]
Using equation (43), for I
= 1 we have:
[Equation]
Using equation (44), for I > 1 , we have:
[Equation]
Since ∥ diag ( σ ′ ( z ( I − 1 ) ))
∥
= max j ∣ σ ′ ( z ( I − 1 ) j )∣ , using equation (68):
[Equation]
Using inequalities (76) and (77), the proof follows using induction.
(c) We have to prove that:
[Equation]
Using Lemma
[Equation]
we have the following equation for ∇ 2 x z ( L ) i :
[Equation]
Using the properties of norm we have:
[Equation]
In the last inequality
[Equation]
we use the property that norm of a diagonal matrix is the maximum absolute value of the diagonal element.
Using the product property of absolute value, we get:
[Equation]
Since ∣ F ( L,I ) i,j ∣ and ∣ σ ′′
( z ( I ) j )∣ are positive terms:
[Equation]
Since ∥ σ ′′ ∥ is bounded by h :
[Equation]
∣ F ( L,I ) i,j ∣)∥ B ( I )
∥
Using inequality (71): ∥∇ 2 x z ( L ) i ∥ ≤ h
L −∑ I = 1 max j ( S ( I ) i,j ) ∥
B ( I ) ∥
Using inequality (72): ∥∇ 2 x z ( L ) i ∥ ≤ h
L −∑ I = 1 ( r ( I ) )
2 max j ( S ( I ) i,j ) ∀ x ∈ R D
E C OMPUTING g, h, h U AND h L FOR DIFFERENT ACTIVATION FUNCTIONS E. S OFTPLUS ACTIVATION
For softplus activation, we have the following.
We use S ( x ) to denote sigmoid:
[Equation]
To bound S ( x )( 1 − S ( x )) , let α denote S ( x ) .
We know that 0 ≤ α ≤ 1 :
[Equation]
Thus, S ( x )( 1 − S ( x )) is maximum at S ( x ) = 1 / 2 and minimum at S ( x ) = 0 and S ( x )
= 1 .
The maximum value is 0.25 and minimum value is 0.
[Equation]
Thus, h U = 0 .
25 , h L =
(for use in Theorem 3) and g = 1 , h = 0 .
25 (for use in Theorem 4).
E. S IGMOID ACTIVATION
For sigmoid activation, we have the following.
We use S ( x ) to denote sigmoid:
[Equation]
The second derivative of sigmoid ( σ ′′
( x )) can be bounded using standard differentiation.
Let α denote S ( x ) .
We know that 0 ≤ α ≤ 1 :
[Equation]
To solve for both h L and h U , we ﬁrst differentiate α ( 1 − α )( 1 − 2 α ) with respect to α : ∇
[Equation]
[Equation] [Equation]
[Equation] [Equation]
= ∇ α (
[Equation]
3 − 3 α 2 + α ) =
( 6 α 2 − 6 α + 1 )
Solving for 6 α 2 − 6 α + 1 = 0 , we get the solutions:
[Equation]
Since both ( 3 + √ 3 / 6 ) , ( 3 − √ 3 / 6 ) lie between 0 and 1, we check for the second derivatives:
[Equation] [Equation]
[Equation] [Equation]
At α = ( 3 + √ 3 )/ 6 , ∇ 2 α = 6 ( 2 α − 1 ) = 2 √ 3 > 0 . At α = ( 3 − √ 3 )/ 6 , ∇ 2 α = 6 ( 2 α − 1 ) = − 2 √ 3 < 0 .
Thus α = ( 3 + √ 3 )/ 6 is a local minima, α
= ( 3 − √ 3 )/ 6 is a local maxima.
Substituting the two critical points into α ( 1 − α )( 1 − 2 α ) , we get h U
= 9 .
623 × 10 − 2 , h L = − 9 .
623 × 10 − 2 .
Thus, h U = 9 .
623 × 10 − 2 , h L = − 9 .
623 × 10 − 2 (for use in Theorem 3) and g = 0 .
25 , h =
.(for use in Theorem 4).
E. T ANH ACTIVATION
For tanh activation, we have the following:
[Equation]
The second derivative of tanh , i.e ( σ ′′ ( x )) can be bounded using standard differentiation.
Let α denote tanh ( x ) .
We know that − 1 ≤ α ≤ 1 :
[Equation]
To solve for both h L and h U , we ﬁrst differentiate − 2 α ( 1 − α )( 1 + α ) with respect to α :
[Equation]
Solving for 6 α 2 − 2 = 0 , we get the solutions:
[Equation]
Since both − 1 /√ 3 , 1 /√ 3 lie between -1 and 1, we check for the second derivatives:
[Equation]
At α = − 1 /√ 3 , ∇ 2 α = 12 α = − 4 √ 3 < 0 .
At α = 1 /√ 3 , ∇ 2 α = 12 α = 4 √ 3 > 0 .
Thus α = 1 /√ 3 is a local minima, α = − 1 /√ 3 is a local maxima.
Substituting the two critical points into − 2 α ( 1 − α )( 1 + α ) , we get h U = 0 .
76981 , h L = − 0 .
Thus, h U = 0 .
76981 , h L = − 0 .
76981 (for use in Theorem 3) and g = 1 , h = 0 .
76981 (for use in
Theorem 4).
F Q UADRATIC BOUNDS FOR TWO - LAYER R E LU NETWORKS
For a 2 layer network with ReLU activation, such that the input x lies in the ball ∥ x − x ( 0 ) ∥
≤ ρ , we can compute the bounds over z ( 1 ) directly:
[Equation]
Thus we can get a lower bound and upper bound for each z ( 1 ) i .
We deﬁne d i and u i as the following:
[Equation]
We can derive the following quadratic lower and upper bounds for each a ( 1 ) i :
[Equation]
⎧⎪⎪⎪⎪⎪⎪⎨⎪⎪⎪⎪⎪⎪
[Equation]
The above steps are exactly the same as the quadratic upper and lower bounds used in (Zhang et al., 2018a).
Usingthe above two inequalities and the identity:
[Equation]
we can compute a quadratic lower bound for z ( 2 ) y − z ( 2 ) t in terms of z ( 1 ) i by taking the lower bound for a ( 1 ) i when ( W ( 2 )
y,i − W ( 2 ) t,i ) > 0 and upper bound when ( W ( 2 )
y,i − W ( 2 ) t,i ) <
= 0 .
Furthermore since z ( 1 ) i
= W ( 1 ) i x + b ( 1 ) i , we can express the resulting quadratic in terms of x .
Thus, we get the following quadratic function :
[Equation]
The coefﬁcients P , q and r can be determined using the above procedure.
Note that unlike in (Zhang et al., 2018a), RHS can be a non-convex function.
Thus, it becomes an optimization problem where the goal is to minimize the distance 1 / 2 ∥ x − x ( 0 ) ∥subject to RHS (which is quadratic in x ) being zero.
That is both our objective and constraint are quadratic functions.
In the optimization literature, this is called the S-procedure and is one of the few non-convex problems that can be solved efﬁciently (Boyd & Vandenberghe, 2004).
We start with two initial values called ρ low (initialized to 0) and ρ high (initialized to 5).
We start with an initial value of ρ , initialized at 1 / 2 ( ρ low + ρ high ) to compute d i (eq.
and u i (eq. (79)).
If the ﬁnal distance after solving the S-procedure is less than ρ , we set ρ low
= ρ .
if the ﬁnal distance is greater than ρ , we set ρ high = ρ .
Set new ρ = 1 / 2 ( ρ low + ρ high ) .
Repeat until convergence.
G A DDITIONAL EXPERIMENTS
Empirical accuracy means the fraction of test samples that were correctly classiﬁed after running a
PGD attack (Madry et al., 2018) with an l 2 bound on the adversarial perturbations.
Certiﬁed accuracy means the fraction of test samples that were classiﬁed correctly initially and had the robustness certiﬁcate greater than a pre-speciﬁed attack radius ρ .
For both empirical and certiﬁed accuracy, we use ρ = 0 .
Unless otherwise speciﬁed, we use the class with the second largest logit as the attack target for the given input (i
[Table]
e. the class t ).
All experiments were run on the MNIST dataset while noting that our results are scalable for more complex datasets.
The notation ( L × [ 1024 ] , activation) denotes a neural network with L layers with the speciﬁed activation function, ( γ = c ) denotes standard training with γ set to c , (CRT, c ) denotes CRT training with γ = c .
Certiﬁcates CROWN and CRC are computed over 150 correctly classiﬁed images.
G. R ESULTS ON C ONVOLUTIONAL NEURAL NETWORKS
We use a 2 layer (1 hidden layer network).
We use 64 ﬁlters in the convolution layer and softplus activation function and stride of length 2.
This is followed by reshaping and a fully connected layer.
[Table]
for different values of the regularization parameter γ [-= 
 (no curvature regularization), networks adversarially trained-]for a single hidden layer convolutional neural network with softplus activation function
G. R ESULTS FOR T ANH NETWORKS
[Table]
outperforms CROWN signiﬁcantly for G
[Table]
C OMPARING R ANDOMIZED S MOOTHING AND TRADES WITH CRT
Randomized smoothing is designed to work in untargeted attack settings while CRT is for targeted attacks.
Thus, to do a fair comparison of CRT with randomized smoothing, we make the following changes in randomized smoothing.
First, we use n 0 = 100 initial samples to select the label class ( l ) and false target class ( t ).
The samples for estimation were n = 100 , 000 and failure probability was α = 0 .
Then we use the binary version of randomized smoothing for estimation, i.e classify between y and t .
To ﬁnd the adversarial example for adversarial training, we use the cross entropy loss for 2 classes ( y and t ). For TRADES
[Table] we select the class with second highest logit as the target class t and use the 2 class version of the cross entropy loss for ﬁnding the adversarial example. [Table] [Table]
[Table]
G. M EASURING THE IMPACT OF CURVATURE REGULARIZATION
In Table 12, we measure how the standard accuracy, empirical accuracy, certiﬁed accuracy, upper bound on the curvature K ub , lower bound on the curvature K lb , changes as we increase the regularization parameter γ and the network is trained with CRT.
In Table 13, we measure how the standard accuracy, empirical accuracy, certiﬁed accuracy, CROWN and CRC changes as we increase the regularization parameter γ and the network is trained without any adversarial training.
G. C OMPUTING K lb AND K ub
First, note that K does not depend on the input, but on network weights W ( I ) , label y and target t .
Different images may still have different K because label y and target t may be different.
To compute K lb in the table, ﬁrst for each pair y and t , we ﬁnd the largest eigenvalue of the Hessian of all test images that have label y and second largest logit of class t .
Then we take the max of the largest eigenvalue across all test images.
This gives a rough estimate of the largest curvature in the vicinity of test images with label y and target t .
We can directly take the mean across all such pairs to compute K lb .
However, we ﬁnd that some pairs y and t were infrequent (with barely 1,2 test images in them).
Thus, for all such pairs we cannot get a good estimate of the largest curvature in vicinity.
We select all pairs y and t that have at least 100 images in them and compute K lb by taking the mean across all such pairs.
To compute K ub in the table, we compute K for all pairs y and t that have at least 100 images, i.e at least 100 images should have label y and target t .
And then we compute the mean across all K that satisfy this condition.
This was done to do a fair comparison with K lb .
{+Figure 2 shows a plot of the K ub and K lb with increasing γ for a tanh network. When γ = 
 (no curvature regularization), networks adversarially trained with CRT or PGD both have high curvatures.+}
However,
CRT with even a small γ leads to a signiﬁcant decrease in curvature bounds. Also, curvature bounds are higher with Tanh than with Sigmoid. Results are similar to Sigmoid in
Figure 1. [Table] [Table] [Table]
[Table] [Table] [Table]
[Table] [Table] [Table]
[Table] [Table] [Table] [Table]
[Table] [Table] [Table] [Table]
[Table] [Table] [Table] [Table]
[Table] [Table] [Table] [Table]
[Table] [Table] [Table] [Table]
[Table] [Table] [Table] [Table]
[Table] [Table] [Table] [Table]
[Table] [Table] [Table] [Table]
[Table] [Table] [Table] [Table]
[Table] [Table] [Table] [Table]
[Table] [Table] [Table] [Table]
[Table] [Table] [Table] [Table]
[Table] [Table] [Table] [Table]
[Table] [Table] [Table] [Table]
[Table] [Table] [Table] [Table]
[Table] [Table] [Table] [Table]
[Table]
In this section, we empirically compare our Curvature-based attack optimization to the PGD method of Madry et al. (2018) (200 steps of size 0 . 01 ). We should note that the objectives of our attack and PGD are different.
The PGD attack (with sufﬁciently large number of steps) will always ﬁnd a solution x ( attack ) such that ∥ x ( attack ) − x ( 0 ) ∥ 2 ≈ ρ . However, if our curvature bound is loose (or even if the curvature bound is tight but the network itself has large curvature), our method may converge to a point inside the ball where ∥ x ( attack ) − x ( 0 ) ∥ 2 < ρ . The primary beneﬁt of our method is that if it converges to a point on the surface of the ball (i.e. ∥ x ( attack ) − x ( 0 ) ∥ 2 = ρ ), then x ( attack ) is provably the worst case perturbation in the l 2 ball of radius ρ around the input x ( 0 ) (since primal=dual using Theorem 2).
Thus, to have the same perturbation magnitude between our attack optimization and PGD, we consider a subset of samples where our attack method converges to a point on the surface of the ball. In Table 14, we show a comparison between average values of z y − z t computed using PGD and our method where y is the correct label and t is the attack target. In this table, Mean CBA (Curvature-Based Attack) is computed by taking the average of z y − z t for all test samples satisfying ∥ x ( attack ) − x ( 0 ) ∥ 2 = ρ (with curvature based attack optimization). The relative gain is computed as (Mean PGD - Mean CBA)/(Mean PGD).
[Table]