T RAINING P ROVABLY R OBUST M ODELS BY P OLYHEDRAL E NVELOPE R EGULARIZATION
Anonymous authors
Paper under double-blind review
A BSTRACT
Training certiﬁable neural networks enables one to obtain models with robustness guarantees against adversarial attacks.
In this work, we use a linear approximation to bound model’s output given an input adversarial budget.
This allows us to bound the adversary-free region in the data neighborhood by a polyhedral envelope and yields ﬁner-grained certiﬁed robustness than existing methods.
We further exploit this certiﬁer to introduce a framework called polyhedral envelope regularization (PER), which encourages larger polyhedral envelopes and thus improves the provable robustness of the models.
We demonstrate the ﬂexibility and effectiveness of our framework on standard benchmarks; it applies to networks with general activation functions and obtains comparable or better robustness guarantees than state-of-the-art methods, with very little cost in clean accuracy, i.e., without over-regularizing the model.
I NTRODUCTION
Despite their great success in many applications, modern deep learning models are vulnerable to adversarial attacks: small but well-designed perturbations can make the state-of-the-art models predict wrong labels with very high conﬁdence (Goodfellow et al., 2014; Moosavi-Dezfooli et al., 2017;
Szegedy et al., 2013).
The existence of such adversarial examples indicates unsatisfactory properties of the deep learning models’ decision boundary (He et al., 2018), and poses a threat to the reliability of modern machine learning systems.
As a consequence, studying the robustness of deep learning has attracted growing attention, from the perspective of both attack and defense strategies.
Popular attack algorithms, such as the fast gradient method (FGM) (Goodfellow et al., 2014), the CW attack (Carlini & Wagner, 2017) and the projected gradient descent (PGD) algorithm (Madry et al., 2017), typically exploit the gradient of the loss w.r.t.
the input to generate adversarial examples.
To counteract such attacks, many defense algorithms have been proposed (Buckman et al., 2018; Dhillon et al., 2018;
Guo et al., 2017; Ma et al., 2018; Samangouei et al., 2018; Song et al., 2017; Xie et al., 2017).
However, it was shown in Athalye et al.
that most of them depend on obfuscated gradients for perceived robustness.
In other words, these methods train models to fool gradient-based attacks but do not achieve true robustness.
As a consequence, they become ineffective when subjected to stronger attacks.
The only exception is adversarial training (Madry et al., 2017), which augments the training data with adversarial examples.
Nevertheless, while adversarial training yields good empirical performance, it still provides no guarantees of a model’s robustness.
In this work, we focus on constructing certiﬁers to ﬁnd certiﬁed regions of the input neighborhood where the model is guaranteed to give the correct prediction, and on using such certiﬁers to train a model to be provably robust against adversarial attacks.
In this context, complete certiﬁers can either guarantee the absence of an adversary or construct an adversarial example given an adversarial budget.
They are typically built on either Satisﬁability Modulo Theories (SMT) (Katz et al., 2017) or mixed integer programming (MIP) (Tjeng et al., 2017; Xiao et al., 2018).
The major disadvantages of complete certiﬁers are their super-polynomial complexity and applicability to only piecewise linear activation functions, such as ReLU.
By contrast, incomplete certiﬁers are faster, more widely applicable but more conservative in terms of certiﬁed regions because they rely on approximations.
Techniques such as linear approximation (Kolter & Wong, 2017; Wong et al., 2018; Zhang et al., 2018; Weng et al., 2018), symbolic interval analysis (Wang et al., 2018b), abstract transformers (Gehr et al., 2018; Singh et al., 2018; 2019), semideﬁnite programming (SDP) (Raghunathan et al., 2018a;b) and randomized smoothing (Cohen et al., 2019; Salman et al., 2019a) are exploited to offer better certiﬁed robustness.
Some of these methods enable training provably robust models (Raghunathan et al., 2018a; Kolter & Wong, 2017; Wong et al., 2018;
Cohen et al., 2019;
Salman et al., 2019a) by optimizing the parameters so as to maximize the area of the certiﬁed regions.
While effective, all the above-mentioned methods, in their vanilla version, only provide binary results given a ﬁxed adversarial budget.
That is, if a data point is certiﬁed, it is guaranteed to be robust in the entire given adversarial budget; otherwise no guaranteed adversary-free region is estimated.
To overcome this and search for the optimal adversarial budget that can be certiﬁed, Kolter & Wong (2017)
; Weng et al.
; Zhang et al.
(2018) use either Newton’s method or binary search.
By contrast, Croce et al.
(2018) take advantage of the geometric property of ReLU networks and gives ﬁner-grained robustness guarantees.
Given the piecewise linear ReLU function, any input is located in a polytope where the ReLU network can be considered linear.
Based on geometry, robustness guarantees can thus be calculated using the input distance to the polytope boundary and the decision boundary constraints.
Unfortunately, in practice, the resulting certiﬁed bounds are trivial because such polytopes are very small even for robust models.
Nevertheless, Croce et al.
(2018) introduce a regularization scheme based on these bounds to effectively train provably robust models.
In this paper, we construct a stronger certiﬁer, as well as a regularization scheme to train provably robust models.
To this end, building upon the geometry-inspired idea of Croce et al.
(2018), we estimate a linear bound of the model’s output given a predeﬁned adversarial budget.
Then, the condition to guarantee robustness inside this budget is also linear and forms a polyhedral envelope of the model’s decision boundary.
Similar to Croce et al.
(2018), the region contained in the polyhedral envelope and the adversarial budget is certiﬁed to be adversary-free.
We then further exploit this certiﬁer to design a regularization framework to train provably robust neural network models.
In short, our contributions are twofold.
First, we design a certiﬁcation method that gives better bounds than its counterparts (Kolter & Wong, 2017; Weng et al., 2018; Zhang et al., 2018) with little overhead and applies to networks with general activation functions.
Second, we propose a geometry-inspired regularization scheme to train provably robust models.
Compared with existing provable training methods, which have been found to over-regularize the model (Zhang et al., 2019), our training framework yields competitive robustness guarantees while maintaining good performance on clean inputs.
In the remainder of the paper, we call our certiﬁcation method Polyhedral Envelope Certiﬁer (PEC) and our regularization scheme Polyhedral Envelope Regularizer (PER) .
P RELIMINARIES 2. N OTATION AND T ERMINOLOGY
For simplicity, we discuss our approach using a standard N -layer fully-connected network.
Note, however, that it straightforwardly extends to any network topology that can be represented by a directed acyclic graph (DAG) in the same way as in (Liu et al., 2019).
A fully-connected network parameterized by { W ( i ) , b ( i ) } N − 1 i =1 can be expressed as
[Equation]
where z ( i ) and ˆ z
( i ) are the pre- & post-activations of the i -th layer, respectively, and ˆ z (1) def =
= x is the input of the network.
An l p norm-based adversarial budget S ( p ) (cid:15) ( x ) is deﬁned as the set { x (cid:48) |(cid:107)
x (cid:48) − x (cid:107) p ≤ (cid:15) } .
x (cid:48) , z (cid:48)
( i ) and
ˆ z (cid:48)
( i ) represent the adversarial input and the corresponding pre- & post-activations.
For layer i having n i neurons, we have W ( i ) ∈ R n i +1 × n i and b ( i
) ∈ R n i +1 .
Throughout this paper, underlines and bars are used to present lower and upper bounds of adversarial activations, respectively, i.e., z ( i ) ≤ z (cid:48)
( i ) ≤ ¯ z
( i ) .
A “ + ” or “ − ” subscript indicates the positive or negative part of a tensor.
We use [ K ] as the abbreviation for the set { 1 , 2 , ..., K } .
2. M ODEL L INEARIZATION
Given an adversarial budget S ( p ) (cid:15) ( x ) , we study the linear bound of the output logits z (cid:48) ( N ) , given by
[Equation]
The linear coefﬁcients introduced above can be calculated by iteratively estimating the bounds of intermediate layers and linearizing the activation functions.
In Appendix A.1, we discuss this for several activation functions, including ReLU, sigmoid, tanh and arctan.
Note that our method differs from Zhang et al.
(2018) because we need the analytical form of the linear coefﬁcients for training, which removes numerical methods such as binary search.
The bounding algorithm trades off computational complexity and bound tightness.
In this work, we study two such algorithms.
One is based on Fast-Lin / CROWN (Weng et al., 2018; Zhang et al., 2018).
It has tighter bounds but high computational complexity.
Another is inspired by the interval bound propagation (IBP)
(Gowal et al., 2018), which is faster but leads to looser bounds.
The details of both algorithms are provided in Appendices A.2 and A.3.
We brieﬂy discuss the complexity of both algorithms in Section 5.
A LGORITHMS 3. R OBUSTNESS G UARANTEES BY P OLYHEDRAL E NVELOPE
For an input point x with label c ∈ [ K ] , a sufﬁcient condition to guarantee robustness is that the lower bounds of z (cid:48) ( N ) c − z (cid:48)
( N ) i are positive for all i ∈
[ K ] .
Here, we use a trick called elision of the last layer (Gowal et al., 2018) to merge the subtraction of z (cid:48)
( N ) c and z (cid:48)
( N ) i with the last linear layer and obtain the lower bound of z (cid:48)
( N ) c − z (cid:48)
( N ) i : z (cid:48)
( N ) c − z (cid:48)
( N ) i def =
=
U i x (cid:48)
+ p i .
Then, the sufﬁcient condition to ensure robustness within a budget S ( p ) (cid:15) ( x ) can be written as
[Equation]
The constraint is trivial when i
= c , so there are K − 1 such linear constraints, corresponding to K −hyperplanes in the input space.
Within the adversarial budget, these hyperplanes provide a polyhedral envelope of the true decision boundary.
In the remainder of the paper, we use the term d ic to represent the distance between the input and the hyperplane deﬁned in (3) and deﬁne
d c = min i ∈
[ K ] ,i (cid:54) =
c d ic as the distance between the input and the polyhedral envelope boundary.
The distance can be based on different l p norms, and d ic = 0 when the input itself does not satisfy the inequality (3).
Since (3) is a sufﬁcient condition for robustness given the adversarial budget S ( p ) (cid:15) ( x ) , there is no adversarial examples in the intersection of S ( p ) (cid:15) ( x ) and the polytope deﬁned in (3).
The theorem below formalizes our robustness certiﬁcation.
We defer its proof to Appendix C.1.
Theorem 1 (PEC in Unconstrained Cases) .
Given a model f : R n 1 → [ K ] and an input point x with label c , let U and p in (3) be calculated using a predeﬁned adversarial budget S ( p ) (cid:15) ( x ) .
Then, there is no adversarial example inside an l p norm ball of radius d centered around x , with d = min { (cid:15), d c } where d ic = max (cid:110) 0 , U i x + p i (cid:107) U
i (cid:107) q (cid:111) .
l q is the dual norm of the l p norm, i.e., 1 p + 1 q = 1 .
Based on Theorem 1, when (cid:15) < d c , PEC has the same robustness guarantees as KW (Kolter & Wong, 2017),
Fast-Lin (Weng et al., 2018) and CROWN (Zhang et al., 2018) if we use the same model linearization method.
When 0 <
d c < (cid:15) , KW / Fast-Lin / CROWN cannot certify the data point at all, while PEC still gives non-trivial robustness guarantees thanks to the geometric interpretability of the polyhedral envelope.
Figure 1(a) compares the certiﬁed bounds of KW / Fast-Lin and PEC on a randomly picked input for different values of (cid:15) .
Figure 1(b) shows a 2D sketch of the two cases mentioned above.
If (cid:15) is too small, as in the left portion of the ﬁgure, the linear bounds in (3) are tight but only valid in a small region S ( p ) (cid:15) ( x ) .
Therefore, the certiﬁed robustness is (cid:15) at most.
If (cid:15) is too large, the linear bounds are valid in a larger region but more pessimistic because of linear approximation.
1 This is depicted by the right portion of the ﬁgure, where the distances between the input and the hyperplanes are smaller and the certiﬁed robustness is then d c . The hyperplane segments inside the adversarial budget (green lines) never exceed the decision boundary (dark blue lines), by deﬁnition of the polyhedral envelope.
[Figure]
C e r t i ﬁ e d
B o und
PECKW
[Figure]
To search for the optimal (cid:15) , i.e., the peak in Figure 1(a), Kolter & Wong (2017) use Newton’s method, which is an expensive second-order method.
By contrast, Weng et al.
; Zhang et al.
(2018) use binary search, and our PEC can accelerate their strategy.
The acceleration comes from the partial credit when 0 < d c < (cid:15) , in which case we can rule out all values of (cid:15) below d c and obtain a better lower bound of the optimal (cid:15) .
A more detailed discussion is provided in Appendix B.1.
Under many circumstances, the input is constrained in a hypercube [ r ( min ) , r ( max ) ] n 1 .
For example, for images with normalized pixel values, an attacker should not perturb the image out of the hypercube [0 , 1] n 1 .
Because of this constraint for the attacker, the certiﬁed regions become larger.
To obtain robustness guarantees in this scenario, we need to recalculate d ic , which is now the distance between the input and the hyperplanes in (3) within the hypercube.
The value of d ic then is the minimum of the optimization problem
[Equation]
where, to simplify the notation, we deﬁne a = U i ,
b = U i x + p , ∆ ( min ) =
r ( min ) − x and ∆ ( max ) =
r ( max ) − x .
When b ≤ 0 , the minimum is obviously 0 as the optimal ∆ is an all-zero vector.
In this case, either we cannot certify the input at all or even the clean input is misclassiﬁed.
When b > 0 , by H ¨ older’s inequality, a ∆ + b ≥ −(cid:107) ∆ (cid:107) p (cid:107) a (cid:107) q + b , with equality reached when ∆ p and a q are collinear.
Based on this, the optimal ∆ of minimum l p norm to satisfy a ∆ + b ≤ 0 is
[Equation]
where sign ( · ) returns +1 for positive numbers and − 1 for negative numbers.
To satisfy the constraint ∆ ( min ) ≤ ∆ ≤ ∆ ( max ) , we use a greedy algorithm that approaches this goal progressively.
That is, we ﬁrst calculate the optimal (cid:98) ∆ based on Equation (5) and check if the constraint ∆ ( min ) ≤ ∆ ≤ ∆ ( max ) is satisﬁed.
For the elements where it is not, we clip their values within [∆ ( min ) , ∆ ( max ) ] and keep them ﬁxed.
We then optimize the remaining elements of ∆ in the next iteration and repeat this process until the constraint is satisﬁed for all elements.
The pseudo-code is provided as Algorithm 2 in Appendix B.2.
The output of Algorithm 2 is the optimum of Problem (4), i.e., d ic .
We provide the proof in Appendix C.2.
The certiﬁed bound in this constrained case is then min { (cid:15), d c } just as demonstrated by Theorem 1.
3. G EOMETRY -I NSPIRED R EGULARIZATION
As in (Croce et al., 2018), we can incorporate our certiﬁcation bounds in the training process so as to obtain more robust models.
To this end, we design a regularization term that encourages larger values of d c .
We ﬁrst introduce the signed distance ˜ d
ic : when d ic > 0 , the clean input satisﬁes (3) and ˜ d ic = d ic ; when d ic = 0 , the clean input does not satisfy (3) and there is no certiﬁed region; ˜ d ic in this case is a negative number whose absolute value is the distance between the input and the hyperplane deﬁned in (3).
If the input is unconstrained, we have ˜ d
ic
=
U
i x + p i (cid:107) U i (cid:107) q .
Otherwise, following the notation of (4), ˜ d ic = sign ( b ) (cid:107) (cid:98) ∆ (cid:107) p , where (cid:98)
[Equation]
=
[Equation]
This problem can be solved by a greedy algorithm similar to the one in Section 3.1.
Now, we sort { ˜ d ic } K − 1 i =0 ,i (cid:54) =
c as ˜ d j 0 c ≤ ˜ d j 1 c ≤ ... ≤ ˜ d j K
[Equation]
3 c ≤ ˜
d
j K − 2 c
and then deﬁne the Polyhedral
Envelope Regularization (PER) term, based on smallest T distances, as
[Equation]
Note that, following Croce et al.
(2018), to accelerate training, we take into account the smallest T distances.
When ˜ d j i c ≥ α , the distance is considered big enough, so the corresponding term will not contribute to the gradient of the model parameters.
This avoids over-regularization and allows us to maintain accuracy on clean inputs.
In practice, we do not activate PER in the early training stages, when the model is not well trained and the corresponding polyhedral envelope is meaningless.
Such ‘warm up’ trick is commonly used in the deep learning practice (Gotmare et al., 2018).
Note that we can further incorporate PER with adversarial training in a similar way to Croce et al.
The only change w.r.t.
the original PER consists of replacing ˜ d
j i c with ˜ d (cid:48)
j i c , the distance between the polyhedral envelope and the adversarial example generated by PGD (Madry et al., 2017) instead of the clean input.
We call the corresponding method PER+at .
Calculating the polyhedral envelope is expensive in terms of both computation and memory because of the need to obtain linear bounds of the output logits.
To prevent a prohibitive computational and memory overhead, we use the stochastic robust approximation of Wang et al.
(2018a).
For a mini-batch of size B , we only calculate the PER or PER+at regularization term for B (cid:48) <
B instances sub-sampled from this mini-batch.
Moosavi-Dezfooli et al.
(2017) empirically points out the geometric correlation of high-dimensional decision boundaries near the data manifold.
Thus, in practice, a B (cid:48) much smaller than B provides a good approximation of the full-batch regularization.
E XPERIMENTS
To validate the theory and algorithms above, we conducted several experiments on two popular image classiﬁcation benchmarks: MNIST and CIFAR10.
Each of these experiments can be completed on a single GPU machine within hours.
We will make the code and checkpoints publicly available.
4. T RAINING AND C ERTIFYING R E LU N ETWORKS
As the main experiment, we ﬁrst demonstrate the beneﬁts of our approach over the state-of-the-art training and certiﬁcation methods.
To this end, we use the same model architectures as in Croce et al.
; Kolter & Wong (2017): FC1 , which is a fully-connected network with one hidden layer of 1024 neurons; and CNN , which has two convolutional layers followed by two fully-connected layers.
For this set of experiments, all activation functions are ReLU.
We focus on training and certiﬁcation method based on model linearization and compare 7 different training schemes using 5 evaluation metrics under the same and ﬁxed adversarial budgets.
All the certiﬁcation results here are one-shot results as the binary search for optimal (cid:15) is not used.
Thetraining schemes are plain training (plain), adversarial training (at) (Madry et al., 2017), KW (Kolter & Wong, 2017), MMR (Croce et al., 2018), MMR plus adversarial training (MMR+at)
(Croce et al., 2018), and our PER and PER+at of Section 3.2.
For certiﬁcation, we compare Fast-Lin (Weng et al., 2018), KW (Kolter & Wong, 2017) and our PEC introduced in Section 3.1.
KW is algorithmically equivalent to Fast-Lin with elision of the last layer (Salman et al., 2019b), so we consider them as one certiﬁcation method.
Based on the discussions in Section 3.1, all three methods have the same certiﬁed robust error, which is the proportion of the inputs whose certiﬁed regions are smaller than the adversarial budget.
Therefore, we report the clean test error (CTE), empirical robust error against PGD (PGD), the certiﬁed robust error (CRE Lin), the average certiﬁed bound by Fast-Lin / KW (ACB
[Table] KW) and by PEC (ACB PEC) on the test set.
We do not report the certiﬁed bound by MMR (Croce et al., 2018), because, in practice, it only gives trivial results.
As a matter of fact, Croce et al.
(2018) emphasize their training method and report certiﬁcation results using KW and MIP (Tjeng et al., 2017).
For methods that are not based on model linearization, such as IBP (Gowal et al., 2018) and CROWN-IBP (Zhang et al., 2019), we put more results and discussions in Table 5 of Appendix D.2.1.
We downloaded the KW, MMR and MMR+at models from the checkpoints provided by Croce et al.
To be fair, we use the same model linearization method for our method.
For CNN models, we use the warm up trick consisting of performing adversarial training before adding our PER or
PER+at regularization term.
The running time overhead of pre-training is negligible compared with computing the regularization term.
We train all models for 100 epochs and provide the detailed hyper-parameter settings in Appendix D.1.
We consider robustness in both the l ∞ and l 2 case, and the attackers are constrained to perturb the images within [0 , 1] n 1 .
Table 1 shows the results with the same adversarial budget used in Croce et al.
Consistently with Section 3.1, our geometry-inspired PEC has better average certiﬁed bounds than Fast-Lin / KW given the same adversarial budget.
For CIFAR10 model against l ∞ attack, there are 10% − 20% test points not certiﬁed by Fast-Lin / KW but have non-trivial bounds by PEC.
Figure 5 of Appendix D.2.1 shows the distribution of these certiﬁed bounds.
Among the training methods, PER / PER+at give better performance in all cases than MMR / MMR+at.
When compared with KW, their certiﬁed robust error is comparable, but PER+at has better clean test accuracy, particularly on CIFAR10.
In fact, PER+at maintains a similar clean test error to adversarial training (at) while achieving competitive certiﬁed robust error.
In other words, a model trained by PER+at is not as over-regularized as other training methods for provable robustness. Figure 6 of
Appendix D.2.1 shows the parameter value distribution of models trained by KW/MMR+at/PER+at.
It is clear that parameters of PER+at models have the largest norms and thus better preserve the model capacity.
The better performance of PER+at over PER, and of MMR+at over MMR, evidences the beneﬁts of augmenting the training data with adversarial examples.
[Figure]
To demonstrate the trade-off between clean test error and certiﬁed robust error, we evaluate our approach with different regularizer strength γ in (6).
Figure 2 shows the results of PER+at in the l 2 case for CIFAR10.
When γ is small,
PER+at becomes similar to adversarial training (at) and has low clean test error but high certiﬁed robust error.
As γ grows, the model is increasingly regularized towards large polyhedral envelopes, which inevitably hurts the performance on the clean input.
By contrast, the certiﬁed robust error ﬁrst decreases and then increases.
This is because training is numerically more difﬁcult when γ is too large and the model is over-regularized.
The results of KW are shown as horizontal dashed lines for comparison, which evidences that PER+at is less over-regularized in general than KW, with much lower clean test error for the same certiﬁed robust error.
The experimental results of PER / PER+at in Table 1 rely on the same model linearization methods as Fast-Lin / KW.
Since our framework is applicable to any linearization strategy, we present the results using IBP-inspired linearization in Table 6 of Appendix D.2.2.
Compared with Table 1, we ﬁnd IBP-inspired linearization has competitive performance during training, although it leads to a looser polyhedral envelope.
In addition, IBP-inspired linearization has less memory consumption and computational complexity, so it is suitable for machines of limited computational resources.
4. T RAINING AND C ERTIFYING N ON -R E LU N ETWORKS
To validate our method’s applicability to non-ReLU networks, we replace the ReLU function in FC with either sigmoid, or tanh, or arctan function.
While Wong et al.
(2018) claim that their methods apply to non-ReLU networks, their main contribution is rather the extension of KW to a broader set of network architectures, and their public code 3 does not support non-ReLU activations.
Instead, we thus compare the average certiﬁed bound of CROWN (Zhang et al., 2018)
(ACB CRO), which generalizes Fast-Lin to non-ReLU functions, with our PEC (ACB PEC).
Since any linearization of activation functions can be plugged in both methods 4 , we use the method in Appendix A.1.
CTE(%)
PGD(%) CRELin (%)
ACBCRO
ACBPEC
CTE(%)
PGD(%)
CRELin (%)
ACBCRO
ACBPEC
MNIST - FC1, Sigmoid, l ∞ , (cid:15) = 0 .
MNIST - FC1, Sigmoid, l 2 , (cid:15) = 0 .plain 2.
0.2539 at 1.
0.2839 PER 2.
0.2849 PER+at 2.
MNIST - FC1, Tanh, l ∞ , (cid:15) = 0 .
MNIST - FC1, Tanh, l 2 , (cid:15) = 0 .plain 2.
0.1789 at 1.
0.2735 PER 2. PER+at 2.
MNIST - FC1, ArcTan, l ∞ , (cid:15) = 0 .
MNIST - FC1, ArcTan, l 2 , (cid:15) = 0 .plain 1.
0.1869 at 1.
0.2784 PER 2. 0.2828 PER+at 2. According to Table 2, in contrast to plain training and adversarial training, our PER / PER+at can train provably robust models, despite a slightly lower performance on the clean test data.
For certiﬁcation, PEC has better average certiﬁed bounds than CROWN given the same adversarial budget.
3 Repository: https://github.com/locuslab/convex adversarial
For certiﬁcation, there is no need to require an analytical form
4. F INDING THE O PTIMAL B UDGET
To obtain the biggest certiﬁed bound, we need to search for the optimal value of (cid:15) , i.e., the peak in
Figure 1(a).
While Kolter & Wong (2017) use Newton’s method to solve a constrained optimization problem, which is expensive, Fast-Lin / CROWN (Weng et al., 2018; Zhang et al., 2018) apply a binary search strategy to ﬁnd the optimal (cid:15) .
To validate the claim in Section 3.1 that PEC can ﬁnd the optimal (cid:15) faster than Fast-Lin / CROWN, we compare the average number of iterations needed to ﬁnd the optimal value given a required precision (cid:15) ∆ .
Using (cid:15) and ¯ (cid:15) to deﬁne the initial lower and upper estimates of the optimal value, then we need (cid:100) log 2 ¯ (cid:15) − (cid:15) (cid:15) ∆ (cid:101) steps of bound calculation to obtain the optimal value by binary search in Fast-Lin /
CROWN.
By contrast, the number of iterations needed by PEC depends on the model to certify.
The pseudo-code of the search algorithm is provided in Appendix B.1.
We brieﬂy discuss the experimental results here, and defer their details to Appendix D.2.3.
Note that, because PEC has almost no computational overhead compared with Fast-Lin and CROWN, the number of iterations reﬂects the running time to obtain the optimal certiﬁed bounds.
Altogether, our results show that PEC can save on average 25% of the running time for FC1 models with ReLU, 15% for FC1 models with non-ReLU activations and 10% for CNN models.
D ISCUSSION
Let us consider an N -layer neural network model with k -dimensional output and m -dimensional input.
For simplicity, let each hidden layer have n neurons and n (cid:29) max { k, m } .
In this context, the bounding algorithm based on Fast-Lin / CROWN need O ( N 2 n 3 ) FLOPs to obtain linear bounds of output logits.
However, the complexity can be reduced to O ( Nn 2 m ) at the cost of bound tightness when we use the IBP-inspired algorithm.
In MMR (Croce et al., 2018), the complexity to calculate the expression of the input’s linear region is also O
( Nn 2 m ) .
On the training side, KW needs a back-propagation to calculate the bound, which costs O ( Nn 2 ) .
MMR needs to calculate the distance between the input and O ( Nn ) hyper-planes, costing O ( Nnm ) FLOPs, while PER only calculates
O ( k ) hyperplanes, thus requiring O ( km ) FLOPs.
Overall, we can see that the estimation of the output logits or decision boundary dominates the complexity of all training algorithms and is the main barrier towards scalable provably robust training.
The complexity of PER is similar to Fast-Lin / CROWN if we use the same model linearization method as theirs.
When we use an IBP-inspired algorithm instead, the complexity of PER is similar to MMR.
Note that the FLOP complexity of PGD with h iterations is O ( Nn 2 h ) and typically h (cid:28) min { m, n } , so the overhead of the warm up phase is negligible for PER training.
No matter which linearization method we use, the bounds of the output logits inevitably become looser for deeper networks, which can be a problem for large models.
Furthermore, the linear approximation implicitly favors the l ∞ norm over other l p norms because the intermediate bounds are calculated in an elementwise manner (Liu et al., 2019).
As a result, our method performs better in l ∞ cases than in l 2 cases.
Designing a training algorithm with scalable and tight certiﬁed robustness is highly non-trivial and worth further exploration.
C ONCLUSION
In this paper, we have studied the robustness of neural networks from a geometric perspective.
In our framework, linear bounds are estimated for the model’s output under an adversarial budget.
Then, the polyhedral envelope resulting from the linear bounds allows us to obtain quantitative robustness guarantees.
Our certiﬁcation method can give non-trivial robustness guarantees to more data points than existing methods.
Furthermore, we have shown that our certiﬁed bounds can be turned into a geometry-inspired regularization scheme that enables training provably robust models.
Compared with existing methods, our proposed framework can be applied to neural networks with general activation functions.
Unlike many over-regularized methods, it can achieve provable robustness at very little loss in clean accuracy.
Extending this framework to larger networks will be the focus of our future research.
A M ODEL L INEARIZATION A. L INEARIZATION OF A CTIVATION F UNCTIONS
In this section, we discuss the choice of d , l and h in the linear approximation dx + l ≤ σ ( x ) ≤ dx + h for activation function σ when x ∈ [ x, ¯ x ] .
Method used here is slightly different from Zhang et al.
First, the slope of the linear upper and lower bound is the same for simplicity.
Second, all coefﬁcients need to have analytical form because we need to calculate the gradient based on them during training.
Note that Zhang et al.
(2018) use binary search for optimal d , l , h .
A.1. R E LU
As Figure 3 shows, the linear approximation for ReLU σ ( x ) = max(0 , x ) , which is convex, is:
[Equation]
[Figure]
A.1. S IGMOID , T ANH , A RCTAN
Unlike ReLU function, Tanh function σ ( x ) =
e 2 x − [Equation] 2 x +1 , Sigmoid function σ ( x ) =
[Figure]
and Arctan function σ ( x ) =
arctan ( x ) are not convex.
However, these three functions are convex when x <and concave when x > 0 .
(Left and right sub-ﬁgures of Figure 4)
Therefore, when x ≤ ¯ x ≤ 0 or 0 ≤ x ≤ ¯ x , we can easily obtain the tight linear approximation.
When x ≤ 0 ≤ ¯ x , we do not use the binary research to obtain the tight linear approximation like Zhang et al.
(2018), because the results would not have an analytical form in this way.
Instead, we ﬁrst calculate the slope between the two ends i.e. d = σ (¯ x ) − σ ( x ) ¯ x − x .
Then, we bound the function by two tangent lines of the same slope as d (middle sub-ﬁgure of Figure 4).
[Figure]
we can calculate
[Equation]
The coefﬁcient t 1 < 0 < t 2 are the position of tangent points on both sides of origin.
The deﬁnitions of t 1 and t 2 for different activation functions provided in Table 3.
[Table]
[Equation]
[Table]
[Equation]
[Table]
[Figure]
A. B OUNDS BASED ON F AST -L IN / CROWN
Based on the linear approximation of activation functions above, we have D ( i ) z (cid:48)
( i ) +
l ( i ) ≤ σ ( z (cid:48)
( i ) ) ≤
D ( i ) z (cid:48)
( i ) + u ( i ) where D ( i ) is a diagonal matrix and l ( i ) , u ( i ) are vectors.
We can re-write this formulation as follows:
[Equation]
We plug (9) into (1) and the expression of z (cid:48) ( i ) can be rewritten as
[Equation]
This is a linear function w.r.t x (cid:48) and { m ( j ) } i − 1 j =2 .
Once given the perturbation budget S ( p ) (cid:15) ( x ) and the bounds of { m ( j ) } i − 1 j =2 , we can calculate the bounds of z (cid:48)
( i ) and the bias term in (10).
This process can be repeated until we obtain the bound of output logits (2).
The derivation here is the same as Weng et al.
Liu et al.
(2019), we encourage interested readers to check these works for details.
A
. B OUNDS I NSPIRED BY I NTERVAL B OUND P ROPAGATION
Interval Bound Propagation (IBP) introduced in Gowal et al.
(2018) is a simple and scalable method to estimate the bounds of each layer in neural networks.
IBP is much faster than algorithm introduced in Appendix A.2 because the bounds of any intermediate layer are calculated only based on the information of its immediate previous layer.
Therefore, the bounds are propagated just like the inference of network models, which cost only O ( N ) matrix-vector multiplications for a N -layer network deﬁned in (1).
In our work, we need linear bounds of output logits in addition to general numeric bounds, so the linearization of activation functions deﬁned in (9) is necessary.
We deﬁne linear bounds U ( i ) x (cid:48)
+ p ( i ) ≤ z (cid:48) ( i ) ≤ U ( i ) x (cid:48) + q ( i ) , (cid:98) U ( i ) x (cid:48)
+ ˆ p ( i ) ≤ ˆ z (cid:48) ( i ) ≤ (cid:98) U ( i ) x (cid:48) + ˆ q ( i ) .
According to Section A.1, we use the same slope to linearize activation functions, so the slopes of both bounds here are the same.
Plug (9) into this formulation, we have:
[Equation]
Here, we assume activation functions are monotonic increasing, so elements in D ( i ) are non-negative.
Similarity, compare the linear bounds of ˆ z (cid:48)
( i ) and z (cid:48)
( i +1) , we have:
[Equation]
By deﬁnition, we have (cid:98) U (1)
=
I and ˆ p (1) =
ˆ q (1) = 0 .
Apply (11) and (12) iteratively, we can obtain the values of coefﬁcients U ( N ) , V ( N ) , p ( N ) and q ( N ) in (2).
B A LGORITHMS B. A LGORITHMS FOR SEARCH OPTIMAL (cid:15)
The pseudo code for ﬁnding optimal (cid:15) is demonstrated as Algorithm 1 below.
M , x , (cid:15) ∆ , (cid:15) , ¯ (cid:15) represent the classiﬁcation model, the input point, precision requirement, predeﬁned estimate of lower bound and upper bound respectively.
Typically, (cid:15) is set 0 and ¯ (cid:15) is set a large value to which degree the image perturbation is perceptible.
f is a function mapping a model, an input point and a value of (cid:15) to a certiﬁed bound.
f is a generalization form of algorithm Fast-Lin, CROWN, PEC etc.
During the search for optimal (cid:15) , the lower bound is updated by current certiﬁed bound, while the upper bound is updated only when current certiﬁed bound is smaller than the choice of (cid:15) .
In Fast-Lin and CROWN, we update either lower or upper bound in one iteration since the certiﬁed bound is either 0 or current choice of (cid:15) .
However, it is possible for PEC to update both lower and upper bound in one iteration, which leads to a faster convergence of (cid:15) . Algorithm
Input: x , (cid:15) , ¯ (cid:15) , (cid:15) ∆ , f , M
Set the bounds of (cid:15) : (cid:15) up = ¯ (cid:15) , (cid:15) low = (cid:15) ; while (cid:15) up − (cid:15) low > (cid:15) ∆ do (cid:15) try = 12 ( (cid:15) low + (cid:15) up ) ; (cid:15) cert = f
( M , x , (cid:15) try ) ; Update lower bound: (cid:15) low = max { (cid:15) low , (cid:15) cert } ; if (cid:15) try > (cid:15) cert then
Update upper bound: ¯ (cid:15)
=
(cid:15) try ; end end
Output: 12 ( (cid:15) low + (cid:15) up )
B. G REEDY ALGORITHM IN C ONSTRAINED C ASES
We provide the pseudo code of greedy algorithm to solve problem (4) as Algorithm 2 below.
To prevent prohibitive computational overhead, we set the maximum iteration number I ( max ) of the while-loop.
If this maximum number is met and the iteration is broken, then the output is a lower bound of the solution in problem (4) and thus still a valid but suboptimal robustness guarantee.
In practice, we set the maximum iteration number to be 20, although the while-loop breaks withiniterations in almost all cases.
C P ROOFS C. P ROOF OF T HEOREM
Proof.
We let x (cid:48)
= x + ∆ be a point to break the condition (3).
not
Set
Output: (cid:107) (cid:98) ∆ (cid:107) p
[Equation]
The = ⇒ comes from the H ¨ older’s inequality.
(13) indicates a perturbation of l p norm over d ic = max (cid:110) 0 ,
U i x
+ p i (cid:107) U i (cid:107) q (cid:111) is needed to break the sufﬁcient condition of z (cid:48)
( N ) c − z (cid:48)
( N ) i ≥ 0 .
Based on the assumption of adversarial budget S ( p ) (cid:15) ( x ) when linearizing the model, so the l p norm of a perturbation to produce an adversarial example is at least min { (cid:15), d c } .
C. P ROOF OF THE O PTIMALITY OF G EEDY A LGORITHM
Proof.
We use primal-dual method to solve the optimization problem (4), which is a convex optimization problem with linear constraints.
It is clear that there exists an image inside the allowable pixel space which makes the model predict wrong label.
That is to say, the constrained problem (4) is strictly feasible:
[Equation]
[Equation]
Thus, this convex optimization problem satisﬁes Slater’s Condition i.e. strong duality holds.
We then rewrite the primal problem as:
[Equation]
[Equation] We minimize (cid:107) ∆ (cid:107) pp instead of directly (cid:107) ∆ (cid:107) p in order to decouple all elements in vector ∆ .
In addition, we consider ∆ ( min ) ≤ ∆ ≤ ∆ ( max ) as the domain of ∆ instead of constraints for simplicity.
We write the dual problem of (15) by introducing a coefﬁcient of relaxation λ ∈ R + .
[Equation]
To solve the inner minimization problem, we let the gradient ∂g (∆ ,λ ) ∂
∆ i = sign (∆ i ) p | ∆ i
| p − 1 + λ a i to be zero and obtain ∆ i = − sign ( a i ) (cid:12)(cid:12)(cid:12) λ a i p (cid:12)(cid:12)(cid:12) 1 p − 1 .
Based on the convexity of function g (∆ , λ ) w.r.t.
∆ , we can obtain the optimal ˜∆ i in the domain.
[Equation]
i [Equation] Based on the strong duality, we can say the optimal ˜∆ is chosen by setting a proper value of λ .
Fortunately, (cid:107) ˜∆ (cid:107) p increases monotonically with λ , so the smallest λ corresponds to the optimum.
As we can see, the expression of (cid:98) ∆ in (5) is consistent with ˜∆ i in (17) if λ is set properly.
5 Greedy algorithm in Algorithm 2 is the process to gradually increase λ to ﬁnd the smallest value satisfying constraint a ∆ + b ≤ 0 .
With the increase of λ , elements in vector ∆ remain unchanged when they reach either ∆ ( min ) or ∆ ( max ) , so we keep such elements ﬁxed and optimize the others.
D A DDITIONAL E XPERIMENTS D. D ETAILS OF E XPERIMENTS D.1. M ODEL A RCHITECTURE
FC1 and CNN network used in this paper is identical to ones used in Croce et al.
FC network is a fully-connected network with one hidden layer of 1024 neurons.
CNN network has two convolutional layers and one additional hidden layer before the output layer.
Both convolutional layers have a kernel size of 4, stride of 2 and padding of 1 on both sides, so the height and width of feature maps are halved after each convolutional layer.
The ﬁrst convolutional layer has 32 channels while the second one has 16.
The hidden layer following convolutional layers has 100 neurons.
D.1. H YPER - PARAMETER S ETTINGS
In all experiments, we use Adam optimizer (Kingma & Ba, 2014) with the initial learning rate 10 − 3 and train all models for 100 epochs with a mini-batch of 100 instances.
For CNN models, we decrease the learning rate to 10 − 4 for last 10 epochs.
When we train CNN models on MNIST, we only calculate the polyhedral envelope of 20 instances subsampled from each mini-batch.
When we train CNN models on CIFAR10, this subsampling number is 10 .
For PER and PER+at, the value of
T is always 4 .
We search in the logarithmic scale for the value of γ and in the linear scale for the value of α .
For the value of (cid:15) , we ensure its ﬁnal values are close to the ones used in the adversarial budget S ( p ) (cid:15) ( x ) .
We compare the constant values with exponential growth scheme for (cid:15) but always use constant values for α and γ .
The optimal value we ﬁnd for different settings are provided as
Table 4 below.
[Table]
D. E XTRA E XPERIMENT R ESULTS D.2. A DDITIONAL R ESULTS OF THE M AIN E XPERIMENT
We show the additional results which include IBP (Gowal et al., 2018) and CROWN-IBP (CIBP) (Zhang et al., 2019) in Table 5.
We also report the certiﬁed robust error by IBP (CRE IBP) and corresponding average certiﬁed bound (ACB IBP).
Consistent with Zhang et al.
[Table]
we ﬁnd IBP has non-trivial certiﬁcation results only on models that are trained by IBP or CROWN-IBP.
For these models, IBP has even better certiﬁcation results than Fast-Lin / KW / PEC, whose complexity is signiﬁcantly higher.
As mentioned in Liu et al.
[Table]
IBP and Fast-Lin are actually complementary.
Compared with IBP and CROWN-IBP, our proposed PER and PER+at are still better.
[Equation]
CREIBP (%)
ACBKW
ACBIBP
ACBPEC
CTE(%)
PGD(
[Table]
CRELin (%)
CREIBP (
[Table]
ACBKW
ACBIBP
ACBPEC
MNIST - FC1, ReLU, l ∞ , (cid:15)
= 0 .
MNIST -
[Table]
l
[Table]
, (cid:15)
[Table]
plain
[Table]
[Table]
[Table]
[Table]
[Table]
MMR+at 1.
[Table]
[Table]
0.2780 PER 1.
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]
2882 PER+at 0.
[Table]
[Table]
[Table]
[Table]
CIFAR10 - CNN,
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]
plain 24.
[Table]
[Table]
[Table]
at 27.
[Table]
[Table]
[Table]
[Table]
KW 39.
[Table]
[Table]
IBP 46.
[Table]
[Table]
[Table]
[Table]
IBP 58.
[Table]
The distribution of certiﬁed bound of CIFAR10 model against l ∞ attack are shown in Figure 6.
We only consider the certiﬁed bounds strictly larger than 0 but smaller than (cid:15) , which are non-trivial bound
[Figure]
[Table]
not obtained by Fast-Lin / KW.
For CIFAR10 test set, there are about 10% − 20% such points and their certiﬁed bounds are close to uniform between 0 and (cid:15) .
We show the histogram of parameter values for CIFAR10 models trained by KW, MMR+at and
PER+at.
It is clear that the parameter values of KW models are much more concentrated aroundthan PER+at.
In both l ∞ and l 2 attack cases, the PER+at model has the biggest norms and thus better uses the model capacity.
D.2. T RAINING BY IBP-I NSPIRED M ODEL L INEARIZATION
We use ReLU network cases to compare the performance of IBP-inspired model linearization and the ones used in Fast-Lin / CROWN.
We ﬁnd the hyper-parameter settings in Table 4 do not work here.
Since the bounds estimation of output logits is looser, we ﬁnd the gradient of the PER term based on
IBP-inspired linearization is larger than the main experiments.
As a result, the value of γ used isor 10 times smaller than ones in Table 4.
In addition, starting with large (cid:15) might not work, because [Figure] the bounds will be further looser under a big adversarial budget.
So, we adopt a ‘warm up’ scheme: increase the value of (cid:15) linearly until it reaches the desired value in the end.
We show the results in Table 6 below.
Note that, we still use the tighter linearization method as one in Fast-Lin / CROWN when we run our certiﬁcation method (PEC).
Despite looser polyhedral envelope during training, the IBP-inspired model linearization has competitive performance with ones in Fast-Lin / CROWN.
In some cases, it has even better performance on the test set.
Because of lower computational complexity and less memory consumption, the IBP-inspired method is suitable for machines of limited computational resources. CTE(%) PGD(%) Change(%)
CRE(%) Change(%) ACBKW Change
ACBPEC
Change
MNIST - FC1, ReLU, l ∞ .
(cid:15) = 0 .
PER 1.
+0.
PER+at 1.
+0.
+0.
MNIST - CNN, ReLU, l ∞ .
(cid:15) = 0 .
PER 1.
+0.0005 0.
+0.0002 PER+at 0.
+0.0002 0.
+0.
CIFAR10 - CNN, ReLU, l ∞ .
(cid:15)
= 2 /
PER 29.
+0.
+7.
-0.0002 PER+at 26.
+1.
MNIST - FC1, ReLU, l 2 , (cid:15)
= 0 .
PER 1.
+0.
0.0000 PER+at 0.
+0.
+2.
MNIST - CNN, ReLU, l 2 , (cid:15)
= 0 .
PER 1.
+0.
+1.
PER+at 0.
+0.
+2.
CIFAR10 -
[Table]
D.2. S EARCH FOR O PTIMAL V ALUE OF (cid:15)
Table 7 and 8 show the number of bound calculations in binary search for optimal (cid:15) in PEC and FastLin / CROWN.
In l ∞ cases, the original interval [ (cid:15), ¯ (cid:15) ] is [0 , 0 . 4] for MNIST and [0 , 0 .
1] for CIFAR10.
In l 2 cases, the original interval [ (cid:15), ¯ (cid:15) ] is [0 , 1 . 2] for MNIST and [0 , 0 . 4] for CIFAR10.
The bound of number calculation does not depend on the model in Fast-Lin / CROWN and is model-dependent in
PEC as discussed in Section 4.3.
[Table] [Table]