E XPECTED T IGHT B OUNDS FOR R OBUST D EEP N EURAL N ETWORK T RAINING
Anonymous authors
Paper under double-blind review
A BSTRACT
Training Deep Neural Networks (DNNs) that are robust to norm bounded adversarial attacks remains an elusive problem.
While veriﬁcation based methods are generally too expensive to robustly train large networks, it was demonstrated by
Gowal et al.
(2019) that bounded input intervals can be inexpensively propagated from layer to layer through deep networks.
This interval bound propagation (IBP) approach led to high robustness and was the ﬁrst to be employed on large networks.
However, due to the very loose nature of the IBP bounds, particularly for large/deep networks, the required training procedure is complex and involved.
In this paper, we closely examine the bounds of a block of layers composed of an afﬁne layer, followed by a ReLU, followed by another afﬁne layer.
To this end, we propose expected bounds (true bounds in expectation), which are provably tighter than
IBP bounds in expectation.
We then extend this result to deeper networks through blockwise propagation and show that we can achieve orders of magnitudes tighter bounds compared to IBP.
Using these tight bounds, we demonstrate that a simple standard training procedure can achieve impressive robustness-accuracy trade-off across several architectures on both MNIST and CIFAR10.
I NTRODUCTION
Deep neural networks (DNNs) have demonstrated impressive performance in many ﬁelds of research with applications ranging from image classiﬁcation (Krizhevsky et al., 2012;
He et al., 2016) and semantic segmentation (Long et al., 2015) to speech recognition (Hinton et al., 2012), just to name a few.
Despite this success, DNNs are still susceptible to small imperceptible perturbations, which can lead to drastic performance degradation, especially in visual classiﬁcation tasks.
Such perturbations are best known and commonly referred to as adversarial attacks .
Early work showed that simple algorithms ( e.g. maximizing the classiﬁcation loss with respect to the input using a single optimization iteration (Goodfellow et al., 2014)) can easily construct such adversaries.
Since then, a research surge has emerged to develop simple routines to construct adversarial examples consistently.
For instance, Moosavi-Dezfooli et al.
(2016) proposed a simple algorithm, called DeepFool, which ﬁnds the smallest perturbation that fools a linearized version of the network.
Interestingly, the work of
Moosavi-Dezfooli et al.
(2017) demonstrated that such adversaries can be both network and input agnostic, i.e. universal deterministic samples that fool a wide range of DNNs across a large number of input samples.
More recently, it was shown that such adversaries can also be as simple as Gaussian noise (Bibi et al., 2018).
Knowing that DNNs are easily susceptible to simple attacks can hinder the public conﬁdence in them, especially for real-world deployment, e.g. in self-driving cars and devices for the visually impaired.
Such a performance nuisance has prompted several active research directions, in particular, work towards network defense and veriﬁcation.
Network defense aims to train networks that are robust against adversarial attacks through means of robust training or procedures at inference time that dampen the effectiveness of the attack (Madry et al., 2018; Wong & Kolter, 2018; Raghunathan et al., 2018; Alfadly et al., 2019).
On the other hand, veriﬁcation aims to certify/verify for a given DNN that there exists no small perturbation of a given input that can change its output prediction (Katz et al., 2017;
Sankaranarayanan et al., 2016; Weng et al., 2018a).
However, there are also works at the intersection of both often referred to as robustness veriﬁcation methods, which use veriﬁcation methods to train robust networks.
Such algorithms often try to minimize the exact (or upper bound) of the worst adversarial loss over all possible bounded energy (often measured in (cid:96) ∞ norm) perturbation around a given input.
Although veriﬁcation methods prove to be effective in training robust networks (Wong & Kolter, 2018), they are computationally expensive, thus limiting their applicability to only small, at best medium, sized networks.
However, Gowal et al.
(2019) recently demonstrated that robustly training large networks is possible by leveraging the cheap-to-compute but very loose interval-based veriﬁer, known as interval domain from Mirman et al.
In particular, they propagate the (cid:15) - (cid:96) ∞
norm bounded input centered at x ∈ R n , i.e. [ x − (cid:15) 1 n , x + (cid:15) 1 n ] , through every layer in the network at a time.
This interval bound propagation (IBP) is inexpensive and simple; however, it results in very loose output interval bounds, which in turn necessitates a complex and involved training procedure.
Closer to our work, there has been several prior arts that propose to perform veriﬁcation differently.
In particular, Webb et al.
(2018) presents a statistical approach to assessing robustness of neural networks.
As opposed to veriﬁcation methods, which in many cases can be hard to scale, that provide a binary measure of robustness per sample, they propose to frame veriﬁcation as a probability of violation instead.
That is to say, they investigate the probability of failure over a violation rather than conﬁrming that this probability is exactly zero.
Moreover, (Weng et al., 2018b) propose CLEVER which estimates a lower bound to the minimum perturbation rather that ﬁnding the lower bounds exactly.
In both works, the veriﬁcation is tackled by, and closely related to our direction of work, probabilistically estimating the bounds.
In this paper, we are interested in improving the tightness of output interval bounds (referred to as bounds from now on).
We do so by closely examining the bounds for a block of layers composed of an afﬁne layer, followed by a ReLU nonlinearity, followed by another afﬁne layer under (cid:15) - (cid:96) ∞ bounded input.
In fact, we propose new expected bounds for this block of layers, which we prove to be not only supersets to the true bounds of this block in expectation but also very tight to the true bounds.
Lastly, we show how to extend such a result to deeper networks through blockwise bound propagation leading to several orders of magnitude tighter bounds as compared to IBP.
Contributions.
Our contributions are three-fold.
( i )
We propose new bounds for the block of layers composed of an afﬁne layer, followed by a ReLU, followed by another afﬁne layer.
We prove that these bounds are in expectation, under a distribution of network parameters, supersets to the true bounds of this block.
Moreover, we prove that these bounds are much tighter, in expectation (will be formalized later) than the IBP bounds (Gowal et al., 2019) generated by propagating the input bounds through every layer in the block.
Our bounds get even tighter as the number of hidden nodes in the ﬁrst afﬁne layer increases.
( ii ) We show a practical and efﬁcient approach to propagate our bounds (for the block of layers) through blocks (not through individual layers) of a deep network, thus resulting in magnitudes tighter output bounds compared to IBP.
( iii )
We conduct experiments on synthetic networks and on real networks, to verify the theory, as well as the factors of improvement over IBP.
Due to the tightness of our proposed expected bounds, we show that with a simple standard training procedure, large/deep networks can be robustly trained on both MNIST (LeCun, 1998) and CIFAR10 (Krizhevsky & Hinton, 2009) achieving state-of-art robustness-accuracy trade-off compared to IBP.
In other words, we can consistently improve robustness by signiﬁcant margins with minimal effect on test accuracy as compared to IBP.
R ELATED W ORK
Training accurate and robust DNNs remains an elusive problem, since several works have demonstrated that small imperceptible perturbations (adversarial attacks) to the DNN input can drastically affect their performance.
Early works showed that with a very simple algorithm, as simple as maximizing the loss with respect to the input for a single iteration (Goodfellow et al., 2014), one can easily construct such adversaries.
This has strengthened the line of work towards network veriﬁcation for both evaluating network robustness and for robust network training.
In general, veriﬁcation approaches can be coarsely categorized as exact or relaxed veriﬁers.
Exact Veriﬁcation.
Veriﬁers of this type try to ﬁnd the exact largest adversarial loss over all possible bounded energy (usually measured in (cid:96) ∞ norm) perturbations around a given input.
They are often tailored for piecewise linear networks, e.g. networks with ReLU and LeakyReLU nonlinearities.
They typically require mixed integer solvers (Cheng et al., 2017; Lomuscio & Maganti, 2017; Tjeng & Tedrake, 2019) or Satisﬁability Modulo Theory (SMT) solvers (Xiaowei Huang & Wu, 2017;
Ehlers, 2017).
The main advantage of these approaches is that they can reason about exact adversarial robustness; however, they generally are computationally intractable for veriﬁcation purposes let alone any sort of robust network training.
The largest network used for veriﬁcation with such veriﬁers was with the work of Tjeng & Tedrake (2019), which employed a mixed integer solver applied to
Relaxed Veriﬁcation.
Veriﬁers of this type aim to ﬁnd an upper bound on the worst adversarial loss across a range of bounded inputs.
For instance, a general framework called CROWN was proposed by Huan Zhang & Daniel (2018) to certify robustness by bounding the activation with linear and quadratic functions, thus, enabling the study of generic, not necessarily piecewise linear, activation functions.
By utilizing the structure in ReLU based networks, the work of Weng et al.
(2018a) proposed two fast algorithms based on linear approximation on the ReLU units.
Moreover, Wang et al.
(2018c) proposed ReluVal for network veriﬁcation based on symbolic interval bounds, while
Wang et al.
(2018b) proposed Neurify with much tighter bounds.
Several other works utilized the dual view of the veriﬁcation problem (Wong & Kolter, 2018; Wong et al., 2018).
More recently,
Salman et al.
(2019) uniﬁed a large number of recent works in a single convex relaxation framework and revealed several relationships between them.
In particular, it was shown that convex relaxation methods that ﬁt this framework suffer from an inherent barrier compared to exact veriﬁers.
For completeness, it is important to note that there are also hybrid methods that combine both exact and relaxed veriﬁers and have shown to be effective (Rudy Bunel, 2018).
Although relaxed veriﬁers are much more computationally friendly than exact veriﬁers, they are still too expensive for robust training of large/deep networks (with more than 5 hidden layers).
However, very loose relaxed veriﬁers can possibly still be exploited for this purpose.
For instance, Wang et al.
(2018a) leveraged symbolic interval analysis to veriﬁably train large networks.
More recently, the work of Gowal et al.
(2019) proposed to use an inexpensive but very loose interval bound propagation (IBP) certiﬁcate to train (for the ﬁrst time) large robust networks with state-of-the-art robustness performance.
This was at the expense of a complex and involved training routine resulting from the loose nature of the bounds.
To remedy these training difﬁculties, we instead propose expected bounds, not for each layer individually, but for a block of layers jointly.
Such bounds are slightly more expensive to compute but are much tighter in expectation.
We then propagate these bounds through every block in a deeper network to attain much tighter bounds overall as compared to layerwise IBP.
The tighter bounds enable the use of simple standard training routines for robust training of large networks, resulting in state-of-art robustness-accuracy trade-off.
E XPECTED T IGHT I NTERVAL B OUNDS
We analyze the interval bounds of a DNN by proposing expected true and tight bounds for a two-layer network (Afﬁne-ReLU-Afﬁne).
Then, we propose a mechanism to extend them for deeper networks.
First, we detail the interval bounds of Gowal et al.
(2019) to put our bounds in context.
3. I NTERVAL B OUNDS FOR A S INGLE A FFINE L AYER
For a single afﬁne layer parameterized by A 1 ∈ R k × n and b 1 ∈ R k , it is easy to show that its output lower and upper interval bounds for an (cid:15) - (cid:96)
∞ norm bounded input ˜ x ∈ [ x − (cid:15) 1 n , x + (cid:15) 1 n ] are:
[Equation]
Note that | .
| is an elementwise absolute operator.
In the presence of any non-decreasing elementwise nonlinearity ( e.g. ReLU), the bounds can then be propagated by applying the nonlinearity to { l 1 , u 1 } directly.
As such, the interval bounds can be propagated through the network one layer at a time, as proposed by Gowal et al.
While this interval bound propagation (IBP) mechanism is a very simple and inexpensive approach to compute bounds, these bounds can be extremely loose for deep networks, requiring a complex and involved robust network training procedure.
3. P ROPOSED I NTERVAL B OUNDS FOR AN A FFINE -R E LU-A FFINE B LOCK
Here, we consider a block of layers of the form Afﬁne-ReLU-Afﬁne in the presence of (cid:96) ∞ perturbations at the input.
The functional form of this network is: g ( x ) = a (cid:62) 2 max ( A 1 x + b 1 , 0 ) + b 2 , where max( . ) is an elementwise operator.
The afﬁne mappings can be of any size, and throughout the paper, we take A 1 ∈ R k × n and without loss of generality the second afﬁne map is a single vector a 2 ∈ R k .
Note that g also includes convolutional layers, since they are also afﬁne mappings.
Layerwise Interval Bound Propagation (IBP) on g .
Here, we apply the layerwise propagation strategy of Gowal et al.
(2019) detailed in Section 3.1 on function g ( ˜ x ) with ˜ x ∈ [ x − (cid:15) 1 n , x + (cid:15) 1 n ] to obtain bounds [ L IBP , U IBP ] .
We use these bounds for comparison in what follows.
[Equation]
Note that max ( l 1 , 0 k ) and max ( u 1 , 0 k ) are the result of propagating [ x − (cid:15) 1 n , x + (cid:15) 1 n ] through the ﬁrst afﬁne map ( A 1 , b 1 ) and then through the ReLU nonlinearity, as stated in (1).
Expected Tight Interval Bounds on g .
Our goal is to propose new interval bounds for g , as a block, which are tighter than the IBP bounds [ L IBP , U IBP ] , since we believe that tighter bounds for a two-layer block, when propagated/extended to deeper networks, can be tighter than applying
IBP layerwise.
Denoting the true output interval bounds of g as [ L true , U true ] , the following inequality holds L true ≤ g (˜ x ) ≤ U true ∀
˜ x ∈
[ x − (cid:15) 1 n , x + (cid:15) 1 n ] .
Deriving these true (and tight) bounds for g in closed form is either hard or results in bounds that are generally very difﬁcult to compute, deeming them impractical for applications such as robust network training.
Instead, we propose new closed form expressions for the interval bounds denoted as [ L M , U M ] , which we prove to be true bounds and tighter than [ L IBP , U IBP ] in expectation under a distribution of the network parameters Aand a 2 .
As such, we make two main theoretical ﬁndings.
(i)
We prove that L M and U M are true bounds in expectation, i.e. E A 1 , a 2 [ L M ]
≤ E
A 1 , a 2 [ L true ] and E A 1 , a 2 [ U M ] ≥ E A 1 , a 2 [ U true ] hold for a sufﬁciently large input dimension n .
(ii)
We prove that [ L M , U M ] can be arbitrarily tighter than the loose bounds [ L IBP , U IBP ] in expectation, as the number of hidden nodes k increases.
Analysis.
To derive L M and U M , we study the bounds of the following function ﬁrst:
[Equation]
Note that ˜ g is very similar to the Afﬁne-ReLU-Afﬁne map captured by g with the ReLU replaced by a diagonal matrix M constructed as follows.
If we denote u 1 =
A 1 x + b 1 + (cid:15)
|
A 1 | 1 n as the upper bound resulting from the propagation of the input bounds [ x − (cid:15) 1 n , x + (cid:15) 1 n ] through the ﬁrst afﬁne map ( A 1 , b 1 ) , then we have M = diag ( 1 { u 1 ≥ 0 k } ) where 1 is an indicator function.
In other words, M ii = 1 when the i th element of u 1 is non-negative and zero otherwise.
Note that for a given u 1 , ˜ g
(˜ x ) is an afﬁne function with the following output interval bounds for ˜ x ∈ [ x − (cid:15) 1 n , x + (cid:15) 1 n ] :
[Equation]
To compare L M and U M to L true and U true , respectively, and since having access to L true and U true is not feasible, we make the following mild key assumption.
Assumption 1.
(Key Assumption).
Consider an (cid:96) ∞ bounded uniform random variable ˜ x , ˜ x ∈ [ x − (cid:15) 1 n , x + (cid:15) 1 n ] where A 1 and a 2 have elements that are i.i.d.
Gaussian of zero mean and σ Aand σ a 2 standard deviations, then there exists a sufﬁciently large m , such that:
[Equation]
Under Assumption 1 and to show that L M and U M are true bounds in expectation, it is sufﬁcient to show that L approx ≥ E A 1 , a 2 [ L M ] and similarly for U M .
However, since it is generally difﬁcult to compute the right hand sides of Assumption 1, they can be well approximated by Lyapunov Central
Limit Theorem.
More formally, follows the two propositions.
Proposition 1.
For a ∈ R n ∼ N ( 0 , σ 2 a I ) and a uniform random vector ˜ x ∼ U [ x − (cid:15) 1 n , x + (cid:15) 1 n ] where both a and ˜ x are independent, we have that Lyapunov Central Limit Theorem holds such that
[Equation]
where
[Equation] [Equation]
where → d indicates convergence in distribution.
Proposition 2.
For a random matrix A 1 with i.i.d.
Gaussian elements of zero mean and σ A 1 standard deviation and a uniform random vector ˜ x ∼ U [ x − (cid:15) 1 n , x + (cid:15) 1 n ] we have that Covariance ( A ˜ x ) =
( 1 3 (cid:15) 2 σ 2 A 1 n + σ 2 A 1 trace ( xx (cid:62) ))
I .
[Figure]
Following Propositions 1 and 2, we have that for a sufﬁciently large input n :
[Equation]
where the output of the ﬁrst afﬁne layer
A 1 ˜ x + b 1 is approximated by Lyapunov Central Limit
Theorem as ˜ y ∼ N ( b
[Equation]
[Equation] [Equation]
+
[Equation]
trace
[Equation]
Theorem 1.
(True Bounds in Expectation) Let Assumption 1 hold.
For large input dimension n , E
[Equation]
Theorem 1 states that the interval bounds for function ˜
g are simply looser bounds to the function of interest g in expectation under a plausible distribution of A 1 and a 2 .
Now, we investigate the tightness of these bounds as compared to the IBP bounds [ L IBP , U IBP ] .
Theorem 2.
(Tighter Bounds in Expectation) Consider an (cid:96) ∞ bounded uniform random variable input ˜ x ∈ [ x − (cid:15) 1 n , x + (cid:15) 1 n ] to a block of layers in the form of Afﬁne-ReLU-Afﬁne (parameterized by A 1 , b 1 , a 2 and b 2 for the ﬁrst and second afﬁne layers respectively) and a 2 ∼ N ( 0 , σ a 2 I ) .
Under the assumption that
[Equation]
j , we have: E a 2 [( U IBP − L IBP ) − ( U M − L M )] ≥ 0 .
Theorem 2 states that under some assumptions on A 1 and under a plausible distribution for a 2 , our proposed interval width can be much smaller than the IBP interval width, i.e. our proposed intervals are much tighter than the IBP intervals in expectation.
Next, we show that the inequality assumption in Theorem 2 is very mild.
In fact, a wide range of ( A 1 , b 1 ) satisfy it, and the following proposition gives an example that does so in expectation.
Proposition 3.
For a random matrix A 1 ∈ R k × n with i.i.d elements A 1 ( i, j ) ∼ N (0 , 1) , then
[Equation]
Proposition 3 implies that as the number of hidden nodes k increases, the expectation of the right hand side of the inequality assumption in Theorem 2 grows more negative, while the left hand side of the inequality is zero in expectation when b 1 ∼ N ( 0 k , I ) .
In other words, for Gaussian zero-mean weights ( A 1 , b 1 ) and with a large enough number of hidden nodes k , the assumption is satisﬁed.
All proofs and detailed analyses are provided in the appendix .
Comment on the Assumptions on A 1 and a 2 .
Generally speaking, our proposed bounds [ L M , U M ] computed using Equation 3 can be very loose and much worse compared to IBP if the network weights
A 1 , a 2 do not follow the Gaussian assumption.
We leave for the appendix an example to such failure.
However, while the Gaussian i.i.d.
assumption can be strong for general networks trained on real data, it is not far from being reasonable due to commonly accepted training procedures.
This is since it is common to regularize network weights while training deep neural networks with an (cid:96) 2 regularizer encouraging the weights to follow a zero mean Gaussian distribution not to mention that networks in many cases are initialized in such manner.
We dwell on this further in the appendix where we show the histogram of the weights of networks trained on real data (MNIST, CIFAR10, CIFAR100).
with and without (cid:96) 2 regularization See appendix for details.
[Figure] [Figure]
3. E XTENDING O UR E XPECTED T IGHT B OUNDS TO D EEPER N ETWORKS
To extend our proposed bounds to networks deeper than a two-layer block, we simply apply our bound procedure described in Section 3.2, recursively for every block.
In particular, consider an
L -layer neural network deﬁned as f ( x ) =
A L ReLU ( A L − 1 ReLU ( · · · A 2 ReLU ( A 1 x ))) and an (cid:15) - (cid:96) ∞
norm
bounded input centered at x
, i.e. ˜ x ∈ [ x − (cid:15) 1 n , x + (cid:15) 1 n ] .
Without loss of generality, we assume f is bias-free for ease of notation.
Then, the output lower and upper bounds of f are
L M = G L − 1 x − (cid:15) | G L − 1 | 1 n and U M =
G L − 1 x + (cid:15) | G L − 1 | 1 n , respectively.
Here, G L − 1 is a linear map that can be obtained recursively as follows: G i
[Equation]
A i
[Equation]
1 M i G i − 1 ( with G 0 = A 1 ) and M i = diag ( 1 { ( G [Equation] 1 x +
[Equation]
| G [Equation] − 1 | 1 n )
[Equation]
Note that G i − 1 x + (cid:15) | G i − 1 | 1 n is the output upper bound through a linear layer parameterized by
G i − 1 for input ˜ x as in (1).
With this blockwise propagation, the output interval bounds of f are now estimated by the output intervals of ˜ f
(˜ x ) =
G L − 1 ˜ x .
E XPERIMENTS
True Bounds in Expectation.
Here, we validate Theorem 1 with several controlled experiments.
For a network g (˜ x ) =
A 2 max ( A 1 ˜ x + b 1 , 0 ) + b 2 that has true bounds [ L true , U true ] for ˜ x ∈ [ x − (cid:15) 1 n , x + (cid:15) 1 n ] , we empirically show that our proposed bounds [ L M , U M ] , under the mild assumptions of Theorem 1, indeed are true, i.e. they are a super set to [ L true , U true ] in expectation.
We also verify this as a function of the network input dimension n (as predicted by Theorem 1).
We start by constructing a network g where the biases b 1 ∈ R k and b 2 ∈ R are initialized following the default Pytorch initialization (Paszke et al., 2017).
As for the elements of the weight matrices
A 1 ∈ R k × n and A 2 ∈ R 1 × k , they are sampled from N (0 , 1 / √ n ) and N (0 , 1 / √ k ) , respectively.
We estimate L true and U true by taking the minimum and maximum of 10 6 + 2 n Monte-Carlo evaluations of g .
For a given x ∼ N ( 0 n , I ) and with (cid:15) = 0 .
1 , we uniformly sample 10 6 examples from the interval [ x − (cid:15) 1 n , x + (cid:15) 1 n ] .
We also sample all 2 n corners of the hyper cube [ x − (cid:15) 1 n , x + (cid:15) 1 n ] n .
To show that the proposed interval [ L M , U M ] is a super set of [ L true , U true ]
( i.e. they are true bounds), we evaluate the length of the intersection of the two intervals over the length of the true interval deﬁned as Γ = | [ L M , U M ] ∩ [ L true , U true ] | / | [ L true , U true ] | .
Note that Γ = 1 if and only if [ L M , U M ] is a super set of [ L true , U true ] .
For a given n , we conduct this experiment 10 3 times with varying A 1 ,
A 2 , b 1 , b 2 and x
and report the average Γ .
Then, we run this for a varying number of input size n and a varying number of hidden nodes k , as reported in Figure 1a.
As predicted by Theorem 1,
[Table]
Figure 1a demonstrates that as n increases, the proposed interval will be more likely to be a super set of the true interval, regardless of the number of hidden nodes k .
Note that networks that are as wide as k = 1000 , require no more than n = 15 input dimensions for the proposed intervals to be a super set of the true intervals.
In practice, n is much larger than that, e.g. n ≈ 3 × 10 3 in CIFAR10.
In Figure 1b we empirically show that the above behavior also holds for deeper networks.
We propagate the bounds blockwise as discussed in Section 3.3 and conduct similar experiments on fully-connected networks.
We construct networks with varying depth, where each layer has the same number of nodes equal to the input dimension k = n .
These results indeed suggest that the proposed bounds are true bounds and are more likely so with larger input dimensions.
Here, n = 20 performs better than n = 10 across different network depths.
Tighter Bounds in Expectation.
Here, we experimentally afﬁrm that our bounds can be much tighter than IBP bounds (Gowal et al., 2019).
In particular, we validate Theorem 2 by comparing the interval length of our proposed bounds, W M =
U M − L M , with that of IBP, W IBP =
U IBP − L IBP , on networks with functional form g .
We compute both the difference and ratio of widths for varying values of k , n , and (cid:15) .
Figure 2 reports the average width difference and ratio over 10 3 runs in a similar setup to the previous section.
Figures 2a and 2b show that the proposed bounds indeed get tighter than IBP, as k increases across all (cid:15) values (as predicted by Theorem 2).
Note that we show tightness results for (cid:15) = { 0 . 01 , 0 . 1 } in Figure 2b as the performance of (cid:15) = { 0 . 5 , 1 . 0 } was very similar to (cid:15)
= 0 .
Similar improvement occurs with increasing n , as in Figures 2c and 2d.
We also compare the tightness of our bounds to those of IBP with increasing depth for both fullyconnected networks (refer to Figures 3a and 3b) and convolutional networks (refer to Figures 3c and 3d).
For all fully-connected networks, we take n = k = 500 .
Our proposed bounds get consistently tighter as the network depth increases over all choices of (cid:15) .
In particular, the proposed bounds can be more than 10 6 times tighter than IBP for a 10 layer DNN.
A similar observation can also be made for convolutional networks, where it is expensive to compute our bounds using the procedure in
Section 3.3, so instead, we obtain matrices M i using the easy-to-compute IBP upper bounds.
Despite this relaxation, we still obtain very tight expected bounds.
Note that this slightly modiﬁed approach reduces exactly to our bounds for two-layer networks.
True and Tight Bounds on Real Networks.
Moreover, we also train small network, the architecture is similar to Gowal et al.
(2019), on the MNIST dataset ∼ 99% .
To show that our bounds are also true on real networks, and since the input dimension is too large for Monte-Carlo sampling, we use the MIP formulation by Tjeng & Tedrake (2019).
We then report Γ over varying testing (cid:15) .
Tabledemonstrates that indeed even on real networks beyond two layers and without the Gaussian weight distributed assumption, our bounds are still a super set to the true bounds computed with an MIP formulation.
Moreover, Table 1 also demonstrates that our bounds are much tighter than IBP bounds.
Note that the reported results are averaged over 100 random MNIST images.
Qualitative Results.
Following previous work (Wong & Kolter, 2018; Gowal et al., 2019) we visualize examples of the proposed bounds in Figure 4 and compare them to the true ones for several choices of (cid:15) ∈ { 0 . 05 , 0 . 1 , 0 . 25 } and a random ﬁve-layer fully-connected network with architecture n - 100 - 100 - 100 - 100 - 2 .
We also show the results of the Monte-Carlo sampling for an input size n = 20 .
More qualitative visualizations for different values of n are in the appendix .
Training Robust Networks.
Here, we conduct experiments showing that our expected bounds can be used to robustly train DNNs.
We compare our method against models trained nominally ( i e. only the nominal training loss is used), and those trained robustly with IBP (Gowal et al., 2019).
Given the wellknown robustness-accuracy trade off, robust models are often less accurate.
Therefore, we compare all methods using an accuracy vs. robustness scatter plot.
Following prior work, we use Projected
Gradient Descent (PGD) (Madry et al., 2018) to measure robustness.
We use a loss function similar to the one proposed in (Gowal et al., 2019).
In particular, we use L = (cid:96) ( f θ ( x ) , y true ) + κ(cid:96) ( z , y true ) , where (cid:96) , f θ ( x ) , y true , and κ are the cross-entropy loss, output logits, true class label, and regularization hyperparameter respectively.
z represents the “adversarial" logits that combine the lower bound of the true label and the upper bound of all other labels, as in (Gowal et al., 2019).
Nominal training occurs when κ = 0 .
Due to the tightness of our bounds, in contrast to IBP, we follow a standard training procedure that avoids the need to vary κ or (cid:15) train during training.
Speciﬁcally, we train three network models (small, medium, large) provided by Gowal et al.
(2019) on both MNIST and CIFAR10.
See appendix for more details.
Following the same setup in (Gowal et al., 2019), we train all models with (cid:15) train ∈ { 0 . 1 , 0 . 2 , 0 . 3 , 0 . 4 } and (cid:15) train ∈ { 2 / 255 , 8 / 255 , 16 / 255 , 0 . 1 } on MNIST and CIFAR10, respectively.
In all experiments, and for stronger baselines and fair comparison between IBP training and our bounds employed in training, we grid search over { 0 . 1 , 0 . 001 , 0 . 0001 } learning rates and employ a temperature over the logits with a grid of { 1 , 1 / 5 } as in (Hinton et al., 2015) and report the best performing models for both.
Then, we compute PGD robustness for every (cid:15) train of every model for all (cid:15) test ∈ { 0 . 1 , 0 . 2 , 0 . 3 , 0 . 4 } for MNIST and for all (cid:15) test ∈ { 2 / 255 , 8 / 255 , 16 / 255 , 0 . 1 } for CIFAR10.
To compare training methods, we compute the average PGD robustness over all (cid:15) test and the test accuracy, and report them in a 2D scatter plot.
We report the performance results on MNIST and CIFAR10 for the small, medium, and large architectures in Figure 5.
For all trained architectures, we only report the results for those that achieve at least a test accuracy of 97 .
5% and 40% on MNIST and CIFAR10, respectively; otherwise, it is an indication of failure in training.
Interestingly, our training scheme can be used to train all architectures for all (cid:15) train .
This is unlike IBP, which for example was only able to successfully train the large architecture with (cid:15) train = 0 .
1 on MNIST.
Moreover, models trained with our bounds always achieve better PGD robustness than the nominally trained networks on all architectures while preserving similar if not higher accuracy (on large networks).
Models trained with IBP achieve high robustness but their test accuracy is drastically affected.
Several other experiments are left for the appendix .
C ONCLUSION
In this work, we proposed new interval bounds that are tight, relatively cheap to compute, and true in expectation.
We analytically showed that for a Afﬁne-ReLU-Afﬁne block with large input and hidden layer sizes, our bounds are true in expectation and can be several orders of magnitude tighter than the bounds obtained with IBP.
We conduct extensive experiments verifying our theory, even for deep networks.
As a result, we are able to train large models, with simple standard training routines while achieving excellent trade-off between accuracy and robustness.
A M ORE Q UALITATIVE R ESULTS OF THE N EW B OUNDS
We conduct several more experiments to showcase the tightness of our proposed bounds to the true bounds and compared them against propagating the bounds layerwise [ l DM , u DM ] for random n − 100 − 100 − 100 − 100 − 2 networks initialized with N (0 , 1 / √ n ) similar to Wong & Kolter (2018).
We show our bounds compared to the polytobe estimated from MonteCarlo sampling on results for n ∈ { 2 , 10 , 20 } and (cid:15) ∈ { 0 . 05 , 0 . 1 , 0 . 25 } .
The layer wise bound propagation is shown in the tables as the bounds were too loose to be presented visually.
(cid:15) , Figure Number
[Figure]
(cid:15)
= 0 .
05 , Figure Number =-10.
(cid:15)
= 0 .
05 , Figure Number =-12.
(cid:15)
= 0 .
05 , Figure Number =-12.
(cid:15)
= 0 .
05 , Figure Number =-17.
(cid:15)
= 0 .
05 , Figure Number =-12.
(cid:15)
= 0 .
1 , Figure Number =-27.
(cid:15)
= 0 .
1 , Figure Number =-23.
(cid:15)
= 0 .
1 , Figure Number =-35.
(cid:15)
= 0 .
1 , Figure Number =-31.
(cid:15)
= 0 .
1 , Figure Number =-25.
(cid:15)
= 0 .
25 , Figure Number =-54.
(cid:15)
= 0 .
25 , Figure Number =-59.
(cid:15)
= 0 .
25 , Figure Number =-50.
(cid:15)
= 0 .
25 , Figure Number =-52.
(cid:15)
= 0 .
25 , Figure Number =-89.
Note how likely
(cid:15) , Figure Number l 1 DM u 1 DM l 2 DM u 2 DM (cid:15) = 0 .
05 , Figure Number =-16.
(cid:15)
= 0 .
05 , Figure Number =
(cid:15)
= 0 .
05 , Figure Number =-41.
(cid:15)
= 0 .
05 , Figure Number =-32.
(cid:15)
= 0 .
05 , Figure Number =-45.
(cid:15)
= 0 .
1 , Figure Number =-48.
(cid:15)
= 0 .
1 , Figure Number =-51.
(cid:15)
= 0 .
1 , Figure Number =-51.
(cid:15)
= 0 .
1 , Figure Number =-71.
(cid:15)
= 0 .
1 , Figure Number =-48.
(cid:15)
= 0 .
25 , Figure Number =-152.
(cid:15)
= 0 .
25 , Figure Number =-196.
[Table]
[Table] [Table]
(cid:15)
= 0 .
25 , Figure Number =-141.
[Table] [Table]
(cid:15)
= 0 .
25 , Figure Number =-
[Table]
[Table] [Table]
(cid:15)
= 0 .
25 , Figure Number =
[Table]
(cid:15) , Figure Number l 1 DM u 1 DM l 2 DM u 2 DM (cid:15) = 0 .
05 , Figure Number =
(cid:15)
= 0 .
05 , Figure Number =-53.
(cid:15)
= 0 .
05 , Figure Number =
(cid:15)
= 0 .
05 , Figure Number =
(cid:15)
= 0 .
05 , Figure Number =
(cid:15)
= 0 .
1 , Figure Number =-83.
(cid:15)
= 0 .
1 , Figure Number =-64.
(cid:15)
= 0 .
1 , Figure Number =-70.
(cid:15)
= 0 .
1 , Figure Number =-78.
(cid:15)
= 0 .
1 , Figure Number =-91.
(cid:15)
= 0 .
25 , Figure Number =-188.
(cid:15)
= 0 .
25 , Figure Number =-219.
[Table]
(cid:15)
= 0 .
25 , Figure Number =-
[Table]
[Table] [Table]
(cid:15)
= 0 .
25 , Figure Number =-
[Table]
(cid:15)
= 0 .
25 , Figure Number =
[Table]
B E XPERIMENTAL S ETUP FOR T RAINING DNN S
[Table]
C PGD R OBUSTNESS ON S PECIFIC I NPUT B OUNDS ON MNIST
D PGD R OBUSTNESS ON S PECIFIC I NPUT B OUNDS ON CIFAR
E F AILURE E XAMPLE FOR O UR B OUNDS
In this section, we show a faliure example to when the network weights do not follow Gaussian
Assumption 1, our bounds can be far too loose even compared to IBP.
Consider a two layer neural network
where
A 1 = 1000 I n × n , b 1 = − 999 1 n , a 2 = − 10 1 n and b 2 = 0 for the interval [ − 1 n , 1 n ] .
Then, we have that
[Equation]
Since u ≥ 0 , then M = I n × n .
Thereafter our estimated bounds as given by Equation 3 are given as:
[Equation]
As for IBP bounds, they are given as follows:
[Equation]
Under this construction of weights to the network, different of the i.i.d Gaussian assumption, it is clear that our bounds can be orders of magnitude looser to IBP.
In the following section, we demonstrate that networks trained on real data do have weights that are not far from the Gaussian assumption by empirically investigating the histogram of the network weights.
F H ISTOGRAM OF W EIGHTS OF T RAINED N ETWORKS
To quantify how reasonable is the assumption of Gaussian i.i.d weights in trained networks, we train medium CNN (discussed in previous section) and show the histogram of the weights of the ﬁrstlayers.
The network is trained with and without (cid:96) 2 regularization on three datasets namely MNIST,
CIFAR10 and CIFAR100.
[Figure] [Figure]
[Figure]
G R EBUTTAL G. R EVIEWER
We thank R2 for their very thorough review of our paper.
Regarding the disclaimer, we want to clarify to other reviewers too that we have indeed tried to address all R2’s previous concerns in this submission.
We have conducted some extra experiments from the previous submission; we have also done some serious changes to the structure and presentation of the paper as previously suggested by
R2.
In what follows we try addressing R2’s current concerns.
• The logical argument explaining the "why" these are valid is harder to understand and could be clariﬁed.
My understanding so far ...
The overall description of R2 to the logical argument is correct.
The proof is based on showing that our bounds are ’supersets’
to
some approximate bounds L approx and U approx .
The approximate bounds are indeed related to the unknown true bounds through Assumption 1.
Note that Assumption 1 states that there exists some m where the Assumption inequality holds.
As we pointed out to R1, the larger the constant m , the larger n has to be for Theorem 1 to hold.
Please refer to the last inequality in the proof of Theorem 1.
• The caption of Figure 1 is misleading.
It says that it shows that the proposed ...
True.
This was a mistake of phrasing on our part.
We have addressed this in the revised version.
• The reporting of Figure 5 and 6 is weird because according to the text, each datapoint seems to be the average robustness of ...
We have addressed this in the revised version.
The reported models had only their robustness averaged over multiple (cid:15) test .
Please note that the robustness results over each individual (cid:15) test were left for the appendix.
The best models from the grid search over the hyperparameters are reported for both our bound training and IBP training for fair comparison.
• I appreciate the effort of the author to include experiments involving a MIP solver returning the true bounds.
This is very helpful in building conﬁdence that the bounds generated are correct.
How is the real network trained?
Is this based on a robustly trained network or is it just standard training?
Is 99 % the nominal accuracy?
Yes, 99% is the nominal accuracy of the trained models.
The network was trained normally without any special regularization.
• In general, I think that the paper would be better if there was more discussion of the failure modes of the method.
There are easy to identify failure cases where the proposed bound is incorrect or loose.
My opinion is that the paper would be stronger if it acknowledged them ...
We agree with R2.
We have addressed this in the revised version.
In particular, we reﬂected this in the paragraph just before section 3.3.
We have added in the appendix the example of the failure case pointed out by R2.
Moreover, as suggested by R1, we have investigated, at least empirically, the histogram of the weights of a normally trained network on 3 real dataset in appendix F.
In what follows, we re-state our response to R1 in regards to the histogram results of the appendix to R for completion. "
For the MNIST experiments, even without (cid:96) 2 regularization, the weights seem to follow a bell shaped.
With (cid:96) 2 regularization, the histogram tend to be much pointy.
Of course, the regularization parameter used in training interpolates between the two histograms, the larger the regularization the more ’pointy’ the histogram will tend to be.
A similar observation can be noted for CIFAR10 and CIFAR100.
We want to emphasize that while this is not a proof that the Gaussian assumption holds in practice, we believe that at least the distribution of the weights does not seem to be far from Gaussian and thereafter sheds light on why perhaps such an assumption may not be utterly unreasonable on real networks."
• The assumption about gaussian weights in A1 and a2 seems strong but is at least partially motivated at the end of 3.2 (although not all networks are ... R2 is correct.
The i.i.d assumption over all elements of the weights matrix A breaks in the presence of topelitz/circulant structure convolutional layers have.
This, unfortunately, is as far as the current theory can go.
We ﬁnd though that this have little impact on practically upon training the networks.
• I think that the paper would beneﬁt from having some discussion of other methods that can derive "bounds" which may not actually be bounds.
The works by Stefan Webb
(A Statistical ...
Indeed, the two referenced works are related to our direction since both aim at either probabilistically carrying out the veriﬁcation or by estimating the lower bound of the minimum perturbation.
We elaborated on this in the revised version and added a paragraph in the introduction.
• While reviewing the paper, I also spotted some strong similarities between the methods proposed (particularly subsection 3.3) and the Fastlin method.
The computation mechanism of Fastlin (propagate recursively from the end through the linear layers and through diagonal matrices that replace the ReLU activation function) is exactly the same as the proposed one (Fastlin goes through the ...
The description of R1 for the intuition of the construction of the diagonal matrix M is correct.
Indeed, there are similarities between both our work and Fastlin.
However, we do not believe that the works are limited.
Both works still provide an excellent trade-off between efﬁciency and computational complexity, even when over estimating the intermediate upper bounds for the intermediate M , the approximation for the superset bounds are still effective.
• Minor comments, typos and references.
We have addressed the issues in the revised version.
We have also tried to address other issues and writing style highlighted by R2.
G. R EVIEWER
We thank R1 for the time spent in reviewing our paper.
Follows our response to R1’s concerns.
• Some assumptions seem pretty unrealistic to me.
For instance, the fact that the neural network should have Gaussian i.i.d weights: it is thus ...
As correctly pointed out by R
Gowal et al do not make such an assumption.
They provide
loose
’true’ bounds for deep architectures as a function of network parameters by simply computing worst case bound per layer.
As suggested by R1, we have conducted an experiment where we trained Medium
CNN with identical structure to previous experiments on MNIST, CIFAR10 and on CIFAR100 with and without (cid:96) 2 regularization.
We then plot the histogram of
the weights of the 6 layers in the network (3 convolutional and 3 fully connected).
We report the results in Section E of the appendix in the revised version.
For the MNIST experiments, even without (cid:96) 2 regularization, the weights seem to follow a bell shaped.
With (cid:96) 2 regularization, the histogram tend to be much pointy.
Of course, the regularization parameter used in training interpolates between the two histograms, the larger the regularization the more ’pointy’ the histogram will tend to be.
A similar observation can be noted for CIFAR10 and CIFAR100.
We want to emphasize that while this is not a proof that the Gaussian assumption holds in practice, we believe that at least the distribution of the weights does not seem to be far from Gaussian and thereafter sheds light on why perhaps such an assumption may not be utterly unreasonable on real networks.
• A major difference with (Gowal et al) is that: here, the bounds are in expectation whereas the bound in (Gowal et al) are deterministic.
In this paper, there are some approximation assumptions ...
This is true.
Unfortunately, as far as the analysis goes, we do not yet have concentration results.
One direction of interest is to study the asymptotic behaviour (in terms of n and k ) of the variance of L M and U M over the distribution of network parameters.
So far we have some little progress in that direction as the analysis becomes untraceable.
• Most of the proofs are long and complicated, and several equations of 7-8 lines could be summarized with up to 2-3 lines maximum.
I had a hard time to understand the proof of Theorem 1, which is simply some algebra.
This could be improved.
The equations were, for most part purposely, left to span multiple lines as we wanted to elaborate on each step of the analysis.
• I’m curious of the imagenet performance: couldn’t this technique be easily applied to AlexNet, which is nowadays simple to manipulate?
Is there a technical issue to do so?
While we were interested for the most part in the analysis, we do believe that the paper can beneﬁt from the ImageNet experiments vastly.
They were left out for time/computation capacity related reasons.
We will try to address this within our capacity.
• Is the assumption 1 really an assumption?
given a,a’ and b,b’ there always exists m such that a>=b-m and a’<=b’+m, like m>=max(|a-b|,|a’-b’|).
Am I wrong?
I think I do not understand this assumption...
Yes, Assumption 1 is trivial.
The key element to note from Assumption 1 is that there is some m (that could be very large) where this inequality holds.
Now, the question is how does this constant m affect the analysis?
Larger m results into requiring a larger input dimension n for Theorem 1 to hold.
To see this, refer to the last equation of the proof of Theorem 1.
Note that the second term of the right hand side of the inequality is a function of m .
This shows that the larger the constant m for the Assumption 1 to hold, the larger the input dimension for the inequality to be positive and thereafter for Theorem 1 to hold.
• How simple is it to extend those theoretical results for NNs to the case of CNNs?
The results hold for generic linear operators A and a 2 .
For convolutional layers, the kernel can indeed be represented with a structured topelitz/circulant matrix ﬁtting into our analysis.
However, we ﬁnd a trick to avoid the inefﬁcient construction of such massive matrices.
Note that to compute our bounds, one only needs to compute Eq (3) efﬁciently where A 1 is convolutional kernel.
For A 1 x , this can be computed by simply performing convolution to the input center x without constructing A 1 (which is a forward pass through the layer).
The tricky part comes upon computing the term | a (cid:62) 2 MA 1 | 1 n .
As one need to compute | a (cid:62) 2 MA 1 | ﬁrst where M is a diagonal matrix without constructing A 1 .
The way we go about this is by simply taking a backward pass through the layer that computes a (cid:62) 2 MA 1 for some input center x as a function of x .
That is we perform a forward pass for some input center x
and then compute the gradients as a function of x resulting in having access to a (cid:62) 2 MA 1 and thereafter computing | a (cid:62) 2 MA 1 | 1 n becomes efﬁcient.
Note that this has been employed in the experiments as the networks used did have convolutional layers.
G. R EVIEWER
We thank R3 for their time reviewing the paper.
Follows our response.
• However, these results shown in
Figs
and
appear to be somewhat inconsistent with those reported in the paper of IBP
[Gowal et al., 2018)].
We would like to raise to the attention of R3 that the metric used here is PGD robustness and not
PGD accuracy.
While they are to some extent related, the PGD robustness is the percentage of the testing samples where the network prediction was not altered under PGD attacks from the prediction of the network on the original samples regardless if the prediction is correct.
Since the pretrained models of IBP were not made available, we trained models with IBP by running the code provided by the authors on their github page (with the same exact setup).
Moreover, for a fair comparison and stronger baselines, all IBP trained models were trained over different learning schedules.
The best models with the highest accuracy-robustness pair trade-off for IBP are reported.
• I am not sure if it really makes sense to discuss which method is better in such ranges of failed defense.
We believe that so long the model accuracy on the noise free-samples is still preserved (captured on the x-axis of Figures 5 and 6) with an increase in robustness, this is good evidence for improvement.
For instance, for CIFAR10 on the small architecture (Figure 6), our models have improved the average robustness over all (cid:15) test (captured by y-axis) while preserving similar accuracy to the nominal model.
This is unlike IBP, where despite that the robustness improved 30% the accuracy dropped by 20% compared to the nominal model.
• Sources for differences to Gowal et.
al.
Indeed, as correctly pointed by R3, the reported robustness is averaged over all tested (cid:15) .
However, note that we report the robustness results for individual (cid:15) test in the supplementary material (Figures 10-17).
The range of (cid:15) test for MNIST is identical to that of Gowal et.
al.
; however, the range for (cid:15) test is indeed larger for CIFAR10 experiments.
• Comment on the metric PGD Robustness and not PGD accuracy.
The focus of our work is to show an efﬁcient approach to estimate the output bounds of a network.
Since the bounds are only ’true’ in expectation they can not be used for certiﬁcation in the absence of concentration results to our bounds.
To that extent, veriﬁed accuracy (accuracy by running exact solvers) and its lower and upper bound estimates, certiﬁed accuracy and PGD accuracy respectively, do not make much sense in our framework.
Therefore, we limit our experiments to showcasing that such simple estimates of the bounds can be utilized to efﬁciently regularize networks to improve their robustness.