Learning Contextual Perturbation Budgets for Training Robust Neural Networks
Authors: Jing Xu
Zhouxing Shi
Huan Zhang
Jinfeng Yi
Cho-Jui Hsieh
Liwei Wang
Venue:  (2021)
Keywords: adversarial robustness
certified robustness
certfied robust training

Abstract: Existing methods for training robust neural networks generally aim to make models uniformly robust on all input dimensions. However, different input dimensions are not uniformly important to the prediction. In this paper, we propose a novel framework to train certifiably robust models and learn non-uniform perturbation budgets on different input dimensions, in contrast to using the popular $\ell_\infty$ threat model. We incorporate a perturbation budget generator into the existing certified defense framework, and perform certified training with generated perturbation budgets. In comparison to the radius of $\ell_\infty$ ball in previous works, the robustness intensity is measured by robustness volume which is the multiplication of perturbation budgets on all input dimensions. We evaluate our method on MNIST and CIFAR-10 datasets and show that we can achieve lower clean and certified errors on relatively larger robustness volumes, compared to methods using uniform perturbation budgets. Further with two synthetic datasets constructed from MNIST and CIFAR-10, we also demonstrate that the perturbation budget generator can produce semantically-meaningful budgets, which implies that the generator can capture contextual information and the sensitivity of different features in input images.

LEARNING CONTEXTUAL PERTURBATION BUDGETS FOR TRAINING ROBUST NEURAL NETWORKS
Jing Xu', Zhouxing Shi?, Huan Zhang", Jiinfeg YI', Cho-Jui Hsieh", Liwei Wang' Peekin University UCLA 'JD Al Research
1700012451 @pku..pku... edu. en; zshi@cs uula edu; huanGhuan-zhame com; com; yi yijinfenge com, chohsiehe cs. ucla. edu; wanglw@pku edu.cn
ABSTRACT
Existing methods for training robust neural networks generally aim to make models uniformly robust on all input dimensions. However, different input dimensions not uniformly important to the prediction. In this paper, we propose a novel framework to train certifiably robust models and learn non-uniform perturbation budgets on different input dimensions, in contrast to using the popular foo threat model. We incorporate perturbation budget generator into the existing certified defense framework, and perform certified training with generated perturbation budgets. In comparison to the radius of too ball in previous works, the robust- ness intensity is measured by robustness volume which the multiplication of perturbation budgets on all input dimensions. We evaluate our method on MNIST and CIFAR- 10 datasets and show that we can achieve lower clean and certified errors relatively larger robustness volumes, compared methods using uniform perturbation budgets. Further with two synthetic datasets constructed from MNIST and CIFAR-10, also demonstrate that the perturbation budget generator produce maeaiiinlly-mmaingfflllll budgets, which implies that the generator can capture contextual information and the sensitivity of different features in input images.
INTRODUCTION
has been demonstrated that deep neural networks, although achieving impressive performance on various tasks, are vulnerable to adverarial perturbations (Szegedy et al. 2013). Models with high accuracy on clean and unperturbed data can be fooled to have extremely poor performance when input data are adversarially perturbed. The existence of adversarial perturbations causes concerns in safety-critical applications such as self-driving cars, face recognition and medical diagnosis.
A number of methods have been proposed for training robust neural networks that resist to adversarial perturbations to some extent. Among them, adversarial training (Goodfellow et al. 2015
Madry et al. 2018) and certified defenses (Wong al. 2018 Gowal et al. 2018] /hang al. 2020) are of the most reliable ones so far, and most of them are trying to make the network robust to any perturbation within an l, norm ball. Taking the commonly used -aal defense an example, robust training methods aim to make model robust to perturbation on any pixel, which means the model is uniformly robust on all the input dimensions. But this valid assumption we should make?
As we know, human perception is non-uniform (humans focus on important features even though these features can be sensitive small noise) and context dependent (what part of image is important heavily depends on what is on the image). We expect a robust model to be close to human perception, rather than learn to defend against a particular fixed threat model, e.g., the traditional l -nom one. Intuitively, we expect a good model to be more sensitive to important features and less sensitive to unimportant features, and the importance of features should context-dependent. Taking the MNIST hand-written digit classification problem as example, the digit 9 can be transformed to 4 simply by modifying just a few pixels on its head, so those pixels should be considered more important, and enforcing them to be robust to large perturbation may not be correct. On the other hand, the pixels on the frame of such an input image can be safely modified without changing the ground-truth label of the image. Therefore, uniform budget in robust training greatly hamper the performance of

---

neural networks certain tasks, and will foree network to ignore some important features that are important for classification. Robustness certification with non uon-umffomm perturbation budgets has been discussed in a prior work (Liu et al. 2019), but training robust models and learning context-dependent perturbation budgets has not been addressed in prior works, which is more challenging and important for obtaining robust models. A detailed discussion Sec. [.2
difference with Liu al. (2019) is in
In this paper, we propose the first method that can learn context-dependent non-uniform perturbation budgets in certified robust training, based on prior certified defense algorithms on l--norm threat models (Zhang et al.) 2020; Xu al.) 2020). To learn a comtext-dependent budget without introducing too many learnable parameters, we introduce a perturbation budget generator with an auxiliary neural network, to generate the context-dependent budgets based on the input image. We also impose constraints on the generator to make generated budgets satisfy target robustness volumes and ranges of budgets, where robustness volume is defined as the multiplication of budgets on all input dimensions. We then train the classifier with linear-relaxation-basss -aased certified defense algorithm, auto nuto_LiRPA (Xu et al. 2020) generalized from CROWN-IBP (Zhang et al. 2020), to minimize the verified error under given budget constraints. The gradients of the loss function can be back pack-propagtted to the perturbation budgets, allowing training the classification network and budget generator jointly in robust training.
Our contribution can be summarized below:
We propose novel algorithm to train robust networks with contextual perturbation budgets rather than uniform ones. We show that it can be incorporated into certified defense methods with linear relaxation- based robustness verification.
We demonstrate that method can effectively train both the classifier and the perturbation generator jointly, and we able to train models on relatively larger robustness volumes and outperform those trained with uniform budgets.
We also show that the learned perturbation budgets are semantically meaningful and align well with the importance of different pixels in the input image. We further confirm this with two synthetic tasks and datasets constructed from MNIST and CIFAR-10 respectively.
2 BACKGROUND AND RELATED WORK
2.1 TRAINING ROBUST NEURAL NETWORKS
Since the discovery of adversarial examples (Szegedy et al. (2013). Biggio et al. (2013)) ). a great number of works has been devoted improving the robustness of neural networks from both attack and defense perspectives (Moosavi-Dezfooli et al. 20066] Carlini & Wagner 2017 Papernot et al.) 2016. -oosaai-Dezfoolii al. [017]]owall Gowal et al. 2019). On K-way classification task, training an ad adversarially robust neural network fwr with weight w can generally be formulated as solving the following min-max optimization problem:
min
E max (x (x.I)~ SEA
(1)
where D is the data distribution, and A is a threat model defining the space of the perturbations, and a loss function.
Adversarial training (Goodfellow et al. 2015 Madry et al. 2018) applies adversarial attacks to solve the inner maximization problem and train the neural network on generated adversarial examples, with efficiency advanced in some recent works (Shafahi et al. 2019: Wong et al.) (020). However, robustness improvements from adversarial training do not have provable guarantees.
Some other recent works seek to train networks that have provable robustness, namely certified defense methods. Such methods solves the inner maximization by computing certified upper bounds that provably hold true for any perturbation within the threat model, including abstract interpretation (Singh et al. 2018), interval bound propagation (Gowal et al. 2018 Mirman et al. 2018). randomized smoothing (Cohen et al. 2019; Salman et al. 2019 Zhai et al. (020), and finear-relaxate based methods (Wong& & Kolter 2018: Mirman et al. 2018 Wong et al. 22188] Zhang et al.] 2020): Xu et al.
2020). However, nearly all existing certified defense methods treat all input features equally in the
2

---

threat model, such by-ball threat models, A []] 81: ct, especially the &oo threat model commonly used in many previous works. But different input pixels are not uniformly important to the prediction, and thus propose to learn contextual perturbation budgets for different pixels.
2.2 ROBUSTNESS CERTIFICATION WITH NON-UNIFORM BUDGETS
There has been recent work Liu et al. (2019) studying robustness certification with non-uniform budgets. Their work aims to maximize the robustness volume while ensuring the network prediction is certifiably correct, and this optimization problem is solved using augmented Lagrangian method. To highlight, their method has several limitations: 1) The perturbation budgets are obtained by solving constrained optimization problem for each input example, which is very time-consuming. This is not only too inefficient for training as can bring a large overhead in each training step, but also incapable for learning perturbation budgets jointly with the classifier. In contrast, have significantly different scheme we aim to maximize the accuracy under given target robustness volumes. And our perturbation budgets is obtained with lightweight nuura-nettwrrk-bsee generator, and it can be jointly trained with the classifier in an end-to-end way. 2) Their work only focuses on certifying trained models due to the inherent limitation of the method, while address of the problem of training robust neural networks and learning contextual perturbation budgets simultancously. Consequently, we are able empirically demonstrate that we can effectively train robust models with much larger robustness volumes and achieve lower errors under given target robustness volumes. 3) We further have experiments with synthetic tasks on MNIST and CIFAR-IO in Sec. [.2 respectively to demonstrate that the learned perturbation budgets are semantically meaningful and can capture contextual information.
[.2 and Sec. 4.3
3 PROPOSED METHOD
3.1 PROBLEM SETTING
Variable threat model Unlike many previous works that use foo threat model with uniform on all input dimensions, we allow different pixels to have different perturbation budgets, but they need to meet some constraints as we will define below. For an n-dimensional input x, when the perturbation budget of pixel x; is the threat model is
A 62, ... ,.)) = {6 6.1 S c, 1slsis
And thereby the too threat model special case with 6 = <0(v] s i S n). We define the robustness volume of threat model A the multiplication of all c( sisn).
V(A) II G. i.e., log V(A) = Σ log 1=1
1=1
In principal, we have target robustness volume, Vo = c) and fw is considered to be provably robust under this target robustness volume on instance (x. y) and only if:
AA D, Vk y, min([f 8)1, If 5)]x) > 0. SEA
This means that the predicted score the ground-trute truth class y is certifiably larger than any other class l * y under threat model A. And instead of using fixed A, in our framework A be taken from a threat model space ID, which consists of threat models A(E1, "2, satisfying the following two constraints:
Volume constraint:
Σ log = nl cD.
(4)
Range constraint:
15 < = aeo. u min (Teo. 1).
(5)
The volume constraint states that the robustness volume of A is equal to the target robustness volume e (written in log domain above). For the range constraint, there relative factors and a controlling the perturbation budget range of each pixel, namely [1, u). This constraint can be set to guarantee minimum robustness and also prevent the model from over-invariant each pixel.
3

---

Perturbation budget generation A classifier can be accompanied by perturbation budget gener- ator <(x) which tries to find perturbation budgets {1 62, ... En and the corresponding threat model (xx) that can minimize the verified loss of the classifier while satisfying constraints (4) and (5), so that 3 more likely to hold true with A generated by < c(x). We will state the optimization problem in the next paragraph.
Robust classification Accompanied by perturbation budget generator, we aim to learn a robust classifier Ja with the following min-max optimization problem:
min
max L(f 0), (x
Note that a key difference between this problem and the traditional one in (T) is that now the threat model (xx) is variable and dependent the input x. This A(x) is generated by e(x), under the given volume and range constraints. We evaluate the robustness a classifier fur by computing an average verified error all the test instances, where the verified correctness each instance is evaluated similarly as 3). where is taken as the generated A(x).
3.2 ALGORITIIM FRAMEWORK
mix c(x))
x
fw((x
Verifier
e(x)
Farward pass
Backward pass
96 (x)
Perturbation Generator
Figure 1: flowchart of training a certifiably robust network with a perturbation budget generator. The dashed rectangle consists of the perturbation generator, which is based on an auxiliary neural network ge (x). The blue arrows and red arrows represent the forward pass and the vack-propagation pass for gradient computation respectively.
Figure Dillustrates our algorithm framework. Given an input x, the perturbation budget generator outputs perturbation budgets e(x) based robustness verifier takes the unperturbed input x and perturbation budgets c(x) as input and computes robust loss function Lyob (x, c(x)). During training, this loss function is back propagated to compute its gradients w.r.t. the parameters of the classifier fw and also perturbation budgets e(x), and the gradients e(x) further propagated the parameters of go go(x). f and go are jointly trained to minimize the robust loss. During test, the verifier also outputs a verified error for evaluation by computing the lower bound of the margin, m(x, e(x)) (see Sec. 3.4).
auxiliary network ga (x) parameterized by 0. Then
3.3 PERTURBATION BUDGET GENERATOR
For the perturbation budget generator E(x), generate perturbation budgets in two steps. In the first step, we generate initial perturbation budgets that are only required to meet the range constraint. And in the second step, refine the perturbation budgets to meet the volume constraint. These two steps are detailed below.
Initial perturbation budgets We first use a neural network gx(x) [0, 1]a generate an initial budget distribution. Specifically, we assign [ē(x)]; exp(logl + [go (x))] log I)) as the initial budget of the i-th pixel, and thereby [((x)]; [7, u). our work, we employ a two-layer
convolutional model with a sigmoid activation in the last as (x).
Refining the perturbation budgets the volume constraint. We compute the log volume of these budgets, log V n log + (log u
We then refine the perturbation budgets to make them meet
4

---

log 1) Ex((()),, and compare it with the target log robustness volume log Vú = log co. If < Vo. we need to allocate (Vo - V) to the log perturbation budgets of different pixels, while still ensuring that each budget is no larger than u. This allocation is done proportionally according to the distance between the current log perturbation budget log[ū(x)]; and the upper limit log u of each pixel, for each pixel i, and then budget (g]((()); is obtained with:
log u - log (x)] nlg V
log[c(x)]; = log[ + (n log co V)
for V <<> Vo = n nog co-
V > n log 60, we remove redundant perturbation budgets. Similar to previous case, this refinement is done proportionally according to the distance between log[e(x)]; and the lower limit log(:
log(e(x)]{ log/ V n nllo
log[c(x)]; log[((x)]; (V log cu)
for V> n log Co-
3.4 CERTIFIED TRAINING WITH NON-UNIFORM PERTURBATION BUDGETS
For the robustness verifier, we adopt auto_ (Xu et al. 2020). It is based on linear relaxations and generalized from CROWN-IBP (Zhang et al. 2020), and there is also loss fusion technique for more efficient robust training. In CROWN-IBP and also the default setting in auto LiRPA, there are generally two passes for each input batch, excluding the gradient back back-propage propagation pass. the first pass, Interval Bound Propagation (IBP) (Mirman et al. 2018 Gowal et al. 2018) is used to compute the output bounds of intermediate layers, which is required by linear relaxations. Given an input x and uniform perturbation budget e, the input of the IBP pass is the lower and upper bounds of the input image, [Clip(x - c), Clip(x + e)). where Clip(-) stands for clipping the image bounds into domain [0, 1]" In the second pass, the CROWN algorithm (Zhang et al. 2018) is applied on the output layer to compute its linear bounds w.r.t. perturbed input (x + d), utilizing intermediate bounds by IBP for linearly relaxing activation funetions. Let h(x + 6) represent the output layer, then its linear bounds w.r.t. (x + 5) and concrete (final) bounds without (x + 5) are (Zhang et al. 2018):
Ax + b- A ll
- A(x + 8) + b < 1x(x + 5) S A(x + 6) + b S Ax + b || |||||
(7)
where A, A, b. 5 are parameters of linear bounds obtained by CROWN. We refer readers to Xu et al. (020)) for their detailed algorithm.
When non-uniform perturbation budgets are used, e(x) from the perturbation generator is used rather than scalar c. This causes differences mainly on the input lower and upper bounds for the IBP pass and the concretization of linear bounds in
In inference, we compute the lower bound of the minimum margin between the ground truth label y and any other label / y:
mx, exx)) = min m(x + 5), where m + 6) minn[f + o)ly [f + 8)]+), (EA(X)
(8)
ify
by taking the margin functions as the output layer in robustness verification. m(x, e(x)) is the lower bound of the margin for all possible perturbation 5 E A. The classifier is verifiably robust on this
instance and the threat model A(x) from the generated perturbation budgets, if m e(x)) > 0, as defined in (3)
During training, we use loss fusion (Xu et al. 2020). to compute and optimize the robust loss Trat (xx)):
Lyob(x, (())) Z max (x
BEA(x)
where the robust loss is an upper bound of the right-band-side obtained from the verifier. The auto LiRPA verifier differentiable and thus we are able to back-propagate the gradients of the loss function w.r.t. the parameters of the both classifier and perturbation budget generator for training. In contrast, this joint training appears to be more difficult for adversarial training methods such as Projected Gradient Descent (PGD) attack (Madry et al. 2018). The PGD loss is technically differentiable to the perturbation budgets. However, take N N-step PGD training as an example, each step add noise roughly 5 so the is used by every step of PGD. Eventually, to get the correct gradient w.r.t e, we will need to backprop through the N PGD update steps, which makes the gradient complicated and hard to calculate.
5

---

4 EXPERIMENTS
We conduct experiments mainly on MNIST (LeCun et al. 2010) and CIFAR-I0 (Krizhevsky et al. 2009) demonstrate the effectiveness of our proposed method. We show that our method can effectively train certifiably robust models with learned contextual perturbation budgets, and the models achieve lower verified errors compared to those using uniform budgets. We also constructed two synthetic datasets to demonstrate that the learned perturbation budgets are semantically meaningful. We report implementation details in Appendix A
4.1 EXPERIMENTS ON MNIST AND CIFAR-10
For MNIST, consider robustness volumes with 60 0.4, 0.6, 0.8 respectively and a to 0.5 and π = 2.0, and for CIFAR-10, 10, we consider en = 8/255, 32/25/ respectively and set Q to 0.125 and 2.0. We reproduce baseline models with uniform budgets following the best setting in CROWN-IBP (Zhang et al., 2020), and in particular, the models with uniform perturbation budgets co 0.4 on MNIST and co 8/255 on CIFAR-10 10 are directly downloaded from their released m
We evaluate the baseline models and our models with learned perturbation budgets mainly using clean error and verified error with the corresponding type of perturbation budgets used for training, i.e., baseline models are verified with uniform budgets and our models are verified with learned budgets. In addition, we also report verified error computed with Liu et al. (2019) where the model is considered verified by] Liu et al. (2019) is at least en-
example if and only if the volume of perturbation budgets
Table 1: Results of using uniform perturbation budgets (Zhang et al. 2020) and our learned ones respectively on MNIST for certified defense with various target robustness volumes. "_" denotes that training with uniform perturbation budgets co = 0.6 or 0.8 totally fails, and thus we also evaluate models trained with co 0.4 on test volumes co 0.6 and co 0.8 respectively. For verified errors in evaluation, we provide results evaluated with the method by Liu et al. (2019), on uniform budgets, and on our learned budgets respectively.
Verified error (%)
Target Volume
Training budgets
Clean error (%)
By Liu et
2019
Uniform
Learned
Uniform cq 0.4
2.17
5.88
12.06
100.0
6o = 0.4
Learned 6o 0.4 (ours)
1.41
7.97
58.10
7.81
Uniform to 0.4
2.17
100.0
100.0
100.0
co 0.6
Uniform co 0.6
Learned co
(ours)
14.13
100.0
13.93
Uniform <g 0.4
2.17
100.0
100.0
100.0
6o =
Uniform <q 0.8
Learned 6g 0.8 (ours)
12.15
25.77
100.0
26.37
Table 2: Results on CIFAR- 10, in a similar format with Table !
Verified error (%)
Target Volume
Training budgets
Clean error (%)
By Liu et al.
2019
Uniform
Leamed
<u 8/255 Uniform co = 8/255
54.02
64.11
67.11
67.75
Learned eu -8/255 (ours)
49.34
65.99
86.42
60.72
Uniform ca = -8/255
54.02
83.83
99.57
99.96
32/255
Uniform c -32/255
80.63
84.73
85.03
85.11
Learned co = -32/255 (ours)
63.34
82.0
98.56
69.31
Results are shown in Table for MNIST and Table 2 for CIFAR-10, On MNIST, on the smaller 60 0.4 target volume, our model has lower clean error and lower verified error learned budgets, compared to the model trained and evaluated with uniform budgets, though the verified error of our model higher than the one trained with uniform budgets when evaluating with Liu et al. (2019). When using larger target volumes, we find that training with uniform budgets totally fail on 0.6 and 60 0.8, because we are able totally break the classification by perturbing every pixel to
8P
formHounds

---

Input
7
2
4
7
S
Budget
Label
9
Figure 2: Visualization of learned perturbation budget distributions on MNIST with cy = 0.6. The budget images have pixels ranging from 0 to representing a per-pixel, per-example Lighter colors indicate larger perturbations and darker colors indicate smaller perturbations.
Input
Budget
Label
ship
ship
irplane
frog
frog
automobile
automobblle
Figure 3: Visualization of learned perturbation budget distributions on CIFAR- 10 with Eo 16/255. For better visualization, the visualized budgets are normalized by the upper limit of the perturbation budgets, 32/255, since the budgets on CIFAR-10 10 are much smaller than 1.
0.5 under uniform perturbation budgets e = 0.6 or e = 0.8, and the model trained with eg = 0.4 uniform budget
able to generalize 6o 0.6 and eo 0.8 target robustness volumes test. In contrast to using uniform budgets, with our learned budgets, we are able to handle cp 0.6 and co 0.8, achieving reasonable verified errors. Moreover, on CIFAR-10, our models using learned budgets achieve lower verified errors than models with uniform budgets on both
8/ 255 and
to 32/255. The experiments demonstrates the effectiveness of our method for training more robust models under non-uniform perturbation budgets.
We note that our models have higher verified errors when evaluated on uniform budgets, but evaluation on uniform budgets is not the major goal of this paper, nor it is the ultimate goal of robust machine learning, as we have argued that non-uniform budgets are more practical. Nevertheless, in case that expects maintain performance uniform budgets while training with son-uniform learned budgets, this can be achieved by setting a which controls the minimum budget of each pixel, as we demonstrate in Appendix C
We also visualize our learned perturbation budget distributions in Figure 2 and Figure 3 for MNIST and CIFAR-10 10 respectively. Our perturbation budget generator able identify important and sensitive features in the input image and assign relatively smaller budgets to the corresponding pixels, while insensitive pixels are assigned with relatively larger perturbation budgets. Note that the perturbation budgets also highlight sensitive pixels that are missing from input images. Therefore, learned perturbations contain important features from both the ground truth class and the classes that are close to it. For example, as visualized in Figure 2 perturbation budgets of digit combine features from both and 9, and digit 5 looks like 5 overlapped with 8.
4.2 EXPERIMENTS ON WATERMARKED MNIST
To demonstrate another benefit of using different perturbation budgets for different pixels, we artificially constructed a dataset from MNIST, namely the Watermarked MNIST, For each image in the original MNIST dataset, shown in the first row of Figure we add small watermark to the left or right margin of the image, which decides the gold label jointly with the main digit the image. The watermark is darker than the main digit, and thus we expect to see the perturbation generator assign smaller perturbation budgets to the watermark, to retain the information carried by the watermark, while training uniform budgets is likely to fail for breaking the watermark. Specifically, the watermark a digit zero or one, chosen randomly, from an image with the corresponding label in the original dataset. We scale down the image 12 12, and crop out pixels on each border and retain the
7

---

Input 8
3
7
7 4
Budget
Label
18
11
12
17
Figure 4: Images in the artificial Watermarked MNIST dataset and the visualization of our learned perturbation budget distributions. The watermarked pixels must use smaller perturbation to be correctly recognized, and our learned perturbation correctly chooses a smaller perturbation budget for these pixels.
inner 8 x 8 part. The opacity of the watermark is reduced to 30%. We randomly put the watermark either on the left or right 28 x 8 border, and the vertical position span is randomly chosen from [0, 7] [1, 8). [20, 27]. The gold label of the new image is the main digit itself, if the watermark O, otherwise the gold label is the main digit plus 10. In this way, we construct a 20-class digit classification problem.
We then perform certified defense on this synthetic dataset, with 60 = 0.4. For our learned perturbation budget generator, we set = 0. 125, ह 2.5. We compare the results with the method using uniform perturbation budgets, as shown in Table 3 When uniform perturbation budgets are used, the clean error and the verified error are very large (higher than 50%). This is because the values of the pixels with the watermark is around 30% of those in an original digit, and thus when a perturbation budget of 2 0.3 is uniformly added to all the pixels, it is enough to perturb the image and make the watermark disappear, and thus the model cannot achieve low errors. In contrast, as shown in Figure 4 by learning contextual perturbation budgets, the budget generator tends to assign relatively small budgets to pixels with the watermark. Pixels with the main digit tend to have budgets that are somewhat larger, and there are empty pixels which receive even larger budgets. With such perturbation budgets, we achieve much lower verified error (19.04 v.s. 59.29 on = 0.4) and clean error (3.48 v.s. 51.75 on to 0.4) under the same robustness volume, as shown Table This experiment demonstrates the importance of learning different perturbation budgets for different pixels and the effectiveness of our proposed method.
Table 3: Results on the artificial Watermarked MNIST dataset. Using an uniform budget resulting in a > 50% error, as the classifier are not sensitive enough to these watermarked pixels.
Uniform budgets (Zhng et al. 2020
Learned budgets (Ours)
Target Volume
Clean error (%)
Verified error (%)
Clean error (%)
Verified error (%)
Eg 0.3
50.90
55.54
1.99
11.00
0.4
51.75
59.29
3.48
19.04
4.3 EXPERIMENTS ON DOUBLED CIFAR-10
To further demonstrate the ability of the perturbation generator in identifying important input features, we create a harder synthetic dataset, namely Doubled CIFAR-10, based on CIFAR-10. An original input image in CIFAR- 10 has a size of 32 32. In this setting, each new input image consists of two original images on the left and the right respectively. Two images combined in this way yield new one with size of 32 64. We also add 4 x 64 border to the top of the new image and make the border either red or green. If the border if red, the gold label is taken as the gold label of the left original image, otherwise is the gold label of the right original image. The border serves as a hint for the model to tell which original 32 x 32 image it should look at for prediction. Thereby the size of each new image is 36 x 64. And with such a dataset, we expect to further study whether the perturbation budget generator can identify important features in the input image, since the two component image in each new image does not affect the gold label, and thus the model is expected to assign large perturbation budgets on this component image.
of
We use 6g = 16/255, a = 0.0625, a = 1.5. Results are shown in Table 4 Again, the errors of the model with learned perturbation budgets are much lower than those of the model with uniform
8

---

Table 4: Results on the synthetic Doubled CIFAR-10 dataset.
Uniform budgets
Zhang al., 2020
Learned budgets (Ours)
Clean error (%)
Veriedd error (5c
Clean error (%)
Verified error (%)
73.16
81.29
56.11
63.24
Input
Budget
Label
ship
cat
truck
automobile
bird
automobile
ship
frog
deer
dog
Figure 5: Images in the synthetic Doubled CIFAR-I0 dataset and the visualization of our learned perturbation budget distributions. The visualized budgets are normalized by 1.5 16/255 similarly as Figure 3 The generated perturbations correctly correlate with border colors.
budgets. We also visualize the learned perturbation budgets in Figure We observe that when the border is in red, large perturbation budgets assigned to the right side, which is consistent with that the gold label is determined only by the left side in this case. This observation also holds true for the case when the border is green. Moreover, we also observe that the perturbation budget distributions of the other side are semantically meaningful. This experiment further demonstrates that our proposed method is able identify important features the input and learn semantically meaningful distributions of perturbation budgets.
CONCLUSIONS
In this paper, we propose a novel approach to learn contextual perturbation budgets and successfully incorporate it into linear-relaxation-baaee certified training methods. We show its superiority over methods with uniform perturbation budgets via experiments on MNIST and CIFAR-1O 10. We also demonstrate that the perturbation budget generator can capture semantic information and align well with features in the input image with synthetically constructed Watermarked MNIST and Doubled CIFARIO datasets. Our proposed method can be used in settings where the perturbation budget of dataset cannot easily determined as in toy datasets, and can handle context dependent perturbations.
REFERENCES
Battista Biggio, Igino Corona, Davide Maiorca, Blaine Nelson, Nedim Šrndić, Pavel Laskov, Giorgio Giacinto, and Fabio Roli. Evasion attacks against machine learning test time. In Joint European conference on machine learning and knowledge discovery in databases, pp. 387-402. Springer, 2013.
Patrick Bilic, Patrick Ferdinand Christ, Eugene Vorontsov, Grzegorz Chlebus, Hao Chen, Qi Dou, Chi-Wing Fu, Xiao Han, Pheng-Ann Heng. Jürgen Hesser, et The liver tumor segmentation benchmark (lits). arXiv preprint arXiv:1901.0440 2019.
Nicholas Carlini and David Wagner. Adversarial examples detection methods. In Proceedings of the 10th ACM Workshop on Artificial Intelligence and Security, pp. 3-14. ACM, 2017.
easily detected: Bypassing
Jeremy M Cohen, Elan Rosenfeld, and Zico Kolter. Certified adversarial robustness via randomized smoothing. In ICML,, 2019.
Ian J Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. In ICLR, 2015.

---

Sven Gowal, Krishnamurthy Dvijotham, Robert Stanforth, Rudy Bunel, Chongli Qin, Jonathan Uesato, Timothy Mann, and Pushmeet Kohli. On the effectiveness interval bound propagation for training verifiably robust models. arXiv preprint arXiv:1810.1271: 2018.
Sven Gowal, Jonathan Uesato, Chongli Qin, Po-Sen Huang, Timothy Mann, and Pushmeet Kohli. An alternative surrogate loss for pgd-based adversarial testing. arXiv preprint 19 2019.
Alex Krizhevsky, Geoffrey Hinton, al. Learning multiple layers of features from tiny images. Technical Report TR-2009, 2009.
Yann LeCun, Corinna Cortes, and CJ Burges. Mnist handwritten digit database. ATT Labs [Online]. wwt../////////iiisttttmm http://aam..eec...coo//b//mm///
Available: http://yam.lecun..com
2, 2010.
Chen Liu, Ryota Tomioka, and Volkan Cevher. On certifying non-umit non-uniform bound against adversarial attacks. arXiv preprint 1rXiv:1903.06660 2019.
Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. In ICLR, 2018.
Matthew Mirman, Timon Gehr, and Martin Vechev. Differentiable abstract interpretation for provably robust neural networks. In International Conference on Machine Learning, pp. 3575-3583, 201 8.
Seysd-Mohsen Moosavi-Dezfoni Alhussein Fawzi, and Pascal Frossard. Deepfool: simple and accurate method to fool deep neural networks. In Proceedings of the IEEE Conference Computer Vision and Pattern Recognition, pp. 2574-2582, 2016.
Seyed-Mohsen Moosavi-Dezfoolli Alhussein Fawzi, Omar Fawzi, and Pascal Frossard. Universal adversarial perturbations. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 1765-1775 2017.
Nicolas Papernot, Patrick McDaniel, and Ian Goodfellow. Transfcrability in machine learning: from phenomena to black-box attacks using adversarial samples. arXiv preprint arXiv:1605.0771 2016.
Hadi Salman, Jerry Li, Ilya Razenshteyn, Pengchuan Zhang, Huan Zhang, Sebastien Bubeck, and Greg Yang. Provably robust deep learning via adversarially trained smoothed classifiers. In Advances in Neural Information Processing Systems, pp. 11289-11304 2019.
Ali Shafahi, Mahyar Najibi, Mohammad Amin Ghiasi, Zheng Xu, John Dickerson, Christoph Studer, Larry S Davis, Gavin Taylor, and Tom Goldstein. Adversarial training for free! In Advances in 2019.
Neural Information Processing Systems, pp.
Gagandeep Singh, Timon Gehr, Matthew Mirman, Markus Püschel, and Martin Vechev. Fast and effective robustness certification. In Advances in Neural Information Processing Systems, pp. 10825-10836, 2018
Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus. Intriguing properties of neural networks. ICLR, 2013.
Eric Wong and Zico Kolter. Provable defenses against adversarial examples via the convex outer adversarial polytope. In International Conference on Machine Leurning, pp. 5283-5292, 2018.
Eric Wong, Frank Schmidt, Jan Hendrik Metzen, and J Zico Kolter. Scaling provable adversarial defenses. In NIPS. 2018.
Eric Wong, Leslie Rice, and J Zico Kolter. Fast is better than free: Revisiting adversarial training. arXiv preprint arXiv:2001.0399. 2020.
Kaidi Xu, Zhouxing Shi, Huan Zhang, Yihan Wang, Kai-Wei Chang, Minlie Huang, Bhavya Kailkhura, Xue Lin, and Cho-Jui Hsich. Provable, scalable and automatic perturbation anal- ysis on general computational graphs, 2020.
10

---

Xuanang Xu, Fugen Zhou, Bo Liu, Dongshan Fu, and Xiangzhi Bai. Efficient multiple organ localization et image using 3d region proposal network. IEEE transactions on medical imaging. 38(8):185518888 2019.
Jiancheng Yang, Rui Shi, and Bingbing Ni. Medmnist classification decathlon: lightweight automl benchmark for medical image analysis. arXiv preprint arXiv:2010. /4925, 2020.
Runtian Zhai, Chen Dan, Di He, Huan Zhang, Boqing Gong, Pradeep Ravikumar, Cho-Jui Hsieh, and Liwei Wang. Macer: Attack-free and scalable robust training via maximizing certified radius, 2020.
Huan Zhang, Tsui- Wei Weng, Pin-Yu Chen, Cho-Jui Hsieh, and Luca Daniel. Efficient neural network robustness certification with general activation functions. Advances in neural information 2018.
processing systems, pp.
Huan Zhang, Hongge Chen, Chaowei Xiao, Bo Li, Duane Boning, and Cho-Jui Hsich. Towards stable and efficient training of verifiably robust neural networks. In International Conference on Learning Representations, 2020.

---

A IMPLEMENTATION DETAILS
We conduct experiments on classification models with similar structures as were used in prior works (Gowal et al. 2018 Zhang et al. 2020). Specifically, we adopt the D
small 1 and DM-1arr arg
models used in CROWN- IBP (Zhang et al. 2020). They are both convolutional models with 2 and 5 convolutional layers respectively, and their details have been reported in Zhang et al. (2020). On CIFAR- 10, we use DM-1a Myrlarge, and on MNIST, we find that DM-large has a serious overfitting problem when training with learned perturbation budgets, and thus we use DM-sma 11 instead. Nevertheless, we still use DM- 1a a rgg to report the results of models with uniform budgets following the best setting in CROWN-IBP.
For the perturbation budget generator, it is a two-layer convolutional model with ReLU activations after each convolutional layer, and the two convolutional layers have 4 and 8 channels respectively and kernel size of 3. After that, there is a linear layer mapping the hidden layer to each pixel in the image, with a Sigmoid activation for computing initial perturbation budgets.
For training with certified defense, we mainly follow settings in CROWN-IBP (Zhang et al. 2020), but we use the loss fusion technique proposed in auto LiRPA for faster training (Xu et al. 2020). There is scheduling on the robustness volume during training, similar as the scheduling of uniform perturbation radius in CROWN-IBP. On MNIST, we use a linear scheduler, where the robustness volume linearly increases from the final target robustness volume in 60 epochs starting from the 2nd epoch. The model is trained for 100 epochs. On CIFAR-1O, we use a smoothed scheduler with a length of 160 epochs starting from the 32nd epoch, which slowly morphs into a linear schedule (Gowal et al. 2018). The model is trained for 320 epochs.
B VISUALIZATION ON MEDMNIST DATASET
MedMNIST YYang et al. 2020) is recently released dataset consisting of 10 subsets of lightweight 28 28 medical images. In this section, we utilize the OrganMNNIST(xiaal) subset consisting of CT images of body organs cropped from Abdominal CT images (Bilic et al. 2019 Xu et al. 2019) as an example, to further justify why non-uniform contextual perturbation budgets are needed in training robust models in more practical scenario. In such medical images, classifiers need to identify and classify abnormal regions in body organs. In these complex medical images, some regions should be rather sensitive to adversarial perturbations, e.g., if regions right correspond to specific abnormality is perturbed, the image label is very likely to change, while some other regions may be able to tolerate larger perturbations. This is practical situation where it is supposed to especially useful to leverage non-uniform and contextual perturbation budgets for robust training. For an initial demonstration, we train model with 6 16/255 using learned contextual perturbation budgets and uniform budgets respectively. For the model with learned budgets, the clean error is 21.8 and the verified error is 50.6, while for the model with uniform budgets, the clean error is 28.0 and the verified error is 57.5. Using learned contextual perturbation budgets, we are able to obtain a model with lower clean and verified errors under the same target robustness volume. We also visualize several input images and our learned perturbation budgets in Figure 6 The learned perturbation budgets appear to be semantically meaningful and align with the input images.
C CONTROLLING PERFORMANCE ON UNIFORM PERTURBATION BUDGETS
In our experiment results shown in Table D and Table we find that models trained with our learned budgets tend to have higher verified errors when the models are evaluated with uniform budgets. Although we have argued that evaluation on uniform budgets is not our major goal and it is less practical than learned non-uniform budgets, in this section, we demonstrate that it is still possible to maintain a performance on uniform budgets while training with learned budgets by controlling a. We train model on MNIST with 6 0.5 and = 0.6, which means the minimum budget of each pixel is 0.3. As a result, verified error on non-uniform budgets is 10.46, while the verified error on uniform 0.3 budget is 9.96, which is close the 9.40 verified error reported in CROWN-IBP ZZhang et al. 2020) on the same "DM-small" model trained with uniform budgets. And if eo = 0.6, = 0.5, the verified error on uniform 0.3 is 1 11.22 which is also not far away from 9 9.40, while now our model has a robustness on a volume as large as 0.6".
12

---

Input
Budget
Label
Input
Budget
Label
10
Input
Budget
Label
10
Figure 6: Visualization of input images and learned perturbation budgets on the Organ))NIST(axial) subset in the MedMNIST dataset.
13


Reviews:
[{"review_id": "SDtweLl0xEg", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper proposes to change the perturbation budget for adversarial attacks to a non-uniform setting where differet input pixels have different perturbation budgets. To achieve this, an additional network is trained to learn the perturbation budget for each part of the input. The approach seems to perform better than a uniform perturbation budget and also learns semantically meaningful budgets for the input.\n\nI am not an expert on this topic and will, therefore, keep my review quite short.\n\nThe idea that not all parts of the input should be treated equally makes sense and is well motivated.\n\nQuestions/remarks:\n- What is the exact architecture of the network that learns the perturbation budged? Is it purely convolutional? Will it easily scale to larger inputs?\n- It would be interesting to see the performance of your model on more complicated datasets, e.g. Tiny-ImageNet\n- How much overhead (training time, model size, etc) does the training of the additional network for the perturbation budget introduce?\n- For the visualization of learned pertubation budget: it would be interesting to also run some analysis/visualization of what parts of the input have the strongest effect on the final prediction to see if this correlated with your learned perturbation budged (i.e. if the parts of the input that have the strongest effect on the final classification also have the smallest perturbation budget)", "extra_scores": {}, "extra_text": {}}, {"review_id": "QICd9osZivN", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper address the problem of training robust neural networks with non-uniform perturbation budgets on different input pixels. In practice, a perturbation budget generator is introduced to generate the context-aware perturbation budget (i.e. conditioned on the input) for each pixel of the input image. A “robustness volume” constraint on generated perturbation budgets to control the robustness intensity is also proposed. Extensive experiments on MNIST and CIFAR10 demonstrate the proposed outperform SOTA method under various uniform perturbation budgets.\n\n\nFrom my perspective, the writing of this paper is good, and contributions are well presented and demonstrated by extensive experiments. So I vote for accepting\n\nComments:\n\n- How to determine the hyper-parameters \\bar{alpha} and \\underline{alpha} for each benchmark is still unknown. Are the final results sensitive to these hyper-parameters? Does it take a high cost to adjust these hyper-parameters for different benchmarks?\n- How about the performance by using IBP?\n\n- In Eqn.2, The index i indicates the i-th pixel. But in Eqn.3, it denotes the i-th category label. Please modify this to avoid misunderstanding.\n\n- Since I'm not very well versed with the current baseline and state-of-art for variable robust training of DNN, it would be good to compare with other SOTA methods.", "extra_scores": {}, "extra_text": {}}, {"review_id": "KMVUjUsdNpx", "reviewer": "AnonReviewer5", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The paper proposes using a budget generator to generate non-uniform radii for certification. The budget generator is trained jointly with the certified defense to change the shape of the perturbation set while maintaining the volume to improve certified error. \n\nOn motivation: the paper could use some more justification or motivation for why we would want to change our perturbation radius during training to maximize certified performance. Typically this the other way around: we have a set we want to defend against, and so the certified defense optimizes for this specified set since we're trying to defend against a particular attack. The setup here is strange in this regard, because rather than adapting a defense to a threat model, the threat model is being adapted to the defense, where the defense is defined by a fixed volume but is otherwise whatever the defender trains it to be. Since an attacker would not conveniently restrict themselves to  the radius learned during training, this doesn't really make much sense from the point of view of certifying robustness to adversarial examples (since it doesn't defend against *all* perturbation sets with the specified volume, it only defends against one which isn't specified a priori). \n\nOn comparisons: The authors compare their certified defense with non-uniform budgets to certified defenses with uniform budgets. In its current form, this is completely incomparable: the uniform budgets are trained and certified with uniform radius, while the learned budgets are trained and certified with learned budgets. Since the learned budgets are almost certainly different from the uniform budgets, these are completely different threat models. It would be much more informative to the reader to report results on *both* types of budgets for *both* models instead, rather than only showing half of the story. Specifically, this means \n(a) evaluating all the models trained with learned budgets using the uniform budgets that are more typical in the literature and reflect what they were actually trained for (e.g. <= 0.4 for MNIST, <= 8/255 for CIFAR10)\n(b) evaluating the models trained with uniform budgets using the learned budgets  to compare against the models trained with learned budgets\nIt's quite possible that the learned budgets, while capable of certifying more volume due to the changed threat model, comes at the cost of worse certified performance for a uniform bound of a *smaller* volume (e.g. at uniform radius 0.1 or 0.3 for MNIST, as is commonly reported in the literature). This would also help provide a more realistic and fair comparison to certified defenses with uniform budgets: the current tables report certified accuracy at an extremely large radii well beyond what they were trained for, and so this winds up being a rather a misleading comparison that is not very useful. \n\nOn related work and comparisons thereof: The authors seem to be unaware of the ICML 2019 publication \"On Certifying Non-Uniform Bounds against Adversarial Attacks\" by Liu et al., which has studied the problem of certifying non-uniform bounds which maximize the volume (exactly the same type of bound studied in this work). There is still a difference, in that they optimize the budget to maximize the volume rather than use a generator to produce perturbation budgets, and do not train. Nonetheless, this is arguably the most relevant work and has been out for quite a while, and so it would be fair to expect some sort of comparison here. For example, a reasonable experiment could be to simply calculate the non-uniform bounds from Liu et al. on the model trained with a non-uniform budget vs the uniform budget. \n\nMinor comment: The authors mention that the joint training of the classifier and the perturbation budget generator is somehow more difficult for PGD adverarial training \"as it is not fully differentiable w.r.t. perturbation budgets\". I don't quite get what the authors are trying to say here. My understanding is that the authors perform joint training by backpropagating the robust loss through both the classifier and the perturbation budget generator, since there is no auxiliary loss for the perturbation budget generator. Shouldn't this imply that the standard loss is in fact differentiable with respect to the perturbation budgets, and so PGD is just as applicable as before? \n\nUpdate: \nI thank the authors for their response. I've read the other reviews as well, and indeed R2 had similar concerns to my own. I'm glad to see the more comprehensive comparison to Liu et al., which paints a fuller picture of the effects and trade-offs of the approach. \n\nThe argument behind the motivation, however, feels much like setting up a straw man for Lp robustness. For example, the authors argue that their approach is label and semantics preserving unlike uniform perturbations; however this is quite frankly only the case for extremely large perturbations in MNIST-like settings which are unrealistic by design (most papers do not consider such large radii for exactly this reason). Uniform perturbations seen commonly in CIFAR10/Imagenet settings are practically invisible and consequently are equally semantics preserving and close to human perception. If the authors do wish to pursue this argument that these are truly more semantics preserving, then this needs to be backed by evidence. The authors weakly suggest this is the case because the budgets look similar to the content in the images. However, this does not imply that an adversarial attack within this budget is label preserving (i.e. many of the presented examples have large budgets in the background directly adjacent to the label-content of the image, which can easily change how the content looks), and so this needs to be justified carefully if this claim is to be made. \n\nThe authors also incorrectly equate the restrictions imposed on an attacker from learned perturbation radii to that of a uniform radius. These are *not* equivalent, especially in the security setting where these are night and day; the first amounts to the defender choosing the rules of the game that work optimally for them, whereas the latter is a *defense agnostic* rule that both the defender and attacker must obey. This is a significantly easier setting for the defender that needs to be properly motivated, as restricting an adversary to a fixed perturbation set is inherently different from restricting the adversary to a fixed perturbation set that the defender gets to choose. The reason why one would want to maximize certification volume needs to be properly motivated, as it is no longer applicable to the usual adversarial security setting and comes at a cost to the usual robustness considerations. \n\nTo recognize the addition of the necessary comparison to past work, I have improved my score slightly. However, I would still argue that this is below the threshold, as their central claim of learning *semantic preserving* perturbation budgets is not justified despite being a central component of the paper, as well as the motivation for why it's considered beneficial to choose the most easily certified volumes for robustness in the first place (and certainly not helpful from a security perspective). ", "extra_scores": {}, "extra_text": {}}, {"review_id": "0NJiIASOXIW", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Update after the rebuttal:\n\nI read response from the authors and other reviews. I have increased my score to 5 given that authors now performed some\ncomparison with Liu et al. However, I still believe that the threat model is not realistic and that attacker can not be bounded by the \nbudget that is produced by generator network here. While authors wrote quite detailed response, I did not find it convincing enough. As R5 points out similar issues, I would encourage authors to think more on how to tackle the problem better. Perhaps the threat model can limit the attacker to all possible perturbations under the certain volume budget, but that would be quite different from the idea in this paper.  Thus, I can not recommend acceptance at the current state.\n\n============================================================================\n\n-> Summary:\n\nAuthors propose to train provably robust models with respect to the threat model\nthat assumes non-uniform perturbation budget for the attacker. Under this threat model,\nattacker is allowed to perturb some pixels more than the others, unlike for the standard\nl_p threat model here each pixel has a maximum perturbation value. In this work, authors\npropose perturbation generator which produces bounds for each pixel and certified defense\nto train provably robust model with respect to the generated perturbation bounds. They show\nthat trained models have lower natural and verified error than models trained with uniform\nbounds of equivalent volume.\n\n-> Reasons for score:\n\nI vote for rejecting this paper. The biggest issue I have is that authors do not compare (and do not cite) \nwith the work of Liu et al. [1] whose contributions I believe substantially overlap with contributions of\nthis submission. The other concern I have is that the idea of learning a perturbation budget which\nthe attacker should obey seems somewhat unrealistic.\n\n-> Pros:\n\n- I like the general idea of a threat model where maximum perturbations is different for each pixel.\n- Paper is well written, easy to follow and manages to bring across the main points.\n- The authors perform experiments on several datasets and present visualizations which help to understand\nwhat the method has actually produced.\n\n-> Cons:\n\n- The biggest issue I have with this paper is omission of a prior work of Liu et al. [1] which \nwas the first paper to consider non-uniform perturbation bounds. Liu et al. consider same\nvolume constraint as in Equation (4) in this submission and propose a Lagrangian method to maximize\nlog of the volume. I think it is essential that authors compare their perturbation generator in section 3.2\nwith the Lagrangian method from Liu et al. and check which of the two methods can generate larger volume. \nOne contribution that this submission has and Liu et al. does not is that of certified defense as Liu et al.\nfocus only on certification of already trained networks (I am not sure if their method can be trivially extended \nfor training). However, given that authors simply use auto_lirpa library for certified defense, contributions there\nare also limited.\n- I am not sure that idea of generating perturbation budget is feasible in practice. What happens here\nis that we generate the threat model under which attacker operates by ourselves and then we somehow expect \nthat attacker should obey this threat model. For example, let's say that proposed procedure results in \neps(x) = [0.3, 0.1, 0.1]. Why would attacker have to obey the fact that third pixel is perturbed by at most 0.1?\nEssentially, the method here learns eps(x) such that attacker can't find adversarial example, but I don't understand\nin what scenario would attacker be limited by the *particular* eps(x) that this method has produced?\nI think it would be more sensible to have a defense which guarantees that model is robust under all threat models\nwhich have volume below some constant V_0, but on the other hand this would be strictly more difficult than the \nuniform baseline.\n- Another thing I think is missing is some baseline for computing perturbation budget introduced in 3.3.\nThis relates to work of Liu et al. who proposed Lagrangian method to do this. Given that algorithm in 3.3 \nis perhaps the biggest contribution of this work, I would have expected there to be at least some baseline to compare with.\n\n[1] Liu, Chen, Ryota Tomioka, and Volkan Cevher. \"On certifying non-uniform bound against adversarial attacks.\" ICML 2019.\n\n\n-> Questions:\n\n- Can authors comment on the work of Liu et al. [1] and what they think are main contributions of their work\nwhich are not already present in Liu et al.? Ideally, authors should also compare to their method (at least\ncertification part as they have no training).\n- Can you explain what would be realistic scenario where attacker has to obey perturbation budget generated by this method?\n\n-> Minor comments:\n\n- typo: \"generailized\"\n- In \"Refining the perturbation budgets\", third line, should it be [g_\\theta(x)]_i under the summand?\n", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: Reviewers raised various concerns about the motivation, unclear justification of the idea and claim, insufficient comparison with related work, and weak experimental results. While authors had made efforts to improve some of these issues in the rebuttal, the revision was not satisfied for publication quality. Overall, the paper has some interesting idea, but is not ready for publication.  
Decision: Reject