GSmooth: Certified Robustness against Semantic Transformations via Generalized Randomized Smoothing
Authors: Hao Zhongkai
Chengyang Ying
Yinpeng Dong
Hang Su
Jun Zhu
Venue: ICLR 2022 Submitted (2022)
Keywords: Randomized Smoothing
Adversarial Robustness
Semantic Transformations
Machine Learning

Abstract: The vulnerability of deep learning models to adversarial examples and semantic transformations has limited the applications in risk-sensitive areas. The recent development of certified defense approaches like randomized smoothing provides a promising direction towards building reliable machine learning systems. However, current certified defenses cannot handle complex semantic transformations like rotational blur and defocus blur which are common in practical applications. In this paper, we propose a generalized randomized smoothing framework (GSmooth) for certified robustness against semantic transformations. We provide both a unified and rigorous theoretical framework and scalable algorithms for certified robustness on complex semantic transformations. Specifically, our key idea is to use a surrogate image-to-image neural network to approximate a transformation which provides a powerful tool for studying the properties of semantic transformations and certify the transformation based on this neural network. Experiments on multiple types of semantic perturbations and corruptions using multiple datasets demonstrate the effectiveness of our approach.

Under review as a conference paper at ICLR 2022
GSMOOTH: CERTIFIED ROBUSTNESS AGAINST SEMANTIC TRANSFORMATIONS VIA GENERALIZED
RANDOMIZED SMOOTHING
Anonymous authors Paper under double-blind review
ABSTRACT
The vulnerability of deep learning models to adversarial examples and seman- tic transformations has limited the applications in risk-sensitive areas. The re- cent development of certified defense approaches like randomized smoothing pro- vides promising direction towards building reliable machine learning systems. However, current certified defenses cannot handle complex semantic transforma- tions like rotational blur and defocus blur which common in practical applica- tions. In this paper, we propose a generalized randomized smoothing framework (GSmooth) for certified robustness against semantic transformations. We provide both a unified and rigorous theoretical framework and scalable algorithms for cer- tified robustness on complex semantic transformations. Specifically, our key idea is to use a surrogate tion which provides a powerful tool for studying the properties of semantic trans- formations and certify the transformation based on this neural network. Experi- ments on multiple types of semantic perturbations and corruptions using multiple datasets demonstrate the effectiveness of our approach.
neural network to approximate transforma-
INTRODUCTION
Although deep learning models have achieved remarkable success on various applications (LeCun et al., 2015), they are vulnerable to adversarial examples (Biggio et al., 2013; Szegedy et al., 2013; Goodfellow et al., 2014) and semantic transformations (Hendrycks & Dietterich, 2019). The vulner- ability of deep learning models can limit their applications on many important tasks. For example, the autonomous driving system can be misled even by a small adversarial patch on the road mark (Jing et al., 2021). Compared with the maliciously crafted adversarial examples, semantic transfor- mations are more practical in real-world scenarios, such as rotation, translation, blur, bad weather, and on. Such transformations do not damage the semantic features of images and can be easily recognized by humans, but they also degrade the performance of deep learning models. Therefore, it is imperative to improve model robustness against semantic transformations.
To develop more reliable machine learning systems, many efforts have been made to design defense techniques against adversarial attacks or semantic transformations. The existing defense methods can be categorized into empirical defenses and certified defenses. Adversarial training (AT) (Madry al., 2017; Zhang et al., 2019a) is one of the most effective empirical defenses against fp-norm bounded adversarial examples. Moreover, methods based on data augmentation (Hendrycks et al., 2019: Wang et al., 2019; Calian al., 2021) have been proposed to empirically improve the perfor- mance under semantic transformations. However, the performance of empirical defenses is difficult to be fully justified and these defenses can be further broken by new adaptive attacks (Athalye al., 2018; Tramer et al., 2020). In contrast, the certified defenses aim to theoretically provide certified region where the model theoretically safe under any attack or perturbation (Wong & Kolter, 2018; Cohen et al., 2019; Gowal et al., 2018; Zhang et al., 2019b). Along this line, developing certified defense methods is a crucial step towards reliable machine learning systems.
Although certified defenses have achieved great success, most of them are limited to defend against f,,-norm bounded attacks. However, the L, distance between the original image and its corrupted counterpart by semantic transformation (e.g., translation, rotation) would be large when the

---

Under review as a conference paper at ICLR 2022
corrppiion is slight. Therefore, the current methods are incapable of certifying robustness against such semantic perturbations. To solve this problem, several recent works (Fischer et al., 2020; Mohapatra et al., 2020; Li et al., 2021) attempt to extend the certified defenses to several simple semantic corruptions, including translation, rotation, and Gaussian blur. However, these works are not scalable to certify robustness against complex and general semantic perturbations. First, deter- ministic certified defenses (Mohapatra et al., 2020) based on convex relaxation for the activation function require solving complex optimization problem for computing bound which is computa- tionally expensive. Second, probabilistic approaches based on randomized smoothing also demand a handcrafted Lipschitz bound (Li et al., 2021). which intractable for complicated semantic trans- formations. For example, many semantic transformations such as glass blur and pixelate do not have a closed form expression or they are black boxes and hard to be analyzed theoretically, but they are common in real-world scenarios. Therefore, it is still highly challenging to certify robustness against these complex and realistic semantic transformations.
To address the aforementioned challenges, we propose generalized randomized smoothing frame- work (GSmooth). First, we provide a unified framework of GSmooth for certifying general seman- tic transformations. Then we categorize the transformations into resolvable transformations (e.g., translation) and non-resolvable transformations (e.g.. rotational blur) similar with Li et al. (2021). As mentioned above, most non-resolvable transformations are complex and the existing methods cannot provide their certified radius. To handle the challenge, we propose to use an image-to-image translation neural network to approximate all these transformations. Due to the strong capacity of neural networks, our method is flexible and scalable to model these complex semantic transforma- tions. By introducing an augmented noise in the layers of the surrogate model, we can theoretically provide the certified radius for the proxy neural networks which can be used for certifying the orig- inal transformations. Next, we provide theoretical analysis and error bounding for the approxima- tion. Finally, we validate the effectiveness of our methods Extensive experimental results demonstrate that our methods are effective for certifying complex semantic transformations including different types of blur or image quality corruptions.
several publicly available datasets.
2 RELATED WORK
2.1 ATTACKS AND DEFENSES FOR SEMANTIC TRANSFORMATIONS
Unlike Ep perturbation which adds small noise to every pixel of an image, semantic attacks or phys- ical attacks are usually unrestricted. Brown et al. (2017); Song et al. (2018) use a small patch added to the image to mislead the classifier or the object detector. Engstrom et al. (2019; 2018); Xiao et al. (2018) construct adversarial examples using spatial transformations like rotation transla- tion. Hendrycks & Dietterich (2019) show that a wide variety of semantic perturbations degrade the performance for many deep learning models. Many works (Cubuk et al., 2019; Hendrycks al., 2019; 2020; Robey et al., 2020) propose diverse data augmentation techniques to enhance robustness under semantic perturbations. Calian et al. (2021) propose adversarial data augmentation that can be viewed as adversarial training for defending semantic perturbations. Beyond empirical defenses, several works (Mohapatra et al., 2020; Madry et al., 2017: Singh et al., 2019; Balunović et al., 2019) attempt to cerúffy some simple geometric transformations. However, all of them belong to deterministic certification approaches and their performance on realistic datasets are unsatisfactory.
2.2 RANDOMIZED SMOOTHING
Randomized smoothing is novel certification method originated from differential privacy (Lecuyer et al., 2019). Cohen et al. (2019) then improve the certified bound and apply it to large scale deep neural networks and datasets. Yang et (2020) exhaustively analyze the robust radius by using different noise distribution and norms. Hayes (2020); Yang et al. (2020) point out that randomized smoothing suffers from curse of dimensionality for the Lo norm. Salman et al. (2019) adopt adver- sarial training to train smoothed classifiers to obtain better robustness guarantees. Li et al. (2021); Fischer et al. (2020) extend randomized smoothing to certify some simple semantic transforma- tions, e.g., image translation and rotation. It shows that randomized smoothing could be generalized to certify more diverse attacks or corruptions. However, their methods are limited to simple semantic transformations, which are easy analyze their mathematical properties.
2

---

Under review as a conference paper at ICLR 2022
3 PROPOSED METHOD
In this section, we present the framework and theoretical analyses of our Generalized Randomized Smoothing (GSmooth). We first introduce the basic notations. Then we divide the semantic trans- formations into resolvable transformations and non-resolvable transformations similar with Li et al. (2021). Next we introduce the details of our GSmooth for these two types of semantic transforma- tions, respectively. Finally, we show the theoretical insight and proof sketch of our main results.
3.1 NOTATIONS
We first introduce the notations and formulation of the task. Given the input of x E R" and the labels of = {11, 2........ p}. probabilities over all p classes. The prediction of f is arg max fz);, where f();, denotes the i-th element of f()). Let (f,, x) R" x R + I= be semantic transformation of raw input z with
we denote the classifier as f(x) R" + [0, 1|2. which outputs predicted
parameter 0 E RT. We define the smoothed classifier as
G(x) gg (T(,
(1)
which is the average prediction for the samples under smoothing distribution g(0) where g(0)
exp(-r((0) and (0) is smooth function from IR - R. Let ||ul| = be any vector with unit
norm and random variable 7a = (u, V (o)( where 5 9 and V the gradient operator of
function. The complementary CDF is Pa (c) = Pltu > c] and the inverse complementary CDF is
Pu (p) inf(c/ > c) < p). Following Yang al. (2020), we define function & as
(p)) = u ElyuT(% > Pp (p))).
(2)
which will be used to represent the certified radius. Let y4= arg max G(z); be the predicted label by the smoothed classifier (zx) and yB mag, G(z)), is the runner-up class. With- out causing confusion, we use G(r)) to denote the probability of the top class G(r))x likewise for G(r))B-
3.2 CERTIFIED BOUND FOR RESOLVABLE SEMANTIC TRANSFORMATIONS
We first discuss a class of transformations that are resolvable - the composition of two transforma- tions with parameters belonging to perturbation set e,g & P I is still a transformation with
a new parameter 70, E) E P C Rm where 7(:.)) P x P + is a function depending on these parameters. For resolvable semantic transformations,
have the following theorem.
Theorem 1. Let f(x) be any classifier and G(z) be the smoothed classifier defined in Eq. (1). If there exists function M M(,-) P P - IR, the transformation T(:-)) satisfies
@r(0. {) 0)(0,E) M (0,E))
0c and there exist two constants PA PB satisfying
00
G(z) > PA > PB 2 G(x) By
then y = arg gy G(T(E,z) holds for any ||El| < R where
PA 1
R
dp,
2M
Pm (pp)
and M mazq,err|| 0)I
Remark. The settings of Theorem I are similar with Li et al. (2021) for resolvable semantic transfor- mations. But here we adopt different presentation and proof for the theorem which could be easier to extend to our GSmooth framework for general semantic transformations. Specifically, we show two examples of the theorem which are additive transformations and commutable transformations. A transformation is additive if 7(8, T(E, x)) = 7(5 + 6, .²) for any 0, ç E P; or it is commutable if E P. For these two types of transformations, it is straight- forward to verify that they satisfy the property proposed in Theorem 1. As an example, we simply
T00,r(E,x)) x)) = T(C, (0,, x)) for any E
apply Theorem for isotropic Gaussian distribution g0) = ~N(0, o²1) !) and get the certified radius
R
(AA)
V (pg)
(4)
2
3

---

Under review as a conference paper at ICLR 2022
Surrogate Model
Base Classifier
Transforation Parameters
(())
R
H(-)
(xx)
Ingul. Image
Aummented Noisy Images
Cartified Radius
Nolse
Augmented Noise
A graphical illustration of our GSmooth. We use a surrogate taage-to-image translation network to accurately fit semantic transformation. Then add new augmented noise into the surrogate model and construct the GSmooth classifier. The augmented noise are sampled to ensure the transformation to resolvable in the semantic space. We theoretically calculate the certification bound for the surrogate model to certify the original semantic transformation.
Figure 1:
where is the inverse CDF of the standard Gaussian distribution. These two kinds of transforma- tions include image translation and Gaussian blur, which are basic semantic transformations and widely discussed in previous works (Li et al., 2021; Fischer et al., 2020). The certification of these simple transformations only requires applying translation Gaussian blur to the sample and gets the average classification score under the noise distribution.
3.3 CERTIFIED BOUND FOR GENERAL SEMANTIC TRANSFORMATIONS
Translation and Gaussian blur are two specific cases of semantic transformations. In practice, most semantic transformations are not commutable or even not resolvable. Therefore, we need to de- velop better methods for certifying more types of semantic transformations. However, the existing methods like Semanify-NN (Mohapatra et al., 2020) based convex relaxation and TSS (Li al., 2021) based on randomized smoothing require to develop a specific algorithm or bound for each in- dividual semantic transformation. This is not scalable and might be infeasible for more complicated transformations without explicit mathematical forms.
To address the challenge, draw inspiration from the fact that neural networks able to ap- proximate functions including a complex and unknown semantic transformation (Zhu et al., 2017). First, we propose
surrogate image-to-image translation model accurately fit the semantic transformation. Then we theorectially show that by introducing an augmented noise in the layers of the surrogate model, as shown Fig randomized smoothing can be extended to handle these transformations. Specifically, we define the surrogate model as the following form that will lead to a simple certification bound we shall see:
x0, = (FF(()) + F2(x)),
(5)
where Fi(.) R" + RR. F2(:) R + Rd, and H(-) Rd + R" are three individual neural networks. F-(-) and F-(-) are the encoders for transformation parameters and images respectively, and their encodings are added together in the semantic space which is critical for its theoretical certification. We find that the surrogate neural network is much easier to analyze and can be certified by introducing dimensional augmentation strategy for both noise parameters and input images.
As illustrated in Fig. 1, an augmented noise is added to the semantic layers H(-) in the surrogate model. Our key insight is that the transformation could be viewed the superposition of resolvable part and a non-resolvable residual part in the augmented semantic space. Then we could use the augmented noise to control the non-resolvable residual part if the augmented dimension d > m + n. This dimension augmentation is the key step our technique. The augmentation for noise is from R. Rdd To keep the dimension consistent, also augment data to Rdd by padding 0 entries.
4

---

Under review as a conference paper at ICLR 2022
By certifying the transformation based on the surrogate model, are able to certify the original transformation if the approximation error is within an acceptable region (detailed analysis in Theorem 3). Our method is flexible and scalable since the surrogate neural network has uniform mathematical form for theoretical analysis and they are trained automatically. Next, we discuss the details of GSmooth.
Specifically, we introduce the augmented data ã E Rd and the augmented parameter E Rd as
x 000 )
0 0r
where the additional parameters 0' E R" sampled from g( and the joint distribution of #' and 0 is 9g where g(0) g (0')g(0) Moreover, the augmented data x' R". We define the generalized smoothed classifier as
= = [()(
where is the "augmented target classifier" that equals the original classifier when constrained on the original input which means f()) fzz). This can be achieved by setting the weights of additional dimensions to O. Note that now all the functions are augmented for a dimensional
input. We further augment our surrogate neural network to represent the augmented transformation T Rd Rdd
where H(),, 7O)A())) Rd Rd are parts of the augmented surrogate model. By carefully designing the interaction between the augmented parameters and the original parameters, we could turn the transformation to resolvable one and does not change the original surrogate model when constraining to the original input x and 0. Specifically, we design the function F; and F2 as follows:
(0) ( Fi + 0' ) FL(())
In-n
(
(9)
(2(
H(x)
Before stating our main theorem, we introduce several notations म = 2n F2( De = (v/ TT) =
and ÿo = (V6- yo) = (0)+ for simplicity.
Then we theoretically prove that the GSmooth classifier is certifiably robust within given range:
Theorem 2. Suppose f(x) is classifier and G(t) is the smoothed classifier defined in Eq. (7), if there exist PA and pB satisfying
G(T)A > PA > PB G(
then ya arg max G(F(E,))) for any ||El12 R, where
PA
R
dp,
(10)
2M
{pp)
PB
and the coefficient M is defined as
1 FF
(F) 2
M
max E
(11)
ag
00
As the main result of our GSmooth, we have several observations about it. First, we see that the certified radius is similar to the result in Theorem 1. Second, compared with resolvable transforma- tions, we need to add new type of noise when constructing the GSmooth classifier. This isotropic noise has the same dimension as the data and is added to the intermediate layers of surrogate neural networks. The theoretical explanation behind this is that this isotropic noise makes the Jacobian matrix of the semantic transformation be invertible which is crucial for the proof. Third, we ob- serve that the coefficient M depends on the norm of the difference of two Jacobian matrices and is independent with the target classifier, later will discuss the meaning of this term in detail.
Before diving into our theoretical insight and proof of the theorem, we introduce a specific case of Theorem 2 which is more convenient for practical usage. We empirically found that taking
5

---

Under review as a conference paper at ICLR 2022
linear transformation (8) = A10 b1 where A1 Ruxm, b1 R"" does not sacrifice the
precision of the surrogate network and we have (0) A1. After substituting the term in Eq. (44)
06
only need to optimize & for calculating M and we make the bound tighter. Additionally, use two gaussian distributions for the noise distribution and the augmented noise distribution, i.e.
g(0) (0 I) and g' (0") N(0,, Formally, we have the following corollary.
Corollary 1. Suppose f(x) classifier and (āž) is the smoothed classifier defined in Eq. (7), if the layer (0) in the surrogate neural network has the following form:
(i = A,0 + b
(12)
where A Ruxm, bj E R are the parameters; and if there exists PA and pg satisfying
G(+)a > PA > PB > G(Ø)
then yA arg max max JEY G(F(E, for any ||El|2 < R where
R
(v (PA) v (TB)),
(13)
2M
where v()) is the inverse CDF of standard Gaussian distribution, and the coefficient M is defined as,
1 1 02
(F2(UE)
M max &EP
A1
(14)
4 PROOF SKETCH AND THEORETICAL ANALYSIS
4.1 PROOF SKETCH OF OUR MAIN THEOREM
In this section, we briefly summarize the main idea for proving the Theorem and the theoretical insight of our GSmooth. The key idea is to prove that the gradient of the smoothed classifier can be bounded by a function of the classification confidence and the parameters of the noise distribution. Formally, we calculate the gradient to the perturbation parameter for our augmented smoothed classifier as
VeG(F(E, z)) (F(00,
(15)
We expand the expectation into integral and see that
ffr 0i(7(0, ūe)) g())dā
V (((((EE)))
(16)
Janid The key step is to climinate the gradient @F(F((.5T)) and replace with (f()) 0c
a
Then we
06
integrate it by parts to get the following obejective.
V(O(F(( = -f-
F(F ūc) O ( (M( 08
(17)
JR
After that, we could bound the gradient of the GSmooth classifier using the technique similar to randomized smoothing (Yang al., 2020). More details of the proof can be found in Appendix A.
4.2 THEORETICAL INSIGHT
Next, we provide the theoretical insight for our augmentation scheme on transformation parameters and data. First, the key is to expand the transformation space by adding additional dimensions to form closed space. In the augmented space, the Jacobian matrix of the semantic transformation became invertible which is crucial for our proof. Second, as influenced by two factors. One is the standard deviation of two noise distributions. The other the norm of the Jacobian matrix @Fz(ve) Ai. It can be viewed the residual of the non-resolvable BE part of the transformation. Along this line, our method decomposes the unknown semantic transfor- mation into a resolvable part and a residual part. The non-resolvable residual part could be handled by introducing additional noise with standard deviation d2.
in Eq. (14) that M is

---

Under review as a conference paper at ICLR 2022
4.3 ERROR ANALYSIS FOR SURROGATE MODEL APPROXIMATION
In this subsection, we theoretically analyze the effectiveness of certifying real semantic transforma- tion due to the existence of approximation error of surrogate neural networks.
Theorem 3. Suppose the simulation of the semantic transformation has an small enough evror
IIF(E, T(5 (15.2)112 S,
Then there exists constant ratio A = A(I)F)(()) ( (F2(2)))la) > 0 does not depend on the target classifier, we have the certified radius for the real semantic transformation satisfies that
R > R(1 AE)
where R is the certified radius for surrogate the neural network in Theorem 2 and
PA
R
dp,
2M
((p)
We find that the reduction of the certified radius is influenced by two factors. The first one the approximation error between the surrogate transformation and the real semantic transformation. The second one the ratio A is about the norm of the Jacobian matrix for some layers of the surrogate model which is also inherent property of the semantic transformation itself and does not depend on the target classifier.
5 EXPERIMENTS
5.1 EXPERIMENTAL SETUP AND EVALUATION METRICS
In this section, conduct extensive experiments to show the effectiveness of GSmooth various types of semantic transformations. We use MNIST, CIFAR- 10 and CIFAR-100 (Krizhevsky al., 2009) datasets to verify methods. We train resnet with 110 layers (He et al., 201 from scratch. Similar to prior works, we apply moderate data augmentation (Cohen et al., 2019) improve the generalization of the classifier. For the surrogate image-to-image translation model for simulating semantic transformations, we adopt the U-Net architecture (Ronneberger et al., 2015) for (-) layers and several simple convolutional or linear layers for (-(+) and E.(.). All models are trained using Adam optimizer with an initial learning rate of 0.001 that decays every 50 epochs until convergence. The algorithms for calculating M and other details in our experiments are listed Appendix B due to limited space. The evaluation metric the certified accuracy, which the percentage of samples that are correctly classified and has a larger certified radius than the given range. We use ||a|| to indicate the preset certified radius.
5.2 MAIN RESULTS
To demonstrate the effectiveness of our GSmooth on certifying complex semantic transforma- tions, we measure the certified correct accuracy for different semantic transformations on differ- datasets in Table We compare the results of our GSmooth with several baselines, including randomized smoothing for some simple semantic transformation of TSS (Li et al., 2021) and Indi- vSPT/distSPT (Fischer et al., 2020), and our GSmooth is natural and powerful extension of their methods. We also compare our method with the deterministic certification approaches, including DeepG (Balunović et al.. 2019) that uses linear relaxations similar Wong Kolter (2018), Inter- val (Singh et al., 2019) that is based on interval bound propagation, VeriVis (Pei et al., 2017) that enumerates possible outcomes for semantic transformations with finite values of parameters, and Semanify-NN (Mohapatra et al., 2020) which uses new preprocessing layer to turn the problem into a 4p norm certification.
We have the following observations for the experimental results. First, only method achieves non-zero accuracy on certifying some complex semantic transformations and the results verify our Theorem 2. This is breakthrough that greatly extends the boundary of randomized smoothing based methods. Second, we see the performance of GSmooth is similar to the sfatc-of-the-ar ran- domized smoothing approaches like TSS on several simple semantic transformations like Gaussian
7

---

Under review as a conference paper at ICLR 2022
Cortified Accuracy(%)
Cent Aco(%)
Type
Dataset
Allack Range
GSmooth (Ours)
TSS
DeepGi
Interval
VeriVii
Semanity- NN
IndivSPTY
disiSPT
MNIST
Ialla <
91.0
90.6
Additive
||el|2
CIFAR- 100
22.1
21.0
MINITT
|ella <
98.7
99.6
0.0
0.0
98.8
96.8
99.6
Translation
Additive
||a||a < 20
0.0
78.8
CIAAR 100
42.2
41.3
24.2
24.2
MNIST
||0 0.5
97.7
97.6
6,0.4
0.0
5,74
Brightness, Contrast
Resolvable
CIAAR 10
||
82.5
82.4
0.0
0.0
CIFAR-100
42.3
41.4
0.0
0.0
|
<,8.88
6660
$,2.48
576
Rotation
Non-reselvable
CIFAR-10
||a||2 10
64.6
70.6
62.5
20.2
6,49.37
$34
CIFAR-100
33.2
36.7
0.0
0.0
$21.7
1.88
MNIST
|
93.9
97.2
85.0
164
Scaling
Non-resolvable
CIFAR-10
||a||a 0.3
54.3
58.8
0.0
0.0
CIFAR- 100
MNIST
||o||2
95.9
Rotational Blur
Non-resolvable
CIFAR-10
||o||2 10
39.7
CIFAR-100
MNIST
||l|2 <.6
89.2
Defocus Blur
Non-resolvable
CIFAR-10
||a||| <
25.0
CIFAR-10
13.1
MNIST
|| 0.5
93.9
Non-resolvahle
||o||2 < 0.5
CIFAR-100
14.2
MNIST
||o|]a < 0.5
87.1
Pixelate
Non-resolvable
CIFAR-I0
||o:]|2 <
45.3
CIFAR-100
30.2
Table Our main results of certification accuracy on several datasets and multiple types of semantic transformations. or 0.0% means the method fails to certify this type of semantic transformation.
folded gaussian
exponential
uniform
certifed accuracy
0.50
0.90
certified
01-
0.00
025
330
6.75
1.00
1.25
1.50
3.75
7.00
0.00
alpha
alpha
Figure 2: Results of ablation experiments on the influence of smoothing distribution for zoomed blur on CIFAR-10 10 dataset. The horizontal axis ||al] is the certified raidus.
blur, translation. This is a natural result since our method works similarly for resolvable transfor- mations. For two specific non-resolvable transformations, i.e. rotation and scaling, accuracy slightly lower. The possible reason that in TSS (L.i et al., 2021) they derive more elaborate Lipschitz bound for rotation which is better than us. Third, those inherently non-resolvable transfor- mations like image blurring (except Gaussian blur) and pixelate are more difficult than resolvable or approximately resolvable (rotation) transformations, Thus their certified accuracy is also lower.
5.3 ABLATION STUDY
Ablation study on the influence noise distribution. The choice of noise distribution is important for randomized smoothing based methods. Since our GSmooth could certify different types of semantic transformations that exhibit different properties. Understanding the influence of different smoothing distributions for different semantic transformations is necessary. We choose (folded) Gaussian, uniform, and exponential distribution and compare the certified accuracy on zoom blur transformation for both CIFAR-10 and CIFAR-100 datasets. As shown in Fig. 2, We found that
8

---

Under review as a conference paper at ICLR 2022
Cert Acc
CIFAR-T0
CIFAR- 100
0'2
0.05
0.10
0.15
0.25
02
0.05
0.10
0.15
0.20
0.1
44.3
46.4
35.5
16.1
0.1
23.1
26.2
17.2
10.4
Rotational Blur
0.25
45.7
47.1
38.3
18.9
0.25
23.2
27.2
20.3
11.3
46.5
48.4
38.8
18.3
0.5
22.2
26.1
18.6
11.8
0.75
42.1
45.5
36.6
17.0
0.75
24.0
25.5
18.3
13.2
Table Results ablation study on the influence of standard deviation of smoothing distributions, Lc. transformation noise øi and augmented noise d2 for certification accuracy rotational blur.
Figure 3: Visualization of difference between the augmented noise the semantic layers and the noise raw images. Left: original images from CIFAR-10. Middle: images with augmented noise of 02 = 0.2. Right: images with additive Gaussian noise a = 0.2.
the impact of smoothing distributions depends on datasets. Uniform distribution is better for small radius certification while exponential distribution is more suitable for certifying large radius on average.
Ablation study on the influence noise variance for certification. Since our GSmooth contains two different variances for controlling the resolvable part and the residual part for non-resolvable semantic transformation. Here we investigate the effect of different noise variance on the certified accuracy. The results shown in Table 2. We found that using medium transformation noise and augmented noise achieves the best certified accuracy. The fact is consistent with results in (Cohen al., 2019). An explanation is that there is trade-off since higher the noise variance decreases the coefficient M but it might also degrade the clean accuracy.
Visulization experiments: comparsion between augmented semantic noise and image noise. Our GSmooth needs to add new noise in the semantic layers of the surrogate model. Here we compare the difference between these two types of noise and visualize them. We random sample images from CIFAR-10 and add gaussian noise with = 0.2 to both the semantic layer of the surrogate model simulating zoomed blur transformation and raw images. Results are shown in Fig. 3. Both two types of noise severely blur the images. But we found that the augmented semantic noise is more placid which can therefore keep the holistic semantic features better, e.g., shapes.
CONCLUSIONS
In this paper, we proposed a generalized randomized smoothing framework (GSmooth) for certifying robustness against general semantic transformations. We proposed novel idea that using a surrogate neural network to fit semantic transformations. Then we prove tight certified robustness bound for the surrogate model and use it for certifying semantic transformations. Extensive experiments verify the effectiveness our method and we achieved state-of-the-ar performance on various types of semantic transformations. In the future, plan to extend our method real-world scenarios more diverse semantic transformations.
9

---

Under review as a conference paper at ICLR 2022
7 REPRODUCIBILITY STATEMENT
We ensure the reproducibility of our paper from three aspects. (1) Experiment: The implementation of our experiment is described in Sec. 5.1. Ablation study for experiments is in Sec. 5.3. Further details are in Appendix B. (2) Code: Our code is included in supplementary materials. (3) Theory and Method: A complete proof of the theoretical results described is provided in Appendix A.
ETHICS STATEMENT
Machine learning models easily attacked by adversarial examples and semantic transformations. Thus it is fundamental problem to develop certified robust machine learning methods. This paper proposed GSmooth certify against semantic transformations. It may promote the development of safe and reliable machine learning models in the future.
REFERENCES
Anish Athalye, Nicholas Carlini, and David Wagner. Obfuscated gradients give a false sense se- curity: Circumventing defenses to adversarial examples. In International Conference on Machine Learning (ICML), pp. 274-283, 2018.
Mislav Balunović, Maximilian Baader, Gagandeep Singh, Timon Gehr, and Martin Vechev. Certify- ing geometric robustness of neural networks. Advances in Neural Information Processing Systems 32, 2019.
Battista Biggio, Igino Corona, Davide Maiorca, Blaine Nelson, Pavel Laskov, Giorgio Giacinto, and Fabio Roli. Evasion attacks against machine learning test time. In Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 387-402, 2013.
Tom B Brown, Dandelion Mané, Aurko Roy, Martín Abadi, and Justin Gilmer. Adversarial patch. arXiv preprint arXiv: 17l2.0961...9 2017.
Dan A Calian, Florian Stimberg. Olivia Wiles, Sylvestre-Alve Alvise Rebuffi, Andras Gyorgy, Timothy Mann, and Sven Gowal. Defending against image corruptions through adversarial augmentations. arXiv preprint arXiv: 100:0:20400006,, 2021.
Jeremy Cohen, Elan Rosenfeld, and Zico Kolter. Certified adversarial robustness via randomized smoothing. In International Conference on Machine Learning, pp. 1310-1320. PMLR, 2019.
Ekin D Cubuk, Barret Zoph, Dandelion Mane, Vijay Vasudevan, and Quoc Le. Autoaugment: Learning augmentation strategies from data. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 113-123, 2019.
Foivos Diakogiannis, François Waldner, Peter Caccetta, and Chen Wu. Resunet-a: A deep learning framework for semantic segmentation of remotely sensed data. ISPRS Journal of Photogrammetry and Remote Sensing, 162:94-111 114,44114220 2020.
Logan Engstrom, Brandon Tran, Dimitris Tsipras, Ludwig Schmidt, and Aleksander Madry. A rotation and translation suffice: Fooling enns with simple transformations. 2018.
Logan Engstrom, Brandon Tran, Dimitris Tsipras, Ludwig Schmidt, and Aleksander Madry. Ex- ploring the landscape of spatial robustness. In International Conference on Machine Learning, pp. 1802-1811. PMLR, 2019.
Mare Fischer, Maximilian Baader, and Martin Vechev. Certified defense to image transformations via randomized smoothing. arXiv preprint arXiv:2002.12463. 2020.
Ian J Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. arXiv preprint arXiv: |412v:|412.6572 2014.
Sven Gowal, Krishnamurthy Dvijotham, Robert Stanforth, Rudy Bunel, Chongli Qin, Jonathan Ue- sato, Relja Arandjelovic, Timothy Mann, and Pushmeet Kohli. On the effectiveness of interval bound propagation for training verifiably robust models. arXiv preprint /810. 120. 2018.
10

---

Under review as a conference paper at ICLR 2022
Jamie Hayes. Extensions and limitations of randomized smoothing for robustness guarantees. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Work- shops, pp. 786-787, 2020.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In Proceedings of the IEEE conference computer vision and paitern recognition, pp- 770-778, 2016a.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Identity mappings in deep residual networks. In European conference on computer vision, pp. 630 630-645. Springer. 2016b.
Dan Hendrycks and Thomas Dietterich. Benchmarking neural network robustness to common cor- ruptions and perturbations. arXiv preprini arXiv: 1903. /2261, 2019.
Dan Hendrycks, Norman Mu, Ekin D Cubuk, Barret Zoph, Justin Gilmer, and Balaji Lakshmi- narayanan. Augmix: A simple data processing method to improve robustness and uncertainty. arXiv preprint arXiv:1912.027800 2019.
Dan Hendryeks, Steven Basart, Norman Mu, Saurav Kadavath, Frank Wang, Evan Dorundo, Rahul Desai, Tyler Zhu, Samyak Parajuli, Mike Guo, al. The many faces of robustness: A critical analysis of out-of-distribution generalization. arXiv preprint arXiv:2006. 16241, 2020.
Pengfei Jing, Qiyi Tang, Yuefeng Du, Lei Xue, Xiapu Luo, Ting Wang, Sen Nie, and Shi Wu. Too good to be safe: Tricking lane detection in autonomous driving with crafted perturbations. In 30th {USENIX} Security Symposium ((USENIX) Security 21). 2021.
Alex Krizhevsky et al. Learning multiple layers of features from tiny images. 2009.
Yann LeCun, Yoshua Bengio, and Geoffrey Hinton. Deep learning. nature, (21(333))4436 444, 2015.
Mathias Lecuyer, Vaggelis Atlidakis, Roxana Geambasu, Daniel Hsu, and Suman Jana. Certified robustness to adversarial examples with differential privacy. In 2019 JEEE Symposium Security and Privacy (SP), pp. 656-672. IEEE, 2019.
Linyi Li, Maurice Weber, Xiaojun Xu, Luka Rimanic, Bhavya Kailkhura, Tao Xie, Ce Zhang, and Bo Li. Tss: Transformation-speeffie smoothing for robustness certification, 2021.
Bee Lim, Sanghyun Son, Heewon Kim, Seungjun Nah, and Kyoung Mu Lee. Enhanced deep resid- ual networks for single image super-resolution. Proceedings of the IEEE conference on com- puter vision and pattern recognition workshops, pp. 136-144, 2017.
Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. arXiv preprint arXiv:1706.0660 2017.
Jeet Mohapatra, Tsui- Wei Weng, Pin-Yu Chen, Sijia Liu, and Luca Daniel. Towards verifying robustness of neural networks against family of semantic perturbations. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 244-252, 2020.
Kexin Pei, Yinzhi Cao, Junfeng Yang. and Suman Jana. Towards practical verification of machine learning: The case of computer vision systems. arXiv preprint 1rxiv:1712.01785, 2017.
Alexander Robey, Hamed Hassani, and George J Pappas. Model-based robust deep learning: Gen- eralizing to natural, distr--distriuutionn data. arXiv preprint arXiv:2005. 10247, 2020.
Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-net: Convolutional networks for biomedi- cal image segmentation. In International Conference Medical image computing and computer- assisted intervention, pp. 234-241. Springer. 2015.
Hadi Salman, Greg Yang. Jemyy Li. Pengchuan Zhang, Huan Zhang, Ilya Razenshteyn, and Sebastien Bubeck. Provably robust deep learning via adversarially trained smoothed classifiers. arXiv preprint arXiv: 1906. 04584, 2019.
11

---

Under review as a conference paper at ICLR 2022
Gagandeep Singh, Timon Gehr, Markus Püschel, and Martin Vechev. An abstract domain for cer- tifying neural networks. Proceedings of the ACM on Programming Languages, 3(POPL): 2019.
Dawn Song, Kevin Eykholt, Ivan Evtimov, Earlence Fernandes, Bo Li, Amir Rahmati, Florian Tramer, Atul Prakash, and Tadayoshi Kohno. Physical adversarial examples for object detectors. In 12th {USENIX} Workshop on Offensive Technologies ({ (WOOT) 18), 2018.
Christian Szegedy, Wojeiech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus. Intriguing properties of neural networks. arXiv preprint arXiv:1312.6199 2013.
Florian Tramer, Nicholas Carlini, Wieland Brendel, and Aleksander Madry. On adaptive attacks to adversarial example defenses. Advances in Neural Information Processing Systems, 33, 2020.
Yulin Wang, Xuran Pan, Shiji Song. Hong Zhang, Gao Huang, and Cheng Wu. Implicit semantic data augmentation for deep networks. Advances in Neural Information Processing Systems, 32: 12635-12644, 2019.
Eric Wong and Zico Kolter. Provable defenses against adversarial examples via the convex outer adversarial polytope. In International Conference on Machine Learning, pp. 5286-5295. PMLR, 2018.
Yuxin Wu and Kaiming He. Group normalization. In Proceedings of the European conference computer vision (ECCV), pp- 3-19, 2018.
Chaowei Xiao, Jun Zhu, Bo Li, Warren He, Mingyan Liu, and Dawn Song. Spatially trans- formed adversarial examples. arXiv preprint arXiv:1801.0220 2018.
Greg Yang, Tony Duan, J Edward Hu, Hadi Salman, Ilya Razenshteyn, and Jerry Li. Randomized smoothing of all shapes and sizes. In International Conference Machine Learning, pp. 10693- 10705. PMLR, 2020.
Hongyang Zhang, Yaodong Yu, Jiantao Jiao, Eric Xing, Laurent El Ghaoui, and Michael Jordan. Theoretically principled trade-off between robustness and accuracy. In International Conference on Machine Learning, pp. 7472-7482. PMLR, 2019a.
Huan Zhang, Hongge Chen, Chaowei Xiao, Sven Gowal, Robert Stanforth, Bo Li, Duane Boning, and Cho-Jui Hsieh. Towards stable and efficient training of verifiably robust neural networks. arXiv preprint arXiv: 1906.06666, 2019b.
Jun Yan Zhu, Taesung Park, Phillip Isola, and Alexei A Efros. Unpaired image-to-image translation using cycle-consistent adversarial networks. Proceedings of the IEEE international conference 2017.
computer vision, pp.
12

---

Under review as a conference paper at ICLR 2022
A PROOF OF THEOREMS
In this section, we will provide detailed proofs theorems in our paper.
First, restate the theorem of randomized smoothing for additive noise and binary classifiers f() : R" - [0, 1].
G(x) Ee~gg (J 0)]
(18)
Theorem 4. Let fz) be any classifier and Gxx) be the smoothed classifier defined in Eq. (18), if G(x) 4. then G(x + 8) for any
|
dp
(19)
G(a) 4(p)
where (-)) is a function about smoothing distribution defined in Eq. (2).
Proof. We first calculate the gradient of the smoothed classifier
0
VG(x) =
f(x + ())
Oz
km Ô
Or f(r + ())(0)d)
00 fx ()g(0)d0
Then we multiple any vector with unit norm u E B, = {u : ||||= 1, R*).
θ
KVG(x), u)l
fæ + ()g(0) 2
Rm 00
[x + 0), v g(0)a0
00
0f( + 0)
Σ R"
(g(())d
00;
Σ Jg--1 (1
1t
0s(r 0)
u.g(0)d0;
II d0,
08;
ESer (
Og(()) 00,-
[xx + 0)u
d0,
II d0,
i
Σ / f(x 0)
0g(0)
u:d0 00;
R"
I I(x () d0
JAm
u)
[(==)) "I]l
(20)
To bound the gradient of the smoothed classifier, we use the following inequality,
KVG(=). < sup Eggg [ + ()(()) f:G(=)=((z)
(21)
As shown in Yang et al. (2020), the optimal fxx achieves at.
1, (f, (0)) (G(x)
+x =
(22)
0, else
13

---

Under review as a conference paper at ICLR 2022
Then,
Rewg [f + (),
E bru > (G(2))
+(G(z))
(23)
which means that,
(VG(t), & (GG(2))
(24)
and this is true for all E (1), so we have,
(VG(x), (G(())
(25)
MEB,(1)
Consider path from & : [|]]]] Rd with Eo and وΙدl = x & and Ei [8] we have
dG(E.) di
(VG(E)), < (GG(E))
(26)
the norm of 5 satisfies that,
||0|| < /
G(x) (pp) dp,
(27)
right hand side exists, we name it ||6o|| = Jā(w)) 1 1/D(p)dp. Without loss of generality, we assume that G(E,) is increasing in then cound calculate the minimal twhen G(Ez) increase to }
T =
dp = 1
(28)
G(Ea) ((p)
By the generality of 8 we have,
1
G(x + 5) 8) < 2
(29)
for any 5 < [|6o||-
This theorem is naturally extended to problems with p > 2 classes by considering the two top classes which are G(*), and G(x) This turns the problem binary classification problem.
A.1 THE PROOF OF THEOREM
Theorem Let fxx) be any classifier and G(x)) be the smoothed classifier defined in Eq. (1), if
there exists function M.), and the transformation T(:--) satisfies that
() a-(0, 5) M(0
00
and there exists two constants PB that satisfies that
G(r) > PA
> pB
> G(x)
then yA = arg max;ey G(T(E, x) holds for any |||||
< where
1
PA
1
R
dp,
(30)
2M
(pp)
PF
here M maa..ol ({)
14

---

Under review as a conference paper at ICLR 2022
Proof. WLOG, we only prove it for binary cases that f(-) R² [0, 1].
())
0f()()(0.)),))
(T()(0,E).z) g(0)d
0r()(0,(),,)
d5
(r()(0,E), ()) (0,5) g
(r()(0.E).z)
()( 0g
(31)
0f() 0r()(0,5),z)
2)() (0, M (0, ()g(O)d
dx(0,E)
0f(7(7(0,)),))) ((0)E)g(0)d0
00
For u E Rd and ||u||| = 1, we have:
|/ 0f(r()(0,)))2))
(VEG(((E)
(0.0000)))))))) ())g(O)d0
(32)
00
()),
M max
ug (0)0
(33)
||||
00
/ (()
0g
M
max
vdd
(34)
|=||=1
00
(35)
here M maxe,5 (0, E)ll and we assume that g(0) cxp(->/(0))
/ ((T(7(0,))))) x)) 0g(0)
(V
M max
vdo
(36)
||||=1
00
f [(T(7(0, 2))
M
max |ell=1
(37)
M
max
Eang [f(T(T(0, x) 2)
(38)
|=||=1
M max
sup =
Eeny (
||||=1
Similar with Theorem 4, the optimal f achieves at
f(7(7(0,E),z)) {
1, (v))()))), u) > -: (G((((((E,))))
(40)
0, else
Then we have
(VVGG(((E) u\) < (((((((,2))
(41)
Consider path from G. [[]]] - Rd with Co and وeوl rEE,2)) and C T8T we have
8
dG(()) (VG(S), & ((G(S))
(42)
dt
The last part of proof is the same with Theorem 4.
A.2 THE PROOF OF THEOREM
Theorem 2. Suppose f(x) is any classifier and G(ă) is the smoothed classifier defined in Eq. (7).
if there exists PA PB that satisfies that
Ō(F) > PA PB ((T) B
15

---

Under review as a conference paper at ICLR 2022
then yA mry G(F any llEll2
R. where
1
1
R
dp,
(43)
2M
$
④(p)
PB
and the coefficient M is defined as
(F2(
(F)
2
M
max E
1+
(44)
dg
00
Proof. In this part, we will prove Theorem 2, which the main result in this paper. WLOG, we prove it for binary cases where f(-) : R -> [0, 1]. First, we will calculate the gradient of ((f( to a:
((G(F((,)) ()) g() (())))))
(45)
We expand the expectation into integral and use chain rule that
0f ūe))
05 UE)
Oūg
V V((((EE))) Janta 05 ūt)
g
dd.
(46)
Tye
a
The key step is to eliminate the gradient of (f(F((. and replace it with 0f(7(0.0c)) (F(..
Since:
80
0F (ō we)
Toi (z6)
F (2((
(47)
Oigc
(202
09g
(x()))
La FF((E)
0
H1)()))
In
(48)
We have:
OF(7(0,TE) TE))
(H(())
F21 (vt))
V V(O(F(E,E)
J
TE)) (2()))
H2(2)
F'ge (ve)
la
3(0)d0
(49)
Hb(zE)
FI((E) In
OF De)) (F De)
[H(((b))
da H2(20)
I
Fi(()
In
(((
In
Ruta
(22(((
HA(EE) Hb
Ia FI(E)
In z(0)do
(50)
(VE(
OF TE))
Ia
Fe2(16)
-F( In ō(()dĀ
Fb2(uz)
00
La
(51)
H2(
Fi(())
In
( ūc))
ME, ())(ō)
(52)
Jania
09
here we define
M( 0) A
La (0)
Ia
(53)
In
FbZ(yE)
Hb((z))
Fi(E)
In
We consider the Unit enlargement, which means:
16

---

Under review as a conference paper at ICLR 2022
H1(z') = z', F2(x') = x²'
(54)
thus:
Ia ( 0) = F/21 (uE)) Fi(0)
O Oax (ve (2
(55)
Since H' is the virtual parameter introduced, which can be taken as 0 in case of actual disturbance. Thus only need to consider the projection of V VūG((e) in the space of &. Thus set
u Onx1
)
(56)
here Rd and ||u||
1. Assume
((0) exp(-v((0)
(57)
0g(0) 5(0) -
(58)
00
And have
Of(F(0, ūE))
(V (VEC()E). a}
ME, ())
(59)
00
J
af(F(0, ūE))
Ia. Fiz(ye) F((0),
Oax
ag(O)do
(60)
00
Duxn
S
aj(F(0, ūe))
M(S, ()
(61)
00
af(F(0, (0.0e)
M max
ig(0)
(62)
||0||==1
Ra+a
00
0g(
M max
f(F(0, ūe))
(63)
||
Raad
00
M max
f(F(0, (0)
(64)
Raad
M max
Eggng
[ ())
(65)
||
We bound the right hand side by
vV (VeG()))))) < sup
M =
Bang ((= m))
(66)
||]|2=]
and the optimal f as follows.
1(5(0,7($, = {
1. iff(((0), ū) > pT (ā((((((
(67)
0, else
here
Ia. M(C. 0) F(e( - F(()),
Oaxn
(68)
Onxn
17

---

Under review as a conference paper at ICLR 2022
and M is
Ta:
dnx
M = max
g
F22 (ee)
FY(0),
Onxn
max E
(v 0
(F)
(69)
20
D2 (yc)
0
2
max 8
dg
00
Notice that here (-2 is the same as F2(.) the notations in the main text. Then we could apply the techniques used in Theorem and Theorem 4, we have:
1
PA 1
R
dp,
(70)
2M
FB 4(p)
Thus we have proven this Theorem.
A.3 THE PROOF OF THEOREM
Theorem 3. Suppose the simulation of the semantic transformation has an small enough error
11F(E.2) - F(E.2)) <E,
Then there exists constant ratio A = I5) F1 III (E)))) O does not depend the target classifier, we have the certified radius for the real semantic transformation satisfies that
R2 > R(1 Ae)
where R is the certified radius for surrogate the neural network in Theorem 2 and
1
PP 1
R
dp.
2M
PB 4(p)
Proof. We set
)
Onxi
u
(71)
here u Rdd and ||u|| = 1. Then we have
/ (F((((,F(EE
0f(7(0, ((,))))
v
ūg(ō)dō
ac
= / (F(E,)) - -(E,8)
0a 0F R)))
üg
(0)10
OE07
(72)
Set L 2 ((.(
we have
0ç0+
L a 05(7(0,
a (() 07 0((0,+(E,))
(73)
de
07
0E 0F(0,
07
Set ua =0, =0, have:
18

---

Under review as a conference paper at ICLR 2022
0F(0,F(E,5)
(0) + +((((E
0f
(F2(f(()
it' (R:00 (2(
Of
it' (F:(0) + (((F(E))))
(F((O)
Ia
(F2(f(E,
(74)
In
Oi
Ia
Fg ()((,()))
00
-F((0)
In
OF, 33)
A1
09
here A,
La
Fs' F2((((E))) 8)). By the proof above, we have a 8 A2, thus we have:
(0) In
0
(((((O)(((E))
(F(0,
L
0
(()
07
0
@f((F(0, (({,))) 0F (((0,((E,))) A1
A2
(75)
a0
0F(0,F(E,ã))
00
a
@ff(F(0, ( +(E,a)))
A1 A2.
ab
00
V V&G(T(E,))) v V,C(F(E, a)
/ (f(E,)) - =(5,,))
82 j((( ((E(0,,(E, ()), (0)
agat
a 05(F(0,=(E,)))) +(E, x))) A1
(76)
/H165.0 =($,,))
Azūg(0)a0
ab
00
<Ac. f F(F(O, (()(0,
where À is a constant depending on (( (3 11)
Then there exists and we have
R > R( A))
(77)
where
R 2M pn (p)) dp
B IMPLEMENTATION DETAILS AND EXPERIMENTAL SETTINGS
B. PRACTICAL ALGORITHMS FOR CALCULATING M
For resolvable transformations in Theorem 1, the M is defined
M max [I #)] E.EEP
(78)
Since if we have verified that the semantic transformation is resolvable, most of time have closed form of M like contrast/brightess transformation and we are able to calculate it analytically shown in Li et al. (2021).
19

---

Under review as a conference paper at ICLR 2022
For non-resolvable transformations in Corollary 1, M is defined
1 o. +
1
( Al
M
max CEP
(79)
a
This ratio is similar with the Lipschitz bound for semantic transformation in Li et (2021).For low dimensional semantic transformations, we are able to interpolate the domain to find a maximum M and corresponding But this remains a challenge for high dimensional semantic transformations. Specifically, for given ç, we need to compute the norm @Fa(ve) A₁. The Jacobian matrix is
n X n. Caculating it requires n times of backpropagation. Thus it is inefficient to store the matrix or directly compute its norm. To solve the problem, we noitice that
FF2(Ue) Dg
FF2(uc)
A1
max
A1
(80)
2
||a||==
05
And then we have
F A1
(F2(UE) ) - -A,,
(81)
0
0
Since tranposing a matrix does not change its norm, we could calculate its norm by optimizing that,
max |||||==1 a
(F2(Us) - A1, u)
(82)
Using this formulation, we only need to multiply the output with an unit vector and perform one backpropagation. This is a simple convex optimization problem. Then we could use any iterative algorithm to find its solution which is very fast to compute. This trick is crucial and it makes the matrix norm computation be scalable in practice.
B.2 OTHER DETAILS FOR EXPERIMENTS.
Our GSmooth requires to train neural networks. First randomly generate corrupted images to train tonge-to-image neural network. The training process of classifiers and certification for semantic transformations are done on 2080Ti GPUs. We use U-Net (Ronneberger al., 2015) for the surrogate model and replace all BatchNorm layers with GroupNorm (Wu & He, 2018) since we might use the model in low bacthsize settings. The U-Net could be replace by other networks used in image segmentation or superresolution like UNse (Diakogiannis et al., 2020) or EDSR (Lim et al., 2017). We use L1-loss to train the surrogate model which achieves better accuracy than others which is also reported (Lim et al., 2017).
After train surrogate model to simulate the semantic transformation. Then we train the base clas- sifier for certification with moderate data augmentation like (Li et al., 2021; Cohen et al., 2019) to ensure that training and testing of the classifier is performed on the same distribution. There are two types of data augmentation, one is the semantic transformation and the other is the augmented noise introduced only in our work. Data augmentation based semantic transformation could be done using both the surrogate model or the raw semantic transformation. We can only use the surrogate model to add the augmented noise because this noise is a type of semantic noise in the layers of surrogate model. In our experiments the standard deviation of the augmented noise is chosen from 0.1 ~ 0.4 depending on the performance. The basic network architectures for these datasets are kept the same with (Li et al., 2021). On CIFAR-100 daatsets, we use PreResNet (He al., 2016b) re-implement the method by Li et al. (2021). We also adopt the progressive sampling trick mentioned in TSS (Li al., 2021) which is useful reduce computational cost and certify larger radius. The details could also be found in Li et al. (2021).
C SUPPLEMENTARY EXPERIMENTS
In this section, report the results of our GSmooth under adaptive attacks to verify the tightness of our certified bound. The experiments are conducted on CIFAR-10 dataset. We use expectation of transformation to calculate the gradient of the model. Then apply projected gradient descent
20

---

Under review as a conference paper at ICLR 2022
Cert Ac)(%)
ΣoT attacks(%)
Gaussian blur
67.4
68.1
Tanslation
82.2
87.5
Rotation
64.6
68.4
Rotational blur
39.7
45.0
Defocus blur
25.0
25.0
Pixelate
45.3
49.2
Table 3: Accuracy of our GSmooth under adaptive attacks (PGD using expectation over transforma- tions) on CIFAR-I0 dataset.
Type/Acc(%)
Augmix
TSS
Ours
Zoom blur
70.8
75.2
77.1
Defocus blur
72.2
75.6
76.8
Pixelate
50.9
76.0
76.7
Brightness
82.4
71.8
72.1
Motion blur
68.6
70.2
70.5
Gaussian blur
67.4
75.8
75.2
Table 4: Empirical accuracy on subsets of CIFAR-1O-C.
to find adversarial examples until it converged. Then we report the accuracy of our model on the corrupted dataset. The result is listed in the following table. We found that the empirical attack is no less than the certified accuracy. This shows that the bound for our model is effective. Additional, some empirical results much higher than the certified accuracy, which might indicate the bound still has space for improvement.
We have conducted some experiments under the common benchmark CIFAR-10-C to show the empirical robustness under these corruptions. We conducted the experiments on some subsets of CI-AR-10-C. The corruption types of these subsets are related to the experiments in our paper. The settings of these experiments and the results of baselines are from TSS(Li et al 2021).
21


Reviews:
[{"review_id": "lwfU9HQsETj", "reviewer": "Reviewer_bwyb", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "In my opinion, this work shows ground-breaking results in certified robustness, by providing an elegant and general framework for robustness certification with arbitrary and possibly complex semantic transformations with performance guarantees. I believe it achieves an important milestone in adversarial robustness.\n\nThe paper is also well-written and easy to follow. The comparisons to existing methods are thorough and convincing. The ablation studies (especially the noise distribution part that uniform distribution is quite competitive) also shed some new insights into randomized smoothing.\n\nI have two suggestions that I hope the authors can incorporate into the revised version:\n\n1. In Table 1, since the considered transformation has very few parameters, I would like to see the comparison of empirical robustness by running the actual attack on the certified value to understand the gap between certified and empirical robustness.\n\n2. Since the proposed method can handle composite semantic transformations, can the authors some case studies to demonstrate such an advantage?\n\nMinor comment\"\n1. In Sec. 3.1, \\nabla should be defined, and I believe \\psi function needs to be differentiable\n", "summary_of_the_paper": "This paper proposed GSmooth, a generalized randomized smoothing method for semantic transformations. \n\nThe main technical contributions are:\n(1) Introduce the use of an image-to-image translation network to provide a unified framework for the analysis of non-resolvable semantic transformations.\n\n(2) Theoretical proof on the certified radius and the approximation error from the image-to-image translation network. \n\n(3) The empirical performance is superior to existing methods on most transformations. More importantly, the method can certify many new transformations that are hard to analyze based on existing methods.\n", "summary_of_the_review": "1. Strong theoretical and empirical results\n2. The contributions are significant for expanding randomized smoothing to more complex semantic transformations", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works.", "empirical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works."}, "extra_text": {}}, {"review_id": "ceJwAFC7x7", "reviewer": "Reviewer_HKx4", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "In this work, the authors first construct a smoothed classifier $G(x)$, based on smoothing the parameter of the parametrised transformation $\\tau$. Based on the compositional properties of $\\tau$, they define conditions to obtain a robustness radius (Theorem 1). Then they proceed to construct a surrogate image to image translation model approximating the transformation $\\tau$. Then the parameters of $\\tau$ and inputs $x$ get augmented to $\\tilde{\\tau}$ and $\\tilde{x}$ respectively to divide the transformation into a resolvable and a non resolvable part. The surrogate transformation then leads to a more concrete form of $M^*$ (Theorem 2), which is further simplified by assuming that $F_1$ is an affine form (Corollary 1). Finally, in Theorem 3, the authors provide a correction for the robustness radius $R_r$ for the case where surrogate transformation $\\tilde{\\tau}$ is approximating $\\bar{\\tau}$ dependent on the maximum $\\ell_2$ approximation error. \n\nThe main ideas of the paper are interesting and to the best of my knowledge novel, particularly the surrogate network. However, this paper needs to be written much more clearly to convey the central ideas to the reader. Particularly Section 3 and Section 4 are hard to understand and required considerable effort, after which some aspects remain still unclear. On page 5: What are the dimensions for $x’$ and $\\theta’$ or $\\tilde{x}, \\tilde{\\theta}$ and how is $x’$ chosen? How do you decompose the transformation to get a resolvable and a non resolvable part? Do i guess correctly from Eq. 9, that the resolvable part is the sum between $\\theta$ and $x’$, and the non resolvable part is the sum between $F_1(\\theta) + \\theta’$ and $F_2(x)$? \n\nThe related work Li et al 2021 and Fischer et al 2020 do evaluate on ImageNet. Can the same be done for GSmooth? Table 1 indicates that VeriVis is not able to certify against translations. However, VeriVis can certify translations. Was the certification accuracy 0? What about the other works? Can the other methods potentially be adapted to handle Blur or the Pixelate transformation? Further, the runtimes need to be stated in order to judge the trade off between performance and accuracy. Where do the authors show how they calculate their $\\epsilon$ for the surrogate transformation networks as needed for Theorem 3?\n\nThe authors claim multiple times (abstract and later in the paper) that existing work can not handle complex semantic transformations. Can the authors substantiate this impossibility? I could not find a proof or some further justification for this. Further, in the related work, section 2.1: Apart from Madry et al. 2017, these works do not attempt to certify some simple geometric transformations but actually do certify them. Mardy et al 2017 on the other hand focuses on attacks and defenses. The authors clarify the difficulties the approach of Li et al 2021 presents for generic transformations. Can the authors also clarify the difficulties the approach of Fischer et al. 2020 presents? In Section 3.3. the authors imply that existing methods based on convex relaxations and randomized smoothing require the development of a specific algorithm for each individual transformation. To what extend is this true for Balunovic et al 2019 and Fischer et al 2020? \n\nMinor comments:\n- Eq. 1: It is slightly confusing to write $\\theta \\sim g(\\theta)$. \n- Just before Eq. 8: the dimensions for $\\tilde{\\tau}$ seem off. ", "summary_of_the_paper": "This paper generalizes randomized smoothing to certify robustness against complex semantic transformations. To do that, they construct a surrogate neural network mapping images to images, to approximate complex semantic transformations and certify robustness with respect to this network. ", "summary_of_the_review": "While the ideas in this paper are to the best of my knowledge interesting and novel, many questions remain unanswered and key points in the paper remain unclear. ", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "Eqjqq9eCWKj", "reviewer": "Reviewer_r3hs", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strengths:\n1. The topic of certifying generalized semantic transformations is extremely relevant. The paper is well motivated and the authors clearly point out differences with existing work. \n2. GSmooth is able to certify several non-trivial semantic transformations which include difficult combinations such as rotational and defocus blur. \n3. While the certified accuracy values are not that significant for the more complex transforms, the algorithm is good first attempt which succeeds as compared to other methods.\n\nWeaknesses:\n1. I am not convinced that an $\\ell_2$ bound in the parameter space for compound transformations makes sense. A real world attacker instead would have $\\ell_\\infty$ or $\\ell_2$ type bounds for parameters of each transform. For example in the case of rotational blur, an attacker could independently increase rotations and blur to the max value. Given the very different ranges of the two tranformations, an $\\ell_2$ bound may emphasize one transform over the other. The current algorithm does not seem to take this into consideration.\n2. The idea of augmenting the latent space to decouple resolvable and non-resolvable components is interesting. However, from the text, it is not clear why such a decoupling would occur without additional training objectives for the surrogate. Could the authors clarify this?\n3. It's not clear to me why H would not reflect in the M* term. The proof seems to ignore the effect of H by considering it to be a concatenation of an identity and a zero matrix. However, the decoder itself may not be Lipschitz due to the non-trivial nonlinearities involved. A more clear explanation here is warranted.\n4. The paper lacks a empirical confirmation of the presented bound. Specifically, the GSmooth classifier should be tested against an adaptive attack that has access to the surrogate and the parameter space to find if the bounds do actually hold. The certificate should also be tested against standard Expectation over Transformation attacks. This would also help quantify the tightness of the proposed bound. \n\nMinor comments:\n1. The explanation regarding the dimensionality augementation is confusing and can be presented better-- \"Specifically, we introduce the augmented data $\\tilde{x}$ ... additional dimensions to 0.''. Also, Eq. 9 should include the expression for augmented $\\tilde{H}$ for better clarity. \n2. Table 2 does not mention the radius for the certified accuracies.\n3. Section 3 needs to be more clear. The notation for the augmented parameter space needs some clarity on the dimensions used. \n4. While the authors present a nice simplification for F_1, how about explicitly forcing lipschitzness for F_1, F_2, and H? ", "summary_of_the_paper": "The authors propose a randomized smoothing based certification algorithm  for general semantic transformations. The key idea of the work is to use a neural surrogate for the semantic transformations, and add noise in the latent space of the surrogate for randomized smoothing. The neural surrogate appears to convert the non-linear, multiplicative transformation into an additive operation in the latent space, thus allowing for randomized smoothing methods to be applied. Their approach proposes to decompose the resolvable and unresolvable parts of the semantic transformation by lifting the data+transformation parameters into a larger augmented latent space defined by an image-to-image network. The resolvable parts of the transform can then be certified similar to previous works (Yang'2021, Salman'2019) using the Lipschitzness of the smoothed transforms. The non-resolvable part of the transform however are assumed to be Lipschitz in the latent space. The authors provide theoretical and empirical evidence for GSmooth and show improvement on contemporary methods.", "summary_of_the_review": "Overall, the paper presents a novel and interesting addition to the randomized smoothing literature. This appears to be the first attempt at certifying complex semantic transformations, and therefore has merit. The paper still needs some additional work however, specifically in terms of quantifying the tightness of their proposed bound against semantic attacks. I also believe the authors should take a more careful look at the actual certificates for the compound attacks. An $ell_2$ bound for the entire parameter vector may not capture the worst case for each component of the transform. \n\nThe paper in its current form is marginally below the acceptance threshold. However, if the authors show empirical evidence of their certificates against semantic attacks, I will be glad to improve my score.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "ER_nh7pfXtz", "reviewer": "Reviewer_oUek", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Overall, I would recommend a reject for this paper.\n\nThe paper discusses an important topic (robustness to semantic transformations), but does not tackle the problem from a perspective that I feel is significant. In particular, it is unclear to me how useful certified robustness to semantic transformations will be, especially for non-resolvable transformations. What does it mean to be robust to rotational blur for attack range ||alpha||_2 < 20 for CIFAR-10, for example? Does this correspond to any guarantees on real datasets, or at least empirical improvements on benchmark datasets?\n\nI appreciate that the authors are thorough with comparing to various prior works in their experiments. However, their method usually performs very similarly or slightly worse than prior works when comparisons are possible; thus, the main experimental improvement is that it is now possible to achieve results on certain non-resolvable transformations that prior works are incapable of handling.\n\nStill, in my view, a key question is unanswered - how can we be sure that the generative model is capturing all the possible non-resolvable transformations that we may care about? Because if we can not answer this question, then it seems that certified robustness is not particularly meaningful, and instead, empirical robustness on benchmark datasets (for example, measuring accuracy on CIFAR-C as well) would be a better measure of success.\n\nAdditionally, the paper could be presented better; in particular, the important components should be explained more clearly (e.g. the question I asked above about what ||alpha||_2 < 20 means), whereas other parts of the paper can be moved to the Appendix. For example, Theorems 1 and 2 and Corollary 1 are primarily based on prior work and use up a lot of text, but there is not much description or explanation of how these theorems can be applied to get certified robustness to semantic transformations. The best example of a useful explanation that I found is the short paragraph just under Figure 1 on page 4, which was helpful for me.\n\nAdditionally, I believe the authors are missing very relevant prior work. The work of [1] discusses a similar idea, where semantic transformations are modeled via generative models, and Lp-robustness in the latent space of the generative model is used as a proxy for robustness to semantic transformation.\n\nSome more specific comments and questions:\n- Can the notation section be simplified any further?\n- How large is the error term epsilon in Theorem 3 look for the generative models used in the experiments?\n\n[1] https://arxiv.org/abs/2005.10247 Model-Based Robust Deep Learning: Generalizing to Natural, Out-of-Distribution Data (Alexander Robey, Hamed Hassani, George J. Pappas)\n\n\n\n\n=======================================================================\n\nAfter the rebuttal period, I have read the author's responses, and changed my score from a 3 to a 5. I appreciate the authors taking the time to attempt to respond to the concerns of all reviewers, and for updating and improving their work during the rebuttal process.\n\nI am glad to see that they do provide empirical evidence of improvement to common-corruption robustness, compared to AugMix (one of the state-of-the-art approaches for standard common-corruption robustness) and TSS, although I can not tell how the authors derived their baselines (I can not find references to the AugMix accuracy numbers that the authors provided in their rebuttal in the TSS or AugMix paper).\n\nStill, in my opinion the paper's novelty is limited. As the authors and I agreed upon in our discussion, the main novelty is not improvement for resolvable transformations (prior works that the authors cite perform about the same or better), but rather, is the ability to handle non-resolvable transformations. I agree that robustness to non-resolvable transformations is important; however, certified robustness to non-resolvable transformations is not meaningful to me, because they are only being certified with respect to a neural network that is trained to approximate those non-resolvable transformations. Without MTurk studies to confirm how good the neural network's non-resolvable transforms are, I do not find certified robustness here meaningful, because it does not necessarily correspond to anything concrete that we can understand. On the other hand, empirical improvements on non-resolvable transformations would be meaningful.\n\nThus, the main reason I increased my score to weak reject is due to slight empirical improvements over baselines on CIFAR-10C.", "summary_of_the_paper": "This paper proposes a more generalized form of certified robustness and attempts to provide new results on applying randomized smoothing to semantic transformations such as different types of blurs or distortions. The main idea is to use an image-to-image neural network to approximate semantic transformations, and then certify robustness based on bounds on that neural network. The authors provide empirical results on standard datasets like MNIST and CIFAR showing that their method can achieve improved results on some transformations compared to prior work.", "summary_of_the_review": "The paper leaves some key questions unanswered, and can be presented much more clearly. It is also missing very relevant prior work. Thus, I recommend a reject.\n\n", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper proposes a more generalized form of certified robustness and attempts to provide new results on applying randomized smoothing to semantic transformations such as different types of blurs or distortions. The main idea is to use an image-to-image neural network to approximate semantic transformations, and then certify robustness based on bounds on that neural network. The authors provide empirical results on standard benchmark datasets like MNIST and CIFAR showing that their method can achieve improved results on some transformations compared to prior work.

The review committee appreciates the authors taking the time to attempt to respond to the concerns of all reviewers, and for updating and improving their work during the rebuttal process. The committee is glad to see that they do provide empirical evidence of improvement to common-corruption robustness, compared to AugMix (one of the state-of-the-art approaches for standard common-corruption robustness) and TSS.


However, the reviewers still have concerns about the novelty of the paper. The main novelty is not improvement for resolvable transformations (prior works that the authors cite perform about the same or better), but rather, is the ability to handle non-resolvable transformations. The reviewers agree that robustness to non-resolvable transformations is important; however, the reviewers think certified robustness to non-resolvable transformations is not meaningful, because they are only being certified with respect to a neural network that is trained to approximate those non-resolvable transformations. Without MTurk studies to confirm how good the neural network's non-resolvable transforms are, the reviewers do not find certified robustness here meaningful.
Decision: Reject