TOWARDS NATURAL ROBUSTNESS AGAINST ADVERSARIAL EXAMPLES
Authors: Haoyu Chu
Shikui Wei
Yao Zhao
Venue:  (2021)

Abstract: Recent studies have shown that deep neural networks are vulnerable to adversarial examples, but most of the methods proposed to defense adversarial examples cannot solve this problem fundamentally. In this paper, we theoretically prove that there is an upper bound for neural networks with identity mappings to constrain the error caused by adversarial noises. However, in actual computations, this kind of neural network no longer holds any upper bound and is therefore susceptible to adversarial examples. Following similar procedures, we explain why adversarial examples can fool other deep neural networks with skip connections. Furthermore, we demonstrate that a new family of deep neural networks called Neural ODEs (Chen et al., 2018) holds a weaker upper bound. This weaker upper bound prevents the amount of change in the result from being too large. Thus, Neural ODEs have natural robustness against adversarial examples. We evaluate the performance of Neural ODEs compared with ResNet under three white-box adversarial attacks (FGSM, PGD, DI2-FGSM) and one black-box adversarial attack (Boundary Attack). Finally, we show that the natural robustness of Neural ODEs is even better than the robustness of neural networks that are trained with adversarial training methods, such as TRADES and YOPO.


Under review as a conference paper at ICLR 2021
TOWARDS NATURAL ROBUSTNESS AGAINST ADVERSARIAL EXAMPLES
Anonymous authors Paper under double-blind review
ABSTRACT
Recent studies have shown that deep neural networks are vulnerable to adversar- ial examples, but most of the methods proposed to defense adversarial examples cannot solve this problem fundamentally. In this paper, we theoretically prove that there is an upper bound for neural networks with identity mappings to con- strain the error caused by adversarial noises. However, actual computations. this kind of neural network no longer holds any upper bound and is therefore sus- ceptible adversarial examples. Following similar procedures, we explain why adversarial examples can fool other deep neural networks with skip connections. Furthermore, we demonstrate that new family deep neural networks called Neural ODEs (Chen et al. 2018) holds weaker upper bound. This weaker upper bound prevents the amount of change in the result from being too large. Thus, Neural ODEs have natural robustness against adversarial examples. We evaluate the performance of Neural ODES compared with ResNet under three white-box adversarial attacks (FGSM, PGD, DI2-FGSM) and one black-box adversarial at- tack (Boundary Attack). Finally, we show that the natural robustness of Neural ODEs is even better than the robustness of neural networks that are trained with adversarial training methods, such as TRADES and YOPO.
INTRODUCTION
Deep neural networks have made great progress in numerous domains of machine learning, espe- cially in computer vision. But (Szeedyy et al. (2013) found that most of the existing state-of-the- art neural networks are easily fooled by adversarial examples that generated by putting only very small perturbations to the input images. Since realizing the unstability of deep neural networks, searchers have proposed different kinds of methods to defense adversarial examples, such as adver- sarial training (Goodfellow et al. 2014), data compression (Dziugaite et al.) 2016), and distillation defense (Papernot et al. 2016). But each of these methods remedy for the original problem, and none of these methods can solve it fundamentally. For example, Moosavi-Dezfooli et al.) (2016) showed that no matter how much adversarial examples are added to training sets, there are new adversarial examples that can successfully attack the adversarial trained deep neural network.
So, avoiding adversarial examples technically cannot solve the most essential problem: why such subtle change in adversarial examples can beat deep neural networks? Mcanwhile, leads more important question: how to make deep neural networks have natural robustness so that they can get rid of malicious adversarial examples.
Early explanations for adversarial examples considered that smoothness prior is typically valid for kernel methods that imperceptibly tiny perturbations of a given image do not normally change the underlying class, while the smoothness assumption does not hold for deep neural networks due to its high non-linearity (Szegedy et al., (013)). This analysis underlies plain deep neural networks like AlexNet (Krizhevsky et al.) 2012). But later than that, Goodfellow et al. (2014) claim adversarial examples are a result of models being too linear rather than too non- non-linear, they can be explained property of nigh-dimensional dot products. Unfortunately, both of these explanations seem to imply that adversarial examples are inevitable for deep neural networks.
On the other hand, we notice that skip connections widely used in current deep neural networks after the appearance of Highway Network (Srivastava et al.] (2013) and ResNet (He et al. 2016). turns out that the identity mapping in ResNet is formally equivalent to step of Huler's 's method

---

Under review as a conference paper at ICLR 2021
which has been used to solve ordinary differential equations (Weinan 2017). More than that, other kinds of skip connections used by different network architectures can be considered as different numerical methods for solving ordinary differential equations. The link between numerical ordinary differential equations with deep neural networks can bring us a whole new perspective to explain adversarial examples through the numerical stability analysis.
In this paper, we attempt to utilize the natural property of neural networks to defense adversarial ex- amples. We first analyze how adversarial examples affect the output of neural networks with identity mappings, obtain an upper bound for this kind of neural networks, and find that this upper bound is impractical in actual computations. In the same way, we figure out why adversarial examples can fool commonly used deep neural networks with skip connections. Then, we demonstrate that Neural ODEs hold a weaker upper bound and verify the natural robustness of Neural ODEs under four types of perturbations. Finally, we compare Neural ODEs with three types of adversarial training methods to show that the natural robustness of Neural ODEs is better than the robustness of neural networks that are trained with adversarial training. The main contributions of our work are follows:
We introduce and formalize the numerical stability analysis for deep neural networks with identity mappings, prove that there is an upper bound for neural networks with identity mappings to constrain the error caused by adversarial noises.
We provide new reason why commonly used deep neural networks with skip connections cannot resist adversarial examples.
We demonstrate that Neural ODEs hold a weaker upper bound which limits the amount of change in the result from being too large. Compare with ResNet and three types of adversarial training methods, we show the natural robustness of Neural ODEs.
2 RELATED WORKS
2.1 ADVERSARIAL DEFENSE
Adversarial training typically uses robust optimization to generate adversarial examples for train- ing deep neural networks. Madry et al. (017) take the optimization as a saddle point problem. Zhhang et al. (2019)) cast adversarial training as a discrete time differential game. Adversarial train- ing can be seen as a data augmentation particularly enhance the robustness to white-box attacks (Tramer et al. 2017). Zantedeschi et al. 2017) augmented the training sets with examples perturbed using Gaussian noises which can also enhance the robustness to black-box attacks. Lee et al. (2017) proposed novel adversarial training method using a generative adversarial network framework. Be- sides. Finlay et al. (2018) augmented adversarial training with worst case adversarial training which improves adversarial robustness in the & norm on CIFAR10.
Modifying the neural networks by using auto encoders, input gradient regularization and distillation can result in robustness against adversarial attacks (Bai et al. [0177] Ross & Doshi Velez, 2017 Papernot et al. 2016). Besides, there are some biologically inspired deep learning models designed have natural robustness against adversarial examples. Nayebi & Ganguli (2017) developed scheme similar to nonlinear dendritic computation to train deep neural networks to make them robust to adversarial attacks. Krotov & Hopheld (2018) proposed Dense Associative Memory (DAM) models and suggested that DAM with higher order energy functions are closer to human visual perception than deep neural networks with ReLUs.
In addition to augmenting datasets modifying original neural networks, there exist adversarial defense methods that rely on using external models and detecting adversarial examples. AAkhtar et al. (0018) presented Perturbation Rectifying Network (PRN) as 'pre-input' layers to targeted model. a perturbation is detected, the output of the PRN is used for label prediction instead of the actual image. Xu et al. (2017) proposed a strategy called feature squeezing to reduce the search space available to an adversary by coalescing samples that correspond to many different feature vectors in the original space into single sample.
2

---

Under review as a conference paper at ICLR 2021
3 NUMERICAL STABILITY ANALYSIS FOR DNNS
3.1 RESNET AND EULER'S METHOD
The building block of ResNet defined as
Ynt1 =y + ffyni fn)
(1)
Where n (h)...(h)) 1}, yn and Yn+1 are imput and output vectors the layers considered. The function f': 0n)) represents the residual mapping and 0n are weights to be learned.
In numerical methods for solving ordinary differential equations,, Euler's method defined by taking this to be exact:
Yo+1 = Yn. +
It can be easily seen that Eqn. is special case of Euler's method when the step size h 1. The iterative updates ResNet can be seen as an Euler discretization of a continuous transformation (Lu et al. 2018, Haber & Ruthotto 2017: Ruthotto & Haber] (209).. When the input data is perturbed by adversarial noises e, we denote the adversarial example by
Zg Yo +
To perform stability analysis for ResNet with adversarial example 2o as input, we define a numerical solution zn by
Za+1 = Zn + -hIj((n,, Zn; 0n)
(4)
and provide following theorem to show that the amount of change between yn and Zn holds an upper bound by some assumptions.
Theorem 3.1. Let yt; 0) be a continuous function for to S t and -co < y < oo, and further assume that [(,,, (1,y;)) satisfies the Lipschitz condition. Then compare the two numerical solutions yn and zn as h + O, there is constant e 2 O, such that amount of change between Yn and zn satisfies
Znn -vll S clel N
max
(5)
Proof.
Let e,, = Zn Yyn n 2 0. Then en = e, and subtracting Eqn. from Eqn. we obtain
Cx+1 e, (t Zn; 6n) [t,,, yn; 0n)
Assume that the derivative function (t,, 0) satisfied the following Lipschitz condition: there exists K such that
y1; (t: 0)1 < Kly1 -yl
Taking bounds using Eqn. 7 we obtain
le S |e,l + +hR zn - ynl
(8)
|e41l S (1 + hR) |enl
(9)
where K is the largest Lipschitz constant.
Apply this recursively to obtain
le,l S (1 h&) cul
(10)
Using the inequality (1 + 1)mh S ml for any 1> -1, any m 2 obtain
(1 + hK) < nhk c (b-to)
(11)
and this implies the main result Eqn. (5.
Roughly speaking, theorem 3.1 means that small adversarial perturbation initial value of the prob- lem leads to a small change in the solution, provided that the function ft. y; 0) is continuous and the step size h is sufficiently small.
Obviously, ResNet does not satisfied the assumption of continuity and its step size h always equal to 1. Particularly, the functions [(I,, y; 0) in ResNet are composite function which contains ReLUU
3

---

Under review as a conference paper at ICLR 2021
activation functions. Although ReLU activation functions g(z) maz(0, 2)) break the continuity, due to their contractive property, i.. satisfies ll - g( + c)ll I |<] for all x, e; follows that
9 (2; 0n) - gn(2 +x,
(0a 0, + b,) max(0, 0, + b,)))
(12)
- e.ll - |10 |||e|l
This provides a weaker upper bound for ReLU to mitigate the loss of continuity.
Even if the step size h in ResNet can be adjusted by multiply outputs of convolution layers by chosen constant, but unfortunately, the step size h in ResNet cannot be too small since very small step size decreases the efficiency in actual computations. We can experimentally show that when step size h is small (such as 10-1, 10 2 and 10-3), ResNet has no obvious robustness against adversarial examples, and when step size is very small (such 10- 10-9, 10 10-1)), ResNet is difficult to train, so not only the classification accuracy of adversarial examples but also the accuracy of clean examples is quite low.
To summarize, the main reason for ResNet's failure in adversarial examples is the step size h = destroys the upper bound given by Eqn.(5) and cannot find proper way to deal with it. Thus, when the input of ResNet is perturbed by adversarial noises, the amount of change in the result will become unpredictably large so that ResNet can no longer correctly classify the input.
3.2 NEURAL ODES
The residual block can be described yu+1 = =., +h 9,) with step size h 1. When taking smaller steps and add more layers, which in the limit, Neural ODEs parameterize the continuous dynamics of hidden units using an ordinary differential equation specified by a neural network
lim Ynth No dy dh
(13)
h
The solution of Egn. 113 can be computed using modern ODE solvers such as Runge-Kutte (Runge) 1895 Kutta 1901 Euler's method only uses f(n,, 0z), which means from any point on a curve, we can find an approximation of nearby point on the curve by moving a short distance along a tangent line to the curve. Instead of using single tangent line, modern numerical ODE methods often use lots of different tangent lines and weighted sum all of them to generate a super tangent line, we denote the super tangent line by F(.n. Yn; On, f).
We can show that modern ODE solvers based on Runge- Kutta are stable under some assumptions, the amount of change between yn and Z, holds an upper bound similar to theorem 3.1
Theorem 3.2. Let ft, y; 0) be continuous function for to < t S and -o < < 0o, and further assume that ft, 0), yt; f) satisfies the Lipschitz condition. Then compare the two namerical solutions Yn and Zn as h + O, there is constant é 2 O, such that amount of change between yn and on satisfies
max Izn -v.l - Elel (SngN(()
(14)
Proof. This result can be obtained in analogy with Eqn. in theorem 3.1
The neural network in Neural ODEs also use ReLU activation functions. And we already show that Eqn. (12) provides a weaker upper bound for ReI.U.
Besides, modern ODE solvers, such as Fehlberg method (Fehlberg 1969) and DOPRI method (Dor- mand & Prince 1980). can adaptively adjust the step size until that the desired tolerance is reached. At each step. two different approximations (for example, a fourth order Runge-Kutta and a fifth order Rutte-Kutta) for the solution are made and compared. If the two answers are in close agree- ment, the approximation is accepted. the two answers do not agree to a specified accuracy, the step size is reduced. If the answers agree to more significant digits than required, the step size is increased. So modern ODE solvers provide a weaker upper bound for step size when does not agree with h + 0.
4

---

Under review as a conference paper at ICLR 2021
To summarize, although Neural ODEs cannot hold the strong upper bound provided by Eqn. (14), still holds a weaker upper bound to constrain the error caused by adversarial noises. We will exper- imentally show that Neural ODEs is more robust adversarial examples compared with ResNet.
3.3
OTHER DEEP NEURAL NETWORKS WITH SKIP CONNECTIONS
Following similar procedures, we can figure out the reason why other deep neural networks with skip connections cannot resist adversarial examples.
3.3.1 POLYNET
PolyNet (Zhang et al. 2017) introduced family of models called PolyInception. Each PolyIncep- tion module is a polynomial combination of Inception units that can be described as
+ y y y F(y) + F(F(y))
(15)
Where y denotes the input, / the identity operator, and J the nonlinear transform carried by the residual block, which can also be considered as an operator.
has been shown that Equ. 15) can be interpreted as an approximation to one step of the backward (or implicit) Euler method (Lu et al. 2018):
Yn+1 Yn h1((n+1::n+1
(16)
And from Eqn.(T6) we can get
Yn+1 (I hf) yn
(17)
Where (T - hf)-1 can be formally rewritten as
1+ +... (hf)*
(18)
So, there two differences between the polynomial combination with the backward Euler method:
The polynomial combination is only a second order approximation to the backward Euler method, the truncation error generated when high order terms are ignored.
PolyNet faces the same problem with ResNet that the step size 1. Thus, PolyNet cannot holds the upper bound given by the backward Euler method.
And step size = is the main reason that PolyNet fails to resist adversarial examples.
3.3.2 FRACTALNET
FractalNet [Larsson et al. 2016) introduced a design strategy for neural network macro-architectue based self-similarity The expansion rule generates fractal architecture with C' intertwined columns can be described as
fc+ily) _ (fo fo) fi(y))
(19)
Where fy cynv(y) and denotes composition.
This expansion rule resembles the Runge-Kutta method of order 2 (also known Heun's method)(Lu et al. 2018)
Yst1 Yn + 1 + fln41,,n +hf yn)))
(20)
So, there two differences between the expansion rule with Heun's method:
The expansion rule losses two identity mapping. namely yx. compared to Heun's 's method. FractalNet faces the same problem with ResNet that the step size h = 1. Thus, FractalNet cannot holds the upper bound given by Heun's 's method.
And step size h is the main reason that FractalNet fails to resist adversarial examples.
5

---

Under review as a conference paper at ICLR 2021
3.3.3 REVNET
RevNet (Gomez al. 2017) introduced variant of ResNet where each layer's activations can be reconstructed exactly from the next layer's. s. Each reversible block takes inputs and produces outputs according to the following additive coupling rules
X Xn fi (Yn) Yx+1 = Yn + f2 (X++1)
(21)
This additive coupling rules can be interpreted as Euler's method applies to the following systems of differential equations (Lu et al. 2018)
== -(Y,)) Y f(X,0)
(22)
RevNet faces the same problem with ResNet that the step size Thus, RevNet cannot holds the upper bound given by Euler's method for the dynamic systems.
4 EXPERIMENTAL RESULTS
4.1 RESNET WITH STEP SIZE h + 1
We first evaluate the performance of ResNets with different step sizes on adversarial examples. Adopting the same hyperparameter settings for ResNet56 that presented in He et al. (016), we train ResNet56s on CIFARIO datasets with the step size decreases from 10" 10
The dassification accuracy of ResNet56s with step size hts1
adversarial CIFARIO Imagesic-O.1)
CIFARIO imagee((-0.3)
adversarial CIFAR10 Imagesic-005)
accurmo
step sizetfroms le4l to 1c-10)
Figure 1:
The classification accuracy of 11 ResNet56s both on clean CIFARIO images and three kinds of adversarial CIFARIO images(on the log scale). The adversarial CIFAR1O images are gen- erated by Fast Gradient Sign Method with = 0.1, 0.3, 0.5 on another neural network which has three residual blocks.
Through the numerical stability analysis of ResNet, we know that the classification result should be well behaved when considering small adversarial noises, provided that the step size of suffi- ciently small. In actual computations, however, when we decrease the step size from 10-1 to 10-10 ResNet has no obvious robustness against adversarial examples. And when step size is very small, ResNet is difficult to train. Therefore, the accuracy of ResNet56 is considerably low both on clean and adversarial inputs. We consider that ResNet is impossible to have natural robustness against adversarial examples.
4.2
THE NATURAL ROBUSTNESS OF NEURAL ODES
As we mentioned in 3.2 Neural ODEs define a continuous dynamic system, which is equivalent to ResNet with step size h 0. And Neural ODEs hold weaker upper bound constrain the emror

---

Under review as a conference paper at ICLR 2021
caused by adversarial noises. In this section, evaluate the performance of Neural ODEs under three white-box attacks, FGSM(Goodfellow et al., (2144), PMDdMadry et al. 2017). DI2- Xie
et al. 2019) and one black-box adversarial attack, Boundary Attack(Brende, et al. 2017p.
Without data augmentation, the accuracy of ResNet on clean MNIST and CIFARIO datasets is 99.33% and 89.60% the accuracy of Neural ODEs clean MNIST and CIFAR10 datasets is 99.05% and 69.94%. Experiment details can be seen in appendix
4.2.1 THE EVALUATION ON WHITE-BOX ADVERSARIAL ATTACKS
Table I shows that Neural ODEs have natural robustness under FGSM and PGD attacks. The clas- sification accuracy of adversarial MNIST images generated by FGSM only drops by less than 2% for all e. Even the strongest first order attack method PGD can hardly beat Neural ODEs on MNIST. Although the performance of Neural ODEs clean CIFAR10 10 images is worse than state-of tht-hhe-rt art networks like ResNet, still works well on adversarial CIFAR1O images generated by PGD.
Datasets
Áttack
Models
= 0.2
c= 0.3
<= 0.4
c 0.5
ResNet50
25.20
10.95
9.80
9.75
FGSM
Neural ODEs
97.50
97.46
97.52
97.48
MNIST
ResNet50
0
PGD
Neural ODEs
93.62
90.55
84.64
75.23
Datasets
Áttack
Models
8/255
12/255
16/255
20/ 255
ResNet50
4.65
4.16
3.60
3.26
FGSM
Neural ODEs
60.70
60.65
60.67
60.67
CIFAR
ResNet50
0
PGD
Neural ODEs
59.15
56.93
53.58
49.53
Table 1: The classification accuracy of Neural ODEs and ResNet50 under FGSM and PGD attacks. For both MNIST and CIFAR10, we set the size of perturbation of PGD in an infinite norm sense, the size of PGD step is set to 0.01, the number of PGD) steps is set to 40 and uniform random perturbation is added before performing PGD.
Nearal II AIAAMNNNT
The
(IDEA Boetet DIT-FOMM Amark
Nears 0BC
Neari OLC)
spalbnitrrmm
spilnntirom O 501
Figure 2: The evaluation of ResNet and Neural ODEs under DI--FGSM. 0 represents the accu- racy of ResNet and Neural ODEs without the adversarial attack applied. For MNIST, the maximum perturbation c is set to 0.5 among all experiments, with pixel value in [0.1]. For CIFAR1O, the maximum perturbation is set to 50 among all experiments, with pixel value in [0,255].
As shown in Figure when facing adversarial noises generated by DI--GGMM, for MNIST, Neural ODEs remains strongly resistant to perturbations while the accuracy ResNet drops sharply with larger For CIFAR10, the reduction accuracy of Neural ODEs significantly less than ResNet50 with an increase of e. Overall, Neural ODEs is more stable when facing white-box adversarial noises compared with ResNet.
4.2.2 THE EVALUATION ON BLACK- BOX ADVERSARIAL ATTACKS
Boundary Attack is decision-based attack that starts from large adversarial perturbation and then seeks to reduce the perturbation while staying adversarial(rrende et al.) 2017). For MNIST, Boundary Attack has almost no effect Neural ODEs. For CIFART0, however, seems that Neural

---

Under review as a conference paper at ICLR 2021
ODEs also fails in this gradient- free attack, but the performance of Neural ODEs is still better than ResNet50 when is larger than 33.
Nears OBC) Budhu
spsibillssm (.
spillo/from B 501)
Figure 3: The evaluation of ResNet and Neural ODEs under Boundary Attack. For Boundary Attack, the number of queries is fixed at 1,000. The initial step size and orthogonal step size are set to 0.1.
4.2.3 COMPARING WITH ADVERSARIAL TRAINING METHODS
As one can see, Neural ODEs can remain its robustness and outperform PGD, TRADES, and YOPO without taking any adversarial training methods.
Model PreAct-Res18
Adversarial training method PGD- PGD-10 Madry et al. (017)
Clean
PGD-20(E = 8/255)
84.82
41.61
PreAct-Res 18
TRADES- LOiZsng al. 2019b)
86.14
44.50
PreAct-Res18
YOPO-5-3 TOPO-5-3ZZang Zhang et al. 2019a
83.99
44.72
Neural ODEs
69.94
59.06
Table 2: The comparison of robustness between Neural ODEs with PreAct-Rest Restl trained by three types of adversarial training methods on CIFAR10.
CONCLUSION AND DISCUSSION
In this paper, we highlighted and analyzed the natural robustness of Neural ODEs. We proved there are two similar upper bounds for ResNet and Neural ODEs under assumptions of continuity and step size h - O. We showed that the step size = causes ResNet and the other three deep neural networks with skip connections fail adversarial examples. Neural ODEs define continuous dynamic system with step size h - O. However, in actual computations, Modern ODE solvers adaptively adjust the step size solve the continuous dynamic system and this brings weaker upper bound for Neural ODEs. We experimentally showed that Neural ODEs is more robust on adversarial examples compared with ResNet.
According to Theorem 3.1 and 3.2 the upper bound cle and ž|e| are related to Lipschitz constant and independent with the step size h. Cisse al. (2017) showed that the robustness DNNs can be improved by constraining the Lipschitz constant. In this paper, however, we are more concerned with the step size h. because the upper bound cannot be guaranteed if - is not satisfied. We experimentally demonstrated that the difference of ResNet with Neural ODEs in step size h sufficient to make significant difference robustness, even without constraining constant. Nevertheless, constraining the Lipschitz constant is expected to make neural networks to be more robust, which we would like to consider as our future work.
Lipschitz
Due to the similarity between DNNs and numerical methods for solving ODEs, we believe that can learn from the numerical stability analysis to design more robust deep neural networks.
8

---

Under review as a conference paper at ICLR 2021
REFERENCES
Naveed Akhtar, Jian Liu, and Ajmal Mian. Defense against universal adversarial perturbations. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3389- 3398, 2018.
Wenjun Bai, Changqin Quan, and Zhiwei Luo. Alleviating adversarial attacks via convolutional au- toencoder. In 2017 18th IEEE/ACIS International Conference on Software Engineering, Artificial Intelligence, Networking and Parallel/Distriuuted Computing (SNPD), pp. 53-58. IEEE, 2017.
Wieland Brendel, Jonas Rauber, and Matthias Bethge. Decision-based adversarial attacks: Reliable attacks against black-box machine learning models. arXiv preprint arXiv: 171..04212 2017.
Ricky TQ Chen, Yulia Rubanova, Jesse Bettencourt, and David Duvenaud. Neural ordinary differential equations. In Advances in neural information processing systems, pp. 6571-6583, 2018.
Moustapha Cisse, Piotr Bojanowski, Edouard Grave, Yann Dauphin, and Nicolas Usunier. Parseval networks: Improving robustness to adversarial examples. arXiv preprint
2017.
John R Dormand and Peter J Prince. family of embedded runge-kutta formulae. Journal of computational and applied mathematics, 6(1):19-26, 1980.
Emilien Dupont, Arnaud Doucet, and Yee Whye Teh. Augmented neural odes. In Advances in Neural Information Processing Systems, pp. 3140-3150, 2019.
Gintare Karolina Dziugaite, Zoubin Ghahramani, and Daniel M Roy. A study of the effect of jpg compression on adversarial images, arXiv preprint arXiv:I608. 00855, 2016.
Erwin Fehlberg. Low-order classical Runge- Kuttea formulas with stepsize control and their applica- fion to some heat transfer problems, volume 315. National aeronautics and space administration, 1969.
Chris Finlay, Adam M Oberman, and Bilal Abbasi. Improved robustness to adversarial examples using lipschitz regularization of the loss. 2018.
Aidan Gomez, Mengye Ren, Raquel Urtasun, and Roger Grosse. The reversible residual net- work: Backpropagation without storing activations. In Advances in neural information processing systems. pp. 2214-2224, 2017.
Ian J Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. arXiv preprint arXiv:1412.6572, 2014.
Eldad Haber and Lars Ruthotto. Stable architectures for deep neural networks. Inverse Problems, 34(1):014004, 2017.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 770-778, 2016.
Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. Imagenet classification with deep convo- lutional neural networks. Advances in neural information processing systems, pp. 1097-1105, 2012.
Dmitry Krotov and John Hopfield. Dense associative memory is robust to adversarial inputs. Neural computation, 30(12):3151-316 2018.
Wilhelm Kutta. Beitrag zur naherungsweisen integration totaler differentiagglecchungen. Math. Phys., 46:435-453, 1901.
Gustav Larsson, Michael Maire, and Gregory Shakhnarovich. Fractalnet: Ultraddeep neural net- works without residuals. arXiv preprint 16Xvv:1605.07646 2016.
Hyeungill Lee. Sungyeob Han, and Jungwoo Lee. Generative adversarial trainer: Defense to adver- sarial perturbations with gan. arXiv preprint 1105.03705003381 2017.

---

Under review as a conference paper at ICLR 2021
Yiping Lu, Aoxiao Zhong, Quanzheng Li, and Bin Dong. Beyond finite layer neural networks: Bridging deep architectures and numerical differential equations. In Internationall Conference on Machine Learning. pp. 3276-3285. PMLR, 2018.
Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks, arXiv preprint arXiv:1706 1776.06083, 2017.
Seyed-Mohsen Moosavi-Dezfooli, Alhussein Fawzi, and Pascal Frossard. Deepfool: simple and accurate method to fool deep neural networks. In Proceedings of the IEEE conference on com- puter vision and pattern recognition, pp. 2574-2582, 2016.
Aran Nayebi and Surya Ganguli. Biologically inspired protection of deep networks from adversarial attacks. arXiv preprint arXiv: htt::/703.00000 09200 2017.
Nicolas Paperno, Patrick McDaniel, Xi Wu, Somesh Jha, and Ananthram Swami. Distillation defense to adversarial perturbations against deep neural networks. In 2016 IEEE Symposium on Security and Privacy (SP), pp. 582-597. IEEE, 2016.
Andrew Slavin Ross and Finale Doshi-Veless Improving the adversarial robustness and in- terpretability of deep neural networks by regularizing their input gradients. arXiv preprint 17X1v:1711.09404, 2017.
Carl Runge. Über die numerische auflösung von differentialgleihhunge.. Mathematische Annalen, 46(2):167-178, 1895.
Lars Ruthotto and Eldad Haber. Deep neural networks motivated by partial differential equations. Journal of Mathematical Imaging and Vision, pp. 1-13, 2019.
Rupesh Kumar Srivastava, Klaus Greff, and Jürgen Schmidhuber. Highway networks. arXiv preprint arXiv: 00387, 2015.
Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus. Intriguing properties of neural networks. arXiv preprint 1312,6199 2013.
Florian Tramèr, Alexey Kurakin, Nicolas Papernot, Ian Goodfellow, Dan Boneh, and Patrick Mc- Daniel. Ensemble adversarial training: Attacks and defenses. arXiv preprint arXiv:1705.077 072044 2017.
E Weinan. proposal machine learning via dynamical systems. Communications in Mathematics and Statistics, 5(1):1-11, 2017.
Cihang Xie, Zhishuai Zhang, Yuyin Zhou, Song Bai, Jianyu Wang, Zhou Ren, and Alan L Yuille. Improving transferability of adversarial examples with input diversity. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2730-2739, 2019.
Weilin Xu, David Evans, and Yanjun Qi. Feature squeezing: Detecting adversarial examples in deep neural networks. arXiv preprint arXiv:1704.001 01 155, 2017.
Valentina Zantedeschi, Maria- Maria-Irina Nicolae, and Ambrish Rawat. Efficient defenses against adver- sarial attacks. In Proceedings of the 10th ACM Workshop Artificial Intelligence and Security, pp. 39.49, 2017.
Dinghuai Zhang, Tianyuan Zhang, Yiping Lu, Zhanxing Zhu, and Bin Dong. You only propagate once: Accelerating adversarial training via maximal principle. In Advances Neural Information Processing Systems, pp. 227-238, 2019a.
Hongyang Zhang, Yaodong Yu, Jiantao Jiao, Eric P Xing, Laurent El Ghaoui, and Michael I Jordan. Theoretically principled trade-off between robustness and accuracy. arXiv preprint 1rXiv:1901.08573. 2019b.
Xingcheng Zhang, Zhizhong Li, Chen Change Loy, and Dahua Lin. Polynet: A pursuit of structural diversity in very deep networks. In Proceedings of the IEEE Conference on Computer Vision and Patlern Recognition, pp. 718-726, 2017.
10

---

Under review as a conference paper at ICLR 2021
A TRAINING CONHIGURATION OF NEURAL ODES
Table AT summarizes our experiment settings for training Neural ODEs. We only change some hyperparameters in Neural ODEs, without taking any help of adversarial defense methods.
Training Configurations
MNIST
CIFARIO
training method
Adam dopri5
Adam dopri5
ODEslover convolution layers convolution channels start time to stop time & training epochs initial learning rate* mini-batches size
5
128
256
0
100
500
100
200
10
10
32
Table Al: The training configurations of Neural ODEs on MNIST and CIFARIO. *Every 50 epochs, the learning rate is reduced half.
It is worth to mention that there are no common training configurations for Neural ODEs on CI- FARIO, Dupont et al. (2019) had reported that the accuracy of Neural ODEs on CIFARIO test sets is 53.7%=0.2%% our experiment result is better than it.


Reviews:
[{"review_id": "yHBcPTwNtGO", "reviewer": "AnonReviewer5", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": " Summary:\n\nThis paper uses theoretical grounding, starting with Lipshitz continuity-based assumptions on residual connections, to show why such architectures are more susceptible to adversarial inputs. In the process, the authors draw a parallel between these residual connections and neural ODEs, showing how the latter can circumvent the main reason that leads to adversarial susceptibility for the former. Finally, via empirical evaluations, they show how neural ODEs have \"natural\" robustness to adversarial examples: they have a non-trivial performance on adversarial inputs, despite not being explicitly trained for robustness.\n\n \n##########################################################################\n\nReasons for score: \n\nThis draft in its current form lacks strong adversarial evaluation and makes strong claims without any experimental evidence. The title and the body of the paper suggest \"natural adversarial robustness\", but evaluated only against ($L_2$: this is my guess since it is not specified in the paper) PGD and FGSM (which, as has been seen recently in the literature, is not useful to see) attacks. Moreover, the Lipshitz-based assumptions in the proof for deriving conditions for residual connections seem a bit too strong. Keeping strong assumptions and weak empirical evaluations in mind, I feel this paper needs a lot more work before it can be considered for acceptance. \n \n##########################################################################\n\nPros: \n  \n- The analysis of existing variants of network architectures that include residual connections seems interesting. It helps to see why these models are theoretically unfit to achieve adversarial robustness. Some empirical evaluation on these models would also have been nice to see: just to see how one is more/less problematic than the other.\n \n##########################################################################\n\nCons: \n\n- My biggest concern is the weak evaluation of the mentioned models. Even though the paper's title (and most places in the paper) talk about \"natural robustness\", evaluation is performed only against one kind of attack (FGSM is just PGD with the number of steps = 1) and that too for just one norm. Please include more extensive evaluation, perhaps like second-order gradient attacks, gradient-free attacks, and augmentation based adversarial attacks, or change the title to reflect your current findings. \n\n- A large body of related works seems to have been missed out in this paper's literature review. For instance, there have been several works on Lipshitz-constraint-based robustness for neural networks are highly relevant, but seem to have been missed out ([example](https://arxiv.org/pdf/1704.08847.pdf), [another example](https://openreview.net/pdf?id=HkxAisC9FQ))\n\n- The second paragraph sets up the flow of the paper to hint at \"designing a deep neural network that has natural robustness\", whereas the main focus of this paper goes to the extent of only evaluating existing architectures. Additionally, there is no clear evidence to suggest that adversarial robustness is even possible just via a well designed neural network.\n\n- Residual connections are not limited to skipping only one connection (Eq 1), and using this as a base assumption should either be explicitly stated or worked out for the general case.\n\n- The function $f$ is related to one specific layer and is thus parameterized by its associated weights $\\theta_n$. However, Eq 7 talks about the Lipschitz continuity for the general layer. Does this mean that this model assumes that __all__ layers under consideration are Lipschitz continuous, that too with the same constant? In an ideal scenario where all layers somehow indeed, using the same constant $K$ would require taking the largest one of all layers, which can make it ridiculously large. I think this is a fatal flaw in the derivation process, and the authors should address it.\n\n- In what norm are all these examples operating? What are the parameters for the PGD attack? Configuration details like the number of steps, step size, and the number of random restarts, can make a significant difference in evaluation metrics. Also, since these attacks include randomness (especially PGD), please run them multiple times and report mean/std values.\n\n- Many numbers in Table 2 do not make any sense (my guess is this is because of the randomness in these attacks, which makes it even more important to have multiple runs). For instance, accuracy __increases** for neural ODE on CIFAR10 for FGSM when the perturbation budget is **increased**? Attack success rates should be strictly non-decreasing with increasing attack budgets since the adversary can copy-paste smaller-budget attacks and get at least the same attack success rate. The same problem holds for Resnet50 on CIFAR10 ($\\epsilon=0.4$ vs $0.5$), neural ODE on MNIST $\\epsilon=0.3$ to $0.4$. \n\n\nPlease address and clarify these cons.\n\n##########################################################################\n\nMinor issues:\n\n- Page 1, last paragraph \"...which has been used to solve ordinary differential equations\". Reference missing.\n\n- Section 2.1, last paragraph: \"...rely on using external models\". This statement is not true. Popular adversarial defense techniques like feature squeezing([ref](https://arxiv.org/pdf/1704.01155.pdf)) do not augment the dataset or use an external model.\n\n- Ambiguity in notation: next to Eq 1, what does $N(h)$ signify? \n\n- The move from Eq 7 to Eq 8 is a bit non-trivial: please add more steps in between to show the process explicitly \n\n- Section 3.3.1, just above Eq 16: \"It has been show that ...\". Reference missing\n\n- Just above Eq 18: \"can be formally rewrite\" -> \"can be formally rewritten as\". Also, the $(1+x)^{-1}$ Taylor series expansion holds only when $|x| <1$. Is that the case here, *i.e.* is $|hf|<1$ ?\n\n- Section 3.3.2, just above Eq 20 \"resembles the Runge-Kutta method of order 2\". Reference missing\n\n- Is the x-axis for Figure 1 on the log-scale? Please clarify Please do not refer to adversarial inputs as \"adversarial test set\", as it is likely to be confused with an adversarial test set that has been generated offline and is used for evaluation.\n\n- What are the numbers in Table 2? Accuracy ($f(\\hat{x}) = y$), or 1 - error success rate ($f(x) = f(\\hat{x})$)? Please clarify. \n", "extra_scores": {}, "extra_text": {}}, {"review_id": "YjZ2xnF9BbR", "reviewer": "AnonReviewer4", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper uses a high-order ODE solver to take an $h=1$ step of a neural\nnetwork layer.  The mechanics of training with an ODE solver that uses\nparameters to determine the dynamics of producing a layer output is previously\nknown.  This paper notes that outputs produced using an ODE integrator, wrt\nadversarial inputs, have established error bounds. They demonstrate the\nadditional adversarial robustness of operating in a regime better respecting\nsuch bounds.\n\nI feel the paper is well written and clear.  The central theoretical idea is not a huge leap, but is\nnovel in the context of robust machine learning, imho. While presenting a simple, basic idea is\nalways nice, the paper left me unclear about whether the demonstration was primarily intended\nto demonstrate a not terribly surprising theoretical prediction, or whether the technique would\nbe useful in practice.\n\nThey show many common networks with skip connections (resnet, etc.) correspond\nto a forward ODE integration scheme with large step size $h=1$, whereas the\ntheoretical ODE adversarial bounds only hold as $h\\rightarrow 0$.  They first show \nthat using fixed h corresponding to some fixed learning rate either does not\nsatisfy $h\\sim 0$ or has too slow convergence, with no gain in adversarial\nrobustness.  So instead, they use a variable step size integrator instead, to\nproject forward to a larger $h$.\n\nThe idea and theory are simple and fairly well presented, and the demonstration\non a simple dataset nicely shows the benefit of this approach compared to the\noriginal neural network.  However I felt a large number of significant things\nwere left out regarding choice of integrator.  Most obviously, Table 1 lacks\ndopri parameters.  How does adversarial robustness depend on such parameter[s]?\nWhat is the effect on execution time?\n\nThey only use one variable step size integrator.  They use a 4th and 5th order\nvariable step size integrator, to demonstrate the predicted natural ODE-based\nrobustness.  Several times I felt the presentation hinting that the low-order\nof integration schemes is to blame; however, for layers with discontinuities\n(relu), I naively expect lower order (and more evaluations?) might work out\njust as well.\n\nWhile the experiments constitute a simple demonstration, it still remained\ndifficult to judge the practical importance without some data comparing\nexecution time.  And once one begins to consider efficiency, a question\nof what styles of integrator work well in practice would be nice (ex. gear\nvs. Richardson vs. their one chosen integrator).\n\nEven if the ODE method is expensive, compute time can be compared with another\nways to promote robustness, such as adversarial training.  A comment about\nfeasibility of using ODE method in conjunction with other robustness methods\ncould be made.\n\n---\n\nI have read the authors' comments.  The addition of the boundary attack experiment\nwas an excellent step; however, it underscores a requirement for further analysis to\nunderstand *why*, apparently, in some cases the additional theoretical bound *fails* to\nconfer significant robustness.  The suggested \"natural robustness\" is only sometimes\npresent. Often clean accuracy is much reduced, so the method is not yet one I would consider\nuseful yet.  For me, understanding when the method works well (or not so well) would\nbring this work out of the realm of interesting theoretical bounds into one of more\ngeneral interest.\n", "extra_scores": {}, "extra_text": {}}, {"review_id": "EyDkVrjxCaV", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper offers an interesting viewpoint of adversarial robustness by comparing neural networks with skip connections such as ResNet with their Neural ODE counterparts. The authors analyze the different behaviors of the networks through their Lipschitz constants. They also try to support their claims that Neural ODEs are more robust due to their continuity (small step sizes) through experiments.  \n\nOne issue I find with the claim that neural ODE is more robust against adversarial examples compared to other neural networks with skip connections is that it does not appear to be an apples-to-apples comparison. The authors try to demonstrate in Theorems 3.1 and 3.2 that Neural ODE has a smaller Lipschitz constant due to using a smaller step size, but it does not take into account the different representational capacity of the model when we use step size h=1 in ResNet. Unless the authors can show the neural ODE version and the finite step size ResNet computes the same/similar classes of functions, comparing the Lipschitz constant is not very helpful. The same comparison issue is also present in the experiments, where the authors use different architectures for neural ODE and ResNet (many fewer layers for neural ODEs). It is very difficult to draw conclusions on which neural network is more robust if they have different representational capacities. \n\nOther than this main issue, there are also some limitations of the current work. It only analyzes neural networks with skip connections, whereas the phenomenon of adversarial attacks is general for many different neural networks. Also, judging from the accuracies under FSGM and PGD attacks, using a neural ODE cannot beat current state-of-art approaches of adversarial defense such as directing training with PGD examples. \n\nAs an aside, can the authors supply more details on how they generate adversarial examples for the neural ODE model? \n\nBased on the above observations, I believe the current paper is not ready for publication in ICLR yet. The author should focus on improving on their core argument on why the factor of step size directly contributes to robustness of neural ODEs. \n", "extra_scores": {}, "extra_text": {}}, {"review_id": "6odP2itSmB5", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "**Update**: Thank you to the authors for addressing the comments and updating the paper. I decrease my rating from 4 to 3 as the original claims of the paper were disproved by the experiments with black-box attacks on CIFAR10 which showed that Neural ODEs offer little advantage over Resnets. I believe that more experiments are needed to demonstrate that Neural ODEs offer robustness advantages. For example, when increasing the computational budget for one step FGSM attack, the accuracy stays the same, which might indicate that Neural ODEs obfuscates the gradient. Experiments with gradient-free attacks also indicate that Neural ODEs obfuscate the gradient (PGD has higher robust accuracy compared to gradient-free attacks). The authors should do an extensive evaluation with stronger attacks (PGD with DLR loss or CW loss with multiple random restarts up to 100-1000, AutoPGD); the study of the gradient obfuscation (confirm that when $\\epsilon=1.0$, the attacks can always succeed).\n\n#### Summary\nThe paper analyzes the robustness of residual deep neural networks through its relationships to ordinary differential equations. It establishes bound on the output change. It tries to explain the lack of robustness of ResNets. It suggests that Neural ODEs addresses the limitations of residual mappings and are naturally robustness to adversarial perturbations. In the experiments, the authors compared Residual Network and Neural ODEs to confirm that Neural ODEs are naturally robust.\n\n#### Concerns:\n- Missing references, e.g. Towards Robust ResNet: A Small Step but a Giant Leap Zhang et al (shows the limitation of ResNet and searched for optimal $h$ using grid search).\n- Limited technical novelty in the main theoretical results in Theorem 3.1. Similar bounds in terms of Lipschitz constant might be established elsewhere.\n- One of the main claims of the paper is that ResNets are not robust because $h = 1$. However, $c$ depends both on $h$ and $K$. So, it is possible to improve the robustness of ResNets by reducing its Lipschitz constant. However, this was not discussed in the paper.\n- The paper suggests using Neural ODEs, which was published elsewhere. Yet, the experimental evaluation is limited. Models should be evaluated against a wider range of attacks, e.g. black-box attacks and other attack norms ($l_{2}$-norm).\n- The attack parameters are not specified, e.g. number of attack iterations, number of restarts. \n  \n#### Minor comments:\n- Incorrect citation for adversarial training method: cited Ganin et al. instead of Goodfellow et. al.\n- Unfortunately, both of these explanations seem to imply that adversarial examples are inevitable to be avoided by deep neural networks. - the structure of the sentence might be incorrect.\n- In this paper, we attempt to utilize the natural property of neural networks to dense adversarial examples. - the sentence is hard to understand.\n", "extra_scores": {}, "extra_text": {}}, {"review_id": "6j7unwbVUK-", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The paper claims that neural ODEs are more robust to adversarial examples than ResNets and offers both empirical evidence and a theoretical explanation. However, I don't believe the theoretical analysis shows what is claimed; in fact, it doesn't seem to say anything about the magnitudes of Lipschitz constants or a relative comparison between them in the case of ResNets vs. neural ODEs. Below are the main issues with the analysis:\n\nClaims: \n\nIn the statement of Thm. 3.1, it is claimed that in ResNets, the distance between z_n (output corresponding to perturbed input) and y_n (output corresponding to the original input) is bounded above by some constant times the amount of perturbation to the input. The statement does not upper bound the constant and only lower bounds it trivially with 0. Therefore, this says nothing about how robust or not robust the model is to adversarial examples -- if the constant is large, the model could be sensitive to adversarial examples; if it's small, it could be robust. So, this statement is unrelated to the main claims of the paper. \n\nIn the statement of Thm. 3.2, a very similar claim is made about neural ODEs, except that the constant in front of the magnitude of perturbation (\\hat{c}) is different from the constant in Thm. 3.1 (c). No claims on the relative magnitudes of \\hat{c} and c are made, and so nothing can be said about the relative robustness or lack thereof of neural ODEs compared to ResNets. \n\nProofs:\n\nIf I understand correctly, Thm. 3.1 is supposed to make a claim about ResNets, which is when n (the number of steps/layers) is held constant as the step size (h) changes. In this case, the last step of the proof (eqn. 11) is incorrect, since nh does not necessarily equal to b - t_0, i.e. when n is fixed and h decreases, Euler's method will not reach b. \n\nOn the other hand, Thm. 3.2 seems to aim at making a claim about neural ODEs, which is when n increases as h decreases. In this case, nh would be equal to b - t_0. However the claim that the distance between z_n and y_n would be upper bounded in the limit of h -> 0 is odd, because it holds even when no limit is taken: nh is always equal to b - t_0 regardless of how small or large h is. So this cannot serve as an explanation for the differences in behaviours between neural ODEs and ResNets (where h = 1). \n\nSignificance:\n\nThe theoretical analysis is trivial and is a simple application of the definition of Lipschitz continuity, which is assumed. Additionally it has nothing to do with the main claims of the paper. \n\nConclusion: \n\nWhile the empirical phenomenon has been convincingly demonstrated and is intriguing, the theoretical analysis doesn't contain anything substantive and isn't much of an explanation. No other attempt at explaining this phenomenon (empirically or otherwise) was made other than the provided theoretical analysis. So I believe the paper is incomplete and will need substantially more effort at explaining the phenomenon. ", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: I thank the authors and reviewers for the discussions. Reviewers raised major concerns regarding the significance of the results and experiments. Given all, I think the paper needs more work before being accepted. I encourage authors to address comments raised by the reviewers to improve their paper.

- AC
Decision: Reject