On Robustness of Neural Ordinary Differential Equations
Authors: Hanshu YAN
Jiawei DU
Vincent TAN
Jiashi FENG
Venue:  (2020)
Keywords: Neural ODE

Abstract:  Neural ordinary differential equations (ODEs) have been attracting increasing attention in various research domains recently.  There have been some works studying optimization issues and approximation capabilities of neural ODEs, but their robustness is still yet unclear. In this work, we fill this important gap by exploring robustness properties of neural ODEs both empirically and theoretically. We first present an empirical study on the robustness of the neural ODE-based networks (ODENets) by exposing them to inputs with various types of perturbations and subsequently investigating the changes of the corresponding outputs. In contrast to conventional convolutional neural networks (CNNs), we find that the ODENets are more robust against both random Gaussian perturbations and adversarial attack examples. We then provide an insightful understanding of this phenomenon by exploiting a certain desirable property of the flow of a continuous-time ODE, namely that integral curves are non-intersecting. Our work suggests that, due to their intrinsic robustness, it is promising to use neural ODEs as a basic block for building robust deep network models. To further enhance the robustness of vanilla neural ODEs, we propose the time-invariant steady neural ODE (TisODE), which regularizes the flow on perturbed data via the time-invariant property and the imposition of a steady-state constraint. We show that the TisODE method outperforms vanilla neural ODEs and also can work in conjunction with other state-of-the-art architectural methods to build more robust deep networks.

Published as a conference paper at ICLR 2020
ON ROBUSTNESS OF NEURAL ORDINARY DIFFEREN- TIAL EQUATIONS
Hanshu YAN*, Jiawei DU*, Vincent Y. F. TAN & Jiashi FENG Department of Electrical and Computer Engineering National University of Singapore (hanshu. yan, dujiawei juu wew.iaweeluuuuueec.m nus. eus. {vtan, eleffiajenus.eddi sg
ABSTRACT
Neural ordinary differential equations (ODEs) have been attracting increasing at- tention in various research domains recently. There have been some works study- ing optimization issues and approximation capabilities of neural ODEs, but their robustness is still yet unclear. In this work, we fill this important gap by exploring robustness properties of neural ODEs both empirically and theoretically. We first present an empirical study on the robustness of the neural ODE-based networks (ODENets) by exposing them inputs with various types of perturbations and subsequently investigating the changes of the corresponding outputs. In contrast conventional convolutional neural networks (CNNs), we find that the ODENets are more robust against both random Gaussian perturbations and adversarial attack examples. We then provide an insightful understanding of this phenomenon by ex- ploiting certain desirable property of the flow of a continuous- time ODE, namely that integral curves are non-intersecting. Our work suggests that, due to their in- trinsic robustness, it is promising to use neural ODEs as a basic block for building robust deep network models. further enhance the robustness of vanilla neural ODEs, propose the time-invariant steady neural ODE (TisODE), which regu- larizes the flow on perturbed data via the time-invariant property and the imposi- tion of steady-state constraint. We show that the TisODE method outperforms vanilla neural ODEs and also can work in conjunction with other state-of-the-ae architectural methods to build more robust deep networks.
INTRODUCTION
Neural ordinary differential equations (Chen al., 2018) form family of models that approxi- mate nonlinear mappings by using continuous-time ODEs. Due to their desirable properties, such as invertibility and parameter efficiency, neural ODEs have attracted increasing attention recently (Dupont et al., 2019; Liu et al., 2019). For example, Grathwohl et al. (2018) proposed neural ODE-based generative model-the the FFJORD-tE to solve inverse problems; Quaglino et al. (2019) used a higher-order approximation of the states in a neural ODE, and proposed the SNet to acceler- ate computation. Along with the wider deployment of neural ODEs, robustness issues come the fore. However, the robustness of neural ODEs is still yet unclear. In particular, it is unclear how robust neural ODEs are comparison to the widely-used CNNs. Robustness properties of CNNs have been studied extensively. In this work, we present the first systematic study on exploring the robustness properties of neural ODEs.
To do so, we consider the task of image classification. We expect that results would be similar for other machine learning tasks such as regression. Neural ODEs are semeension-preservvie but a classification model transforms a high-dimensional input-such such as an image-ine into an output whose dimension equal the number of classes. Thus, we consider the neural ODE-based classi- fication network (ODENet) whose architecture is shown in Figure 1. An ODENet consists of three components: the feature extractor (FE) consists of convolutional layers which maps an input datum to multi-chamen feature map, neural ODE that serves the nonlinear representation mapping (RM), and the fully-connected classifier (FCC) that generates a prediction vector based on the output of the RM.
mappings,

---

Published as a conference paper at ICLR 2020
Input
The robustness of classification model can be evaluated through the lens of its performance on perturbed images. To comprehen- sively investigate the robustness of neural ODEs, perturb orig- inal images with commonly-used perturbations, namely, random Gaussian noise (Szegedy et al., 2013) and harmful adversarial ex- amples (Goodfellow et al., 2014; Madry et al., 2017). We conduct experiments in two common trainin the model only on authentic non oon-peturbedd images and training the model on authen- tic images as well as the Gaussian perturbed ones. We observe that ODENets are more robust compared to CNN models against all types of perturbations in both settings. We then provide an insight- ful understanding of such intriguing robustness of neural ODEs by exploiting a certain property of the flow (Dupont et al., 2019), namely that integral curves that start at distinct initial states are non- intersecting. The flow of a continuous- time ODE is defined as the family of solutions/paths traversed by the state, starting from dif- ferent initial points, and an integral curve is a specific solution for given initial point. The non-intersecting property indicates that an integral curve starting from some point is constrained by the integral curves starting from that point's 's neighborhood. Thus, in an ODENet, if correctly classified datum is slightly perturbed, the integral curve associated to its perturbed version would not change too much from the original one. Consequently, the perturbed datum could still be correctly classified. Thus, there exists intrinsic robustness regularization in ODENets, which is absent from CNNs.
Feature Extractor
Representation
Mapping (Neural ODE
ConnecCo Classifler
Prediction
Figure 1: The architecture of an ODENet. The neu- ral ODE block serves as a dimension-preserving nonlin- ear mapping.
Motivated by this property of the neural ODE flow, we attempt explore more robust neural ODE architecture by introducing stronger regularization the flow. We thus propose a Steudy neural ODE (TisODE). The TisODE removes the time dependence of the dynamics in an ODE and imposes steady-state constraint on the integral curves. Removing the time dependence of the derivative results in the time-invariant property of the ODE. To wit, given a solution z (t), another solution ū1 (t), with initial state (0) 21 (T") for some T > O, can be regarded as the - shift version of 21 (t). Such a time-invariant property would make bounding the difference between output states convenient. To elaborate, let the ouiput of neural ODE correspond to states at time 7' > 0. By the time-invariant property, the difference between outputs, 1| (7) - z (7)]], equals to |z| (T + T") - zi (T)Il- To control this distance, a steady-state regularization term introduced to the overall objective to constrain the change of state after time exceeds T. With the inne-invariant property and the steady-state term, we show that TisODE even is more robust. We do so by evaluating the robustness of TisODE-based classifiers against various types of perturbations and observe that such models are more robust than vanilla ODE-based models.
In addition, some other effective architectural solutions have also been recently proposed to improve the robustness of CNNs. For example, Xie et al. (2017) randomly resizes or pads zeros into test images to destroy the specific structure of adversarial perturbations. Besides, the model proposed by Xie et al. (2019) contains feature denoising filters to remove the feature- patterns of adversarial examples. We conduct experiments show that our proposed TisODE can work seamlessly and in conjunction with these methods to further boost the robustness of deep models. Thus, the proposed TisODE can be used generally applicable and effective component for improving the robustness of deep models.
In summary, contributions are follows. Firstly, are the first to provide a systematic empir- ical study on the robustness of neural ODEs and find that the neural ODE-based models are more robust compared to conventional CNN models. This finding inspires new applications of neural ODEs in improving robustness of deep models, a problem that concerns many deep learning theo- rists and practitioners alike. Secondly, we propose the TisODE method, which is simple yet effective in significantly boosting the robustness of neural ODEs. Moreover. the proposed TisODE can also be used in conjunction with other state-of-the- robust architectures. Thus, TisODE can serve as drop-in module to improve the robustness of deep models effectively.
2

---

where So R2d x (0) + Rdd denotes the trainable layers that are parameterized by weights 0 and z [0, 00) + Rd represents the d-dimensional state of the neural ODE. We assume that fe is continuous in and globally Lipschitz continuous in In this case, the imput Zin of the neural ODE corresponds to the state at = O, and the output Zout is associated to the state at some E (0, oo). also use fo to denote the Published as a conference paper at ICLR 2020
2 PRELIMINARIES ON NEURAL ODE
has been shown that a residual block (He et al., 2016) can be interpreted as the discrete approxima- tion of an ODE by setting the discretization step be one. When the discretization step approaches zero, it yields a family of neural networks, which are called neural ODEs (Chen et al., 2018). For- mally, in neural ODE, the relation between imput and output characterized by the following set of equations:
dz(l) fo(a((),)), x(0) Zin:
Zout
(T)),
dt
Because fo governs how the state changes with respect to time dynamics of the neural ODE.
Given input Zin the output Zont can be computed by solving the ODE in (1). If is fixed. the output out only depends on the input Vin and the dynamics fo. which also corresponds to the weighted lay- in the neural ODE, Therefore, the neural ODE can be represented the d-dimensional function (r(:)) of the input Zin and the dynamics f8. i.c.,
Zout z ZTT) z(0) + S fo(z(t),t) (Zin fo)-
The terminal time T of the output state (TT) is set be 1 in practice. Several methods have been proposed for training neural ODEs, such as the adjoint sensitivity method (Chen et al., 2018), SNet (Quaglino et al., 2019), and the auto-differentiation technique (Paszke al., 2017). In this work, we use the most straightforward technique, i.e., updating the weights 0 with the auto- differentiation technique in the PyTorch framework.
AN EMPIRICAL STUDY ON THE ROBUSTNESS OF ODENETS
Robustness of deep models has gained increased attention, is imperative that deep models em- ployed in critical applications, such as healthcare, are robust. The robustness of a model is measured by the sensitivity of the prediction with respect to small perturbations on the imputs. In this study, we consider three commonly-used perturbation schemes, namely random Gaussian perturbations, FGSM (Goodfellow et al., 2014) adversarial examples, and PGD (Madry al.. 2017) adversarial examples. These perturbation schemes reflect noise and adversarial robustness properties of the investigated models respectively. We evaluate the robustness via the classification accuracies on perturbed images, in which the original non-perturbed versions of these images are all correctly classified.
For fair comparison with conventional CNN models, we made sure that the number of parameters of an ODENet is close to that of its counterpart CNN model. Specifically, the ODENet shares the same network architecture with the CNN model for the FE and FCC parts. The only difference that, for the RM part, the input of the baEed RM is concatenated with one more channel which represents the time while the RM in CNN model has skip connection and serves as a residual block. During the training phase, all the hyperparameters are kept the same, including training epochs, learning rate schedules, and weight decay coefficients. Each model is trained three times with different random seeds, and we report the average performance (classification accuracy) together with the standard deviation.
3.1 EXPERIMENTAL SETTINGS
We conduct experiments to compare the robustness of ODENets with CNN models on three datasets, i.e., the MNIST (LeCun et al., 1998), the SVHN (Netzer et al., 2011), ,, and a subset of the ImageNet datset (Deng al., 2009). We call the subset ImgNet10 since it is collected from 10 synsets of ImageNet: dog. bird, car, fish, monkey, turtle, lizard, bridge, cow, and crab. We selected 3,000 training images and 300 test images from each synset and resized all images to 128 128.
Dataset:
3

---

Published as a conference paper at ICLR 2020
Architectures: On the MNIST dataset, both the ODENet and the CNN model consists of four convolutional layers and one fully-connected layer. The total number of parameters of the two models is around 140k. On the SVHN dataset, the networks are similar to those for the MNIST; only changed the input channels of the first convolutional layer to three. On the Img dataset, there nine convolutional layers and one fully-connected layer for both the ODENet and the CNN model. The numbers of parameters is approximately 280k. In practice, the neural ODE can be solved with different numerical solvers such as the Euler method and the Runge-Kutta methods (Chen et al., 2018). Here, we use the issill-mpleeented Euler method in the experiments. To balance the computation and the continuity of the flow, we solve the ODE initial value problem in equation (1) by the Euler method with step size 0.1. Our implementation builds on the open-source neural ODE codes.! Details on the network architectures are included in the Appendix.
The experiments are conducted using two settings on each dataset-training models only with original non-perturbed images and training models on original images together with their per- turbed versions. In both settings, added weight decay term into the training objective to reg- ularize the norm of the weights, since this can help control the model's representation capacity and improve the robustness of neural network (Sokolić et al., 2017). In the second setting, images perturbed with random Gaussian noise are used to fine- tune the models, because augmenting the dataset with small perturbations can possibly improve the robustness of models and synthesizing Gaussian noise does not incur excessive computation time.
Training:
3.2 ROBUSTNESS OF ODENETS TRAINED ONLY ON NON-PERTURBEI IMAGES
The first question we are interested in is how robust ODENets are against perturbations if the model is only trained on original non-perturbet images. We train CNNs and ODEnets to perform clas- sification on three datasets and set the weight decay parameters for all models to be 0.0005. We make sure that both the well-trained ODENets and CNN models have satisfactory performances on original non-perturbed images, i.e., around 99.5% for MNIST, 95.0% for the SVHN, and 80.0% for ImgNe110.
Since Gaussian noise ubiquitous in modeling image degradation, we first evaluated the robustness of the models in the presence of zero-mean random Gaussian perturbations. It has also been shown that a deep model is vulnerable to harmful adversarial examples, such as the FGSM (Goodfellow et al., 2014). We are also interested in how robust ODENets are in the presence of adversarial examples. The standard deviation of Gaussian noise and the -nom c of the FGSM attack for each dataset are shown in Table 1.
Table 1: Robustness comparison of different models. We report their mean classification accuracies (%) and standard deviations (mean + std) on perturbed images from the MNIST, the SVHN, and the Nmr datasets. Two types of perturbations are used-7ero-ment Gaussian noise and FGSM adversarial attack. The results show that ODENets much more robust in comparison CNN models.
Gaussian noise
Adversarial attack
MNIST
0 == 50
75
0 100
FGSM-0.15
FGSM-0.3
FGSM-0.5
CNN
98.110.77
85.8.14.3
56.415.6
63.4.12.3
24.0:L8.9
8.3.1..2
ODENet
98.7.10.6
90.6.15.4
73.2.1..6
83.5.10.9
42.1112.4
14.3:12.1
SVHN
a= 15
a = 25
0 35
FGSM-3/255
FGSM-5/255
FGSM-8/255
CNN
9..0+11.2
76.31.2.7
60.913.9
29.2.12.9
13.711.9
5.411.5
ODENet
95.7
8..
7..2.12.1
58.2.1..3
43.0.11.3
3..9
ImgNet10
a 0= 10
a 15
= 25
FGSM-5/255
FGSM-8/255
FGSM-16/255
CNN
80.1
63.3:2.00
40.8=2.7
28.5
18.1
1.4
ODENet
81.912.0
67.5+1..0
48.7-22.6
3..2+:.00
27.211.1
14.4+1.7
From the results in Table 1, we observe that the ODENets demonstrate superior robustness com- pared to CNNs for all types of perturbations. On the MNIST dataset, the presence of Gaussian
4

---

Published as a conference paper at ICLR 2020
perturbations with large of 100, the ODENet produces much higher accuracy on perturbed im- ages compared to the CNN model (73.2% vs. 56.4%). For the FGSM-0.3 adversarial examples, the accuracy of ONEnet is around twice high as that of the CNN model. On the SVHN dataset, ODENets significantly outperform CNN models, e.g., for the FGSM-5/255 examples, the accuracy of the ODENet is 43.0%, which is much higher than that of the CNN model (13.7%). On the ImgNet10, for both cases of o = 25 and FGSM-8/255, ODENet outperforms CNNs by a large margin of around 9%.
3.3 ROBUSTNESS OF ODENETS TRAINED ON ORIGINAL IMAGES TOGETHER WITH GAUSSIAN PERTURBATIONS
Training a model on original images together with their perturbed versions can improve the robust- ness of the model. As mentioned previously, Gaussian noise is commonly assumed to be present in real-world images. Synthesizing Gaussian noise is also fast and easy. Thus, we add random Gaussian noise into the original images to generate their perturbed versions. ODENets and CNN models are both trained on original images together with their perturbed versions. The standard de- viation of the added Gaussian noise is randomly chosen from (50, 75, 100} on the MNIST dataset, {15, 25, 35} on the SVHN dataset, and {10, 15, 25} on the ImgNet10. All other hyperparameters are kept the same as above.
Table Robustness comparison of different models. We report their mean classification accuracies (%) and standard deviations (mean + std) on perturbed images from the MNIST, the SVHN, and the 1mgNet10 datsets. Three types of perturbations are used geed-7ero-mean Gaussian noise, FGSM adversarial attack, and PGD adversarial attack. The results show that ODENets are more robust compared to CNN models.
Gaussian noise
Adversarial attack
MNIST
0 = 100
FGSM-0.3
FGSM-0.5
PGD-0.2
PGD-0.3
CNN
98.7.10.1
54.2111.
15.8
32.911.77
0.0.00.0
ODENet
99.4+-0.1
71.5+1.1
19.9
64.7+11.8
13.0+00.
SVHN
= 35
FGSM-5/255
FGSM-8/255
PGD-3/255
PGD-5/255
CNN
90.6.10.2
25.310.6
12.310.7
32.410.4
14.010.5
ODENet
95.1110.1
49.4
34.7.10.5
50.911..
27.2.1..4
ImgNet10
a 25
FGSM-5/255
FGSM-8/255
PGD-3/255
PGD-5/255
CNN
92.6+00.6
40.9+1.8
26.7+1.7
28.6++1.5
11,2+1.2
ODENet
92.6+00.5
42,0+0044
29.0+
29.8+00.4
12.3+-0.6
The robustness of the models is evaluated under Gaussian perturbations, FGSM adversarial exam- ples, and PGD (Madry al., 2017) adversarial examples. The latter a stronger attacker compared to the FGSM. The lox -nom e of the PGD attack for each dataset is shown in Table 2. Based on the results, we observe that ODENets consistently outperform CNN models on both two datasets. On the MNIST dataset, the ODENet outperforms the CNN against all types of perturbations. In par- ticular, for the PGD-0.2 adversarial examples, the accuracy of the ODENet (64.7%) is much higher than that of the CNN (32.9%). Besides, for the PGD-0.3 attack, the CNN is completely misled by the adversarial examples, but the ODENet can still classify perturbed images with an accuracy of 13.0%. On the SVHN dataset, ODENets also show superior robustness in comparison to CNN models. For all the adversarial examples, ODENets outperform CNN models by a margin of at least 10 percentage points. On the ImgNet10 dataset, the ODENet also performs better than CNN models against all forms of adversarial examples.
3.4 INSIGHTTS ON THE ROBUSTNESS OF ODENETS
From the results in Sections 3.2 and 3.3, we find ODENets are more robust compared to CNN models. Here, we attempt to provide an intuitive understanding of the robustness of the neural ODE. In ODENet, given some datum, the FE extracts informative feature map from the datum. The neural ODE, serving as the RM, takes as input the feature map and performs nonlinear mapping. In practice, use the weight decay technique during training which regularizes the norm of weights
5

---

Published as a conference paper at ICLR 2020
in the FE part, so that the change of feature map in terms of small perturbation the input can be controlled. We aim to show that, in the neural ODE, a small change on the feature map will not lead to large deviation from the original output associated with the feature map.
Theorem (ODE integral curves do not intersect (Coddington & Levinson, 1955; Younes, 2010; Dupont et al., 2019)). Let z1 (t) and (t)(t) be two solutions of the ODE in (1) with different initial conditions, i.e, 21 (0) o (0). In (1), So is continuous in and globally Lipschitz continuous in z. Then, holds that zi (t) + (2) for all E [0, oo).
To illustrate this theorem, considering a simple 1- dimensional system in which the state scalar. As shown in Figure 2, equation (1) has a solution 21 start- ing from A (0, 21 (0)), where (0) is the feature of some datum. Equation (1) also has another two solutions (t) and 2z( whose starting points A (0, (0)))0)) and A3 (0, (0)). both of which are close to Ai. Suppose A, between A and Ag. By Theorem 1, we know that the integral curve 21 (t) is always sandwiched between the integral curves 22 and 2t).).
180
205-
200
3.0
Now, let (2) 21 (0)), |2 (0)0 21 (0)1). Con- sider a solution b④ (t) of equation (1). The integral curve &1 (t) starts from a point A, (0, Σ (0)). The point A is the e-neighborhood of A₁ with (0) (1)-2100
Figure 2: No integral curves intersect.
The integral curve starting from A₁ is always sandwiched between two inte- gral curves starting from A₁ and Ag.
<6 By Theorem I, we know that (T) (T) S (7)
(T)). In other words, if any perturbation smaller than e is added to the scalar (0) in Ar, the deviation from the original output (T') is bounded by the distance between 22 (T) and 23()). In contrast, in CNN model, there is no such bound on the deviation from the original output. Thus, opine that due to this non-intersecting property, ODENets are intrinsically robust.
4 TISODE: BOOSTING THE ROBUSTNESS OF NEURAL ODES
In the previous section, we presented an empirical study on the robustness of ODENets and observed that ODENets more robust compared CNN models. In this section, explore how to boost the robustness of the vanilla neural ODE model further. This motivates the proposal of time-invariant steady neural ODEs (TisODEs).
4.1
TIME-INVARIANT STEADY NEURAL ODES
From the discussion in Section 3.4, the key to improv- ing the robustness of neural ODEs is to control the dif- ference between neighboring integral curves. By Grow- nall's inequality (Howard, 1998) (see Theorem 2 in the Appendix), we know that the difference between two ter- minal states is bounded by the difference between initial states multiplied by the exponential of the dynamics' Lip- schitz constant. However, it is very difficult to bound the Lipschitz constant of the dynamies directly. Alternatively, we propose to achieve the goal of controlling the output deviation by following two steps: (i) removing the time dependence of the dynamics and (ii) imposing certain steady-state constraint.
2,(7)2 -2 2,
k,(7)
2((T)
2,(0)
2,(t)
2,(0)
2.(0)
Figure 3: An illustration of the time- invariant property of ODEs. We can see that the curve Z1 (t) is exactly the hori- zontal translation of z (t) on the inter- val (o),
In the neural ODE characterized by equation (1), the dy- namics fo (z(),,)) t) depends on both the state z(t) at time and the time t itself. In contrast, if the neural ODE is modified to be üime-invariamt the time dependence of the dynamics is removed. Consequently, the dynamics depends only the state So, can rewrite

---

Published as a conference paper at ICLR 2020
the dynamies function fo (z), and the neural ODE is characterized as
dz((t) df
fo(z(t));
(0) = Ziui
Zout = z(T).
Let 21 (t) be solution of (2) on [0, 00) and c > 0 be small positive value. We define the set
M, = { [z (t). t)|t E (1. |z| (t) z, (0)l S c). This set contains all points on the curve of z (t) during 00, T) that are also inside the e-neighborhood of 21 (0). For some element (2) (T"), T') E M,,
let 1 (t) the solution of (2) which starts from 2 (0) Z1 (T"). Then we have
7 (t) = Z1 (t + T'
(3)
for all t in [0, so). The property shown in equation (3) is known as the time-invariane property. It indicates that the integral curve Z (t) is the -T' shift of 2, (t) (Figure 3).
interested in how large the difference between 21 (T) and 21 (T) is. In a robust model, the difference should be small, By
We can regard ū, (0)
slightly perturbed version of 21 (0), and
equation (3), have | (T) z (T) = ||z, (T +T') 2, (.) Since T' E [0, T). the difference
between z (I') and 2 (T)) can be bounded as follows,
1, T4T
J T+T
(fo(z dt s S (, (t))
7 (T)-
fo(z df S
(4)
where all norms are lz norms and [fa denotes the element-wise absolute operation of vector- valued function fo. That is to say, the difference between ž1 (T) and (T) can be bounded by only using the information of the curve z (t). For any 1' E [0, T] and element (z, (t'), t') Mi,, consider the integral curve that starts from zi (t'). The difference between the output state of this curve and z (T) satisfies inequality (4).
Therefore, we propose to add an additional term Lee to the loss function when training the time- invariant neural ODE:
N
f (t)l
2T
Liss Σ
(5)
where N the number of samples in the training set and (t) is the solution whose initial state equals to the feature of the 7th sample. The regularization term Lys is termed as the steady-state loss. This terminology "steady state" is borrowed from the dynamical systems literature. In a stable dynamical system, the states stabilize around a fixed point, known the steady-state, to infinity. If we can ensure that Lag is small, for each sample, the outputs of all the points M, will stabilize around 2,(T), Consequently, the model is robust. This modification of the neural ODE is dubbed Time-invarian steady neural ODE.
time tends
4.2 EVALUATING ROBUSTNESS OF TISODE BASSED CLASSIFIERS
Here, we conduct experiments to evaluate the robustness of our proposed TisODE, and compare TisODE- models with the vanilla ODENets. We train all models with original non-perturbed images together with their Gaussian perturbed versions. The regularization parameter for the steady- state loss Lage is set to be 0.1. All other hyperparameters are exactly the same as those in Section 3.3.
From the results in Table we can see that our proposed TisODE- models are clearly more robust compared to vanilla ODENets. On the MNIST dataset, when combating FGSM-0.3 attacks, the TisODE-based models outperform vanilla ODENets by more than 4 percentage points. For the FGSM-0.5 adversarial examples, the accuracy of the TisODE-based model is 6 percentage points better. On the SVHN dataset, the TisODE-based models perform better in terms of all forms of adversarial examples. On the ImgNet10 dataset, the MooDEE-based models also outperform vanilla ODE-based models on all types of perturbations. In the presence of FGSM and PGD-5/255 exam- ples, the accuracies are enhanced by more than percentage points.

---

Published as a conference paper at ICLR 2020
Table 3: Classification accuracy (mean + std in %) on perturbed images from MNIST, SVHN and ImgNet10. To evaluate the robustness of classifiers, we use three types of perturbations, namely zero-mean Gaussian noise with standard deviation o, FGSM attack and PGD attack. From the results, the proposed TisODE effectively improve the robustness of the vanilla neural ODE.
Gaussian noise
Adversarial attack
MNIST
100
FGSM-0.3
FGSM-0.5
PGD-0.2
PGD-0.3
CNN
98.7+-0.1
54.2+1.1
15.8+1.3
32.9+3.7
0.0+0.0
ODENet
99.4.10.1
71.51111
19.911.2
64.7.1..8
13.0:10.2
TisODE
99.6.10.0
75.7.11.4
26.5.13.8
67.4.11.5
13.2.11.0
SVHN
FGSM-5/255
FGSM-8/255
PGD-3/255
PGD-5/255
CNN
90.6+F0.2
25.3+0.6
12.3+0.7
+2.4
14.0+0.5
ODENet
95.1110.1
49,411.0
34.7.10.5
50,911.3
27.2
TisODE
94.911.11
51.611.2
38.2:1199
52.010.99
28.2.10.33
ImgNet10
25
FGSM-5/255
FGSM-8/255
PGD-3/255
PGD-5/255
CNN
92.6++0.6
40.9+1188
26.7+1.77
28.6+11.5
11.2+1.2
ODENet 92.6++0.5
42,0+00.4
29.0+110
29.8+0.4
12.3+0.6
TisODE 92.8.10.4
44.3
31.41111
31.1111.2
14.5.11.1
4.3 TISODE A GENERALLY APPLICABLE DROP-IN TECHNIQUE FOR IMPROVING THE ROBUSTNESS OF DEEP NETWORKS
In view of the excellent robustness of the TisODE, claim that the proposed TisODE be used as general drop-in module for improving the robustness of deep networks. We support this claim by showing the TisODE can work conjunction with other state-of-the-ar techniques and further boost the models' robustness. These techniques include the feature denoising (FDn) method (Xie al., 2019) and the input randomization (IR) method (Xie et al., 2017). We conduct experiments on the MNIST and SVHN datasets. All models trained with original non- penturetddbed images together with their Gaussian perturbed versions. We show that models using the FDn/IRd technique becomes much more robust when equipped with the TisODE. In the FDn experiments, the dot-product non- local denoising layer (Xie et al., 2019) is added to the head the fully-connecter classifier.
Table 4: Classification accuracy (mean + std in %) on perturbed images from MNIST and SVHN. We evaluate against three types of perturbations, namely zero-mean Gaussian noise with standard deviation a, FGSM attack and PGD attack. From the results, upon the CNNs modified with FDn and IRd, using TisODE can further improve the robustness.
Gaussian noise
Adversarial attack
MNIST CNN
0 100
FGSM-0.3
FGSM-0.5
PGD-0.2
PGD-0.3
98.7-10.1
54.2.11.1
15.8111.
32.9.13.7
0.0.0.00
CNN-FDn
99.0-10.1
74.0:14.1
32.6.15.3
58.9114.0
8.2412.6
TisODE-FDa
99.4+10.0
80.6+12.3
40.4+:5.7
72.6+12.4
28.2433.6
CNN-IRd
95.3
78.1
36.742.1
79.6
55.5+12.9
TisODE
97.6
86,812.3
49.110.2
88.8+0.9
66.0+10.9
SVHN
35
FGSM-5/255
FGSM-8/255
PGD-3/255
PGD-5/255
CNN
90.6:0..2
25.3+00.6
12.340.7
32.4+0.4
14.0#0.5
CNN-FDn
92.4=1011
43.8
31.5++3.0
40.0+t2.6
19.6::3.4
TisODE-FD.
95.2-10.1
57.8+1.7
48.242.0
53,4+22.9
3..
CNN-IRd 84,9+1122
65.8+004
54.7:11.
74.0:10.5
64.51:0.8
TIRODE-IRd 91.7+0.55
74.4++1.2
61.9++1.8
81.6+0.88
71.0++0.5
From Table 4, we observe that both FDn and IRd can effectively improve the adversarial robustness of vanilla CNN models (CNN-FDn, CNN-IRd). Furthermore, combining our proposed TisODE with FDn IRd (TisODE-FDn. TisODE-IRd), the adversarial robustness of the resultant model significantly enhanced. For example, on the MNIST dataset, the additional use of our TisODE increases the accuracies the PGD-0.3 examples
least 10 percentage points for both FDn (8.2% to 28.2%) and IRd (55.5% to 66.0%). However, on both MNIST and SVHN datasets, the IRd technique improves the robustness against adversarial examples, but its performance is worse
8

---

Published as a conference paper at ICLR 2020
on random Gaussian noise. With the help of the TisODE, the degradation in the robustness against random Gaussian noise can be effectively ameliorated.
5 RELATED WORKS
In this section, we briefly review related works on the neural ODE and works concerning improving the robustness of deep neural networks.
Neural ODE:
The neural ODE (Chen et al., 2018) method models the input and output as two states of a continuous-time dynamical system by approximating the dynamics of this system with trainable layers. Before the proposal of neural ODE, the idea of modeling nonlinear mappings using continuous-tims dynamical systems was proposed in Weinan (2017). Lu et al. (2017) also showed that several popular network architectures could be interpreted as the discretization of a continuous- time ODE. For example, the ResNet (He et al., 2016) and PolyNet (Zhang et al., 2017) associated with the Fuler scheme and the FractalNet (Larsson et al., 2016) 6) is related to the Runge- Kutta scheme. In contrast these discretization models, neural ODEs are endowed with intrinsic invertibility property, which yields a family of invertible models for solving inverse problems (Ardizzone et al., 2018), such the FFJORD (Grathwohl al., 2018).
Recently, many researchers have conducted studies on neural ODEs from the perspectives of opti- mization techniques, approximation capabilities, and generalization. Concerning the optimization of neural ODEs, the auto-differentiation techniques can effectively train ODENets, but the train- ing procedure is computationally and memory inefficient. To address this problem, Chen et al. (2018) proposed to compute gradients using the adjoint sensitivity method (Pontryagin, 2018). in which there is no need to store any intermediate quantities of the forward pass. Also in Quaglino et al. (2019), the authors proposed the SNet which accelerates the neural ODEs by expressing their dynamics as truncated series of Legendre polynomials. Concerning the approximation capability, Dupont et al. (2019) pointed out the limitations in approximation capabilities of neural ODEs be- cause of the preserving of input topology. The authors proposed an augmented neural ODE which increases the dimension of states by concatenating zeros so that complex mappings can be learned with simple flow. The most relevant work to ours concerns strategies to improve the generalization of neural ODEs. In Liu et al. (2019), the authors proposed the neural stochastic differential equation (SDE) by injecting random noise to the dynamics function and showed that the generalization and robustness of vanilla neural ODEs could be improved. However, our improvement on the neural ODEs is explored from different perspective by introducing constraints on the flow. We empiri- cally found that our proposal and the neural SDE can work in tandem to further boost the robustness of neural ODEs.
Robust Improvement: A straightforward way of improving the robustness of model is to smooth the loss surface by controlling the spectral norm of the Jacobian matrix of the loss function (Sokolić et al., 2017). In terms of adversarial examples (Carlini & Wagner, 2017: Chen et al., 2017), re- searchers have proposed adversarial training strategies (Madry et al., 2017; Elsayed et al., 2018; Tramèr et al., 2017) in which the model is fine-tuned with adversarial examples generated in real- time. However, generating adversarial examples is not computationally efficient, and there exists a trade-off between the adversarial robustness and the performance on original non-perturbed im- ages (Yan et al., 2018; Tsipras et al., 2018). In Wang et al. (2018a), the authors model the ResNet as a transport equation, in which the adversarial vulnerability can be interpreted as the irregularity of the decision boundary. Consequently, a diffusion term introduced to enhance the robustness of the neural nets. Besides, there are also some works that propose novel architectural defense mechanisms against adversarial examples. For example, Xie et al. (2017) utilized random resizing and random padding to destroy the specific structure of adversarial perturbations; Wang et al. (201 8b) and Wang et al. (2018c) improved the robustness of neural networks by replacing the output layers with novel interpolating functions; In Xie et al. (2019), the authors designed feature denoising filter that can remove the perturbation's 's pattern from feature maps. In this work, we explore the intrinsic robust- ness of a specific novel architecture (neural ODE), and show that the proposed TisODE can improve the robustness of deep networks and can also work in tandem with these state-of-tt of-te-of-the-nn the-art methods Xie et al. (2017; 2019) to achieve further improvements.

---

Published as a conference paper at ICLR 2020
CONCLUSION
In this paper, we first empirically study the robustness of neural ODEs. Our studies reveal that neural ODE-based models are superior in of robustness compared CNN models. We then explore how to further boost the robustness of vanilla neural ODEs and propose the TisODE. Finally, we show that the proposed TisODE outperforms the vanilla neural ODE and also can work in conjunc- tion with other state-of- the-an art techniques to further improve the robustness of deep networks. Thus, the TisODE method is effective drop-in module for building robust deep models.
ACKNOWLEDGEMENT
This work funded by a Singapore National Research Foundation (NRF) Fellowship (R-263-000- D02-281).
Jiashi Feng was partially supported by NUS IDS R-263-000-C67-666, ECRA R- R-263-000-C88--33, MOE Tier-II R- (=223-0000D17-112 and ALSG R-263-000-D99-490
10

---

Published as a conference paper at ICLR 2020
REFERENCES
Lynton Ardizzone, Jakob Kruse, Sebastian Wirkert, Daniel Rahner, Eric W Pellegrini, Ralf' S Klessen, Lena Maier-Hein, Carsten Rother, and Ullrich Köthe. Analyzing inverse problems with invertible neural networks. arXiv preprint arXiv: 1e0Xiv:800..00 04730, 2018.
Nicholas Carlini and David Wagner. Towards evaluating the robustness of neural networks. In 2017 IEEE Symposium on Security and Privacy (SP), pp. 39-57. IEEE, 2017.
Yu Chen, Huan Zhang, Yash Sharma, Jinfeng Yi, and Cho-Jui Hsieh. Zoo: Zeroth order opti- mization based black-box attacks to deep neural networks without training substitute models. In Proceedings of the 10th ACM Workshop Artificial Intelligence and Security, pp. 15-26. ACM, 2017.
Tian Qi Chen, Yulia Rubanova, Jesse Bettencourt, and David K Duvenaud. Neural ordinary differ- ential equations. In Advances in neural information processing systems, pp. 6571-6588 2018.
Earl A Coddington and Norman Levinson. Theory of ordinary differential equations. Tata McGraw-
Hill Education, 1955.
Jia Deng, Wei Dong. Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale scale hi- erarchical image database. In 2009 IEEE conference on computer vision and pattern recognition, pp. leee, 2009.
Emilien Dupont, Amaud Doucet, and Yee Whye Teh. Augmented neural odes. arXiv preprint arXiv:190..01611, 2019,
Gamaleldin F Elsayed, Shreya Shankar, Brian Cheung, Nicolas Papernot, Alex Kurakin, Ian Good- fellow, and Jascha Sohl-Dickstein. Adversarial examples that fool both human and computer 10, 2018.
vision. arXiv preprint
Ian J Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. arXiv preprint arXiv:1412.6572 2014.
Will Grathwohl, Ricky TQ Chen, Jesse Betternourt, llya Sutskever, and David Duvenaud. Ffjord: Free-form continuous dynamics for scalable reversible generative models. arXiv preprint arXiv:1810.01367. 2018.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 770-778, 2016.
Ralph Howard. The gronwall inequality. lecture notes, 1998.
Gustav Larsson, Michael Maire, and Gregory Shakhnarovich. Fractalnet: Ultra-deep neural net- works without residuals. arXiv preprint arXiv:1605.07648, 2016.
Yann LeCun, Léon Bottou, Yoshua Bengio, Patrick Haffner, et al. Gradient-based learning applied to document recognition. Proceedings of the IEFF, 86(1 1):2278-2324, 1998.
Xuanqing Liu, Si Si, Qin Cao, Sanjiv Kumar, and Cho-Jui Hsieh. Neural sde: Stabilizing neural ode networks with stochastic noise. arXiv preprint arXiv: 02355, 2019.
Yiping Lu, Aoxiao Zhong, Quanzheng Li, and Bin Dong. Beyond finite layer neural net- works: Bridging deep architectures and numerical differential equations. arXiv preprint 17X1v:1710.10121, 2017.
Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. arXiv preprint arXiv: 1706.06083, 2017.
Yuval Netzer, Tao Wang, Adam Coates, Alessandro Bissacco, Bo Wu, and Andrew Ng. Reading digits in natural images with unsupervised feature learning. 2011. 1.

---

Published as a conference paper at ICLR 2020
Adam Paszke, Sam Gross, Soumith Chintala, Gregory Chanan, Edward Yang, Zachary Vitito, Zeming Lin, Alban Desmaison, Luca Antiga, and Adam Lerer. Automatic differentiation in pytorch. 2017.
Lev Semenovich Pontryagin. Mathematical theory of optimal processes. Routledge, 2018.
Alessio Quaglino, Marco Gallieri, Jonathan Masci, and Jan Koutník. Accelerating neural odes with spectral elements. arXiv preprint arXiv:
2019.
Jure Sokolić, Raja Giryes, Guillermo Sapiro, and Miguel RD Rodrigues, Robust large margin deep neural networks. IEEE Transactions on Signal Processing, 65(16):4265-4288 2017.
Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus. Intriguing properties of neural networks. arXiv preprint arXiv: 131p.6112669999 2.6999, 2013.
Florian Tramèr, Alexey Kurakin, Nicolas Papernot, Ian Goodfellow, Dan Boneh, and Patrick Mc- Daniel. Ensemble adversarial training: Attacks and defenses. arXiv preprint arXiv:1705.0722 2017.
Dimitris Tsipras, Shibani Santurkar, Logan Engstrom, Alexander Turner, and Aleksander Madry. Robustmess may be odds with accuracy. arXiv preprint
2018.
B. Wang, B. Yuan, Z. Shi, and S. Osher. ResNets Ensemble via the Feynman-Kac Formalism to Improve Natural and Robust Accuracies. arXiv e-prints, art. arXiv: 1.111.1..1755. Nov 2018a.
Bao Wang, Alex T Lin, Zuoqiang Shi, Wei Zhu, Penghang Yin, Andrea L Bertozzi, and Stanley Os- her. Adversarial defense via data dependent activation function and total variation minimization. arXiv preprint arXiv: 1809 08516, 2018b.
Bao Wang. Xiyang Luo, Zhen Li, Wei Zhu, Zuoqiang Shi. and Stanley Osher. Deep neural nets with interpolating function as output activation. In Advances in Neural Information Processing Systems, pp. 743-753, 20.
E Weinan. proposal machine learning via dynamical systems. Communications in Mathematics and Statistics, 5(1):1-11, 2017.
Cihang Xie, Jianyu Wang, Zhishuai Zhang, Zhou Ren, and Alan Yuille. Mitigating adversarial effects through randomization. arXiv preprini 1711v:1711.00 2017.
Cihang Xie, Yuxin Wu, Laurens van der Maaten, Alan Yuille, and Kaiming He. Feature denoising for improving adversarial robustness. In Proceedings the IEEE Conference Computer Vision and Pattern Recognition, pp. 501-509, 2019.
Ziang Yan, Yiwen Guo, and Changshui Zhang. Deep defense: sarial robustness. In Advances in Neural Information Processing Systems, pp. 419-428, 2018.
Training dnns with improved adver-
Laurent Younes. Shapes and diffeomorphisms, volume 171. Springer, 2010.
Xingcheng Zhang, Zhizhong Li, Chen Change Loy, and Dahua Lin. Polynet: A pursuit of structural diversity in very deep networks. In Proceedings of the IEEE Conference Computer Vision and Pattern Recognition, pp. 718-726, 2017.
12

---

Published as a conference paper at ICLR 2020
7 APPENDIX
7.1 NETWORKS USED ON THE MNIST, THE SVHN, AND THE IMGNETI0 DATASETS
Table 5: The architectures of the ODENets different datasets.
MNIST
Repetition
Layer
x
ConviL, 64, 1) + GroupNorm + ReLU
FE
x1
Conv(64, 64, 4, 2) GroupNorm + ReLU
RM
x2
Conv(64, 64, 3, 1) + GroupNorm + ReLU
FCC
AdaptiveAvgPool22 + Linear(64.10)
SVHN
Repetition
Layer
x
Conv(3, 64, 3, 1) + GroupNorm + ReLU
FE
x1
Conv(64, 64, 4, 2) + GroupNorm + ReLU
RM
x2
Conv(64, 64, 3, 1) + GroupNorm + ReLU
FCC
AdaptiveAvyPool22 + Lincar(64,,00
ImgNet10
Repetition
Layer
x
Conv(3, 32, 5, 2) + Group.Norm MaxPooling(() BaiscBlock(32, 64, 2) MaxPooling(2)
x
FE
x1
RM
x3
BaiscBlock(64,, 64, 1)
FCC
x1
Adaptivc-vgPool224 + Lincar(64,,0)
In Table 5, the four arguments of the Conv layer represent the input channel, output channel, kernel size, and the stride. The two arguments of the Linear layer represents the input dimension and the output dimension of this fully-connected layer. In the network on the ImgNet10, the BasicBlock refers to the standard architecture in (He et al., 2016), the three arguments of the BasicBlock repre- sent the input channel, output channel and the stride of the Conv layers inside the block. Note that we replace the BatchNorm layers in BasicBlocks as the GroupNorm to guarantee that the dynamics of each datum independent of other data in the same mini-batch.
7.2 THE CONSTRUCTION OF IMGNETI0 DATASET
Table 6: The corresponding indexes to each class in the original ImageNet dataset
Class
Indexing
dog bird
n02090721.
102091032.
n02088094
n01532829,
n01558993,
n01534433
n02814533,
n03930630,
a03100240
n01484850.
n01491361,
n01494475
monkey
n02483708,
n02484975,
n02486261
turtle
n01664065,
n01665541,
n0166711 144
lizard
n01677366,
m01682714,
n01685808
bridge
n03933933,
n04366367.
n04311004
cow
n02403003,
n02408429.
n02410509
crab
n01980166,
n01978455,
n01981276
7.3 GRONWALL'S 'S INEQUALITY
We formally state the Gronwall's Inequality here, following the version in (Howard, 1998).
13

---

Published as a conference paper at ICLR 2020
Theorem 2. Let U C R be open set. Let f: f x [0, + R be continuous function and let
21, Zz. [0, + U satisfy the initial value problems:
dz (1)
f(zz (t),t),
z1(t) X1
dt
dz2(1) [(z2(1), (), =2(1) = X2
dt
Assume there a constant C 2 0 such that, for all E [).
[l - f(z (1), 0) - (2|z2(()) 21 (1)
Then, for any t E [0,
[1) 22(1)] & [x2 - x.
7.4
MORE EXPERIMENTAL RESULTS
7.4.1
COMPARISON IN THE SETTING OF ADVERSARIAL TRAINING
We implement the adversarial training of the models on the MNIST dataset, and the adversarial examples for training are generated in real-time via the FGSM method (epsilon=0.3) during each epoch (Madry al., 2017). The results of the adversarially trained models are shown in Table 7. We can observe that the neural ODE-based models are consistently more robust than CNN models. The proposed TisODE also outperforms the vanilla neural ODE.
Table Classification accuracy (%) on perturbed images from MNIST. To evaluate the robustness of classifiers, we use three types of perturbations, namely zero-mean Gaussian noise with standard deviation a, FGSM attack and PGD attack.
Gaussian noise
Adversarial attack
MNIST CNN
0 = 100
FGSM-0.3
FGSM-0.5
PGD-0.3
58.0
98.4
21.1
5.3
ODENet
84.2
99.1
36.0
12.3
TisODE
87.9
99.1
66.5
78.9
7.4.2 EXPERIMENTS ON THE CIFARIO DATASET
We conduct experiments CIFAR10 to compare the robustness of CNN and neural ODE-based models. We train all the models only with original nen-uertrrbed images and evaluate the robustness of models against random Gaussian noise and FGSM adversarial attacks. The results Table We can observe that the ONENet is more robust than the CNN model in terms of both the random noise and the FGSM attack. Besides, our proposal, TisODE, can improve the robustness of the vanilla neural ODE.
shown in
Table 8: Classification accuracy (%) on perturbed images from CIFAR1O. To evaluate the robustness of classifiers, we use two types of perturbations, namely zero-mean Gaussian noise with standard deviation and FGSM attack.
Gaussian noise
Adversarial attack
CIFAR10
20
FGSM-8/255
FGSM-10/255
CNN
70.2
57.6
24.3
18.4
ODENet
72.6
60.6
31.2
26.0
TisODE
74.3
62.0
33.6
26.8
Here, we control the number of parameters to be the same for all kinds of models. We use a small network, which consists of five convolutional layers and one linear layer.
14

---

Published as a conference paper at ICLR 2020
Table 9: The architecture of the ODENet on CIFARIO.
Repetition
Layer
Conv(3, 16, 3, Group ReLU
FE
Conv(16. 32, 3, 2) GroupNorm + ReLU
x1
Conv(32, 64, 3, 2) + GroupNorm + ReLU
RM
x2
Conv(64, 64, 3, 1) GroupNorm ReLU
FCC
x1
AdaptiveAv@Poll21 + Linear(64410))
7.4.3 AN EXTENSION ON THE COMPARISON BETWEEN CNNS AND ODENETS
Here, we compare CNN and neural ODE-based models by controlling both the number of parameters and the number of function evaluations. We conduct experiments on the MNIST dataset, and all the models are trained only with original non-perturbed images.
For the neural ODE-based models, the time range is set from 0 to 1. We use the Fuler method, and For the CNN models
the step size is set be 0.05. Thus the number of evaluations 1/0.05 (specifically ResNet), repeatedly concatenate the residual block for 20 times, and these 20 blocks share the same weights. Our experiments show that, in this condition, the neural ODE-based models still outperform the CNN models (FGSM-0.155 87.5% vs. 81.9%, FGSM-0.3: 53.4% vs. 49.7%, PGD-0.2: 11.8% vs. 4.8%).
15


Reviews:
[{"review_id": "SygdqthatS", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The paper is concerned with neural ODE-based networks, specifically their robustness. While ODEs are a classical subject in mathematics with many applications in the sciences and beyond, neural ODEs are a recently proposed family of models for nonlinear mappings in the context of machine learning systems. There they show promise and are an active field of research.\n\nThe paper makes two primary contributions. (1) It studies the robustness of neural ODE, and (2) proposes a more robust variant of neural ODEs. For (1), robustness to both perturbed and adversarial inputs is considered, and theoretical interpretations for the robustness of neural ODEs are given. These theoretical insights form the basis of the contribution in (2).\n\nThe paper is well and clearly written, supplies most of the necessary theoretical background and offers useful contributions. I recommend the paper for publication.\n\nIn terms of improving the paper further, I’d suggest a slightly less casual treatment of the conditions under which the mathematical statements quoted hold. E.g. Theorem 1 is part of the classical Picard-Lindelof theorem and requires similar conditions (or at least the conditions of the necessary and sufficient, but less well known, Okamura's theorem, see [1]). A differentiable counterexample if these conditions don’t hold can be found e.g. in Wikipedia [2]. I see that the authors have responded to that point on the openreview website. I’d suggest however that for a result going back to the early 19th century citing a paper from 2019 (which itself cites a textbook on computational anatomy) seems suboptimal from an educational point of view.\n\nAnother possible improvement of the paper could be to expand the adversarial attacks considered to the gradient-free optimization techniques employed in e.g. [3] which have sharply reduced other defenses against adversarial attacks.\n\n[1] https://www.ams.org/journals/proc/1967-018-04/S0002-9939-1967-0212240-6/S0002-9939-1967-0212240-6.pdf\n[2] https://en.wikipedia.org/wiki/Picard–Lindelöf_theorem#Example_of_non-uniqueness\n[3] https://arxiv.org/abs/1802.05666", "extra_scores": {"experience_assessment": "I have read many papers in this area.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper at least twice and used my best judgement in assessing the paper.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I carefully checked the derivations and theory."}, "extra_text": {}}, {"review_id": "HJeAZTd-KB", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper studied the robustness of neural ODE-based networks (ODENets) to various types of perturbations on the input images. The authors observed that ODENets are more robust to both Gaussian perturbation and adversarial attacks, which the authors explained as non-intersecting of the integral curves for different initial points. Moreover, the authors proposed the time-invariant steady neural ODE (TisODE) to enhance the robustness of ODENets. I list my concerns below:\n\n1. To show the ODENet is more robust, the authors should bound the gap between the integral curves for different inputs. Non-intersecting of the integral curves does not guarantee the robustness. \n\n\n2. The ODENet architecture showed in Figure~1 can be regarded as an augmented CNN. I think the identity map gives a good trade-off between robustness and generalization. To enhance robustness, one might design an expansion map, but this, in general, hurt the accuracy of the model.\n\n3. Why do not perform experiments on the CIFAR10 benchmark dataset? I think it is very important to add these results.\n\n4. To verify the robustness of ODENets and CNNs, the authors should also perform adversarial training besides training on the original and noisy images with Gaussian perturbation.\n\n5. Theorem~1 is wrong. I suggest the authors check the conditions to make it valid. We can construct an ODE of the form (1) that blows up in finite-time, e.g., dx/dt = x^2.\n\n6. Most importantly, the authors should match the number of function evaluations of neural ODE with the depth of the CNN, in addition to matching the number of parameters. Please perform such a comparison in rebuttal. (THIS IS THE MOST IMPORTANT RESULT I WANT TO SEE IN REBUTTAL)\n\n7. The authors did not compare with existing work that tries to improve the robustness of neural nets from a differential equation viewpoint. The related works should be elaborated.\n\n\n======================\nI would like to point out a few related papers that lift the dimension of ODE to a transport equation and improve the neural nets' robustness from the lens of the transport equation's theory. Also, the author should compare their results with some reported results in 1, 3, 4.\n\n1. Bao Wang, Binjie Yuan, Zuoqiang Shi, Stanley J. Osher. ResNets Ensemble via the Feynman-Kac Formalism to Improve Natural and Robust Accuracies, arXiv:1811.10745, NeurIPS, 2019\n\n2. Bao Wang, Xiyang Luo, Zhen Li, Wei Zhu, Zuoqiang Shi, Stanley J. Osher. Deep Neural Nets with Interpolating Function as Output Activation, NeurIPS, 2018\n\n3. Bao Wang, Alex T. Lin, Zuoqiang Shi, Wei Zhu, Penghang Yin, Andrea L. Bertozzi, Stanley J. Osher. Adversarial Defense via Data Dependent Activation Function and Total Variation Minimization, arXiv:1809.08516, 2018\n\n4. B. Wang, S. Osher. Graph Interpolating Activation Improves Both Natural and Robust Accuracies in Data-Efficient Deep Learning, arXiv:1907.06800\n\n=======================\nPlease address the previously mentioned concerns in rebuttal.", "extra_scores": {"experience_assessment": "I have published in this field for several years.", "review_assessment:_checking_correctness_of_experiments": "I carefully checked the experiments.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper thoroughly.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I carefully checked the derivations and theory."}, "extra_text": {}}, {"review_id": "Bkx6D1fx9H", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper investigates the robustness of Neural Ordinary differential equations (ODEs) against corrupted and adversarial examples. The crux of the analysis is based on the separation property of ODE integral curves. The insights from empirical robustness evaluation show that controlling the difference between neighboring integral curves is able to improve neural ODE's robustness. In general, neural ODE is a hot research topic in recent years, and a paper advancing knowledge in this area about understanding its various characteristics is certainly welcome. The paper is well motivated and clearly written. One aspect that confuses me a little originally is the different effects of getting ridding of the dependency on the time t and adding the steady state regularization. It would be nice to elucidate which part makes more contributions? Furthermore, to compare the robustness of the new approach with CNN, the input data consists of original images and their Gaussian-noise based perturbed samples. Since the paper already involves the evaluation using adversarial examples, it will make the paper much more stronger to show that when training both the new approach and the CNN with adversarial training, the proposed regularization can still lead to better robustness. ", "extra_scores": {"experience_assessment": "I have read many papers in this area.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper at least twice and used my best judgement in assessing the paper.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I assessed the sensibility of the derivations and theory."}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper studies the robustness of NeuralODE, as well as propose a new variant. The results suggest that the neuralODE can be used as a building block to build robust deep networks. The reviewers agree that this is a good paper for ICLR, and based on their recommendation I suggest to accept this paper.
Decision: Accept (Spotlight)