Stable Neural ODE with Lyapunov-Stable Equilibrium Points for Defending Against Adversarial Attacks
Authors: QIYU KANG
Yang Song
Qinxu Ding
Wee Peng Tay
Venue: NeurIPS 2021 Poster (2021)
Keywords: ODE
stable
Ordinary Differential Equation
adversarial attack

Abstract: Deep neural networks (DNNs) are well-known to be vulnerable to adversarial attacks, where malicious human-imperceptible perturbations are included in the input to the deep network to fool it into making a wrong classification. Recent studies have demonstrated that neural Ordinary Differential Equations (ODEs) are intrinsically more robust against adversarial attacks compared to vanilla DNNs. In this work, we propose a neural ODE with Lyapunov-stable equilibrium points for defending against adversarial attacks (SODEF). By ensuring that the equilibrium points of the ODE solution used as part of SODEF are Lyapunov-stable, the ODE solution for an input with a small perturbation converges to the same solution as the unperturbed input. We provide theoretical results that give insights into the stability of SODEF as well as the choice of regularizers to ensure its stability. Our analysis suggests that our proposed regularizers force the extracted feature points to be within a neighborhood of the Lyapunov-stable equilibrium points of the SODEF ODE. SODEF is compatible with many defense methods and can be applied to any neural network's final regressor layer to enhance its stability against adversarial attacks. 

Stable Neural ODE with Lyapunov-Stable Equilibrium Points for Defending Against Adversarial Attacks
Qiyu Kang"
Yang Song" School of Electrical and Electronic Engineering
Continental-NTU Corporate Lab Nanyang Technological University 50 Nanyang Avenue, 639798, Singapore kang00808e. ntu. edu.sg
Nanyang Technological University 50 Nanyang Avenue, 639798, Singapore songyentu. seg.y.
Qinxu Ding
Wee Peng Tay School of Electrical and Electronic Engineering Nanyang Technological University 50 Nanyang Avenue, 639798, Singapore wptayEntu. edu. sg
School of Business
Singapore University of Social Sciences 463 Clementi Road, 599494, Singapore ginxudingOsuss. edu.sg
Abstract
Deep neural networks (DNNs) are well-known to be vulnerable to adversarial attacks, where malicious human-imperceptible perturbations are included in the input to the deep network fool it into making wrong classification. Recent studies have demonstrated that neural Ordinary Differential Equations (ODEs) are intrinsically more robust against adversarial attacks compared to vanilla DNNs. In this work, we propose a stable neural ODE with Lyapunov-stable equilibrium points for defending against adversarial attacks (SODEF). By ensuring that the equilibrium points of the ODE solution used as part of SODEF is Lyapunov-stable the ODE solution for an input with small perturbation converges to the same solution as the unperturbed input. We provide theoretical results that give insights into the stability SODEF as well as the choice of regularizers to ensure its stability. Our analysis suggests that our proposed regularizers force the extracted feature points be within a neighborhood of the Lyapunov-stable equilibrium points of the ODE. SODEF is compatible with many defense methods and can be applied to any neural network's final regressor layer to enhance its stability against adversarial attacks.
1 Introduction
Although deep learning has found successful applications in many tasks such as image classification [1, 2], speech recognition [3], and natural language processing [4], the vulnerability of deep learning to adversarial attacks (e.g., [5]) has limited its real-world applications due to performance and safety concerns in critical applications. Inputs corrupted with human human-imperrs imperrcepipilllee perturbations can casily fool many vanilla deep neural networks (DNNs) into mis mis-classifying them and thus significantly impact their performance.
Recent studies [6-8] have applied neural Ordinary Differential Equations (ODEs) [9] to defend against adversarial attacks. Some works like [6] have revealed interesting intrinsic properties of
First two authors contributed equally to this work.
35th Conference on Neural Information Processing Systems (NcurIPS 2021).

---

ODEs that make them more stable than conventional convolutional neural networks (CNNs). The paper [6] proposes a inve-invriinnt steady neural ODE (TisODE) using the property that the integral curves from ODE solution starting from different initial points (inputs) do not intersect and always preserve uniqueness in the solution function space. However, this does not guarantee that small perturbations of the initial point lead to small perturbations of the integral curve output at a later time T. The authors thus proposed a regularizer to limit the evolution of the curves by forcing the integrand to be close to zero. However, neither the non-intersecting property nor the steady-state constraint used in TisODE can guarantee robustness against input perturbations since these constraints do not within neighborhood of Lyapunov-stabl equilibrium points. An example
ensure that the inputs is an ODE that serves as an identity mapping is not robust to imput perturbations but satisfies all the constraints proposed in [6].
In this paper, our objective to design a neural ODE such that the features extracted are within a neighborhood of the Lyapunov-stable equilibrium points of the ODE. We first develop diversity promoting technique applied in the final fully connected (FC) layer to improve the ODE's stability and analyze the reasons why. We then propose stable neural ODE with Lyapunov-stable equilibrium points to eliminate the effects of perturbations in the input. From linear control theory [10], a linear time-invariant system / dt Az()), where A is constant matrix, is exponentially stable if all eigenvalues of A have negative real parts. Specifically, we propose to force the Jacobian matrix of the ODE used in the neural ODE have eigenvalues with negative real parts. Instead of directly imposing constraints on the cigenvalues of the matrix, which lead to high computational complexity when the Jacobian matrix is large, we instead add constraints to the matrix elements implicitly force the real parts of its eigenvalues to be negative.
Our main contributions are summarized as follows:
1. Based on the concept of Lyapumov-stable equilibrium points, we propose a simple yet effective technique to improve the robustness of neural ODE networks by fixing the final FC layer to be a matrix whose rows have unit norm and such that the maximum cosine similarity between any two rows is minimized. Such a FC layer can be constructed off-line.
2.
We propose stable neural ODE for deFending against adversarial attacks (SODEF) to suppress the input perturbations. We derive an optimization formulation for SODEF to force the extracted feature points to be within neighborhood of the Lyapumov-stable equilibrium points of the SODEF ODE. We provide sufficient conditions for learning a robust feature representation under SODEF.
3. We test SODEF on several widely used datasets MNIST [11], CIFAR- 10 and CIFAR-100 [12] under well-known adversarial attacks. We demonstrate that SODEF is robust against adversarial white-box attacks with improvement in classification accuracy of adversarial examples under PGD attack [13] of up to 44.02%, 52.54% and 18.91% percentage points compared to another current sfate-of-the-aar neural ODE network TisODE [6] on MNIST, CIFAR-10 and 100,R-100, respectively. Similar improvements in classification accuracy of adversarial examples of up 43.699 52.38% and 18.99% percentage points compared to ODE net [9] are also obtained.
The rest of this paper organized as follows. We provide essential preliminaries on neural ODE and its stability analysis in Section 2. In Section 3, we present SODEF model architecture and its training method. We show how to maximize the distance between stable equilibrium points of neural ODEs. We propose optimization and present theoretical results on its stability properties. We summarize experimental results Section 4 and conclude the paper in Section 5. The proofs for all lemmas and theorems proposed in this paper are given in the supplementary material. We also refer interested readers to the supplementary material for a more detailed account of related works [14-16, 6] and some popular adversarial attacks [17, 13] that are used verify the robustness of our proposed SODEF. In the paper, we use lowerease boldface characters like z to denote vectors in R", capital boldface characters like A to denote matrices in 8 and normal characters like to denote scalars except that the notation (I. y) are normal characters reserved to denote the input and label pairs. A vector z E R" is represented as (1) z ........ z ())).. The (i.j)-th element of a matrix A is A.y or [A] The Jacobian matrix of function f RR" + R evaluated at z is denoted as V fz(). The set of functions R + H² with continuous first derivatives is denoted as O' (R² R.).

---

2 Preliminaries: Neural ODE and Stability
In neural ODE layer, the relation between the layer input z(0) and output z(T) is described as the following differential equation:
dz(() ()()))),
(1)
dt
where fo: 18 x 00, (0) R" denotes the non- non-linear trainable layers that are parameterized by weights 0 and z [0, 00) + R represents the n-dimensional state of the neural ODE. Neural ODEs are the continuous analog of residual networks where the hidden layers of residual networks can be regarded as discrete-ting difference equations z(t + 1) z(t) + fo (z((t,, t). In this work, for simplicity, we only consider the tiee-nnvriiant (autonomous) case fo(z(t), = fo(z(t)), where the dynamical system does explicitly depend t. For such non-linear dynamical systems, the following theorem shows that under mild conditions, its behaviour can be studied via linearization near special points called hyperbolic equilibrium points.
Theorem 1 (Hartman (Harbman-Grromann Theorem [18]. Consider a system evolving in time with state
dz(() [(z(1)) for some J C*R",, IR"), f(z) (z) (z)) Suppose the map has a hyperbolic equilibrium state R", i.e.
x() E Rhth that satisfies the differential equation dt
f(z') 0 and the Jacobian matrix f = [8)()) / of f evaluated at z has no eigenvalue with real part equal to zero. Then there exists neighbourhood Na of the equilibrium point z² and a homeomorphism g Nz- + R". such that g(z") 0 and in the neighbourhood No. the flow of dlz(() dt f(z(t)) is topologically conjugate by the continuous map z(t) g(z(t)) the
dz(()) V f((z) z(t).
flow of its linearization
dt
The theorem states that when the Jacobian matrix at the zeros of ʃ has no eigenvalue with zero real part, the behaviour of the original dynamical system can be studied using the simpler linearization of the system around those zeros. We next review some definitions and theorems from linear control theory [10].
dz(t) Az(t) with
Definition 1 (Lyapunov Stability [10]). The linear lime-invariant system dt
constant matrix A is marginally stable or stable in the sense of Lyapunov if every finite initial state z(0) excites a bounded response. is asymptotically stable if every finite initial state excites a bounded response, which, in addition, approaches 0 as oc.
Theorem (Lyapunov Stability Theorem [10]). a) The equation dz(() Az(t)) is marginally stable if and only if all eigenvalues of A have zero or negative real parts and those with zero real parts are simple roots of the minimal polynomial of A. b) The equation dz(t)
dl
Az(t) is asymptotically dt
stable if and only if all eigenvalues of A have negative real parts.
In Theorem 1, we say that hyperbolic equilibrium point is Lyapunov-stable all eigenvalues of the Jacobian matrix evaluated at it have negative real parts. From Theorems 1 and 2, we see that small perturbation around the Lyapunov-stable equilibrium point z(0) leads Ztt) + z(0) as t ++ co, i.e., 3 5 > 0 such that for all z()) with [|z(0) 2 l2 < 6, we have (ž(t) z ||2 + 0 as t t- 0o, where (tt) is the ODE solution for the perturbed imput (0). the context of neural network adversarial attacks, if the malicious perturbations around the ODE input z(0) is small, then the output (TT) for large enough will not affected significantly by the perturbation. Consequently, the succeeding network layers after the neural ODE layer can still perform well without being affected by the input perturbation. The perturbation weakening phenomenon around Lyapunov-stable equilibrium points works like a noise filter and acts as defense against adversarial attacks.
We require the following definition and result in our stability analysis. Definition (Strictly diagonally dominant [19]). Let A Caxn, We say that A is strictly diagonally dominant if A I Ejts Aiij for all i 1,...,
Theorem 3 Lleyy-Desplanuee theorem [19]). A E Ca is strictly diagonally dominant and if every main diagonal entry of is real and negative, then A is non-singular and every eigenvalue of has negative real part.
3

---

(z
2
2
feature extractor ha
ODE
fo(z(())
FC
y
dl
Fig. 1: SODEF model architecture.
Given l distinct points 2; E R and matrices A; E R == 1,... k, there exists function f E (R", RR)) such that f(z)) 0 and V [o(z)) A+.
Lemma 1.
SODEF Architecture
We consider classification problem with L classes. The proposed SODEF model architecture is shown in Fig. 1. The input x E X (e.g., an image) is first passed through a feature extractor he X + R" to obtain an embedding feature representation z(0). A neural ODE layer fe follows as nonlinear feature mapping to stabilize the feature representation output z0) from hĄ. The final FC FC layer V serves as linear mapping to generate a prediction vector based on the output z(T) the neural ODE layer. The parameters Φ, 0 and V are parameterized weights for the feature extractor, neural ODE layer and FC layer, respectively.
We provide motivation and design guidance for the FC layer in Section 3.1, which attempts to separate Lyapunov-stable equilibrium points implicitly by maximizing the similarity distance between feature representations corresponding to the L different classes, Experimental results demonstrate the advantages of our diversity promoting FC layer in Section 3.1 with comparisons traditional neural ODEs without diversity promoting.
However, the embedded features after using diversity promoting not guaranteed to locate near the Lyapunov-stable equilibrium points. In Section 3.2, we formulate an optimization problem to force embedding features to locate near the Lyapunov-stable equilibrium points. We introduce optimization constraints to force the Jacobian matrix of the ODE our neural ODE layer to have eigenvalues with negative real parts the syapunov-stable equilibrium points. Instead of directly imposing constraints on the eigenvalue of the matrix, which may be computationally complex especially when the matrix large, we add constraints the matrix elements instead.
3.1 Maximizing the Distance between Lyapunov-Stable Equilibrium Points
From Section 2, we observe that points in small neighbourhood of Lyapunov-stable equilibrium point is robust against adversarial perturbations. We call this neighborhood a stable neighborhood. However Lyapunov-stable equilibrium points for different classes may very well locate near each other and therefore each stable neighborhood may be very small, leading to poor adversarial defense. In this section, propose to add FC layer after the neural ODE layer given by (1) avoid this scenario. The purpose of the FC layer to map the output the neural ODE layer to a feature vector v, if the input x belongs to the class / = 1, ....., L. We design the FC layer that the cosine similarities between different vi's are minimized.
Lemma 2. Given a set of unit vectors Vi... Vk in R", where n 2 k, let alV1..... .K) maxişj v.Vj. Then min alVi....,,...................... VK) 1/(1 k). where the minimun is taken over all possible sets of h unit vectors V1,...,Va Vk-
Corollary 1. Consider a k x k matrix = byll with by and by = 11((1- k), Vi Vi Let the eigen decomposition of B be B = UΣUT. For any 2 k and i 1,.... k. let v; be the i-th column of QΣ where Q is any n x k matrix such that QTQ = I Then, a a(Vi,...,VV) max14) v.v, = 1/(1 k).
Corollary 1 suggests diversity promoting scheme to maximally separate the equilibrium points of the neural ODE layer. The FC layer represented by an x matrix V = [vi,..., vz]. where is the dimension of zTT).. the output from the neural ODE layer. If zT)) is generated from an input from class 1, it is mapped to vi. By minimizing the maximum cosine similarity a(v1,.- V&) = maxiş j v v between the representations from two different classes, we ensure that the output of SODEF robust to perturbations in the input. Corollary provides a way to choose the FC layer weights V.

---

To validate our observations, we conduct experiments to compare the robustness of ODE net [9] and TisODE [6] with and without our proposed FC layer V, on two standard datasets: MNIST [2] and CIFARIO [12] P. On the MNIST dataset, all models consist of four convolutional layers and one fully-connected layer. On the CIFARIO dataset, the networks are similar to those for MNIST except the down-sampling network is stack of ResNet blocks. In practice, the neural ODE can be solved with different numerical solvers such as the Euler method and the Runge- Kutta methods [9]. Here, we use Runge-Kuute of order 5 in our experiments. Our implementation builds on the open-source neural ODE codes.? 3 During training, no Gaussian noise or adversarial examples are augmented into the training set. We test the performance of model in defending against white-box attacks FGSM [17] and PGD [13]. The parameters for different attack methods used in this paper are given in the supplementary material. From Tables and 2, observe that for both datasets, our fixed FC layer improves each network's 's defense ability by a significant margin. We visualize the features before the final FC layer using t-SNE [20] in Figs. 2 and 3. We observe that with the FC layer, the features for different classes are well separated even under attacks.
Table 1: Classification accuracy (%) on adversarial MNIST examples, where the superscript indicates the last FC layer is fixed to be V.
Attack
Para.
ODE
ODE*
TisODE
TisODE
None
99.6
99.7
99.5
99.7
FGSM
0.3
31.4
52.8
45.9
63.5
PGD
0.3
0.29
0.30
0.4
20.20
Table 2: Classification accuracy (%) adversarial CIFARIO examples, where the superscript indicates the last FC layer is fixed to be V.
Attack
Para.
ODE
ODE+
TisODE
TisODE+
None
87.0
85.0
87.4
81.8
FGSM
c= 0.1
12.9
47.6
13.1
41.9
PGD
0.1
7.8
14.7
7.4
16.2
Fig. t-SNE visualization results on the features before the final FC layer. The input is the test set of MNIST. Left: trained with TisODE, middle: TisODE using randomly chosen orthogonal matrix as the final FC, right: TisODE using proposed as the final FC.
3.2 Objective Formulation and Stability
In this subsection, we formulate optimization framework for SODEF to force output features to locate within the stable neighborhood of Lyapunov-stabie equilibrium points. We make the following assumption.
Assumption 1. The imput x takes values in a compact metric space X and has probability distribution μ.. The feature extractor h& is injective and continuous.
Our experiments are run on a GeForce RTX 2080 Ti GPU.
Phttps://e btteps://githubb com//rtgicchen/oorrcddffeeee
5

---

Fig. 3: t-SNE visualization results on the features before the final FC layer. The input is the adversarial examples of the test set of MNIST generated using FGSM method at 0.3. Left: trained with TisODE, middle: TisODE using randomly chosen orthogonal matrix the final FC, right: TisODE using proposed V as the final FC.
The above assumption is satisfied if the input (e.g., an image) resides in a bounded and closed set of Euclidean space. We denote the pushforward measure (still probability distribution) of under the continuous feature extractor mapping h& as V - h where denotes function composition. The conditional probability distribution for the embedding of each class / {l...., }} has compact support Es " since Et is closed and h is bounded in R" In Section 3.1, the FC layer V tries to maximize the distance between E,, I = 1,.... L. In this section for analysis purposes, we also assume the following.
Assumption 2. We have E E if +fllll
+1, I', i.e. the supports of each class are pairwise disjoint.
Our objective function is formulated follows, which is explained in detail in the sequel:
min ((L((V((z(T)) vi) 0.6
s.t. t.. Eng. ll [z(()))) <, fo E (R RR), [fo(((0))) < 0, vi I,...,m,
(3)
Er
(4)
E
[[]]o(()))]]l [[VSO(Z0))s 0, vi 1,...:n,
(5)
z0) = hz(x,, and (T)) is the output of (1) with input z0)).
(6)
Here, is a loss function and > 0 is a positive constant. The constraints (3) to (5) force z(0) to be near the Lyapunov-stable equilibrium points with strictly diagonally dominant derivatives. We limit the So to be in C² (IR",, "IR) to satisfy the condition in Theorem 1. From [21], we also know that standard multi-layer feed forward networks with few as single hidden layer and arbitrary bounded and non-constant activation function are universal approximators for C* IR") functions with respect some performance criteria provided only that sufficiently many hidden units are available.
As a comparison, TisODE [6] only includes a constraint similar to (3), which in general provides no guarantee force z (0) near the Lyapunov-stable equilibrium points. In the extreme case with parameters 0 = 0 for fe such that fe = 0, the ODE degenerates to an identity mapping. No z(0) E R can now be Lyapunov -taale equilibrium point, and no stability can therefore be guaranteed to defend against adversarial attacks even though the ODE curves still possess the non- intersecting property and steady-state constraint, which were cited as reasons for the stability of TisODE.
Instead of directly optimizing the above objective function, in our implementation, optimize the following empirical Lagrangian with a training set { Wk) k = L.....N:: N):
(T) ((VTzz (T), vk) 11l1fo agg1 a2g1 I V (o(zz(
min 0.6
N
K=
+ 0330
(0))aal) +
Σ
(7)
S. t. z (0) = has (xx), and (T) is the output of (1) with input 2n (0), Vk = =1....... -....,N N (8)

---

where 01, a2 and Og are hyperparameter weights. 91 and 92 are chosen monotonically increasing functions bounded below to eliminate the unbounded impact of the two regularizers that can otherwise dominate the loss. In this paper, we set 91 (+) (2 exp(-). We call these two latter terms the SODEF regularizers.
2 (0) is finite. For each i -,..... k, let A, E RR=xa be strictly diagonally dominant matrix with every main diagonal entry be negative such that the eigenvalues for A; all have negative real part. From Theorem 3, each A; is non-singular and every eigenvalue of A, has negative real part. Therefore, from Theorem 2 and Lemma 1, there exists a function fo such that all z are Lyapunov-stable equilibrium points with corresponding first derivative V. = A.. This shows that if there exist only finite representation
Suppose for each class = L,..., L, the embedding feature set E = {z]
points for each class, we can find a function So such that all inputs to the neural ODE layer are Lyapunov-stable equílibrium points for ʃe and
(a) Exg (z()))l2 = 0, (b) Exg [[o((())] < <0, I...,,.., n
(c) Eyg [ - [1662(0)))))) 0, -i= -1,...... n.
If the input space X has infinite cardinality, then an injective and continuous feature extractor his results in us with non-finite support, i.e., at least one E,, = I,... I, is infinite. It is not obvious whether we can obtain a fe where every point in E = U, Ei is a stable equilibrium point. The following result gives negative answer if vs is continuous measure (i.e., absolutely continuous with respect to (W..t.) Lebesgue measure) on some subset.
Lemma 3. If the restriction of Ve some open E' c. E is continuous measure, there is no continuous function fo such that for V& almosmost surely all z E E, fo(z) = 0 and all the eigenvalues of V fo(2)) have negative real parts. In other words, there is continuous function fo such that almost surely all z in E are Lyapunov-stable equilibrium points.
Lemma 3 indicates that it is too much to hope for all points in E to be I. Lyapunov-stable equilibrium points. In the following, we relax this requirement and show that under mild conditions, for all 0, we can find continuous function fo with finitely many stable equilibrium points such that conditions (b) and (c) above hold and condition (a) is replaced by E fo(((()))) < e. This motivates the optimization constraints in (3) to (5).
Theorem 4. Suppose Assumptions 1 and 2. If v. is not a continuous uniform measure on Eg for each | = L, then the following holds: 1) The function space satisfying the constraints in (3) to (5) is non-empty for all e > 0. 2) If additionally the restriction of V to any open set O C E₁ is not continuous uniform measure, there exist functions in this space such that each support E, contains at least one Lyapunov-stade stable equilibrium point.
Experiments
In this section, we evaluate the robustness of SODEF under adversarial attacks with different attack parameters. We conduct experiments to compare the robustness of ODE net [9] and TisODE net [6] on three standard datasets: MNIST [2], CIFAR10 and CIFAR100 [3]. Since SODEF is compatible with many defense methods, it can be applied to any neural network's 's final regressor layer to enhance its stability against adversarial attacks. Our experiment codes are provided in https:// /ttp:::/gitoe github....... com Sam/KANNQIVU//SDEE
4.1
Setup
We use open-source pre-trained models that achieve the top accuracy on each dataset as the feature extractor hg. Specifically for simple MNIST task, we use the ResNet18 model provided in Pytorch. We use the model provided by [22], which obtains nearly 88% clean accuracy on CIFAR100 using EfficientNet [23] and the model provided by [24], which has nearly 95% clean accuracy on CIFARIO. 10. In the neural ODE layer, fo consists of 2 FC layers. During the trainings of SODEF (except in the experiment included in Section 4.2), we train the neural network with the fixed FC introduced in Sec- tion 3.1. In the first 30 epochs, we fixed fo to let the feature extractor h& learn feature representation
7

---

with only the cross-entropy loss L, and in the remaining 120 epochs, we release h& to further train
fo using (7) with a and 02 = ag = 0.05. For CIFARIO and CIFARI00, the pixel values are normalized by (Ir /o/ar where μr = [0.4914, 4, 0.4822, 0.4465] and o [0. 2023, 0. 0.1994, 0.2010] 4. To show that our SODEF compatible with many defense methods and can be applied to any neural network's s final regression layer, we conduct an experiment where we use recently proposed robust network TRADES [25] as the feature extractor in our SODEF. The pretrained model is provided here 5, and we choose the model with architecture "WRN-34-10 to conduct our experiments. Besides the two vanilla white-box attacks FGSM and PGD as metioned in Section 3.1, we also include a strong ensemble attack AutoAttack [26], which sequentially performs attack using all of the following four individual attacks: three white- box attacks APGDCE, APGD and FAB [27], and one black- box Square attack [28]. We refer the reader to the the supplementary material for more details of the attacks used in this paper, where, in additional, more experiments are included.
4.2 Compatibility of SODEF
Adversarial training (AT) is one of the most effective strategies for defending adversarial attacks. TRADES [25] is one of the adversarial training defense methods with combinations of tricks of warmup, early stopping, weight decay, batch size and other hyper parameter settings. In this experiment we fix the pretained TRADES model (except the final FC layer (size 640x10)) as our feature extractor h2s We then append our (trainable) SODEF with integration time T = 5 to the output of the feature extractor. To evaluate model robustness, we use AutoAttack and attack the models using both the L2 norm (c 0.5) and Lo norm (c 8 8/255). The results shown in Table 3. We clearly observe that our SODEF can enhance TRADES's ' robustness under all the four individual attacks and the strongest ensemble AutoAttack. For the strong L2 Auto/ttack, have improved the model robustness from 59.42% to 67.75%. Our experiment show that SODEF can be applied to many defense models' regression layer enhance their stability against attacks.
SODEF
Table 3: Classification accuracy (%) using TRADES (w/ and w/o SODEF) under AutoAttack on adversarial CIFAR 10 examples with La norm (e = 0.5) and Lox. norm (e = 8/255).
Attac/ ModeITRADES c. TRADES Coo TRADES L SODDDE Ca
Clean 85.48
85.18
85,48
85.18
APGDrE
56.08
70.90
61.74
74.35
APGDERR
53.70
64.15
59.22
68.55
FAB 54.18
82.92
60.31
83.15
Square
59.12
62.21
72.65
76.02
AutoAttack
53.69
57.76
59.42
67.75
4.3 Inffuence of Integration Time T
From the discussion after Theorems and 2, we know if the malicious perturbations around the ODE input Lyapumov
equilibrium point z(0) is small, then the output ((T)) for large enough T will not be affected significantly by the perturbation: (((()) z00]]2 - 0 as / -> xc. Consequently, the succeeding network layers after the neural ODE layer can still perform well without being affected by the input perturbation. In this section, we test the influence of the SODEF integration time T using CIFARI00. We use the model EfficientNet provided by [23] as h& (Note, unlike Section 4.2, hss is trainable in this experiments). We use AutoAttack with L2 norm (e = 0.5). We observe that for all the four individual attacks and the strongest ensemble AutoAttack, SODEF performs generally better for large integration time T. We also test larger integration time T > 10, but do not see any obvious improvements.
4.4 Performance Comparison Under AutoAttack
For a comparison, we provide the results of applying AutoAttack to other baseline models mentioned in the paper. We set the same integration time for ODE, TisODE and SODEF. We observe that for the
"To test AutoAttack, we have strictly followed the instruction https: //pp::///tttbbb./p http://ggttuuub../oooooeeee htt.:/cootooeteeeececc..o robustbench to attack the original images before any normalization or resizing. faaps//giitubbcom/22333/// wof-Triccs-oor.
8

---

Table 4: Classification accuracy (%) under AutoAttack on adversarial CIFARI00 examples with C
norm, 0.5 and different integration time 7 for SODEF.
Attack /
3
6
9
10
Clean
88.00
88.12
88.15
88.00
87.92
88.00
88,05
88.10
APGDeE
17.20
21.33
21.05
23.67
69.67
85.33
87.10
86.88
APGDELR
21.02
21.00
22.00
26.00
63.30
86.90
86.20
86.54
FAB
86.33
85.10
86.36
87.70
87.67
86.55
86.22
85.93
Square
84.67
86.22
87.05
87.20
86.90
86.33
87.05
86.75
AutoAttack
2.00
3.53
4.87
4.33
30.66
78.80
78.97
79.10
strongest Auto/ttack, our SODEF outperforms the other baseline models by significant margin. In this case, SODEF achieves 79.10% accuracy while other models only get less than 3% accuracy.
Table Classification accuracy (%) under AutoAttack on adversarial CIFARI00 examples with L2
norm, 0.5 and T 10.
Attack / Model
No ODE
ODE
TisODE
SODEF
Clean
88.00
87.90
88.00
88.10
APGDęE
23.30
6.75
14.32
86,88
APGDOLR
7.33
22.00
24.20
86.54
FAB
79.30
78.67
77.16
85.93
Square
84.52
85.67
86.32
86.75
AutoAttack
0.00
1.33
4.06
79.10
4.5 Performance Under PGD and FGSM Attacks
White-box adversaries have knowledge of the classifier models, including training data, model architectures and parameters. We test the performance of our model in defending against the white- box attacks, PGD and FGSM. We set T parameters for different attack methods used are given the supplementary material. The subsequent experiments use these settings by default, unless otherwise stated.
the integration time for the neural ODE layer. The
Table 6: Classification accuracy (%) on adversarial MNIST examples.
Attack
Para.
no ode
ODE
TisODE
SODEF
None
99.45
99.42
99.43
99.44
FGSM c 0.3
10.03
29.6
36.70
63.36
PGD = 0.3
0.31
1.56
1.82
45.25
The classification results on MNIST are shown in Table We observe that while maintaining the -fate-of-the-- art accuracy normal images, SODEF improves the adversarial robustness as compared the other two methods. For the most effective attack in this experiment, i.e., PGD attack, SODEF shows 45. 25% 1.56% 43.69% improvement over ODE and 45.25% 1.23% 44.02% improvement over TisODE.
Table 7: Classification accuracy (%) adversarial CIFAR10 examples.
Attack
Para.
no ode
ODE
TisODE
SODEF
None
95.2
94.9
95.1
95.0
FGSM
c 0.1
47.31
45.23
43.28
68.05
PGD
0.1
3.09
3.21
3.80
55.59
For CIFAR-10, we see from Table 7 that SODEF maintains high accuracy on normal examples and makes the best predictions under adversarial attacks. In particular, SODEF achieves absolute percentage point improvement over ODE net up to 52.38% and over TisODE up to 52.54% for PGD attack.
For CIFAR- the results in the supplementary material shows that the most effective attack causes the classification accuracy drop relatively by 74.6% 88.0-22.35 for SODEF and by

---

97.3% = -8.3-2.39 for vanilla EfficientNet, which is pre-trained on ImageNet to obtain a top clean 88.3
accuracy. Neither ODE net nor TisODE net can improve the classification accuracy under PGD attack by big margin, e.g. TisODE net only improves the classification accuracy from 2.39% to 3.44%, while SODEF still shows clear defense capability in this scenario.
4.6
Ablation Studies
The impact the ODE with and without the SODEF regularizers in (7) has been presented in the above comparisons between SODEF and ODE, In this section, we show the necessity of diversity promoting using the FC introduced in Section 3.1 and conduct transferability study.
4.6.1 Impact of Diversity Promotion
Table 8: Classification accuracy (%) on adversarial MNIST examples, where the superscript indicates the last FC layer is not fixed to be V and is set to be a trainable layer.
Attack
Para.
SODEF
SODEF
None
95.0
95.1
FGSM
c= 0.1
63.36
51.6
PGD
c 0.1
45.25
34.9
Table shows the difference of the defense performance when fixing the final FC be V or setting it to trainable linear layer. It can be seen that having diversity control improves the robustness. One possible reason for this phenomenon given Scction 3 that diversity promotion with fixed designed FC attempts to make the embedding feature support ES of each class disjoint each other and therefore the Lyapunov-stable equilibrium points for each E₁ are well separated.
4.6.2 Transferability Study
Transferability study is carried out on CIFAR-10, where the adversarial examples are generated using FGSM and PGD attacks using ResNet18 without any ODEs. The classification accuracy drops from 68.05% to 59% for FGSM with = 0.3, and from 55.59% to 34% for PGD with 0.1. One possible reason for this phenomenon is that ODEs have obfuscated gradient masking effect as discussed in [15], and a transfer attack may deteriorate the defense effect. However, we observe from Table even with transfer attack on SODEF, it still performs better than other ODEs without transfer attacks.
5 Conclusion
In this paper, have developed a new neural ODE network, SODEF, to suppress input perturbations. SODEF compatible with any existing neural networks and can thus appended to the state-of-the- art networks to increase their robustness adversarial attacks. We demonstrated empirically and theoretically that the robustness of SODEF mainly derives from its stability and proposed a training method that imposes constraints to ensure all eigenvalues of the Jacobian matrix of the neural ODE layer have negative real parts. When each classification class converges to its own equilibrium points, showed that the last FC layer can be designed in such way that the distance between the stable equilibrium points maximized, which further improves the network's 's robustness. The effectiveness of SODEF has been verified under several popular while-box attacks.
Acknowledgments and Disclosure of Funding
This research supported part by A$STAR under RIE2020 Advanced Manufacturing and Engineering (AME) Industry Alignment Fund - Pre Positioning (IAF-PP) (Grant No. A19D6a0053) and the RIE2020 Industry Alignment Fund - Industry Collaboration Projects (IAF-ICP) Funding Initiative, as well as cash and innkind contribution from the industry partner(s). The computational
10

---

work for this article was partially performed on resources of the National Supercomputing Centre, Singapore (htts:/wwww.mscc.s))
Broader Impact
Our work, which contributes more robust DNNs, supposed mitigate the threat of adversarial attacks. However, on the hand, the reliable deployment of DNNs in automation of tasks will potentially bring mass-scale unemployment and social unrest. As DNNs become more robust and more tasks, especially those whose failures will bring high risks to human lives or large property losses under adversarial attacks, fall into the automatic task category, massive jobs could disappear.
References
[1] A. Krizhevsky, I. Sutskever, and G. E. Hinton, "ImageNet classification with deep convolutional neural networks," in Proc. Advances Neural Inf. Process. Syst., 2012.
[2] Y. Lecun, L. Bottou, Y. Bengio, and Haffner, "Gradient-bare based learning applied document recognition," Proceedings of the IEEE, vol. 86, no. 11, pp. 2278-2324, Nov. 1998.
[3] G. Hinton, L. Deng, D. Yu, G. E. Dahl, Mohamed, N. Jaitly, Senior, V. Vanhoucke, P. Nguyen, T. N. Sainath, and B. Kingsbury. "Deep neural networks for acoustic modeling in speech recognition: The shared views of four research groups." IEEE Signal Process. Mag., vol. 29, no. 6, pp. 82-97, Nov. 2012.
[4] D. Andor, Alberti, D. Weiss, Severyn, Presta, K. Ganchev, Petrov, and M. Collins, "Globally normalized transition-basee neural networks," in Proc. Annu. Meeting Assoc, Comput. Linguistics, 2016.
[5] C. Szegedy, W. Zaremba, I. Sutskever, J. Bruna, D. Erhan, I. Goodfellow, and R. Fergus, "Intriguing properties of neural networks," in Proc. Int. Conf. Learning Representations, 2013.
[6] H. Yan, J. Du, Y. Tan, and J. Feng, "On robustness neural ordinary differential equations," in Proc, Advances Neural Inf. Process. Syst., 2018, pp. 1-13.
[7] E. Haber and L. Ruthotto, "Stable architectures for deep neural networks," Inverse Problems, vol. 34, no. 1, pp. 1-23, Dec. 2017.
[8] X. Liu, S. Si, Q. Cao, S. Kumar, and C.-J. Hsich, "How does noise help robustness? Explanation and exploration under the neural sde framework," in Proc. Conf. Comput. Vision Pattern Recognition, 2020, pp. 282-290.
[9] R. T. Chen, Y. Rubanova, J. Bettencourt, and D. Duvenaud, "Neural ordinary differential equations," arXiv preprint arXiv:1806.005 2018
[10] C.-T. Chen and B. Shafai, Linear system theory and design. New York: Oxford university press New York, 1999.
[11] Y. LeCun, C. Corte, and C. Burges, "MNIST handwritten digit database," " ATT Labs [Online]. Available: http://yann. http://yann.lecun.cco
vol. 2, 2010, (Last accessed: Dec 2020).
[12] A. Krizhevsky and G. Hinton, "Learning multiple layers of features from tiny images," Master's 's thesis, Department of Computer Science, University of Toronto, 2009.
[13] A. Madry, A. Makelov, L. Schmidt, D. Tsipras, and A. Vladu, "Towards deep learning models resistant to adversarial attacks," in Proc. Int. Conf. Learning Representations, 2018.
[14] E. Haber and L. Ruthotto, "Stable architectures for deep neural networks." " Inverse Problems, vol. 34, no. 1, 2018
[15] Y. Huang, Y. Yu, H. Zhang, Ma, and Y. Yao, "Adversarial robustness of stabilized neuralodes might be from obfuscated gradients," arXiv preprint 200.:::0009911111100 2020.
11

---

[16] L. Mingjie, H. Lingshen, and L. Zhouchen, "Implicit Euler skip connections: Enhancing adversarial robustmess via numerical stability," in Proc. Int. Conf. Machine Learning, 2020.
[17] I. J. Goodfellow, Shlens, and C. Szegedy, "Explaining and harnessing adversarial examples," in Proc. Int. Conf. Learning Representations, 2015.
[18] D. Arrowsmith and C. M. Place, Dynamical systems: differential equations, maps, and chaotic behaviour. London: CRC Press, 1992.
[19] R. A. Horn and C. R. Johnson, Matrix analysis. New York: Cambridge university press, 2012.
[20] L. Van der Maaten and G. Hinton, "Visualizing data using t-sne." J. Mach. Learning Res., vol. 9, no. 11, pp. 2580-2605, Nov. 2008
[21] K. Hornik, "Approximation capabilities of multilayer feedforward networks," Neural Networks, vol. 4, no. 2. pp. 251-257, Oct. 1991.
[22] Y. Luo, Y. Wong. S. M. Kankanhalli. and Q. Zhao, "Direction concentration learning: Enhancing congruency in machine learning," IEEE Trans. Puttern Anal. Mach. Intell., vol. 43, no. 1, pp. 1928 1946, Jun. 2021.
[23] M. Tan and Q. Le, "Efficientnet: Rethinking model scaling for convolutional neural networks," in Proc. Int. Conf. Mach. Learning, 2019, pp. 6105-6114.
[24] Github pytorch-cifar repository. Accessed: May 1, 2021. [Online]. Available: https:
//gidhu.ccom/uungiiu//yoooo cifar
[25] T. Pang, X. Yang, Y. Dong, H. Su, and J. Zhu, "Bag of tricks for adversarial training," in Proc. Ini. Conf. Learning Representations, 2021.
[26] Croce and M. Hein, "Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks," in Proc. Int. Conf. Mach. Learning, 2020, pp. 2206-220 2216.
[27] "Minimally distorted adversarial examples with fast adaptive boundary attack," in Proc. Int. Conf. Mach. Learning, 2020, pp. 2196-2205.
[28] M. Andriushchenko, F. Croce, N. Flammarion, and M. Hein, "Square attack: a query-efficient black-box adversarial attack via random search," in Proc. European Conf. Comput. Vision. Springer, 2020, pp. 484-501.
Checklist
1. For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper's
contributions and scope? [Yes]
(b) Did you describe the limitations of your work? [Yes]
(c) Did you discuss any potential negative societal impacts of your work? [Yes]
(d) Have you read the ethics review guidelines and ensured that your paper conforms to them? [Yes]
2. If you including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results? [Yes]. See Assump- tion and Assumption
(b) Did you include complete proofs of all theoretical results? [Yes] See supplementary material.
3. If you experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main experi- mental results (either in the supplemental material or URL)? [Yes].
(b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)? [Yes]. See Sections 3.1 and
12

---

(c) Did you report enror bars (e.g., with respect to the random seed after running exper- iments multiple times)? [No] Having ODE blocks in our model, it would be too computationally expensive to repeat experiments for many times. We repeated each experiment for 2-3 times and we observe the deviation of the classification results is within +3%, though these experimental repetitions are not enough to construct error bars.
(d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, cloud provider)? [Yes] See Section 3.1.
4. If you using existing assets (e.g., code, data, models) or curating/releasing new assets...
(a) If your work uses existing assets, did you cite the creators? [Yes] See Sections 3.1 and 4 for the open-source models we have used from GitHub.
(b) Did you mention the license of the assets? [No]. Please see the licenses given in the GitHub link.
(c) Did you include any assets either in the supplemental material URL? [No] No new assets.
(d) Did you discuss whether and how consent was obtained from people whose data you' 're using/curating? [Yes] MNIST, CIFAR-I0 and CIFAR-100 are all open-source datasets. (c) Did you discuss whether the data you are using/curating contains personally identifiable information offensive content? [No] There no identifiable information or offensive content in the datasets.
5. If you used crowdsourcing or conducted research with human subjects...
(a) Did you include the full text of instructions given participants and screenshots, if applicable? [N/A]
(b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, applicable? [N/A]
(c) Did you include the estimated hourly wage paid to participants and the total amount spent participant compensation? [N/A]
13


Reviews:
[{"review_id": "ju_90Sflfnw", "reviewer": "Reviewer_Bitu", "summary": "This work aims to develop a neural ODE-based model to defend against adversarial attacks. Specifically, the authors propose a stable neural ODE with Lyapunov-stable equilibrium points (SODEF). By ensuring the equilibrium points are Lyapunov-stable, the ODE solution for an input with a small perturbation will converge to the same solution for the original unperturbed input. The paper provides theoretical insights into the stability of SODEF and how to choose the stability regularizers. Besides, the paper also provides design guidance for the final FC layer, that aims to separate Lyapunov-stable equilibrium points. The proposed SODEF can work in conjunction with several defense methods and can be applied to any network's final regressor layer.", "questions": "", "limitations": "", "rating": 7, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "This paper investigates the adversarial robustness from a novel perspective --- stable dynamical system. It is very interesting to study the connection between robust deep networks and the stability of dynamical systems\n\nThis paper aims to learn a stable dynamical system-based feature mapping which can mitigate the effect of perturbation. Stable points in a dynamical systems are fixed points with negative-definite Jacobians. Instead of directly compute the eigenvalues of Jacobians, this paper adds constraints to the Jacobian matrices by encouraging the Jacobian to be strictly diagonally dominant. The levy-Desplanques theorem guarantees the stability of systems of which the Jacobians are strictly diagonally dominant and with real negative diagonal entries.\n\nConcern:\n1. The formulated optimization problem does not impose regularization on the feature extractor $h_{\\phi}$. The optimization-(7) only encourage that the feature extracted by $h_{\\phi}$ is located within the neighborhood (say neighborhood with radius $r$) of stable points.  In this case, a perturbation within the $r$-neighborhood on the extracted feature will gradually vanishes.  However, since no regularization on $h_{\\phi}$, the imperceptible perturbation on the input $x$ may be amplified --- the perturbed feature is highly likely outside the $r$-neighborhood. Then, the perturbed feature won't converge to the original and the robustness is not guarnteed. So, I think it is necessary to discuss this problem.\n\n2. In section 4.3.2, the authors evaluated the robustness of SODEF with transfer-based attacks. I think it is necessary to evaluate the proposed methods via other more black-box attacks, such as the auto-attack (Croce et al. 2020). \n\n3. The authors provided the implementation codes in the supplementary materials. I skim the codes and find there may be mistakes of the implementation of TisODE. In the file \"Tisodenet_cifar10_resnet.py\", see lines 461-461, the second term in the loss is  abs (state_t1 - state_t2), which equals to the abs of integral of dynamics from t1 to t2. However, in TisODE (Eq-5), the L_ss is the integral of the abs dynamics from t1 to t2.\n\n4. Some typos. For example, Neural ODE and TisODE are the main baselines in this paper.  In the Reference section, the citation of TisODE is not correct, TisODE is an ICLR 2020 paper, not NeurIPS-2018. \n\n-----------------------------\nReliable Evaluation of Adversarial Robustness with an Ensemble of Diverse Parameter-free Attacks. Francesco Croce,\nMatthias Hein. ICML 2020", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Hope authors can respond to the questions above.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "6"}, "extra_text": {}}, {"review_id": "TcZ4jAa1bK2", "reviewer": "Reviewer_8u2W", "summary": "This paper tries to improve the robustness of neural ODE via enforcing the Lyapunov stability constraints in the training process. The key idea is as follows: a dynamic system can have multiple equilibrium points, and the input perturbation around the equilibrium point will decay and not significantly influence the result if the equilibrium point is Lyapunov stable. This is an excellent contribution that connects the Lyapunov stability of ODEs with the robustness of neural networks.  ", "questions": "", "limitations": "", "rating": 8, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "This is a very well written paper. The authors proposed a new neural network to enforce the robustness of neural networks. The network consists of three blocks: (1) a feature extractor block, (2) a neural ODE block that satisfies the Lyapunov stability condition, and (3) a diversity promoting FC layer. By enforcing the Lyapunov condition of the neural ODE, the input perturbation is guaranteed to decay and will not significantly influence the prediction. \n\nOn the implementation side, the Lyapunov condition is achieved by enforcing the Jacobian matrix A to be dominantly diagonal and have eigen values with negative real parts. These constraints appear in the regularization term of the training loss function. \n\nThe research results are well justified by theoretical analysis. The proposed Lyapunov stable model has shown significant robustness improvement compared with neural ODE trained without Lyapunov stability constraints. \n\nSome minor comments:\n1. Some important references of neural ODE are missing. The idea of neural ODEs were firstly proposed in the applied math community, and the following papers should be cited:\n--W. E,  A proposal on machine learning via dynamical systems. Communications in Mathematics and Statistics, 5(1):1–11, 2017.\n--Eldad Haber and Lars Ruthotto.  Stable architectures for deep neural networks. Inverse Problems,34(1):014004, 2017.\n\n2, The following papers improve the robustness of neural ODEs via open-loop and close-loop controls. They are ignored in this paper:\n--Chen Z, Li Q, Zhang Z. Towards Robust Neural Networks via Close-loop Control. In International Conference on Learning Representations 2020 Sep 28\n--Dinghuai Zhang, Tianyuan Zhang, Yiping Lu, Zhanxing Zhu, and Bin Dong.  You only propagate once: Accelerating adversarial training via maximal principle. In Advances in Neural Information Processing Systems, pp. 227–238, 2019.\n\n3. The authors assume that the ODE is time-invariant. I wonder how they handle the model parameters (e.g., weight matrices and convolution filters) at different layers: are they the same across different layers?\n\n4. More details should be provided to explain why we need the feature extraction block and how this block should be designed for general cases. \n\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "It would be great if the following points can be discussed/clarified:\n1. The linearization analysis only works if the perturbation is small. How strong is this assumption?\n2. How many equilibrium points should the system have in the training process? How can we control the number of equilibrium points? What's the relationship of the number of equilibrium points with the number of output labels?\n\nA potential limitation: this work only compared with standard neural ODE, and has not been compared with other robust neural ODE, for instance:\n--Chen Z, Li Q, Zhang Z. Towards Robust Neural Networks via Close-loop Control. In International Conference on Learning Representations 2020 Sep 28\n--Dinghuai Zhang, Tianyuan Zhang, Yiping Lu, Zhanxing Zhu, and Bin Dong.  You only propagate once: Accelerating adversarial training via maximal principle. In Advances in Neural Information Processing Systems, pp. 227–238, 2019.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "3 hours"}, "extra_text": {}}, {"review_id": "7nwFSo98Ch3", "reviewer": "Reviewer_dTd1", "summary": "The paper proposes a defense against adversarial attacks using two orthogonal ideas: (i) the design of the final fully connected layer that maximizes the cosine distance between vectors corresponding to different labels, and (ii) design of a neural ODE whose Jacobian has only negative real parts and is hence Lyapunov stable near its hyperbolic equilibrium points (via Hartman-Grobman and linearization).", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "(+) While Hartman-Grobman theorem has been used in the stability analysis and generalization/robustness of neural networks via linearization around hyperbolic equilibrium points (see Section 4 of [1] and Section 5.2 of [2]), its use in training adversarially robust neural networks is new. \n\n(+) The conclusion highlights that the approach is compatible with any existing neural network. It is clear that residual neural nets and their variants can benefit from this approach. It may be useful to comment on how this approach can be naturally extended to transformers without residual layers.\n\n(+) The source code has been made available to the reviewers. Can the authors confirm that the complete pipeline in Figure 1 from x to y is being attacked during experimental studies?\n\n[1] Kim P, Pan L, Wirjanto TS. Local Stability Analysis of Neural Network Models with Application to Exchange-Rate Data. Mimeo, University of Waterloo, Waterloo, Ontario, Canada; 1999. https://www.researchgate.net/profile/Tony-Wirjanto/publication/228936073_Local_Stability_Analysis_of_Neural_Network_Models_With_Application_to_Exchange-Rate_Data/links/00b7d5295fb1709e0a000000/Local-Stability-Analysis-of-Neural-Network-Models-With-Application-to-Exchange-Rate-Data.pdf\n\n[2] Poggio T, Liao Q, Miranda B, Banburski A, Boix X, Hidary J. Theory IIIb: Generalization in deep networks. arXiv preprint arXiv:1806.11379. 2018 Jun 29.  https://arxiv.org/pdf/1806.11379.pdf\n\n\nAfter reading the author's rebuttal, the reviewer's main concern has been met and the score of the paper has been raised.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "(-) The experimental evaluation of the proposed approach, including results in the supplementary material, is limited. A good reference for completing a thorough evaluation is [3]. At the very minimum, PGD attacks with varying L2 and Linf norms should be reported. Modern attacks such as FABS and Square Attack should be evaluated. Can we obtain a report on the results of applying AutoAttack including PGD, FABS and SquareAttacks on these models? \n\n(-) The quality of the reported experimental results is low. For example, Table 1 in Sec. 2 of the Supplementary Material shows that the PGD attack uses a maximum perturbation size of epsilon=0.1 and a step size of 0.1 with 20 iteration steps (not size). The step size should be much smaller such as 2.5*0.1/20 following Madry [4]. AutoAttack https://github.com/fra31/auto-attack provides a parameter-free approach of performing such an evaluation.\n\n(-) The Hartman-Grobman linearization holds only near hyperbolic equilibrium points. Why are these points the only points of interest for robustness of neural networks? Why is the dynamics in the rest of the state space not of interest for robustness analysis?\n\n(-) Figure 3 states that the input is the adversarial examples of the test set of MNIST.  Are these plots obtained by looking at the adversarial examples obtained by attacking each of the three models: TisODE, TisODE and TisODE + V as FC?\n\n\n\nMinor points:\n\n1. Reference 8 focuses on stochastic differential equations and not ordinary differential equations. So, paragraph 2 on page 1 may need to be suitably edited. \n\n2. Table 1 has no results for applying PGD to MNIST while Table 2 applies both FGSM and PGD to CIFAR10. \n\n3. Table 3 does not specify the norm used for the PGD/FGSM attack. \n\n4. The paper or the Supplementary Material does not include investigations that sweep the magnitude of the perturbation (epsilon) and observe the impact on the robustness of the model. \n\nReferences:\n\n[3] Carlini N, Wagner D. Towards evaluating the robustness of neural networks. In 2017 ieee symposium on security and privacy (sp) 2017 May 22 (pp. 39-57). IEEE. https://arxiv.org/abs/1608.04644\n\n[4] https://github.com/MadryLab/robustness\n\n", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "4"}, "extra_text": {}}, {"review_id": "3ZhS-5-DyLO", "reviewer": "Reviewer_eqFz", "summary": "This paper proposes a neural network classifier architecture based on neural ODEs that is designed to be robust to adversarial attacks. In the proposed method, the classes are represented by vectors, chosen to be maximally separated in the output space w.r.t. cosine similarity. The neural ODE system is then encouraged to have equilibrium points at those class vectors via regularization; the idea is that perturbed outputs are pulled towards the equilibria as the ODEs evolve.", "questions": "", "limitations": "", "rating": 7, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "This paper builds on a line of work that connects neural ODEs with adversarial robustness. The approach was interesting - a diagonally dominant condition was used as a scalable approach to enforce stability conditions on the dynamics Jacobian. The various experiments visually and quantitatively show the potential benefits of locally attractive diverse class label targets. In my understanding, the main strength is that the SODEF layer mainly serves to stabilize the output of a pretrained classifier, and does not really require much predictive power of its own - this may make it easier to tune the several regularization hyperparameters that are required. It seems that a pretrained model can be used as the feature extractor $h_\\phi$, but it is necessary that its output $z(0)$ is sufficiently close to the target diversified equilibrium points. Could a neural network layer have been used to map the output of a pretained model to a diversified $z(0)$, without needing to retrain the entire feature extractor? Also, Runge-Kutta order 5 was used to integrate SODEF. Was the proposed method compared with simpler ODE solvers such as forward Euler? ", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Yes", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "2"}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper proposes a neural network classifier architecture based on neural ODEs against adversarial attacks. Some of the reviewers have concerns on the experiments, while in the rebuttal the newly added experiments convince the reviewers. All reviewers finally give positive support to the paper. Thus, I recommend accepting the paper and the authors should include the new experiments in the final version. 
Decision: Accept (Poster)