Stable Neural ODE with Lyapunov-Stable Equilibrium Points for Defending Against Adversarial Attacks
Year: 2021

Abstract:
Deep neural networks (DNNs) are well-known to be vulnerable to adversarial attacks, where malicious human-imperceptible perturbations are included in the input to the deep network to fool it into making a wrong classification. Recent studies have demonstrated that neural Ordinary Differential Equations (ODEs) are intrinsically more robust against adversarial attacks compared to vanilla DNNs. In this work, we propose a stable neural ODE with Lyapunov-stable equilibrium points for defending against adversarial attacks (SODEF). By ensuring that the equilibrium points of the ODE solution used as part of SODEF is Lyapunov-stable, the ODE solution for an input with a small perturbation converges to the same solution as the unperturbed input. We provide theoretical results that give insights into the stability of SODEF as well as the choice of regularizers to ensure its stability. Our analysis suggests that our proposed regularizers force the extracted feature points to be within a neighborhood of the Lyapunov-stable equilibrium points of the ODE. SODEF is compatible with many defense methods and can be applied to any neural network’s final regressor layer to enhance its stability against adversarial attacks.

heading: 1 Introduction
text: Although deep learning has found successful applications in many tasks such as image classification [1, 2], speech recognition [3], and natural language processing [4], the vulnerability of deep learning to adversarial attacks (e.g., see [5]) has limited its real-world applications due to performance and safety concerns in critical applications. Inputs corrupted with human-imperceptible perturbations can easily fool many vanilla deep neural networks (DNNs) into mis-classifying them and thus significantly impact their performance.
Recent studies [6–8] have applied neural Ordinary Differential Equations (ODEs) [9] to defend against adversarial attacks. Some works like [6] have revealed interesting intrinsic properties of
∗First two authors contributed equally to this work.
35th Conference on Neural Information Processing Systems (NeurIPS 2021).
ODEs that make them more stable than conventional convolutional neural networks (CNNs). The paper [6] proposes a time-invariant steady neural ODE (TisODE) using the property that the integral curves from a ODE solution starting from different initial points (inputs) do not intersect and always preserve uniqueness in the solution function space. However, this does not guarantee that small perturbations of the initial point lead to small perturbations of the integral curve output at a later time T . The authors thus proposed a regularizer to limit the evolution of the curves by forcing the integrand to be close to zero. However, neither the non-intersecting property nor the steady-state constraint used in TisODE can guarantee robustness against input perturbations since these constraints do not ensure that the inputs are within a neighborhood of Lyapunov-stable equilibrium points. An example is an ODE that serves as an identity mapping is not robust to input perturbations but satisfies all the constraints proposed in [6].
In this paper, our objective is to design a neural ODE such that the features extracted are within a neighborhood of the Lyapunov-stable equilibrium points of the ODE. We first develop a diversity promoting technique applied in the final fully connected (FC) layer to improve the ODE’s stability and analyze the reasons why. We then propose a stable neural ODE with Lyapunov-stable equilibrium points to eliminate the effects of perturbations in the input. From linear control theory [10], a linear time-invariant system dz(t)/ dt = Az(t), where A is a constant matrix, is exponentially stable if all eigenvalues of A have negative real parts. Specifically, we propose to force the Jacobian matrix of the ODE used in the neural ODE to have eigenvalues with negative real parts. Instead of directly imposing constraints on the eigenvalues of the matrix, which lead to high computational complexity when the Jacobian matrix is large, we instead add constraints to the matrix elements to implicitly force the real parts of its eigenvalues to be negative.
Our main contributions are summarized as follows:
1. Based on the concept of Lyapunov-stable equilibrium points, we propose a simple yet effective technique to improve the robustness of neural ODE networks by fixing the final FC layer to be a matrix whose rows have unit norm and such that the maximum cosine similarity between any two rows is minimized. Such a FC layer can be constructed off-line.
2. We propose a stable neural ODE for deFending against adversarial attacks (SODEF) to suppress the input perturbations. We derive an optimization formulation for SODEF to force the extracted feature points to be within a neighborhood of the Lyapunov-stable equilibrium points of the SODEF ODE. We provide sufficient conditions for learning a robust feature representation under SODEF.
3. We test SODEF on several widely used datasets MNIST [11], CIFAR-10 and CIFAR-100 [12] under well-known adversarial attacks. We demonstrate that SODEF is robust against adversarial white-box attacks with improvement in classification accuracy of adversarial examples under PGD attack [13] of up to 44.02%, 52.54% and 18.91% percentage points compared to another current state-of-the-art neural ODE network TisODE [6] on MNIST, CIFAR-10 and CIFAR-100, respectively. Similar improvements in classification accuracy of adversarial examples of up to 43.69%, 52.38% and 18.99% percentage points compared to ODE net [9] are also obtained.
The rest of this paper is organized as follows. We provide essential preliminaries on neural ODE and its stability analysis in Section 2. In Section 3, we present SODEF model architecture and its training method. We show how to maximize the distance between stable equilibrium points of neural ODEs. We propose an optimization and present theoretical results on its stability properties. We summarize experimental results in Section 4 and conclude the paper in Section 5. The proofs for all lemmas and theorems proposed in this paper are given in the supplementary material. We also refer interested readers to the supplementary material for a more detailed account of related works [14–16, 6] and some popular adversarial attacks [17, 13] that are used to verify the robustness of our proposed SODEF. In the paper, we use lowercase boldface characters like z to denote vectors in Rn, capital boldface characters like A to denote matrices in Rn×n, and normal characters like z to denote scalars except that the notation (x, y) are normal characters reserved to denote the input and label pairs. A vector z ∈ Rn is represented as (z(1), z(2), . . . , z(n)). The (i, j)-th element of a matrix A is Aij or [A]ij . The Jacobian matrix of a function f : Rn 7→ Rn evaluated at z is denoted as ∇f(z). The set of functions Rn 7→ Rn with continuous first derivatives is denoted as C1(Rn,Rn).

heading: 2 Preliminaries: Neural ODE and Stability
text: In a neural ODE layer, the relation between the layer input z(0) and output z(T ) is described as the following differential equation:
dz(t)
dt = fθ(z(t), t) (1)
where fθ : Rn × [0,∞) 7→ Rn denotes the non-linear trainable layers that are parameterized by weights θ and z : [0,∞) 7→ Rn represents the n-dimensional state of the neural ODE. Neural ODEs are the continuous analog of residual networks where the hidden layers of residual networks can be regarded as discrete-time difference equations z(t + 1) = z(t) + fθ(z(t), t). In this work, for simplicity, we only consider the time-invariant (autonomous) case fθ(z(t), t) = fθ(z(t)), where the dynamical system does not explicitly depend on t. For such non-linear dynamical systems, the following theorem shows that under mild conditions, its behaviour can be studied via linearization near special points called hyperbolic equilibrium points. Theorem 1 (Hartman–Grobman Theorem [18]). Consider a system evolving in time with state
z(t) ∈ Rn that satisfies the differential equation dz(t) dt
= f(z(t)) for some f ∈ C1(Rn,Rn), f(z) = (f (1)(z), . . . , f (n)(z)). Suppose the map has a hyperbolic equilibrium state z∗ ∈ Rn, i.e., f(z∗) = 0 and the Jacobian matrix ∇f = [∂f (i)/∂z(j)]ni,j=1 of f evaluated at z = z∗ has no eigenvalue with real part equal to zero. Then there exists a neighbourhood Nz∗ of the equilibrium point z∗ and a homeomorphism g : Nz∗ 7→ Rn, such that g(z∗) = 0 and in the neighbourhood Nz∗ , the flow of dz(t)
dt = f(z(t)) is topologically conjugate by the continuous map z̄(t) = g(z(t)) to the
flow of its linearization dz̄(t)
dt = ∇f(z∗) · z̄(t).
The theorem states that when the Jacobian matrix at the zeros of f has no eigenvalue with zero real part, the behaviour of the original dynamical system can be studied using the simpler linearization of the system around those zeros. We next review some definitions and theorems from linear control theory [10].
Definition 1 (Lyapunov Stability [10]). The linear time-invariant system dz̄(t)
dt = Az̄(t) with
constant matrix A is marginally stable or stable in the sense of Lyapunov if every finite initial state z̄(0) excites a bounded response. It is asymptotically stable if every finite initial state excites a bounded response, which, in addition, approaches 0 as t→∞.
Theorem 2 (Lyapunov Stability Theorem [10]). a) The equation dz̄(t)
dt = Az̄(t)) is marginally
stable if and only if all eigenvalues of A have zero or negative real parts and those with zero real parts
are simple roots of the minimal polynomial of A. b) The equation dz̄(t)
dt = Az̄(t) is asymptotically
stable if and only if all eigenvalues of A have negative real parts.
In Theorem 1, we say that a hyperbolic equilibrium point is Lyapunov-stable if all eigenvalues of the Jacobian matrix evaluated at it have negative real parts. From Theorems 1 and 2, we see that a small perturbation around the Lyapunov-stable equilibrium point z(0) leads to z̃(t) → z(0) as t→∞, i.e., ∃ δ > 0 such that for all z̃(0) with ‖z(0)− z̃(0)‖2 < δ, we have ‖z̃(t)− z(0)‖2 → 0 as t→∞, where z̃(t) is the ODE solution for the perturbed input z̃(0). In the context of neural network adversarial attacks, if the malicious perturbations around the ODE input z(0) is small, then the output z(T ) for large enough T will not be affected significantly by the perturbation. Consequently, the succeeding network layers after the neural ODE layer can still perform well without being affected by the input perturbation. The perturbation weakening phenomenon around Lyapunov-stable equilibrium points works like a noise filter and acts as a defense against adversarial attacks.
We require the following definition and result in our stability analysis. Definition 2 (Strictly diagonally dominant [19]). Let A ∈ Cn×n . We say that A is strictly diagonally dominant if |Aii| > ∑ j 6=i |Aij | for all i = 1, ..., n. Theorem 3 (Levy–Desplanques theorem [19]). If A ∈ Cn×n is strictly diagonally dominant and if every main diagonal entry of A is real and negative, then A is non-singular and every eigenvalue of A has negative real part.
Lemma 1. Given k distinct points zi ∈ Rn and matrices Ai ∈ Rn×n, i = 1, ..., k, there exists a function f ∈ C1(Rn,Rn) such that f(zi) = 0 and ∇fθ(zi) = Ai.

heading: 3 SODEF Architecture
text: We consider a classification problem with L classes. The proposed SODEF model architecture is shown in Fig. 1. The input x ∈ X (e.g., an image) is first passed through a feature extractor hφ : X 7→ Rn to obtain an embedding feature representation z(0). A neural ODE layer fθ follows as a nonlinear feature mapping to stabilize the feature representation output z(0) from hφ. The final FC layer V serves as a linear mapping to generate a prediction vector based on the output z(T ) of the neural ODE layer. The parameters φ,θ and V are parameterized weights for the feature extractor, neural ODE layer and FC layer, respectively.
We provide motivation and design guidance for the FC layer V in Section 3.1, which attempts to separate Lyapunov-stable equilibrium points implicitly by maximizing the similarity distance between feature representations corresponding to the L different classes. Experimental results demonstrate the advantages of our diversity promoting FC layer in Section 3.1 with comparisons to traditional neural ODEs without diversity promoting.
However, the embedded features after using diversity promoting are not guaranteed to locate near the Lyapunov-stable equilibrium points. In Section 3.2, we formulate an optimization problem to force embedding features to locate near the Lyapunov-stable equilibrium points. We introduce optimization constraints to force the Jacobian matrix of the ODE in our neural ODE layer to have eigenvalues with negative real parts at the Lyapunov-stable equilibrium points. Instead of directly imposing constraints on the eigenvalue of the matrix, which may be computationally complex especially when the matrix is large, we add constraints to the matrix elements instead.

heading: 3.1 Maximizing the Distance between Lyapunov-Stable Equilibrium Points
text: From Section 2, we observe that points in a small neighbourhood of a Lyapunov-stable equilibrium point is robust against adversarial perturbations. We call this neighborhood a stable neighborhood. However Lyapunov-stable equilibrium points for different classes may very well locate near each other and therefore each stable neighborhood may be very small, leading to poor adversarial defense. In this section, we propose to add a FC layer after the neural ODE layer given by (1) to avoid this scenario. The purpose of the FC layer is to map the output of the neural ODE layer to a feature vector vl if the input x belongs to the class l = 1, . . . , L. We design the FC layer so that the cosine similarities between different vl’s are minimized.
Lemma 2. Given a set of k unit vectors v1, . . . ,vk in Rn, where n ≥ k, let a(v1, . . . ,vk) = maxi6=j v ᵀ i vj . Then min a(v1, . . . ,vk) = 1/(1− k), where the minimum is taken over all possible sets of k unit vectors v1, . . . ,vk.
Corollary 1. Consider a k × k matrix B = [bij ]ki,j=1 with bii = 1 and bij = 1/(1 − k), ∀ i 6= j. Let the eigen decomposition of B be B = UΣUᵀ. For any n ≥ k and i = 1, . . . , k, let vi be the i-th column of QΣ1/2Uᵀ, where Q is any n × k matrix such that QᵀQ = Ik. Then, a(v1, . . . ,vk) = maxi6=j v ᵀ i vj = 1/(1− k).
Corollary 1 suggests a diversity promoting scheme to maximally separate the equilibrium points of the neural ODE layer. The FC layer is represented by an n × L matrix V = [v1, . . . ,vL], where n is the dimension of z(T ), the output from the neural ODE layer. If z(T ) is generated from an input from class l, it is mapped to vl. By minimizing the maximum cosine similarity a(v1, . . . ,vk) = maxi 6=j v ᵀ i vj between the representations from two different classes, we ensure that the output of SODEF is robust to perturbations in the input. Corollary 1 provides a way to choose the FC layer weights V.
To validate our observations, we conduct experiments to compare the robustness of ODE net [9] and TisODE [6] with and without our proposed FC layer V, on two standard datasets: MNIST [2] and CIFAR10 [12] 2. On the MNIST dataset, all models consist of four convolutional layers and one fully-connected layer. On the CIFAR10 dataset, the networks are similar to those for MNIST except the down-sampling network is a stack of 2 ResNet blocks. In practice, the neural ODE can be solved with different numerical solvers such as the Euler method and the Runge-Kutta methods [9]. Here, we use Runge-Kutta of order 5 in our experiments. Our implementation builds on the open-source neural ODE codes.3 During training, no Gaussian noise or adversarial examples are augmented into the training set. We test the performance of our model in defending against white-box attacks FGSM [17] and PGD [13] . The parameters for different attack methods used in this paper are given in the supplementary material. From Tables 1 and 2, we observe that for both datasets, our fixed FC layer improves each network’s defense ability by a significant margin. We visualize the features before the final FC layer using t-SNE [20] in Figs. 2 and 3. We observe that with the FC layer, the features for different classes are well separated even under attacks.

heading: 3.2 Objective Formulation and Stability
text: In this subsection, we formulate an optimization framework for SODEF to force output features to locate within the stable neighborhood of Lyapunov-stable equilibrium points. We make the following assumption. Assumption 1. The input x takes values in a compact metric spaceX and has probability distribution µ. The feature extractor hφ is injective and continuous.
2Our experiments are run on a GeForce RTX 2080 Ti GPU. 3https://github.com/rtqichen/torchdiffeq
The above assumption is satisfied if the input x (e.g., an image) resides in a bounded and closed set of a Euclidean space. We denote the pushforward measure (still a probability distribution) of µ under the continuous feature extractor mapping hφ as νφ = µ ◦h−1φ , where ◦ denotes function composition. The conditional probability distribution for the embedding of each class l ∈ {1, ..., L} has compact support El ⊂ Rn since El is closed and hφ(X) is bounded in Rn. In Section 3.1, the FC layer V tries to maximize the distance between El, l = 1, . . . , L. In this section for analysis purposes, we also assume the following.
Assumption 2. We have El ⋂ El′ = ∅ if l 6= l′, i.e., the supports of each class are pairwise disjoint.
Our objective function is formulated as follows, which is explained in detail in the sequel:
min θ,φ
Eµ`(Vᵀ(z(T )), yi) (2)
s. t. Eνφ‖fθ(z(0))‖2 < , fθ ∈ C1(Rn,Rn), (3) Eνφ [∇fθ(z(0))]ii < 0, ∀ i = 1, . . . , n, (4)
Eνφ [ |[∇fθ(z(0))]ii| − ∑ j 6=i |[∇fθ(z(0))]ij | ] > 0, ∀ i = 1, . . . , n, (5)
z(0) = hφ(x), and z(T ) is the output of (1) with input z(0). (6)
Here, ` is a loss function and > 0 is a positive constant. The constraints (3) to (5) force z(0) to be near the Lyapunov-stable equilibrium points with strictly diagonally dominant derivatives. We limit the fθ to be in C1(Rn,Rn) to satisfy the condition in Theorem 1. From [21], we also know that standard multi-layer feed forward networks with as few as a single hidden layer and arbitrary bounded and non-constant activation function are universal approximators for C1(Rn,Rn) functions with respect to some performance criteria provided only that sufficiently many hidden units are available.
As a comparison, TisODE [6] only includes a constraint similar to (3), which in general provides no guarantee to force z(0) near the Lyapunov-stable equilibrium points. In the extreme case with parameters θ = 0 for fθ such that fθ = 0, the ODE degenerates to an identity mapping. No z(0) ∈ Rn can now be a Lyapunov-stable equilibrium point, and no stability can therefore be guaranteed to defend against adversarial attacks even though the ODE curves still possess the nonintersecting property and steady-state constraint, which were cited as reasons for the stability of TisODE.
Instead of directly optimizing the above objective function, in our implementation, we optimize the following empirical Lagrangian with a training set {(xk, yk) : k = 1, ..., N}:
min θ,φ
1
N N−1∑ k=0
( ` ( Vᵀzk(T ), yk ) + α1‖fθ ( zk(0) ) ‖2 + α2g1 ( n∑ i=1 [∇fθ(zk(0))]ii )
+ α3g2 ( n∑ i=1 (−|[∇fθ(zk(0))]ii|+ ∑ j 6=i
|[∇fθ(zk(0))]ij |) ) (7)
s. t. zk(0) = hφ(xk), and zk(T ) is the output of (1) with input zk(0), ∀ k = 1, . . . , N (8)
where α1, α2 and α3 are hyperparameter weights, g1 and g2 are chosen monotonically increasing functions bounded below to eliminate the unbounded impact of the two regularizers that can otherwise dominate the loss. In this paper, we set g1(·) = g2(·) = exp(·). We call these two latter terms the SODEF regularizers.
Suppose for each class l = 1, . . . , L, the embedding feature set El = {z(l)1 , . . . , z (l) k } is finite. For each i = 1, . . . , k, let Ai ∈ Rn×n be strictly diagonally dominant matrix with every main diagonal entry be negative such that the eigenvalues for Ai all have negative real part. From Theorem 3, each Ai is non-singular and every eigenvalue of Ai has negative real part. Therefore, from Theorem 2 and Lemma 1, there exists a function fθ such that all z (l) i are Lyapunov-stable equilibrium points with corresponding first derivative∇fθ(z(l)i ) = Ai. This shows that if there exist only finite representation points for each class, we can find a function fθ such that all inputs to the neural ODE layer are Lyapunov-stable equilibrium points for fθ and
(a) Eνφ‖fθ(z(0))‖2 = 0, (b) Eνφ [∇fθ(z(0))]ii < 0, ∀ i = 1, . . . , n,
(c) Eνφ [ |[∇fθ(z(0))]ii| − ∑ j 6=i|[∇fθ(z(0))]ij | ] > 0, ∀ i = 1, . . . , n.
If the input space X has infinite cardinality, then an injective and continuous feature extractor hφ results in a νφ with non-finite support, i.e., at least one El, l = 1, . . . , L, is infinite. It is not obvious whether we can obtain a fθ where every point in E = ⋃ lEl is a stable equilibrium point. The following result gives a negative answer if νφ is a continuous measure (i.e., absolutely continuous with respect to (w.r.t.) Lebesgue measure) on some subset. Lemma 3. If the restriction of νφ to some open set E′ ⊂ E is a continuous measure, there is no continuous function fθ such that for νφ-almost surely all z ∈ E, fθ(z) = 0 and all the eigenvalues of ∇fθ(z) have negative real parts. In other words, there is no continuous function fθ such that almost surely all z in E are Lyapunov-stable equilibrium points.
Lemma 3 indicates that it is too much to hope for all points in E to be Lyapunov-stable equilibrium points. In the following, we relax this requirement and show that under mild conditions, for all > 0, we can find a continuous function fθ with finitely many stable equilibrium points such that conditions (b) and (c) above hold and condition (a) is replaced by Eνφ‖fθ(z(0))‖2 < . This motivates the optimization constraints in (3) to (5). Theorem 4. Suppose Assumptions 1 and 2. If νφ is not a continuous uniform measure on El for each l = 1, . . . , L, then the following holds: 1) The function space satisfying the constraints in (3) to (5) is non-empty for all > 0. 2) If additionally the restriction of νφ to any open set O ⊂ El is not a continuous uniform measure, there exist functions in this space such that each support El contains at least one Lyapunov-stable equilibrium point.

heading: 4 Experiments
text: In this section, we evaluate the robustness of SODEF under adversarial attacks with different attack parameters. We conduct experiments to compare the robustness of ODE net [9] and TisODE net [6] on three standard datasets: MNIST [2], CIFAR10 and CIFAR100 [3]. Since SODEF is compatible with many defense methods, it can be applied to any neural network’s final regressor layer to enhance its stability against adversarial attacks. Our experiment codes are provided in https://github.com/KANGQIYU/SODEF.

heading: 4.1 Setup
text: We use open-source pre-trained models that achieve the top accuracy on each dataset as the feature extractor hφ. Specifically for simple MNIST task, we use the ResNet18 model provided in Pytorch. We use the model provided by [22], which obtains nearly 88% clean accuracy on CIFAR100 using EfficientNet [23] and the model provided by [24], which has nearly 95% clean accuracy on CIFAR10. In the neural ODE layer, fθ consists of 2 FC layers. During the trainings of SODEF (except in the experiment included in Section 4.2), we train the neural network with the fixed FC introduced in Section 3.1. In the first 30 epochs, we fixed fθ to let the feature extractor hφ learn a feature representation
with only the cross-entropy loss `, and in the remaining 120 epochs, we release hφ to further train fθ using (7) with α1 = 1 and α2 = α3 = 0.05. For CIFAR10 and CIFAR100, the pixel values are normalized by (x− µ)/σ where µ = [0.4914, 0.4822, 0.4465] and σ = [0.2023, 0.1994, 0.2010] 4. To show that our SODEF is compatible with many defense methods and can be applied to any neural network’s final regression layer, we conduct an experiment where we use a recently proposed robust network TRADES [25] as the feature extractor in our SODEF. The pretrained model is provided here 5, and we choose the model with architecture "WRN-34-10" to conduct our experiments. Besides the two vanilla white-box attacks FGSM and PGD as metioned in Section 3.1, we also include a strong ensemble attack AutoAttack [26], which sequentially performs attack using all of the following four individual attacks: three white-box attacks APGDCE, APGDTDLR and FAB
T[27], and one black-box Square attack [28]. We refer the reader to the the supplementary material for more details of the attacks used in this paper, where, in additional, more experiments are included.

heading: 4.2 Compatibility of SODEF
text: Adversarial training (AT) is one of the most effective strategies for defending adversarial attacks. TRADES [25] is one of the adversarial training defense methods with combinations of tricks of warmup, early stopping, weight decay, batch size and other hyper parameter settings. In this experiment we fix the pretained TRADES model (except the final FC layer (size 640x10)) as our feature extractor hφ. We then append our (trainable) SODEF with integration time T = 5 to the output of the feature extractor. To evaluate model robustness, we use AutoAttack and attack the models using both the L2 norm ( = 0.5) and L∞ norm ( = 8/255). The results are shown in Table 3. We clearly observe that our SODEF can enhance TRADES’s robustness under all the four individual attacks and the strongest ensemble AutoAttack. For the strong L2 AutoAttack, our SODEF have improved the model robustness from 59.42% to 67.75%. Our experiment show that SODEF can be applied to many defense models’ regression layer to enhance their stability against attacks.

heading: 4.3 Influence of Integration Time T
text: From the discussion after Theorems 1 and 2, we know if the malicious perturbations around the ODE input Lyapunov-stable equilibrium point z(0) is small, then the output z(T ) for large enough T will not be affected significantly by the perturbation: ‖z̃(t)− z(0)‖2 → 0 as t→∞. Consequently, the succeeding network layers after the neural ODE layer can still perform well without being affected by the input perturbation. In this section, we test the influence of the SODEF integration time T using CIFAR100. We use the model EfficientNet provided by [23] as hφ (Note, unlike Section 4.2, hφ is trainable in this experiments). We use AutoAttack with L2 norm ( = 0.5). We observe that for all the four individual attacks and the strongest ensemble AutoAttack, SODEF performs generally better for large integration time T . We also test larger integration time T > 10, but do not see any obvious improvements.

heading: 4.4 Performance Comparison Under AutoAttack
text: For a comparison, we provide the results of applying AutoAttack to other baseline models mentioned in the paper. We set the same integration time for ODE, TisODE and SODEF. We observe that for the
4To test AutoAttack, we have strictly followed the instruction in https://github.com/RobustBench/ robustbench to attack the original images before any normalization or resizing.
5https://github.com/P2333/Bag-of-Tricks-for-AT
strongest AutoAttack, our SODEF outperforms the other baseline models by a significant margin. In this case, SODEF achieves 79.10% accuracy while other models only get less than 3% accuracy.

heading: 4.5 Performance Under PGD and FGSM Attacks
text: White-box adversaries have knowledge of the classifier models, including training data, model architectures and parameters. We test the performance of our model in defending against the whitebox attacks, PGD and FGSM. We set T = 5 as the integration time for the neural ODE layer. The parameters for different attack methods used are given in the supplementary material. The subsequent experiments use these settings by default, unless otherwise stated.
The classification results on MNIST are shown in Table 6. We observe that while maintaining the state-of-the-art accuracy on normal images, SODEF improves the adversarial robustness as compared to the other two methods. For the most effective attack in this experiment, i.e., PGD attack, SODEF shows a 45.25% − 1.56% = 43.69% improvement over ODE and a 45.25% − 1.23% = 44.02% improvement over TisODE.
For CIFAR-10, we see from Table 7 that SODEF maintains high accuracy on normal examples and makes the best predictions under adversarial attacks. In particular, SODEF achieves an absolute percentage point improvement over ODE net up to 52.38% and over TisODE up to 52.54% for PGD attack.
For CIFAR-100, the results in the supplementary material shows that the most effective attack causes the classification accuracy to drop relatively by 74.6% = 88.0−22.3588.0 for SODEF and by
97.3% = 88.3−2.3988.3 for vanilla EfficientNet, which is pre-trained on ImageNet to obtain a top clean accuracy. Neither ODE net nor TisODE net can improve the classification accuracy under PGD attack by a big margin, e.g. TisODE net only improves the classification accuracy from 2.39% to 3.44%, while SODEF still shows clear defense capability in this scenario.

heading: 4.6 Ablation Studies
text: The impact of the ODE with and without the SODEF regularizers in (7) has been presented in the above comparisons between SODEF and ODE. In this section, we show the necessity of diversity promoting using the FC introduced in Section 3.1 and conduct transferability study.

heading: 4.6.1 Impact of Diversity Promotion
text: Table 8 shows the difference of the defense performance when fixing the final FC be V or setting it to a trainable linear layer. It can be seen that having diversity control improves the robustness. One possible reason for this phenomenon given in Section 3 is that diversity promotion with a fixed designed FC attempts to make the embedding feature support El of each class l disjoint to each other and therefore the Lyapunov-stable equilibrium points for each El are well separated.

heading: 4.6.2 Transferability Study
text: Transferability study is carried out on CIFAR-10, where the adversarial examples are generated using FGSM and PGD attacks using ResNet18 without any ODEs. The classification accuracy drops from 68.05% to 59% for FGSM with = 0.3, and from 55.59% to 34% for PGD with = 0.1. One possible reason for this phenomenon is that ODEs have obfuscated gradient masking effect as discussed in [15], and a transfer attack may deteriorate the defense effect. However, as we observe from Table 7, even with a transfer attack on SODEF, it still performs better than other ODEs without transfer attacks.

heading: 5 Conclusion
text: In this paper, we have developed a new neural ODE network, SODEF, to suppress input perturbations. SODEF is compatible with any existing neural networks and can thus be appended to the state-of-theart networks to increase their robustness to adversarial attacks. We demonstrated empirically and theoretically that the robustness of SODEF mainly derives from its stability and proposed a training method that imposes constraints to ensure all eigenvalues of the Jacobian matrix of the neural ODE layer have negative real parts. When each classification class converges to its own equilibrium points, we showed that the last FC layer can be designed in such a way that the distance between the stable equilibrium points is maximized, which further improves the network’s robustness. The effectiveness of SODEF has been verified under several popular while-box attacks.
Acknowledgments and Disclosure of Funding
This research is supported in part by A*STAR under its RIE2020 Advanced Manufacturing and Engineering (AME) Industry Alignment Fund – Pre Positioning (IAF-PP) (Grant No. A19D6a0053) and the RIE2020 Industry Alignment Fund – Industry Collaboration Projects (IAF-ICP) Funding Initiative, as well as cash and in-kind contribution from the industry partner(s). The computational
work for this article was partially performed on resources of the National Supercomputing Centre, Singapore (https://www.nscc.sg).
Broader Impact
Our work, which contributes to more robust DNNs, is supposed to mitigate the threat of adversarial attacks. However, on the hand, the reliable deployment of DNNs in automation of tasks will potentially bring mass-scale unemployment and social unrest. As DNNs become more robust and more tasks, especially those whose failures will bring high risks to human lives or large property losses under adversarial attacks, fall into the automatic task category, massive jobs could disappear.

Review summaries:
This paper proposes a neural network classifier architecture based on neural ODEs that is designed to be robust to adversarial attacks. In the proposed method, the classes are represented by vectors, chosen to be maximally separated in the output space w.r.t. cosine similarity. The neural ODE system is then encouraged to have equilibrium points at those class vectors via regularization; the idea is that perturbed outputs are pulled towards the equilibria as the ODEs evolve.

This work aims to develop a neural ODE-based model to defend against adversarial attacks. Specifically, the authors propose a stable neural ODE with Lyapunov-stable equilibrium points (SODEF). By ensuring the equilibrium points are Lyapunov-stable, the ODE solution for an input with a small perturbation will converge to the same solution for the original unperturbed input. The paper provides theoretical insights into the stability of SODEF and how to choose the stability regularizers. Besides, the paper also provides design guidance for the final FC layer, that aims to separate Lyapunov-stable equilibrium points. The proposed SODEF can work in conjunction with several defense methods and can be applied to any network's final regressor layer.

This paper tries to improve the robustness of neural ODE via enforcing the Lyapunov stability constraints in the training process. The key idea is as follows: a dynamic system can have multiple equilibrium points, and the input perturbation around the equilibrium point will decay and not significantly influence the result if the equilibrium point is Lyapunov stable. This is an excellent contribution that connects the Lyapunov stability of ODEs with the robustness of neural networks.