LAYER-WISE ADVERSARIAL DEFENSE: AN ODE PER-
Year: 2020

Abstract:
Deep neural networks are observed to be fragile against adversarial attacks, which have dramatically limited their practical applicability. On improving model robustness, the adversarial training techniques have proven effective and gained increasing attention from research communities. Existing adversarial training approaches mainly focus on perturbations to inputs, while the effect of the perturbations in hidden layers remains underexplored. In this work, we propose layer-wise adversarial defense which improves adversarial training by a noticeable margin. The basic idea of our method is to strengthen all of the hidden layers with perturbations that are proportional to the back-propagated gradients. In order to study the layer-wise neural dynamics, we formulate our approach from the perspective of ordinary differential equations (ODEs) and build up its extended relationship with conventional adversarial training methods, which tightens the relationship between neural networks and ODEs. In the implementation, we propose two different training algorithms by discretizing the ODE model with the Lie-Trotter and the Strang-Marchuk splitting schemes from the operator-splitting theory. Experiments on CIFAR-10 and CIFAR-100 benchmarks show that our methods consistently improve adversarial model robustness on top of widely-used strong adversarial training techniques.

heading: 1 INTRODUCTION
text: Recent years have witnessed the prosperity of deep learning in many tasks (Hinton & Salakhutdinov, 2006; Sutskever et al., 2014; He et al., 2016; LeCun et al., 2015; Huang et al., 2017; Vaswani et al., 2017). Stacked with multiple layers, neural networks provide an end-to-end solution to all the tasks and prove to be highly effective. However, the seminal study by Szegedy et al. (2013) has shown that deep neural networks (DNNs) can be fragile against attacks: minor perturbations on inputs lead to significant change in model predictions. Regarding the defense approaches, intensive studies on adversarial defense techniques have been proposed (Athalye et al., 2018a; Goodfellow et al., 2014; Zheng et al., 2016; Madry et al., 2018; Zhang et al., 2019b; Kurakin et al., 2017; Pang et al., 2019a; 2020; 2019b; Raff et al., 2019; Guo et al., 2018; Zhang et al., 2020a; Balunovic & Vechev, 2019; Wong et al., 2020; Chan et al., 2020; Zhang et al., 2020b). Among these techniques, adversarial training algorithms (Madry et al., 2018; Zhang et al., 2019b) incorporate the effect of perturbed inputs into the loss function, which are shown to be competent and boasts the dominant impact in the adversarial defense research field.
While adversarial training techniques have gained increasing attention in the robust deep learning research community, most of current approaches concentrate on deriving perturbations on the inputs with gradients back-propagated from the loss function. However, as information flow in neural networks starts from inputs and passes through hidden layers, it is essential to robustify both the inputs and the hidden layers. While previous studies have made successful attempts on introducing damping terms (Yang et al., 2020) or stochastic noise (Liu et al., 2020; Wang et al., 2019) to each layer in neural architectures, they concentrate on improving general model robustness and are less focused on adversarial model robustness. We ask the following question: Can we take the hidden layers of neural networks into account to improve adversarial model robustness?
In this work, we propose layer-wise adversarial defense to improve adversarial training, which enhances adversarial model robustness by stabilizing both inputs and hidden layers. In our method,
the layer-wise perturbations are incorporated into the robust optimization framework of adversarial training. We propose to inject scaled back-propagated gradients into the architecture as layer-wise perturbations. Besides, we formulate our method from the perspective of ordinary differential equations and propose a novel ODE as its the continuous limit in order to study the neural dynamics. Inspired from the rich literature on numerical analysis, we use the Lie-Trotter and the Strang-Marchuk splitting schemes to solve the proposed ODE. We refer to the resulted discrete algorithms as Layerwise Adversarial Defense (LAD) and LAD-SM, respectively. Furthermore, we build up the extended relationship between our methods with current natural training and adversarial training techniques by analyzing the second order dynamics. Our analysis shows that our methods have introduced additional perturbations in the first order initial value of the second order dynamics compared with current adversarial training algorithms.Experiments on the CIFAR-10 and CIFAR-100 benchmarks show that our methods improve adversarial model robustness on top of different widely-used strong adversarial training techniques.
We summarize our contributions as follows:
• We propose layer-wise adversarial defense which generalizes conventional adversarial training approaches with layer-wise adversarial perturbations (Section 3.1);
• We investigate the continuous limit of our layer-wise adversarial defense methods and propose an ODE that integrates the adjoint state into the forward dynamics (Section 3.2);
• We build up the extended relationship between our methods and current adversarial training approaches by analyzing the second order neural dynamics in theory. Experiments have also shown the effectiveness of our methods in practice. (Section 3.3 and Section 4).

heading: 2 RELATED WORK

heading: 2.1 ADVERSARIAL MODEL ROBUSTNESS
text: In this section we review the literature on gradient-based attack and defense approaches in the field of adversarial model robustness. For adversarial attacks, widely-used approaches include Fast Gradient Sign Method (FGSM) (Goodfellow et al., 2015) and Iterated Fast Gradient Sign Method (IFGSM) (Madry et al., 2018). For a given data point, FGSM induces the adversarial example by moving with the attack radius at each component along the gradient ascent direction. Iterated FGSM performs FGSM with inner iteration updates with smaller step size α. Prior studies have inspired multiple adversarial attack techniques (Athalye et al., 2018b; Carlini & Wagner, 2017; Ilyas et al., 2018; Dong et al., 2018; Pang et al., 2018). Adversarial defense techniques can be categorized by training phase (Athalye et al., 2018a; Goodfellow et al., 2014; Zheng et al., 2016; Madry et al., 2018; Zhang et al., 2019b; Kurakin et al., 2017; Pang et al., 2019a; 2020; Zhang et al., 2020a; Balunovic & Vechev, 2019; Wong et al., 2020; Chan et al., 2020; Zhang et al., 2020b) and inference phase (Pang et al., 2019b; Raff et al., 2019; Xie et al., 2018; Guo et al., 2018).
The widely-used approach in training phase is Projected Gradient Descent (PGD) training (Madry et al., 2018), which integrates the effect of the perturbed inputs into its loss function. The current state-of-the-art defense approach in training phase is TRADES (Zhang et al., 2019b), which additionally introduces the boundary error as a regularization term into its loss function. In our experiments, we select PGD training and TRADES as our baselines. While substantially enhancing adversarial model robustness, the gradient-based perturbations in adversarial training are currently only performed on inputs. As cascaded hidden layers comprise the passage for information flow in neural networks, it is essential to stabilize hidden layers as well. In our work, we introduce layerwise gradient-based perturbations to neural architectures to improve adversarial model robustness.

heading: 2.2 ODE-INSPIRED ARCHITECTURE DESIGNS
text: Research about the relationship between neural networks and ODEs starts with the continuous limit formulation of ResNet (E, 2017), which has inspired many novel neural architecture designs (Lu et al., 2018; Zhu et al., 2018; Chang et al., 2018; Haber & Ruthotto, 2017; Chen et al., 2018; Dupont et al., 2019). Regarding model robustness, most prior studies have focused on improving dynamic system stability by Lyapunov analysis, more stable numerical schemes, or imposing regularization.
From Lyaponov stability perspective, Yang et al. (2020) introduce damping terms to residual networks to stabilize dynamic systems. Appropriately adjusting the damping factor introduces damping effect to the dynamic system and enhances model robustness. Similarly, Chang et al. (2019) improve Lyapunov stability of RNNs by imposing antisymmetric constraints on the weight matrix. On more stable numerical schemes, prior studies include taking small step sizes in the forward Euler scheme (Zhang et al., 2019c) or leveraging implicit schemes to enhance stability (Reshniak & Webster, 2019; Li et al., 2020). For imposing regularization, stochasticity is introduced into ODEs for stabilization (Liu et al., 2020; Wang et al., 2019). Hanshu et al. (2019) regularize neural ODE models to be timeinvariant and add a regularization term about the upper bound of the effect from input perturbation. Zhang et al. (2019a) propose a differential game formulation for adversarial training and accelerate the process from the optimal control theory. Our work differs from the prior studies by integrating gradient-based perturbations into the neural dynamics, which proves to be an extension to current approaches on improving adversarial model robustness in both theory and practice.

heading: 3 LAYER-WISE ADVERSARIAL DEFENSE

heading: 3.1 THE MODEL FORMULATION
text: The objective of conventional adversarial training approaches can be formulated into a min-max problem (Madry et al., 2018; Zhang et al., 2019a). We introduce layer-wise perturbations {∆xn}Nn=0 and rewrite the min-max problem as follows:
min {θn}Nn=1 max {∆xn}Nn=0 L(Θ,∆x) := L(xN )
subject to x̃0 = x0 + ∆x0, xn+1 = x̃n + f(x̃n,θn),
x̃n+1 = xn+1 + ∆xn+1, n = 0, 1, 2, · · · , N − 1.
(1)
where N is the number of layers in a neural network, Θ = {θn}Nn=1 represent its trainable parameters and the {∆xn}Nn=0 represent the layer-wise perturbations. In our formulation, we ignore the bounded assumptions on the perturbations for simplicity, since if there are additional bounded constraints on the perturbations, we can project the gradient onto the intervals. It is noted that when ∆xn = 0 for all n = 1, . . . , N , the model (1) reduces to the conventional adversarial training formulation. More specifically, for adversarial training algorithms (Madry et al., 2018; Zhang et al., 2019b), let M be the maximum number of inner iterations for the perturbations, we have the following update rule (ignoring the bounded constraints):
x (m+1) 0 = x (m) 0 + η
∂L
∂x0 ∣∣∣∣ (m) , x (0) 0 = x0, (2)
where the perturbation ∆x0 is defined as x (M) 0 − x0.
In this work, we generalize the above idea to introducing perturbations to hidden layers, i.e. determine ∆xn+1 in Eq. (1) so that the objective in Eq. (1) is maximized. Similar to Eq. (2), we perturb
xn+1 with iterative gradient descent:
x (m+1) n+1 = x (m) n+1 + η
∂L
∂xn+1 ∣∣∣∣ (m) , x (0) n+1 = xn+1, (3)
where η is the step size as a scaling factor over the gradients ∂L/∂xn+1. We set ∆xn+1 = x (M) n+1 − xn+1, where M is the number of the iteration steps. When M = 1, the layer-wise adversarial perturbations are given by
∆xn+1 = x (1) n+1 − xn+1 = η
∂L
∂xn+1 , n = 0, 1, 2, · · · , N − 1. (4)
Replacing the maximization of layer-wise perturbations {∆xn}Nn=0 by Eq. (4), we obtained a simplified problem as follows.
min {θn}Nn=1 L(Θ) := L(xN )
subject to x̃n+1 = xn + f(xn,θn),
xn+1 = x̃n+1 + η ∂L
∂x̃n+1 , n = 0, 1, 2, · · · , N − 1,
(5)
with the inputs x̃0 = x0 +∆x0 determined by Eq. (4). Notice that directly applying Eq. (5) requires alternative computations of x̃n+1 and ∂L/∂x̃n+1 with iterative forward and backward passes, which can be extremely time-consuming. In our implementation, we leverage a two-stage approach: first record the gradients with respect to each layer in a forward and backward pass, then add the recorded gradients to each layer in another forward pass as layer-wise adversarial perturbations. We refer to this algorithm as our Layer-wise Adversarial Defense (LAD) method.

heading: 3.2 ODE-INSPIRED ALGORITHM DESIGN
text: In this section, we explore the continuous formulation of the constraints in Eq. (5) to study the layerwise dynamics from the ODE perspective. Recall that the conventional ODE formulation (E, 2017) takes the following form:
dx̂(t)
dt = f(x̂(t), t), x̂(0) = x̂0, (6)
which is the continuous limit of the discrete ResNet x̂n+1 = x̂n+fn(x̂n) with the time step ∆t = 1. In the conventional ODE, L = L(x̂(T )) is defined as the loss function. In our work, we propose to integrate dL/dx̂(t) into the forward process:
dx(t)
dt = f(x(t), t) + η
dL
dx̂(t) , x(0) = x0. (7)
where x0 is the (perturbed) inputs and η is a scaling factor. The introduced dL/dx̂(t) represents the continuous limit of back-propagated gradients. As we record the gradients from an earlier backward pass, the corresponding forward pass can be approximately treated as solving the original ODE. As shown in Eq. (7), there are two operators on the right hand side. According to the rich literature on the operator-splitting theory from numerical analysis, we have the following proposition. Proposition 3.1. The LAD method is the numerical solution of our proposed ODE with the LieTrotter splitting scheme with step size ∆t = 1.
In the operator splitting theory, the Strang-Marchuk (SM) splitting scheme (Ascher & Petzold, 1998) is also a widely-used technique of solving ODE with multiple terms. Compared with the Lie-Trotter splitting scheme, the Strang-Marchuk splitting scheme enjoys much lower local truncation errors (Bobylev & Ohwada, 2001). Lu et al. (2019) propose to leverage the SM splitting scheme to improve the Transformer architecture, which results in higher model accuracy. We also propose to use the SM splitting scheme to discretize our ODE in Eq. (7), but the direct application of SM method is intractable. With proper approximation, we have the following modified version. Theorem 3.2. An approximated numerical scheme of Eq. (7) with the SM splitting is x̃n = xn + η 2 ∂L ∂x̂n , x̄n = x̃n + fn(x̃n),
xn+1 = x̄n + η 4 ( ∂L ∂x̂n + ∂L∂x̂n+1 ) .
(8)
The proofs and a self-contained introduction to the numerical background are provided in Appendix A. We refer to Eq. (8) as our LAD-SM method.

heading: 3.3 ANALYSIS OF THE SECOND ORDER DYNAMICS
text: In this section, we provide an analysis of the second order dynamics and connect our ODE (Eq. (7)) with the original ODE (Eq. (6)). Define a(t) = dL/dx̂(t), it is known that the dynamics of a(t) (Pontryagin et al., 1962; Chen et al., 2018) satisfies
da dt = −a(t)T ∂f(x̂, t) ∂x̂ . (9)
The next theorem presents the second order dynamics of the proposed Eq. (7).
Theorem 3.3. The second order dynamics of Eq. (7) is given by
d2x
dt2 = ∂f(x, t) ∂x f(x, t) + ∂f(x, t) ∂t + ηa(t)T (∂f(x, t) ∂x − ∂f(x̂, t) ∂x̂ ) (10)
with
x(0) = x0, dx
dt ∣∣∣∣ t=0 = f(x(0), 0) + η dL dx(0) . (11)
Proof. By the direct computation, we know the second order dynamics of Eq. (7) is
d2x
dt2 = ∂f(x, t) ∂x dx dt + ∂f(x, t) ∂t − ηa(t)T ∂f(x̂, t) ∂x̂ (12)
= ∂f(x, t)
∂x
( f(x, t) + ηa(t) ) + ∂f(x, t)
∂t − ηa(t)T ∂f(x̂, t) ∂x̂ (13)
= ∂f(x, t)
∂x f(x, t) +
∂f(x, t)
∂t + ηa(t)T (∂f(x, t) ∂x − ∂f(x̂, t) ∂x̂ ) (14)
where the first equality is from Eq. (9) and the second equlaity is from Eq. (7).
Moreover, the second order dynamics of Eq. (6) is given by
d2x̂
dt2 = ∂f(x̂, t) ∂x̂ f(x̂, t) + ∂f(x̂, t) ∂t ,
with x̂(0) = x0, dx̂
dt ∣∣∣∣ t=0 = f(x0, 0).
(15)
Since x is a small perturbation of x̂ as well as the existence of ηa(t), the last term in Eq. (10) can be negligible and the main difference of the seccond order dynamics of our ODE (Eq. (7)) with the original ODE (Eq. (6)) lies in the first order initial values. The extra momentum of the input in Eq. (11) leads to extra perturbations in all of the hidden layers during the propagation process. The implementation details of our methods can be found in Appendix B.

heading: 4 EXPERIMENTS
text: We evaluate our proposed methods on the CIFAR (Krizhevsky et al., 2009) benchmarks. For each experiment, we conduct 3 runs with different random seeds and report the averaged result to reduce the impact of random variations. We also select Yang et al. (2020) as our baseline methods, which introduce layer-wise damping terms to each layer in neural networks. The details of experimental settings can be found in Appendix B.

heading: 4.1 LAD WITH NATURAL TRAINING
text: We first evaluate our methods in the setting of natural training (He et al., 2016). This setting is equivalent to setting ∆x0 = 0 in our layer-wise adversarial defense framework (Eq. (1)). Table 4.1 shows the accuracy and robustness results of our methods composed with natural training under different attack radii .
Results show that our methods perform best under most attack settings with small radii. While all methods fail under attacks with larger radii because of the vulnerability of natural training, results under attacks with small radii have still shown the enhanced robustness by only introducing perturbations on hidden layers.

heading: 4.2 LAD WITH PGD TRAINING
text: In this section, we report our experiments with PGD Training (Madry et al., 2018). Table 2 shows the results of our methods and the baseline techniques with PGD training.
According to the experimental results, it is shown that our methods consistently outperform the baselines in the robustness performance. Another interesting finding is that in our experiments, our approaches achieve more significant improvements over baselines on the CIFAR-100 benchmark. We provide a possible interpretation: from the discrete neural network perspective, the stacked
layers represent different levels of representation learning. While our methods have perturbed each layers, it can be interpreted as an augmentation with adversarial “examples” in each levels of feature learning. Given that each class in the CIFAR-100 training set consists only 500 images (which is far less than that in the CIFAR-10 training set (5,000 images)), it is inferred that our methods have a potential positive effect in the data-scarce regime.

heading: 4.3 LAD WITH TRADES
text: In this section, we report our experimental results with the TRADES adversarial training approach (Zhang et al., 2019b). Table 3 shows the results of our methods composed with the TRADES method. According to the results shown in Table 2, we only select TRADES with the original ResNet architecture as our baseline methods. Results show that our methods have also surpassed the baseline methods on top of the state-of-the-art TRADES technique. We also provide robustness results of our methods as well as the baselines under stronger attacks in Appendix C.

heading: 4.4 LAD WITH STOCHASTICITY
text: Prior studies have shown that model robustness is improved with layer-wise noise injection as regularization (Liu et al., 2020; Wang et al., 2019). We also experiment on introducing stochasticity to our methods in order to further boost the robustness performance. We augment our experiments with PGD training with additive Gaussian noise, which is proposed by Wang et al. (2019). During training, Gaussian noise n ∼ N(0, 0.01) is added to each layer in the model. During inference, we perform forward pass with noise sampled for 100 times and take the averaged output logits as the expectation outputs. Table 4 shows the results on the CIFAR-10 benchmark. As shown from the results, our approaches consistently surpass the baseline approaches under both the deterministic and the stochastic settings.
4.5 THE EFFECT OF η
In this section, we introduce how we determine the scaling factor η. The hyperparameter η scales the back-propagated gradients, which are further added to the network layer-wisely. We start to get basic knowledge about the proper range of η by comparing the norms of back-propagated gradients with the norms of each layers. According to Eq. (11), our methods have essentially perturbed the first order initial value in the view of the second order dynamics. As a result, we let
ratio = ‖x(0) + f(x(0), 0)‖2∥∥∥ dLdx(0)∥∥∥
2
(16)
and trace the ratio during training.
As Figure 2 shows, the ratio is ultimately large at the beginning of training (around 2×106) and becomes smaller during training. While it achieves its minimum at around the end of the training, the ratio is always larger than 1×105. As a result, the ratio serves as a quantitative comparison between the norm of the back-propagated gradient with the norm of the layer in a network. It suggests that the η should not be too small; otherwise the layer-wise perturbations can be too small to affect the training. Model robustness results under different η’s have also proven the finding. As Table 5 shows, the effect on adversarial model robustness from a small η is marginal compared with that from a larger η. Table 5 also shows that η should not be too large. A too large η may unstabilize the training process and deteriorate the model robustness.

heading: 5 CONCLUSION
text: In this work, we propose layer-wise adversarial defense as an extension to current adversarial training approaches. The hidden layers are robustified by the introduced layer-wise perturbations, which are proportional to the back-propagated gradients. We build up the extended relationship of our methods with conventional adversarial training methods from the ODE perspective by providing the analysis of the second order dynamics. We use the Lie-Trotter and the Strang-Marchuk splitting schemes to discretize the proposed ODE model, resulting in two different training algorithms. Experiments on the CIFAR-10 and CIFAR-100 benchmarks show that our methods improve adversarial model robustness on top of different widely-used strong adversarial training techniques.

heading: A A BRIEF INTRODUCTION TO THE SPLITTING SCHEMES
text: A.1 NUMERICAL BACKGROUND
Consider an ODE with two coupled terms in the right-hand side:
dx dt = F (x(t), t) +G(x(t), t), x(0) = x0. (17)
It is difficult to solve the ODEs with two coupled terms in the right-hand side. The splitting methods provide a natural way to decompose the coupled terms into individual calculation for different differential operators (McLachlan & Quispel, 2002). The simplest splitting scheme is the Lie-Trotter splitting scheme which alternatively calculates F (·) andG(·). The Lie-Trotter splitting scheme with the forward Euler method discretizes Eq. (17) as follows:
x̃(t) = x(t) + ∆tF (x(t), t),
x(t+ ∆t) = x̃(t) + ∆tG(x(t), t). (18)
The Lie-Trotter splitting scheme first solves the ODE with respect to F (·) to acquire the intermediate state x̃(t). Starting from x̃(t), it continues to solve the ODE with respect to G(·) to complete the discretization from time t to time t + ∆t. In the proposed ODE (Eq. (7)), we treat the f function as the operator F (·) and the dL/dx function as the operator G(·). From the formulation of the Lie-Trotter splitting scheme (Eq. (18)), we have the following discretization:{
x̃n+1 = xn + fn(xn), xn+1 = x̃n+1 + η ∂L
∂x̂n+1 ,
(19)
which is equivalent to our two-stage approximation approach (note that since we use the first order forward Euler method for each ODE, it is equivalent to either use ∂L/∂x̂n+1 or use ∂L/∂x̂n in solving G(·)). It is thus straightforward to see that Proposition 3.1 holds. The Strang-Marchuk splitting scheme extends the Lie-Trotter splitting scheme by dividing the onestep solver for G(·) into two half steps. Using the Strang-Marchuk splitting scheme to solve Eq. (17) yields
x̂(t) = x(t) + ∆t
2 G(x(t), t),
x̃(t) = x̂(t) + ∆tF (x̂(t), t),
x(t+ ∆t) = x̃(t) + ∆t
2 G
( x̃(t), t+ ∆t
2
) .
(20)
The Strang-Marchuk splitting scheme enjoys lower local truncation error than the Lie-Trotter splitting scheme (Bobylev & Ohwada, 2001). As a result, the Strang-Marchuk splitting scheme may lead to a more accurate solution in terms of solving the proposed ODE (Eq. (7)). In the next section we provide the proof for Theorem 3.2.
A.2 PROOF OF THEOREM 3.2
Applying the Strang-Marchuk splitting scheme (Eq. (20)) with step size ∆t = 1 to solve our ODE (7), we have the following algorithm: x̃n = xn + η 2 ∂L ∂x̂n , x̄n = x̃n + fn(x̃n),
xn+1 = x̄n + η 2 ∂L ∂x̂(n+1/2) .
(21)
In Equation (21), the essential part is to calculate ∂L/∂x̂(n+ 1/2). As shown in Eq. (9), our goal is to estimate a(n+ 1/2) with a(n) and a(n+ 1). As the learned filters in the deep layers converges, we can mildly treat ∂f(x̂, t)/∂x̂ in the RHS of Eq. (9) as a constant C. In this way, the adjoint dynamics is relaxed as a linear ODE, the solution of which reads as follows:
a(t) = exp(−Ct). (22)
Then we have the following relationship:
a2(n+ 1/2) = a(n) · a(n+ 1). (23)
As shown by Bhatia & Davis (1993), we have that the matrix form of the arithmetic-geometric mean inequality
2‖A1/2B1/2‖ ≤ ‖A+B‖ (24)
holds for any positive definite matrices A,B ∈ Rn×n and unitarily invariant norm ‖ · ‖. We thus further bound above a(n + 1/2) with (a(n) + a(n + 1))/2 in Eq. (23). Substituting the upper bound for a(n + 1/2) into Eq. (21) leads to Theorem 3.2. Notice that while our relaxation may have the potential negative effect on the accuracy of the splitting scheme itself, the formulation we provide differs from the LAD method and is easy to implement. Besides, slightly larger layerwise perturbations may also contribute to the adversarial model robustness. We leave more accurate algorithms implementing the Strang-Marchuk splitting scheme for future work.

heading: B EXPERIMENTAL SETTINGS
text: B.1 GENERAL SETTINGS
Following He et al. (2016), we pad 4 pixels on each side of the image and sample a 32 × 32 crop from it or its horizontal flip. We use pre-activated ResNet-56 as our backbone architecture and experiment with our LAD (Eq. (5)) and LAD-SM (Eq. (8)) methods. For all experiments, we use the SGD optimizer with the batch size of 128. We train for 160 (300) epochs for the CIFAR-10 (-100) benchmark; the learning rate starts with 0.1, and is divided it by 10 at 80 (150) and 120 (225) epochs. We apply weight decay of 1e-4 and momentum of 0.9. We determine the scaling factor η by cross-validation on training set, with η = 50 for CIFAR-10 and η = 1000 for CIFAR-100. For natural training, We set α = 0.5/255 and M = 20 in the IFGSM attack. For PGD training, we set α = 2/255, with iteration times M = 10 during PGD training and M = 20 for the IFGSM attacks. In the TRADES setting, we set λ = 1/6 and γ = 1/2. For other hyperparameters, we follow the settings in the PGD training experiments.
B.2 IMPLEMENTATION DETAILS OF OUR METHODS
In implementation, we set hooks on each layers in the neural architectures and perform a forward pass with the perturbed inputs to calculate the loss. Gradients back-propagated from the loss function are caught by the hooks on each layers. In another forward pass, the recorded gradients are further scaled and added to each layer following Eq. (5) or Eq. (8). Following conventional adversarial training algorithms, we add the infinity norm bounded constraints to the input perturbations.We propose to replace the (adversarial) loss term with the lately calculated loss function. Denote that for a given data point 〈x, y〉, the perturbed input is x′. The natural loss term and the adversarial loss term are L(f(x), y) and L(f(x′), y), respectively. Then for natural training, we replace the term L(f(x), y) with the resulted loss function LLAD/LAD−SM(f(x′), y); for PGD training, we replace the term L(f(x′), y) with LLAD/LAD−SM(f(x′), y); for TRADES training, we replace the term L(f(x), y) with γL(f(x), y) + (1− γ)LLAD/LAD−SM(f(x′), y), where γ is a hyperparameter for weighted averaging.

heading: C STRONGER ATTACKS
text: Table 6 shows the accuracy and robustness results of our methods as well as baselines with adversarial training under stronger attacks on the CIFAR benchmarks. It can be seen that our methods have consistently outrun the baseline methods under attacks of different strength levels.

heading: D RESULTS OF LAD WITH STOCHASTICITY ON CIFAR-100
text: In this section, we provide additional experimental results of LAD with stochasticity on CIFAR-100, shown in Table 7.

Review summaries:
This paper proposed a layer-wise adversarial defense which added perturbations in each hidden layer considering the influence of hidden features in latent space from the ODE perspective. It is essential to enhance the adversarial model robustness by stabilizing both inputs and hidden layers. The proposed method leveraged two operator splitting theory w.r.t. the Lie-Trotter and the Strang-Marchuk splitting schemes to discretize the specially designed ODE formulation by integrating the continuous limit of back-propagated gradients into the forward process. The main contribution of this paper is to generate perturbations with the idea of ODE in each layer. Empirical studies were performed to show the effectiveness of the proposed method on two benchmarks with two attack methods.