Layer-wise Adversarial Defense: An ODE Perspective
Authors: Zonghan Yang
Yang Liu
Chenglong Bao
Zuoqiang Shi
Venue:  (2021)
Keywords: adversarial training
robustness
ODE

Abstract: Deep neural networks are observed to be fragile against adversarial attacks, which have dramatically limited their practical applicability. On improving model robustness, the adversarial training techniques have proven effective and gained increasing attention from research communities. Existing adversarial training approaches mainly focus on perturbations to inputs, while the effect of the perturbations in hidden layers remains underexplored. In this work, we propose layer-wise adversarial defense which improves adversarial training by a noticeable margin. The basic idea of our method is to strengthen all of the hidden layers with perturbations that are proportional to the back-propagated gradients. In order to study the layer-wise neural dynamics, we formulate our approach from the perspective of ordinary differential equations (ODEs) and build up its extended relationship with conventional adversarial training methods, which tightens the relationship between neural networks and ODEs. In the implementation, we propose two different training algorithms by discretizing the ODE model with the Lie-Trotter and the Strang-Marchuk splitting schemes from the operator-splitting theory. Experiments on CIFAR-10 and CIFAR-100 benchmarks show that our methods consistently improve adversarial model robustness on top of widely-used strong adversarial training techniques. 

Under review as a conference paper at ICLR 2021
LAYER-WISE ADVERSARIAL DEFENSE: AN ODE PER- SPECTIVE
Anonymous authors Paper under double-blind review
ABSTRACT
Deep neural networks are observed to be fragile against adversarial attacks, which have dramatically limited their practical applicability. On improving model ro- bustness, the adversarial training techniques have proven effective and gained in- creasing attention from research communities. Existing adversarial training ap- proaches mainly focus perturbations to inputs, while the effect of the perturba- tions in hidden layers remains underexplored. In this work, we propose layer-wise adversarial defense which improves adversarial training by a noticeable margin. The basic idea of our method is to strengthen all of the hidden layers with pertur- bations that are proportional to the back-propage propagated gradients. In order study the layer-wise neural dynamics, we formulate our approach from the perspective of ordinary differential equations (ODEs) and build up its extended relationship with conventional adversarial training methods, which tightens the relationship between neural networks and ODEs. In the implementation, we propose two dif- ferent training algorithms by discretizing the ODE model with the Lie Lie-Trotter and the Strang-Marchie splitting schemes from the operator-splitting theory. Experi- ments on CIFAR-10 10 and CIFAR- 100 benchmarks show that our methods consis- tently improve adversarial model robustness on top of widely-used strong adver- sarial training techniques.
INTRODUCTION
Recent years have witnessed the prosperity of deep learning in many tasks (Hinton & Salakhutdinov 2006 Sutskever et al. 2014 He et af., 2016: LeCun et al. 2015] Huang et al.] Vaswani al,,
2017 Stacked with multiple layers, neural networks provide an end-to-end solution to all the tasks and prove to be highly effective. However, the seminal study by Szegedy et al. (2013) has shown that deep neural networks (DNNs) can be fragile against attacks: minor perturbations on inputs lead to significant change in model predictions. Regarding the defense approaches, intensive studies on adversarial defense techniques have been proposed (Athalye al., 2018a Goodfellow et al. 2014
Zheng et al. 2016 Madry et al. 2201 2018: et al. [2019]] Kurakin et al. 2017] et al. 2019a
2020 2019b: Raff al. 2019] Guo 2018| /hang al. 2020al Balunovic & Vechev 2019
Wong et al. 2020 Chan et al. 2020 Zhang et al. [2220b). Among these techniques, adversarial training algorithms (Madry al. 2018 Zhang et al.) (019b) incorporate the effect of perturbed inputs into the loss function, which are shown to be competent and boasts the dominant impact in the adversarial defense research field.
While adversarial training techniques have gained increasing attention in the robust deep learning research community, most of current approaches concentrate on deriving perturbations on the inputs with gradients pack-propagated from the loss function. However, as information flow in neural networks starts from inputs and passes through hidden layers, it is essential to robustify both the inputs and the hidden layers. While previous studies have made successful attempts on introducing damping terms (Yang et al. 2020) or stochastic noise (Liu et al.) 2020 Wang et al., 2019) to each layer in neural architectures. they concentrate improving general model robustness and are less focused on adversarial model robustness. We ask the following question: Can we take the hidden layers of neural networks into account to improve adversarial model robustness?
In this work, we propose wise adversarial defense to improve adversarial training, which en- hances adversarial model robustness by stabilizing both inputs and hidden layers. In our method,

---

Under review as a conference paper at ICLR 2021
the layer-wise perturbations are incorporated into the robust optimization framework of adversarial training. We propose to inject scaled back- propagate gradients into the architecture as layer-wise perturbations. Besides, we formulate our method from the perspective of ordinary differential equa- tions and propose a novel ODE as its the continuous limit in order to study the neural dynamics. In- spired from the rich literature numerical analysis, we use the Lie-Trotter and the Strang-Marchie splitting schemes to solve the proposed ODE. We refer to the resulted discrete algorithms as Layer- wise Adversarial Defense (LAD) and LAD-SM, respectively. Furthermore, we build up the extended relationship between our methods with current natural training and adversarial training techniques by analyzing the second order dynamics. Our analysis shows that our methods have introduced additional perturbations in the first order initial value of the second order dynamics compared with current adversarial training Expprtmetst on the CIFAR-10 10 and CIFAR- 100 benchmarks show that our methods improve adversarial model robustness on top of different widely-used strong adversarial training techniques.
We summarize contributions follows:
We propose layer-wise adversarial defense which generalizes conventional adversarial training approaches with layer-wise adversarial perturbations (Section 3.1);
We investigate the continuous limit of our layer-wise adversarial defense methods and pro- pose an ODE that integrates the adjoint state into the forward dynamics (Section 3.2);
We build up the extended relationship between our methods and current adversarill training approaches by analyzing the second order neural dynamics in theory. Experiments have also shown the effectiveness of our methods in practice. (Section 3.3 and Section 4
2 RELATED WORK
2.1 ADVERSARIAL MODEL ROBUSTNESS
In this section review the literature gradient-based attack and defense approaches in the field of adversarial model robustness. For adversarial attacks, widely-used approaches include Fast Gradient Sign Method (FGSM) (Goodfellow al. 2015) and Iterated Fast Gradient Sign Method (IFGSM) (Madry et al., 2018). For a given data point, FGSM induces the adversarial example by moving with the attack radius each component along the gradient ascent direction. Iterated FGSM performs FGSM with inner iteration updates with smaller step size a. Prior studies have inspired multiple adversarial attack techniques (Athalye et al., 2018b; Carlini & Wagner 2017 Iyas al., 2018 Dong et al.) 2018 Pang et al., 2018). Adversanil defense techniques can be categorized by training phase (Athalye et al. 20] Goodfellow et al. 2014 Zheng et al. 2016 Madry al. 2018
Zhang et al. 2019b; Kurakin et al. 20.7 et al. 2019a 2020gZhang et al. [2220a] Balunovic &
Vechev 2019: Wong et 2020 Chan et al. 2020 Zhang et al.] 2020b) and inference phase (Pang
al. 2019b Raff et al. 2019; Xie et al. 2018 Guo al. 2018).
The widely-used approach in training phase Projected Gradient Descent (PGD) training (Madry et al., 2018)), which integrates the effect of the perturbed inputs into its loss function. The cur- rent state-of-the-ar defense approach in training phase is TRADES (Zhang et al. 2019b), which additionally introduces the boundary eиrоr regularization term into its loss function. In periments, we select PGD training and TRADES as our baselines. While substantially enhancing adversarial model robustness, the gradient-based perturbations in adversarial training are currently only performed on inputs. As cascaded hidden layers comprise the passage for information flow in neural networks, it is essential to stabilize hidden layers as well. In our work, we introduce layer- wise gradient-based perturbations to neural architectures to improve adversarial model robustness.
2.2 ODE-INSPIRED ARCHITECTURE DESIGNS
Research about the relationship between neural networks and ODEs starts with the continuous limit formulation of ResNet (E) 2017). which has inspired many novel neural architecture designs (Lu 2018/Chang
et al.
2018 Zhu et al. 2018 Chang et al. 2018 Haber & Ruthotto 2017: Chen et al. 2018: Dupont
ct al. 2019). Regarding model robustness, most prior studies have focused on improving dynamic
system stability by Lyapunov analysis, more stable numerical schemes, or imposing regularization.
2

---

Under review as a conference paper at ICLR 2021
-:
|ل|لا hidden layers
Loss
Loss
perturbed inputs
perturbed inputs
perturbed hidden layers
Figure I: Left: Conventional adversarial training techniques. Right: Our layer-wise adversarial de- fense methods. In conventional adversarial training algorithms, the perturbations are only performed on the input side with gradients back propagated through the architecture. In our layer-wise adver- sarial defense methods, the gradients on each layers are recorded along backpropagation, which are further added to each layers layer-wise perturbations in another forward pass.
From Lyaponov stability perspective, Yang et al. (2020) introduce damping terms to residual net- works to stabilize dynamic systems. Appropriately adjusting the damping factor introduces damping effect to the dynamic system and enhances model robustness. Similarly, Chang et al. (2019) improve Lyapunov stability of RNNs by imposing antisymmetric constraints on the weight matrix. On more stable numerical schemes, prior studies include taking small step sizes in the forward Euler scheme (Zhang al. 20T9c) or leveraging implicit schemes enhance stability (Reshniak & Webster 2019 et al. 2020). For imposing regularization, stochasticity is introduced into ODEs for stabilization (Liu et al. 2020: Wang et al. 2019). Hanshu al. (2019) regularize neural ODE models be time- invariant and add regularization term about the upper bound of the effect from input perturbation.
Zhang et al. (2019a) propose differential game formulation for adversarial training and accelerate the process from the optimal control theory. Our work differs from the prior studies by integrating gradient-based perturbations into the neural dynamics, which proves to be an extension to current approaches on improving adversarial model robustness in both theory and practice.
3
LAYER-WISH ADVERSARIAL DEFENSE
3.1 THE MODEL FORMULATION
The objective of conventional adversarial training approaches can be formulated into a min-max problem (Madry et al. 2018 8 Zhang et al. 2019a). We introduce layer-wise perturbations (Ax, ) and rewrite the min- max problem as follows:
min (0, J.
max L(O,, Ax) L(xN)
(Ax, J.
subject to
Xo = X + Axo.
Xn+1 = Xn + f(xn, 0n),
K = Xn+1 Ax,+1, n 0,1,2,
where N is the number of layers in a neural network, @ = {0,) J.m represent its trainable pa- rameters and the [Ax,) J represent the layer-wise perturbations. In our formulation, ignore the bounded assumptions on the perturbations for simplicity, since if there are additional bounded constraints on the perturbations, we can project the gradient onto the intervals. is noted that when Ax,, = 0 for all n I,...,N, .N, the model (I) reduces to the conventional adversarial train- ing formulation. More specifically, for adversarial training algorithms (Madry et al., [018)) /Zang et al. (2019h), let M be the maximum number of inner iterations for the perturbations, we have the following update rule (ignoring the bounded constraints):
(m+1)
(m)
ØL
(0) X
OxL
Xo:
(m)
where the perturbation Ax₂ is defined as (M)
Xg-
In this work, we generalize the above idea to introducing perturbations to hidden layers, i.e. deter- mine 4 in Eq. (1) so that the objective in Eq. (1) is maximized. Similar to Eqq. (2). perturb

---

Under review as a conference paper at ICLR 2021
Xn+1 with iterative gradient descent:
(m++1)_ (m) #+1 -xx+1 + 0x,+1 (m)
ØL
(0) Xa+1 Xn+11
3
where 7) is the step size as a scaling factor over the gradients 0L/ /L/0xx+1- We set Ax,+1 x x
Xn+1, where M is the number of the iteration steps. When M 1, the layer-wise adversarial
perturbations are given by
OL
Ax x+1 Xx+1 = "xx+1"
n = 0, 1,,, .. , NN11.
Replacing the maximization of layer-wise perturbations {Ax, 1. by Eq. (4). we obtained a sim- plified problem follows.
min L(O) := L(xx) (0.
subject Xn+1 Xx, + fx Qn),
(5)
OL
Xx+1 = 8x+1 + 08n+11 n n= ...
with the inputs Xo = Xg x Ax₂ determined by Eq. 4 Notice that directly applying Eq. ( requires alternative computations of Xx+1 and 0L/0x,,1 with iterative forward and backward passes, which can be extremely cime-consuning. In our implementation, we leverage two-stage approach: first record the gradients with respect each layer in forward and backward pass, then add the recorded gradients to each layer in another forward pass as layer-wise adversarial perturbations. We refer to this algorithm our Layer-wise Adversarial Defense (LAD) method.
3.2
ALGORITHM DESIGN
In this section, we explore the continuous formulation of the constraints in Eq. 5 to study the layer- wise dynamics from the ODE perspective. Recall that the conventional ODE formulation E 2017) takes the following form:
dx(t) = (x((),,, x(0) = Xo:
(6)
dt
which is the continuous limit of the discrete ResNet x S2 + fn (Xt, with the time step At 1. In the conventional ODE, = L(x(T)) is defined as the loss function. In our work, we propose to integrate dL/dd((t) into the forward process:
dx(t) dt
AL [x())) x(0) Xo.
f(x((),))
where Xo is the (perturbed) inputs and 17 is scaling factor: The introduced d.L/d(()) represents the continuous limit back- fack-propagated gradients. As we record the gradients from an earlier backward pass, the corresponding forward pass can be approximately treated solving the original ODE. As shown in Eq. Z.. there are two operators on the right hand side. According to the rich literature on the operator sperator-plliting theory from numerical analysis, we have the following proposition.
Proposition 3.1. The LAD method is the numerical solution of our proposed ODE with the Lie- Trotter splitting scheme with step size At
In the operator splitting theory, the Strang-Marchut Marchuk (SM) splitting scheme (Ascher & Petzold 1998) is also a widely-used technique of solving ODE with multiple terms. Compared with the Lie- Trotter splitting scheme. the
splitting scheme enjoys much lower local truncation errors (Bobylev& & Ohwadal 2001). Lu et al. (2019) propose to leverage the SM splitting scheme to improve the Transformer architecture, which results in higher model accuracy. We also propose to use the SM splitting scheme discretize our ODE in Eq. (7). but the direct application of SM method is intractable. With proper approximation, we have the following modified version.
Theorem 3.2. An approximated numerical scheme Eq. with the SM splitting is
Xn Xx, + 1 aL 98.
8n = Xn fn(()))
(8)
Xn+1 Xx, + 7
ar.
BL. 38
4

---

Under review as a conference paper at ICLR 2021
The proofs and self-contained introduction to the numerical background provided in Appendix We refer to Eq. ( as our LAD-SM method.
3.3 ANALYSIS OF THE SECOND ORDER DYNAMICS
In this section, we provide analysis of the second order dynamics and connect our ODE (Eq. 7 with the original ODE (Eq. (6)). Define at) dT)/dx(t), known that the dynamics of a
(Pontryagin al. 1962 Chen et al. 2018) satisfies
da -
0x( t)
a(()
(9)
di
0x
The next theorem presents the second order dynamics of the proposed Eq. 7
Theorem 3.3. The second order dynamics of Eq. 7 given by
d2x
tf(x,t)
0f(x,t) t) 1(x,1)
0f(x t)
0f(x t)
01 na((1)
(10)
d12
0x
8x
0%
with
dx
dL
x(0)
Xo,
dt
f(x(0),0 + +x
dx(0)
(11)
Proof. By the direct computation, we know the second order dynamics of Eq. is
d2x
0f(x,t)
dx 0f(xtt)
01 ( 0f(xtt T((x1))
(12)
d12
0x
di
0&
05(x,1) (f(X, t) +na((t) + 0f(x, -na(t)) (8( ()
(13)
0x
ØL
8%
11(x 1)
0s(x,
+ na(t)² (0](x,1)
(
0f(x; 1)
f(x,t)
(14)
0x
Ot
8x
0&
where the first equality is from Eq.
9 and the second equlaity is from Eq.
Moreover, the second order dynamics of Eq.
66) is given by
d'
i f
0f(x,t)
f(x, t)
dt
0x
(15)
d&
with x(0) =
Xo,
f(xo, 0).
dt
t=
Since is small perturbation of x well the existence of rja(t,, the last term in Eq. (10
be negligible and the main difference of the seccond order dynamics of our ODE (Eq. (7) with the original ODE (Eq. (6) lies in the first order initial values. The extra momentum of the input in
Eq. leads to extra perturbations in all of the hidden layers during the propagation process. The
implementation details of our methods can be found Appendix B
4 EXPERIMENTS
We evaluate proposed methods the CIFAR (Krizhevsky et al. 2009) benchmarks. For each experiment, we conduct 3 runs with different random seeds and report the averaged result to reduce the impact of random variations. We also select Yang et al. (2200) as baseline methods, which introduce layer-wise damping terms to each layer in neural networks. The details experimental settings can be found in Appendix B
5

---

Under review as a conference paper at ICLR 2021
4.1 LAD WITH NATURAL TRAINING
We first evaluate our methods in the setting of natural training (He et al., 2016). This setting is equivalent to setting Axo 0 in layer-wise adversarial defense framework (Eq. ( Table 4.1 shows the accuracy and robustness results of our methods composed with natural training under different attack radii
Table I: Accuracy and robustness results of natural training baselines as well as our methods on CIFAR-10 and CIFAR-I00. The "Avg" column reports the averaged robustness results for each methods under different attacks of all the radii. "Natural" denotes ResNet with natural training;
"damp" and "A-damp" represent In-ResNet and X-In-ResNet in Yang al. (2220). respectively.
FGSM
IFGSM
Benchmark
Method
Acc
Avg
0.5/255
17255
0.5/255
17255
natural
92.89
67.22
49.80
56.98
19.20
48.30
natural + damp
92.77
70.91
53.17
64.26
27.95
54.07
CIFAR-10 10
natural + X-damp
92.90
70.75
52.70
63.73
26.68
53.47
natural + LAD
93.04
71.79
51.18
66.02
28.27
54.32
natural + LAD-SM
93.05
70.91
50.82
65.20
27.36
53.57
natural
71.35
33.27
21.25
24.15
5,54
21.05
natural + damp
70.89
35.75
22.10
28.14
7.35
23.34
CIFAR-100
natural + X-damp
70.44
35.83
22.13
27.94
7.42
23.33
natural + LAD
70.21
39.74
22.95
33.81
9.90
26.60
natural + LAD-SM
70.28
40.33
23.92
34.74
11.30
27.57
Results show that our methods perform best under most attack settings with small radii While all methods fail under attacks with larger radii because of the vulnerability of natural training, re- sults under attacks with small radii have still shown the enhanced robustness by only introducing perturbations on hidden layers.
4.2 LAD WITH PGD TRAINING
In this section, report our experiments with PGD Training (Madry al. 2018). Table shows the results of our methods and the baseline techniques with PGD training.
Table 2: Accuracy and robustness results of baselines as well as our methods with PGD training on the CIFAR-10 and CIFAR-101 henchmarks. The "Avg" column reports the averaged robustness results for each methods under differrntt attacks of all the radii. "PGD damp" and "PGD X-damp" represent Re-ReNNet and -n-ReeNNet in Yang et al. (2020) with adversarial training, respectively.
FGSM
IFGSM
Benchmark
Method
Acc
Avg
8/255
12/255
8/255
12/255
PGD
82.14
53.55
42.48
44.79
25.46
41.57
PGD damp
82.12
53.39
41.96
44.43
25.31
41.27
CIFAR-10 10
PGD X-damp
82.58
53.89
42.40
44.51
25.41
41.55
PGD LAD
82.03
53.99
42.37
46.07
26.88
42.33
PGD LAD-SM
81.67
53.79
42.60
46.21
27.45
42.51
PGD
56.85
23.71
16.10
17.74
8.52
16.52
PGD damp
53.16
19.89
13.22
12.40
5.22
12.68
C00
PGD X-damp
56.14
19.27
13.06
12.03
5.51
12.47
PGD LAD
58.11
26.70
17.99
22.44
11.85
19.75
PGD LAD-SM
57.97
26.79
17.94
22.39
11.58
19.68
According to the experimental results, is shown that our methods consistently outperform the baselines in the robustness performance. Another interesting finding is that in experiments. approaches achieve more significant improvements over baselines on the CIFAR-100 benchmark. We provide possible interpretation: from the discrete neural network perspective, the stacked

---

Under review as a conference paper at ICLR 2021
layers represent different levels of representation learning. While methods have perturbed each layers, can be interpreted as an augmentation with adversarial "examples" in each levels feature learning. Given that each class in the CIFAR- 100 training set consists only 500 images (which is far less than that the CIFAR-10 training set (5,000 images)), is inferred that our methods have a potential positive effect in the data-scarce regime.
4.3 LAD WITH TRADES
In this section, we report our experimental results with the TRADES adversarial training approach (Zhang et al., 2019b). Table 3 shows the results of our methods composed with the TRADES method. According to the results shown in Table 2 we only select TRADES with the original ResNet architecture as our baseline methods. Results show that our methods have also surpassed the baseline methods top of the state-of-the-an TRADES technique. We also provide robustness results of our methods as well as the baselines under stronger attacks in Appendix
Table 3: Accuracy and robustness results of the baselines as well as our methods with TRADES defense on the CIFAR-I0 and CIFAR-100 benchmarks. The "Avg" column reports the averaged robustness results for each methods under different attacks of all the radii.
FGSM
IFGSM
Benchmark
Method
Acc
Avg
8/255
12/255
8/255
12/255
TRADES
79.08
54.70
45.09
49.85
34.69
46.08
CIFAR-10
TRADES + LAD
78.47
55.19
45.53
50.80
35,86
46.85
TRADES LAD-SM
78.49
55.13
45.26
50.68
35.60
46.67
TRADES
52.53
28.51
21.38
25.86
16.87
23.16
CIFAR-100
TRADES + LAD
52.46
29.30
21.70
27.10
17.83
23.98
TRADES LAD-SM
52.60
29.16
21.78
26.97
17.78
23.92
4.4 LAD WITH STOCHASTICITY
Prior studies have shown that model robustness is improved with layer-wise noise injection as reg- ularization (Liu et al. 2020; Wang et al.) 2019). We also experiment on introducing stochasticity to our methods in order to further boost the robustness performance. We augment our experiments with PGD training with additive Gaussian noise, which is proposed by Wang al. (2019). During training. Gaussian noise perform forward pass with noise sampled for 100 times and take the averaged output logits as the expectation outputs. Table 4 shows the results on the CIFAR-10 benchmark. As shown from the results, our approaches consistently surpass the baseline approaches under both the deterministic and the stochastic settings.
(0,, 0.01) is added to each layer in the model. During inference,
Table 4: Comparison of accuracy and robustness results of baselines and our methods with PGD training between the deterministic and the stochastic versions on CIFAR 10. The "Avg" column reports the averaged robustness results for each methods under different attacks of all the radii.
FGSM
IFGSM
Method
Acc
Avg
8/255
12/255
8/255
12/255
PGD
82.14
53.55
42.48
44.79
25.46
41.57
PGD + noise
82.48
53.77
42.54
44.63
25.51
41.61
PGD + LAD
82.03
53.99
42.37
46.07
26.88
42.33
PGD + LAD-SM
81.67
53.79
42.60
46.21
27.45
42.51
PGD + LAD + noise
81.79
54.08
42.41
46.34
27.16
42.50
PGD + LAD-SM + noise
81.99
54.35
43.00
46.49
27.54
42.85

---

Under review as a conference paper at ICLR 2021
4.5 THE EFFECT OF
In this section, we introduce how we determine the scaling factor 1)- The hyperparameter v scales the back-propagated gradients, which further added to the network layer-wisely. We start get basic knowledge about the proper range of 1 by comparing the norms of back-propan pack-propagated gradients with the norms of each layers. According Eq. (II. our methods have essentially perturbed the first order initial value in the view of the second order dynamics. As result, we let
xx(0) (() 0)ll
ratio
(16)
dr
dx(O)
and trace the ratio during training.
As Figure 2 shows, the ratio is ultimately large at the beginning of training (around 2 10%) and becomes smaller during train- ing. While it achieves its minimum at around the end of the training, the ratio is always larger than x 10 As a result, the ratio serves as quantitative comparison between the norm of the back-propagated gradient with the norm of the layer in a network. It suggests that the should not be too small; otherwise the layer-wise per- turbations can be too small to affect the training. Model robustness results under different 17's have also proven the finding. As Table 5 shows, the effect on adver- sarial model robustness from small r is marginal compared with that from a larger n) Table 5 also shows that y) should not be too large. A too large v may unstabi- lize the training process and deteriorate the model robustness.
2000
70
1750
60-
1500
50
(1E3)
1250
1000
30
20
750
10
500
230
240
250
260
270
280
290
300
250
0
50
100
150
200
250
300
Epoch
Figure 2: The change of the ratio (Eq. 16 during
PGD training on the CIFAR-100 benchmark.
Table 5: Robustness results under FGSM and IFGSM attacks with radius 8/255 the CIFAR- 100 benchmarks. Our methods are equivalent to the ResNet baseline when v7 = 0. It can be seen that n should be neither too small nor too large. When 7) grows larger in proper range, the adversarial model robustness is gradually improved.
Attack
Method
10
100
1,000
10,000
LAD
23.02
24.97
23.09
26.70
17.07
FGSM
23.71
LAD-SM
24.49
22.43
24.31
26.79
17.87
LAD
17.42
19.59
18.01
22.44
15.20
IFGSM
17.74
LAD-SM
18.97
15.98
19.54
22.39
15.30
CONCLUSION
In this work, we propose layer-wise adversarial defense as an extension current adversarial train- ing approaches. The hidden layers are robustified by the introduced layer-wise perturbations, which are proportional to the back propapropag gradients. We build up the extended relationship our methods with conventional adversarial training methods from the ODE perspective by providing the analysis of the second order dynamics. We use the Lie-Tree Tee-Treter and the Strang- Marchuk splitting schemes discretize the proposed ODE model, resulting in two different training algorithms. Ex- periments on the CIFAR- 10 and CIFAR- 1-00 benchmarks show that our methods improve adversarial model robustness top of different widely-used strong adversarial training techniques.
8

---

Under review as a conference paper at ICLR 2021
REFERENCES
Uri M Ascher and Linda R Petzold. Computer methods for ordinary differential equations and iifferential-aggerraie equations, volume 61. Siam, 1998.
Anish Athalye, Nicholas Carlini, and David Wagner. Obfuscated gradients give a false sense se- curity: Circumventing defenses to adversarial examples. In International Conference on Machine Learning, pp. 274-283, 2018a.
Anish Athalye, Logan Engstrom, Andrew Ilyas, and Kevin Kwok. Synthesizing robust adversarial examples. In International conference on machine learning, pp. 284-293. PMLR, 2018b.
Mislav Balunovic and Martin Vechev. Adversarial training and provable defenses: Bridging the gap. In International Conference on Learning Representations, 2019.
Rajendra Bhatia and Chandler Davis. More matrix forms of the arithmetic- wrthmetii-geommericc mean inequal- ity. SIAM Journal on Matrix Analysis and Applications, 14(1):132-136) 1993.
Alexander V Bobylev and Taku Ohwada. The error of the splitting scheme for solving evolutionary equations. Applied Mathematics Letters, 14(1):45-44) 48, 2001.
Nicholas Carlini and David Wagner. Towards evaluating the robustness of neural networks. In 2017 ieee symposium on security and privacy (sp), pp. 39-57. IEEE, 2017.
Alvin Chan, Yi Tay, Yew Soon Ong, and Jie Fu. Jacobian adversarially regularized networks for robustness. International Conference on Learning Representations, 2020.
Bo Chang, Lili Meng, Eldad Haber, Lars Ruthotto, David Begert, and Elliot Holtham. Reversible architectures for arbitrarily deep residual neural networks. In AAAI, 2018.
Bo Chang, Minmin Chen, Eldad Haber, and Ed H Chi. Antisymmetricrm:: A dynamical system view recurrent neural networks. In ICLR, 2019.
Ricky T Q Chen, Yulia Rubanova, Jesse Bettencourt, and David Duvenaud. Neural Ordinary Dif- ferential Equations. In NeurIPS, 2018.
Yinpeng Dong, Fangzhou Liao, Tianyu Pang. Hang Su, Jun Zhu, Xiaolin Hu, and Jianguo Li. Boost- ing adversarial attacks with momentum. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 9185-9193, 2018.
Emilien Dupont, Arnaud Doucet, and Yee Whye Teh. Augmented neural odes. In NeurlPS, pp- 3134-3144, 2019.
Weinan E. proposal on machine learning via dynamical systems. Communications in Mathematics and Statistics, 5(1):1-11, 2017.
Ian J Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. arXiv preprint arXiv:1412.6572, 2014.
Ian J Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. In ICLR, 2015.
Chuan Guo, Mayank Rana, Moustapha Cisse, and Laurens van der Maaten. Countering adversarial images using input transformations. In International Conference on Learning Representations, 2018.
Eldad Haber and Lars Ruthotto. Stable architectures for deep neural networks. Inverse Problems, 34(1):014004, 2017.
YAN Hanshu, DU Jiawei, TAN Vincent, and FENG Jiashi. On robustness of neural ordinary differ- ential equations. In ICLR, 2019.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Identity mappings in deep residual networks. In ECCV, pp. 630-645. Springer, 2016.

---

Under review as a conference paper at ICLR 2021
Geoffrey E Hinton and Ruslan R Salakhutdinov. Reducing the dimensionality of data with neural networks. science, 313(5786):504-500 2006.
Gao Huang, Zhuang Liu, Laurens Van Der Maaten, and Kilian Q Weinberger. Densely connected convolutional networks. In Proceedings of the IEEE conference computer vision and pattern recogmition, pp. 4700-4708, 2017.
Andrew Ilyas, Logan Engstrom, Anish Athalye, and Jessy Lin. Black-box adversarial attacks with limited queries and information. In International Conference on Machine Learning, pp. 2137- 2146, 2018.
Alex Krizhevsky et al. Learning multiple layers of features from tiny images. 2009.
Alexey Kurakin, Ian Goodfellow, and Samy Bengio. Adversarial machine learning at scale. In International Conference on Learning Representations, 2017.
Yann LeCun, Yoshua Bengio, and Geoffrey Hinton. Deep learning. nature, 521(7553))3364444 2015.
Mingjie Li, Lingshen He, and Zhouchen Lin. Implicit euler skip connections: Enhancing adversarial robustness via numerical stability. In Proceedings of ICMI., 2020.
Xuanqing Liu, Tesi Xiao, Si Si, Qin Cao, Sanjiv Kumar, and Cho-Jui Hsich. How does noise help robustness? explanation and exploration under the neural sde framework. In Proceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recognition (CVPR), June 2020.
Yiping Lu, Aoxiao Zhong, Quanzheng Li, and Bin Dong. Beyond Finite Layer Neural Networks: Bridging Deep Architectures and Numerical Differential Equations. In /CML, 2018
Yiping Lu, Zhuohan Li, Di He, Zhiqing Sun, Bin Dong, Tao Qin, Liwei Wang, and Tie- Yan Liu. Understanding and improving transformer from a multi-particle dynamic system point of view. 1rXi::1906.02762,1 2019.
Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. In International Conference on Learning Representations, 2018.
Robert McLacchan and G Reinout W Quispel. Splitting methods. Acta Numerica, 11:341, 2002.
Tianyu Pang, Chao Du, Yinpeng Dong, and Jun Zhu. Towards robust detection of adversarial exam- ples. In Advances in Neural Information Processing Systems, pp. 4579-4589, 2018.
Tianyu Pang, Kun Xu, Chao Du, Ning Chen, and Jun Zhu. Improving adversarial robustness via promoting ensemble diversity. In International Conference on Machine Learning, pp. 4970-4979, 2019a.
Tianyu Pang. Kun Xu, and Jun Zhu. Mixup inference: Better exploiting mixup to defend adversarial attacks. In International Conference on Learning Representations, 2019b.
Tianyu Pang, Kun Xu, Yinpeng Dong, Chao Du, Ning Chen, and Jun Zhu. Rethinking softmax cross- entropy loss for adversarial robustness. In International Conference on Learning Representations, 2020.
Lev Semenovich Pontryagin, EF Mishchenko, VG Boltyanskii, and RV Gamkrelidze. The mathe- matical theory of optimal processes. 1962.
Edward Raff, Jared Sylvester, Steven Forsyth, and Mark McLean. Barrage of random transforms for adversarially robust defense. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 6528-6537, 2019.
Viktor Reshniak and Clayton Webster.
Robust learning with implicit residual networks.
190. 10479, 2019.
Ilya Sutskever, Oriol Vinyals, and Quoe V. Le. Sequence to sequence learning with neural networks. In Proceedings of NIPS, 2014.
10

---

Under review as a conference paper at ICLR 2021
Christian Szegedy, Wojeiech Zaremba, llya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus. Intriguing properties of neural networks. arXiv preprint arXiv:1312.6 2013.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. In Advances in neural information processing systems, pp. 5998-6008, 2017.
Bao Wang, Binjie Yuan, Zuoqiang Shi, and Stanley J Osher. Enresnet: Resnet ensemble via the feynman-kac formalism. In Proceedings of NewrIPS, 2019.
Eric Wong, Leslie Rice, and Zico Kolter. Fast is better than free: Revisiting adversarial training.
In International Conference on Learning Representations, 2020.
Cihang Xie, Jianyu Wang, Zhishuai Zhang, Zhou Ren, and Alan Yuille. Mitigating adversarial
effects through randomization. In International Conference on Learning Representations, 2018.
Zonghan Yang, Yang Liu,, Chenglong Bao, and Zuoqiang Shi. Interpolation between residual and non-residual networks. Proceedings of ICML, 2020.
Dinghuai Zhang, Tianyuan Zhang, Yiping Lu, Zhanxing Zhu, and Bin Dong. You only propagate once: Painless adversarial training using maximal principle. In NeurIPS, 2019a.
Hongyang Zhang, Yaodong Yu, Jiantao Jiao, Eric Xing, Laurent El Ghaoui, and Michael Jordan. Theoretically principled trade-off between robustness and accuracy. In Proceedings of ICML, pp. 7472-7482, 2019b.
Jingfeng Zhang, Bo Han, Laura Wynter, Kian Hsiang Low, and Mohan Kankanhalli. Towards robust resnet: A small step but giant leap. In ICAII, 2019c.
Jingfeng Zhang, Xilie Xu, Bo Han, Gang Niu, Lizhen Cui, Masashi Sugiyama, and Mohan Kankan- halli. Attacks which do not kill training make adversarial learning stronger. In International Conference Machine Learning, 2020a.
Jingfeng Zhang, Xilie Xu, Bo Han, Gang Niu, Lizhen Cui, Masashi Sugiyama, and Mohan Kankan- halli. Attacks which do not kill training make adversarial learning stronger. In International Conference on Machine Learning. 2020b.
Stephan Zheng, Yang Song, Thomas Leung, and lan Goodfellow. Improving the robustness of deep neural networks via stability training. In Proceedings of the IEEE Conference Computer Vision and Puttern Recognition, pp. 4480-4485 2016.
Mai Zhu, Bo Chang, and Chong Fu. Convolutional Neural Networks combined with Runge- Kutta Methods. arXiv.org February 2018

---

Under review as a conference paper at ICLR 2021
APPENDIX
A A BRIEF INTRODUCTION TO THE SPLITTING SCHEMES
A.1 NUMERICAL BACKGROUND
Consider an ODE with two coupled terms in the right-hand side:
dx = F(x G(x(), x(0) = X0-
(17)
dt
is difficult to solve the ODEs with two coupled terms in the right-han side. The splitting methods provide a natural way to decompose the coupled terms into individual calculation for different dif- ferential operators (Melachhan & Quispel. 2002). The simplest splitting scheme the Lie-Trotte splitting scheme which alternatively calculates F(.) and G(-). The Lie-Trotter splitting scheme with
the forward Euler method discretizes Eq. (17) as follows:
x(t) x(t) + AtF(((t), t), x(1 AI) x() + AIG(X(1)(1)
(18)
The Lie-Trotter splitting scheme first solves the ODE with respect to F(-) to acquire the intermediate state x(t). Starting from x(t), it continues solve the ODE with respect to G(-) complete the discretization from time t to time + At. In the proposed ODE (Eq. (7). we treat the f function as the operator F-) and the dT/dxx function as the operator G(-). From the formulation of the Lie- iie-Trotter splitting scheme (Eq. 18 ), we have the following discretization:
Xn+1 Nn fn (XC,)
(19)
Xn+1 xn+1 1 + 1) 28.
0L
which is equivalent to our two-stage approximation approach (note that since we use the first order forward Euler method for each ODE, it is equivalent either 0L/08n+1 solving G(-)). is thus straightforward to see that Proposition 3.1 holds.
0L/0%, in
The Strang- Marchuu splitting scheme extends the Lie-Trotter splitting scheme by dividing the step solver for G(-) into two half steps. Using the Strang-Marchuk splitting scheme to solve Eq. 17 yields
x(1) = x(1) - At
2 G(x(1),)),
x(1) x(1) AIP()()),))
(20)
At G (x(t),++ At 2
x(t + At) = x(1
2
The Strang-Marchuk splitting scheme enjoys lower local truncation error than the Lie-Trotter split- ting scheme (Bobylev Ohwadal (2001). As result, the Strang-Marchuk splitting scheme may lead to more accurate solution in terms of solving the proposed ODE (Eq. (7)). In the next section we provide the proof for Theorem 3.2
A.2 PROOF OF THBOREM 3.2
Applying the Strang-Marchie Marchuk splitting scheme (Eq. (20) with step size At = 1 to solve our ODE we have the following algorithm:
Xn Xx, OL
88..
Xn Xn + fn (x,),
(21)
OL X = X, 0x(221/2)
12

---

Under review as a conference paper at ICLR 2021
In Equation 21 the essential part is calculate aL/0 1/2). As shown in Eq. Q our goal is to estimate a + 1/2) with a(n) and a(n + 1). As the learned filters the deep layers converges, mildly treat 0f t)/0& in the RHS of Eq. (9) as constant C. In this way, the adjoint
dynamics is relaxed as a linear ODE, the solution of which reads as follows:
a(t) = cxp( Ct).
(22)
Then we have the following relationship:
a² + 1/2) = =a(n) a(n 1).
(23)
As shown by Bhatia & Davis (1993). have that the matrix form of the arithmetic-geometrie mean inequality
2| S I|A B
(24)
holds for any positive definite matrices A, B E R=x= and unitarily invariant norm ||- We thus further bound above a(n + 1/2) with (a(n) + a(n 1) in Eq. (23). Substituting the upper bound for a(n + 1/2) into Eq. (2I) leads to Theorem 3.2 Notice that while our relaxation may have the potential negative effect on the accuracy of the splitting scheme itself, the formulation we provide differs from the LAD method and is easy to implement. Besides, slightly larger layer- wise perturbations may also contribute to the adversarial model robustness. We leave more accurate algorithms implementing the Strang-Marchhe splitting scheme for future work.
B EXPERIMENTAL SETTINGS
B. GENERAL SETTINGS
Following He al. (016)), we pad 4 pixels on each side of the image and sample 32 crop from it or its horizontal flip. We use pre-activated ResNet-56 as our backbone architecture and experiment with our LAD (Eq. (5) and LAD-SM (Eq. (8) methods. For all experiments, we use the SGD optimizer with the batch size of 128. We train for 160 (300) epochs for the CIFAR-10 (-100) benchmark; the learning rate starts with 0.1, and is divided it by 10 at 80 (150) and 120 (225) epochs. We apply weight decay of lc-4 and momentum of 0.9. We determine the scaling factor by cross-validation on training set, with natural training, We set 0.5/255 and M 20 in the IFGSM attack. For PGD training, we set 2/255, with iteration times M = 10 during PGD training and M = 20 for the IFGSM attacks. In the TRADES setting, we À 1/6 and 7 1/2. For other hyperparameters, follow the settings in the PGD training experiments.
50 for CIFAR-10 and = 1000 for CIFAR- For
B.2 IMPLEMENTATION DETAILS OF OUR METHODS
In implementation, we set hooks on each layers in the neural architectures and perform a forward pass with the perturbed inputs to calculate the loss. Gradients back-propagated from the loss function are caught by the hooks on each layers. In another forward pass, the recorded gradients are further scaled and added to each layer following Eq. (5) Eq. (8). Following conventional adversarial training algorithms, we add the infinity norm bounded constraints to the input perturbations. We propose to replace the (adversarial) loss
with the lately calculated loss function. Denote that for given data point (x. v). the perturbed input is x'. The natural loss term and the adversarial loss term are L (xx(),. y) and L(f(x'), (fxxx)))) .). respectively. Then for natural training. we replace the term (xf(),, y) with the resulted loss function LLAD/LAD sM (f(x'), ((x),,); y); for PGD training, we replace the term L(f(x), y) with LLAD/LAD- (xx)))))(((()))))) for TRADES training, we replace the term L (xxx)) y) with (f(()x)) + (1 LL (x(x)) y). where 7 is a hyperparameter for weighted averaging.
13

---

Under review as a conference paper at ICLR 2021
C STRONGER ATTACKS
Table 6 shows the accuracy and robustness results of our methods as well as baselines with adver- sarial training under stronger attacks on the CIFAR benchmarks. It can be seen that our methods have consistently outrun the baseline methods under attacks of different strength levels.
Table 6: Accuracy and robustness results of our methods as well as baselines under attacks of different strength levels on the CIFAR-10 and CIFAR-100 benchmarks.
Benchmark
Method
Clean
IFGSM-20
IFGSM-100
IFGSM- 1000
PGD
82.14
44.79
44.25
44.18
PGD + LAD
82.03
46.07
45.64
45.58
PGD + LAD-SM
81.67
46.21
45.90
45.82
CIFAR-10 10
TRADES
79.08
49.85
49.67
49.62
TRADES + LAD
78.47
50.80
50.66
50.64
TRADES + LAD-SM
78.49
50.68
50.57
50.54
PGD
56.85
17.74
17.39
17.27
PGD + LAD
58.11
22.44
22.29
22.26
PGD + LAD-SM
57.97
22.39
22.21
22.17
CIFAR-100
TRADES
52.53
25.86
25.80
25.78
TRADES + LAD
52.46
27.10
27.05
27.05
TRADES LAD-SM
52.60
26.97
26.92
26.92
D RESULTS OF LAD WITH STOCHASTICITY ON CIFAR-100
In this section, we provide additional experimental results of LAD with stochasticity on CIFAR-100 shown in Table 7
Table 7: Comparison of accuracy and robustness results of baselines and our methods with PGD training between the deterministic and the stochastic versions on CIFAR-100, The "Avg" column reports the averaged robustness results for each methods under different attacks of all the radii.
FGSM
IFGSM
Method
Acc
Avg
8/255
12/255
8/255
12/255
PGD
56.85
23.71
16.10
17.74
8.52
16.52
PGD + noise
55.86
23.35
15.54
17.71
8.21
16.20
PGD + LAD
58.11
26.70
17.99
22.44
11.85
19.75
PGD + LAD-SM
57.97
26.79
17.94
22.39
11.58
19.68
PGD + LAD + noise
58.14
26.48
17.98
22.44
11.77
19.67
PGD + LAD-SM noise
57.54
26.28
17.70
22.19
11.35
19.38
14


Reviews:
[{"review_id": "jHGVFM8A1KZ", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The authors proposed a new adversarial training scheme based on ODE techniques. The standard adversarial training seeks for small perturbations in the input space. But the perturbations can be found in the feature space. From the observation and the similarity between the layer-wise adversarial attacks and ODE formulation, the optimization scheme was developed.\n \nClarity:\nThe paper is written well. Overall, it reads well. If the pseudocode of the two proposed methods is provided, it would be easier to understand the proposed schemes. The authors may save some space leaving out the proof of Theorem 3.3.\n \nStrengths/Quality/Significance (pros):\nThe authors studied an interesting construction. The proposed methods show that the adversarial training can be generalized to hidden representations. The generalization can be viewed as ODE formulations.\n \nTo optimize the model parameters efficiently, using the operator-splitting theory from numerical analysis,  \nefficient numerical schemes can be developed using ODE theories. Discussion about Lie-Trotter Splitting Scheme and Strang-Marchuck splitting scheme was interesting.\n \nIn addition, the resulting algorithms achieved competitive performance compared to state-of-the-art methods.\n \nThe effectiveness of the proposed method with natural training methods is significant compared to vanilla natural training.\n \nWeaknesses (cons) & Questions:\nAuthors claimed that  Eq 1. is inefficient to optimize and derive their numerical schemes based on techniques in the literature of ODE/Numerical analysis. The authors did not show the efficiency of the proposed methods. Even with a small dataset and small neural networks, if authors provide the comparison with Naïve approaches in terms of elapsed time/memory consumption and time/space complexity, it will be easier to evaluate the value of the proposed methods.\n \nThe effect of approximated schemes in terms of numerical stability and adversarial robustness needs to be discussed.\n \nSince the bound for perturbation is omitted, the starting point (the optimization formulation) is unbounded. If authors tighten the loose end and bridge between the optimization problem and ODE-inspired numerical schemes, it will be more useful. Unlike the input space (image space), it is much trickier to define small perturbations to preserve the labels of samples. If the authors can fill this gap, it might be possible to provide a new definition of adversarial attacks.\n \nIt is confusing. In Figure 2, the smallest number (ratio) looks smaller than 10^5, unlike the discussion. Clarify this.\n \nThe performance gain against TRADES is table 3 is marginal.\n \nNo experimental results are available to verify the theoretical analysis of the second-order dynamics. Is it possible to design experiments to check whether the models/numerical schemes show the behavior?", "extra_scores": {}, "extra_text": {}}, {"review_id": "huPKGfzAdO", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper proposed a layer-wise adversarial defense which added perturbations in each hidden layer considering the influence of hidden features in latent space from the ODE perspective. It is essential to enhance the adversarial model robustness by stabilizing both inputs and hidden layers. The proposed method leveraged two operator splitting theory w.r.t. the Lie-Trotter and the Strang-Marchuk splitting schemes to discretize the specially designed ODE formulation by integrating the continuous limit of back-propagated gradients into the forward process. The main contribution of this paper is to generate perturbations with the idea of ODE in each layer. Empirical studies were performed to show the effectiveness of the proposed method on two benchmarks with two attack methods.\n\nThere are several concerns suggesting that this paper may not be accepted at its current form.\n1.\tNovelty is a concern. \na)\tTo improve the limitation that the existing adversarial training approaches mainly focus on perturbations to inputs, this paper added perturbations in each hidden layer with the ODE perspective, which is not convincing enough to me. \nb)\tThe authors did not state the significance compared with  the other layer-wise perturbation methods. Why it is efficient to use the ODE method adding perturbation in each layer. \nc)\tAt least, the authors should compare with the other adversarial training methods in a layer-wise way, e.g. the following two references.\n\nRef1. Sankaranarayanan, S.; Jain, A.; Chellappa, R.; and Lim, S. N. 2018. Regularizing deep networks using efficient layerwise adversarial training. In 32nd AAAI Conference on Artificial Intelligence. \n\nRef2. Kumari, Nupur, et al. \"Harnessing the Vulnerability of Latent Layers in Adversarially Trained Models.\" IJCAI. 2019\n2.\tAnother concern is that the paper may be in lack of sufficient and convincing experimentation to support the usefulness of the proposed method.\na)\tThe authors only selected one baseline method. As for the defense against attack method, this paper just used FGSM and IFGSM which are quite weak attacking methods. The proposed method may also be compared under strong attack methods, such as PGD and CW methods, since this paper targets defense. \nb)\tFurthermore, the results may not be significantly enough to verify the effectiveness of the proposed method.\nc)\tIn Table1, did you use the same ResNet with the baseline, ResNet-110 or ResNet-164? It appears that the results in the paper are quite different from those reported in the original paper of Yang et al.\n3.\tThe authors mentioned that they leveraged a two-stage approach to solve the time-consuming problem during training in section 3.1. There are not clear descriptions about this approach how it works and the authors did not explain why it could save time.\n4.\tIn equation (1) and (5), some notations of x are different from each other. The authors did not give clear annotation which is confusing to me. The function f is not defined as well.\n5.\tIn the experimental part, the authors did not conduct black-box experiments which are import in adversarial training.\n6.\tThe paper could be further polished. There are quite a few typos. Figure 1 is not mentioned in the full paper. Table 4.1 should be Table 1 in the first paragraph in section 4.1.\n", "extra_scores": {}, "extra_text": {}}, {"review_id": "OGi1coe8XVg", "reviewer": "AnonReviewer4", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "### Summary\nThe paper proposes to improve the robustness of residual networks (ResNet) by adversarial training with layer-wise and gradient-based perturbations. It further offers an interpretation of the proposed perturbations as ordinary differential equations with the lenses of the operator-splitting theory, resulting in two techniques (either using Lie-Trotter or Strang-Marchuk splits) for approximately computing the perturbations and training the model.\n\n### Novelty and significance\nContrary to what stated in the paper, **layer-wise and gradient-based techniques have already been proposed** and analyzed in the past. For instance, layer-wise and gradient-based perturbations were first proposed in [1] and with a two stage approach, similar to the LT split approximation described here. Also in [2] the authors analyze the link between input perturbations and layer-wise perturbations.\n**The real novelty of the paper is its interpretation of such perturbations on ResNet using ODE, and its link to split decompositions.** \nThis interpretation is interesting and significant as it provides theoretical foundation to the approximation schemes. However **its analysis in comparison with input perturbations falls short**:\n1. it assumes that the model's output and the loss don't change much for small perturbations, which is not satisfied by common architectures, otherwise we wouldn't need to study adversarial examples and defenses;\n2. it doesn't provide insights on why perturbing the intermediate representations improves robustness.\n\n### Experiments\nThe paper carries out several experiments in the white box setting, where **the proposed techniques implemented with state-of-the-art adversarial attacks are shown to improve model robustness** overall and to perform better than methods that are tailored to  improve generic robustness. \nThe setup and results seem sound and the study of the perturbation scale $\\eta$ is useful.\nOf course, also in the experiments the proposed methods should be compared to existing gradient-based and layer-wise defenses that have been ignored.\nFinally, it's not clear from the main text why the SM approximation works better than the LT one in practice.\n\n### Quality and clarity\nThe paper is generally well written. It sometimes lacks of mathematical rigor and some important notations are missing:\n1. In Problem (1) $L$ and $f$ are not defined and it should be clearly stated that the formulation is specific to ResNets. \n2. In Equation (6) $\\hat{x}$ is not defined. \n3. $x$ and $\\widetilde{x}$ are interchangeably used across the paper for original or perturbed inputs, which is confusing.\n\n[1] Sankaranarayanan, Swami, et al. \"Regularizing deep networks using efficient layerwise adversarial training.\" AAAI (2017)\n[2] Chen, Xiaoyi and N. Zhang. “Layer-wise Adversarial Training Approach to Improve Adversarial Robustness.” IJCNN (2020).", "extra_scores": {}, "extra_text": {}}, {"review_id": "912ssCqJXz0", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Summary\n\nThis paper proposes to extend the commonly used input perturbation in adversarial training to layer-wise perturbation, which adversarially perturbs all hidden layer inputs. It further analyses the perturbed dynamics from the perspective of ODE, and proposes two implementations based on different numerical schemes. These algorithms are empirically tested on CIFAR-10 and CIFAR-100 datasets.\n\nI think this paper contains many interesting ideas, but they are not yet presented as a coherent story. In addition, the empirical results are not convincing enough to justify the authors claims. Therefore, I think the ideas here need further development before the paper can be accepted at ICLR.\n\nPros\n1. This paper is overall clearly written, and provides enough background material on numerical analysis.\n2. The ODE-based analysis is interesting, and this direction may provide further insight into related work.\n\nCons\n1. A major problem is that the layer-wise perturbation is not well motivated and justified. Since adversarial attacks are only performed on the inputs, it is unclear to me why perturbing all hidden layers is necessary (deemed “essential” in multiple places in the paper) for the defense.  E.g., would hidden representations adapt from input perturbation through training?\n2. The analysis, although interesting, is a bit handwavy. For example, why the last term in eq. 10 “can be negligible”? This claim needs to be justified by either proofs or numerical simulation that shows this term is indeed much smaller than others. In addition, what does the conclusion (“the main difference … lies in the first order initial values”) imply, and can it be experimentally verified?\n3. The empirical results are not convincing enough. First, the reported numbers were averaged from 3 random seeds, while the deviations were not given. This is problematic given that the improvements from LAD or LAD-SM are relatively small -- without the standard deviation, it is difficult to evaluate their significance. In addition, as an extension of input perturbation, is LAD(-SM) comparable with PGD? It would be helpful to show columns cross Table 1, 2 and 3 with the same attack radii for such comparison.\n\nOther comments\n\nThe comparison of different splitting schemes is interesting, but it is unclear why the Strang-Marchuk splitting scheme should be better in practice, since the ODE (eq. 7) is obtained from the limit of a discrete system. The experiments indeed show that LAD-SM was not superior in many cases. I would be helpful to discuss this.", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: This submission aims to improve adversarial training by making it involve also layer-wise (instead of only input-wise) perturbations. This is an interesting idea and it is accompanied by an interesting ODE-based perspective on the resulting dynamics. However, as the comments and reviews detail, the current manuscript misses the discussion of very relevant previous work, does not specify important details of the approach (e.g., how to bound the extent of the perturbations used), and relies on weak primitives (FGSM vs PGD). 

The consensus is that this would be an interesting and valuable contribution but only after addressing the above shortcomings. 
Decision: Reject