Adversarially Robust Neural Networks via Optimal Control: Bridging Robustness with Lyapunov Stability
Authors: Zhiyang Chen
Hang Su
Venue:  (2020)
Keywords: adversarial defense
optimal control
Lyapunov stability

Abstract: Deep neural networks are known to be vulnerable to adversarial perturbations. In this paper, we bridge adversarial robustness of neural nets with Lyapunov stability of dynamical systems. From this viewpoint, training neural nets is equivalent to finding an optimal control of the discrete dynamical system, which allows one to utilize methods of successive approximations, an optimal control algorithm based on Pontryagin's maximum principle, to train neural nets. This decoupled training method allows us to add constraints to the optimization, which makes the deep model more robust. The constrained optimization problem can be formulated as a semi-definite programming problem and hence can be solved efficiently. Experiments show that our method effectively improves deep model's adversarial robustness.

Under review as a conference paper at ICLR 2020
ADVERSARIALLY ROBUST NEURAL NETWORKS VIA OPTIMAL CONTROL: BRIDGING ROBUSTNESS WITH LYAPUNOV STABILITY
Anonymous authors Paper under double-bling brindd review
ABSTRACT
Deep neural networks are known to be vulnerable to adversarial perturbations. In this paper, we bridge adversarial robustness of neural nets with Lyapunov stability of dynamical systems. From this viewpoint, training neural nets is equivalent finding an optimal control of the discrete dynamical system, which allows one to utilize methods of successive approximations, on Pontryagin's maximum principle, to train neural nets. This decoupled training method allows model more robust. The constrained optimization problem can be formulated as semi-definite programming problem and hence can solved efficiently. Ex- periments show that our method effectively improves deep model's ' adversarial robustness.
optimal control algorithm based
add constraints the optimization, which makes the deep
INTRODUCTION
Deep neural networks achieve state-of-the-art performances on a variety tasks 2015). However, neural nets known be vulnerable to adversarial examples. Imperceptibly perturbed inputs can induce erroneous outputs neural nets (Szegedy et al. 2013). In image clas- sification problems of computer vision, previous work has proposed various methods to attack deep models and induce low accuracy (Goodfellow et al. 2015 Madry et al. 2017 Papernot et al. 2016a Carlini & Wagner 701 Whereas multiple delenses against adversarial attacks developed, they don't I ensure safety faced with strong attacking methods. There are also theories that explain the existence of adversarial examples (Ilyas et al., 2019 [Shamir al.) 2019), but they often fail to fully explain the features and behaviors of this phenomenon. This makes the study of adversarial attacks important that is a threat real-life machine learning systems (Kurakin et al., 2016).
(LeCun et al,
In this paper, we propose dynamical system view on the adversarial robustness of the models, as well as new method that significantly defense adversarial attacks.
Recent works have shown the connection between deep neural networks and dynamical systems (L. 2017 Li et al. 2017 Haber & Ruthotto 2017 Lu al. 2017). If we regard the neural net as discretization of ordinary differential equation (ODE), then training neural nets becomes finding an optimal control of the corresponding discrete dynamical system. Traditionally, we often treat training neural networks unconstrained non-convex optimization problem
min J(0) + R(0), 8E0
where 0 denotes the parameters of the model, denotes the loss function and denotes the reg- ularizer term, and we solve the problem with (stochastic) gradient-descent based methods (Bottou) 2010 Ruder) 2016). the training process, we feed the network with a batch training data, and compute the gradient with forward and backward propagation (E. Rumelhart et al.) 1986). The propagation process resembles solving optimal control problems that tune the parameters to make the output be close to target states. This viewpoint motivates to bridge adversarial robustness with Lyapunov stability of a dynamical system, and to train robust networks with algorithms that find stable optimal control. We will formulate the discussion in later sections.

---

Under review as a conference paper at ICLR 2020
2
RELATED WORK
2.1 ADVERSARIAL DEFENSE
Many defense methods have been proposed improve the models' adversarial robustness. The de- fenses mainly fall into three types: adversarial training (Szegedy et al. 2013 Zhang et al.) 2019).
modifying the networks (Gu & Rigazio 2015; Lyu al. 2015 Papernot | 2016b Nayebi &
Canguli, 2017: Ross & Deshi-Velez 2017). and adding external models (Lee et al. 2017 Akhtar et al., 2017 Gebhart & Schrater 2017 Xu et al. 2018) Sun et al. 2019). Although various de- fense methods have been developed, defended deep model often successfully attacked by newly developed attacks or specific counter-counter measures (Carlini & Wagner (017b). Therefore, be hoped that defenses against general attacks will be devised to make deep learning models (adversarially) robust to real-life threats.
2.2 NEURAL ODES AND OPTIMAL CONTROL
Recent works have bridged deep neural networks with ODEs and dynamical systems. On the one hand, deep residual networks (He et al. 2015) can be illustrated forward Fuler scheme approx- imating an ODE EE, 2017). which motivates us to design effective network structures (Lu et al.] 2017). On the other hand, regarding the network as dynamical system allows to set up op- timal control viewpoint of neural nets. Pontryagin' 's Maximum Principle (Boltyanskii et al. 1960)
has been applied to train neural nets (Li et al. 2017 Li & Hao 2018).
ADVERSARIAL ROBUSTNESS AND LYAPUNOV STABILITY
3.1 DYNAMICS OF DEEP NEURAL NETS
Given a T-yayer neural net, we let the dynamical system [fe 01) 0. O..... T) represents the network, where x; the imput of f-th layer, 0, is the parameter, and ft : Rd. O₁ 12 denotes the 1-th layer's transformation, which is usually non-linear function a0,,r + b.) for fully-connected layers, convolution layers and batch normalization layers, etc. Therefore, training the neural net can be regarded as controlling the parameters to let the dynamics fit the training data. Specifically, the training optimization problem can be formulated as a typical optimal control problem follows:
T
min Σ
L(..)
==1
subj. 10/44 2441 Selet, (c), 1- 0....., 1,
where we use to denote the i-th input in the batch and B denote the batch size. J/ and L are the loss function and the regularizer, respectively. Specially, if the model is deep residual network with structure r++1 = xt + (1,,, 0.), we can regard the problem as the forward Fuler discretization of the following continuous optimal control problem:
min [(x(T)) + [ I(((t)) dt, 0
subj. i f(t, (t), 0(t)), =00 x, O5t5T,
where st)) a continuous trajectory from the input to the output logits.
3.2 LYAPUNOV STABILITY
Adversarial examples are usually clean images added by small calculated perturbation 77. The model predicts correct labels fed with clean inputs so. while the output is completely different when it is fed with perturbed input ny n). The dynamical system view of neural nets motivate to
characterize this sensitivity with Lyapunov stability of system (Hirsch et al. 2004].
Definition 1 (Lyapunov Stability). For a given dynamical system & f(x), = Zg. F. is an equilibrium, then
2

---

Under review as a conference paper at ICLR 2020
The system is Lyapunov stable, if. V > 0, 38 0 such that, if [0(00) xell < 6, then for
every 2 0, ||[(t zell <
The system is asymptoricallly stable if it is Lyapunov stable and 3 6 > such that if [l=(00) Tell < 6, then lime [|x(()) Fell 0.
The system is exponentially stable if it is asymptotically stable and J > 0, B 0, such that if [|= Zell < 6, then x=(() - Iell S al|z((0) - xelle- 8t, for all t 2 0.
The definitions can be easily extended to discrete-time systems.
Intuitively, the Lyapunov stability states that for any small perturbation y), the trajectory is still "close enough" to the original one. If we regard a neural net as a dynamical system, and ensure the network Lyapunov stable, then the model is robust to all (adversarial) perturbations.
3.3 ADVERSARIALLY ROBUST NEURAL NETS
Due to the connection between numerical ODEs and residual networks, we first consider robustness (i.e. Lyapunov stability) of continuous ODEs.
Theorem (Stable ODEs). For a given ODE ft, (Ax b), where is the activation function, e.g., Sigmoid function or ReLU function, it is stable if Re)).(A)) I O, Vi, where Re denotes the real part, and X; denotes the i-th eigenvalue.
One can see, e.g. Hirsch et al. (2004), for the proof of this theorem.
Theorem provides a of conditions for stable ODEs. However, deep residual network is only a forward Euler discretization scheme of continuous ODE. To ensure numerical stability, we require 11- (A)h] S 1 (Ascher & Petzold 1998), where the step size by the identity mapping in residual networks, we can get the stable conditions for discrete dynamics.
in residual networks. Added
Theorem 2 (Stable Discrete Networks). For a discrete neural network, i.e. discrete dynamics (fe(x,, 0.) o,...,T Th where fa(z,, 0₂) (000z) (we omit the bias term for simplicity), the network is stable if the p(0,) S 1, where p(A) = max, ((a(()) is the spectral radius.
If the conditions are added to the unconstrained optimization problem of training, we can greatly improve the adversarial robustness of neural nets. The methods will be discussed in the following section.
TRAINING ROBUST NEURAL NETS
4.1 PMP AND MSA
For deterministic systems, the Pontryagin's Maximum Principle (PMP) (Boltyanskii et al. 1960)
provides set of necessary conditions for optimal control of the system. Various algorithms have been proposed to solve the deterministic optimal control problem based on PMP. Among them, the Method of Successive Approximations (MSA) (Krylov & Chernous' ko 1963) one of the simplest algorithms. In the field of deep learning, previous work has utilized MSA to train neural networks eL al. 2017 Li & Hao 2018).
Formally, consider the optimal control problem for training neural nets in section For dynam- ics [f((xz 0t) = O,..., T), assume
07 {0,....,0% a solution to the optimal con- trol problem. Also, we define the Hamiltonian function H Rd x Rdde+s x O₂ x [T] + TR by H (2 0, t) p- f( 0) L(O,), where the dot denotes the inner product. We have the following necessary conditions for 0*
Theorem 3 (Pontryagin's Maximum Principle for Discrete Systems). Assume ft and J are suffi- ciently smooth. There exists co-states p² (pi .T.} s.t. the following conditions hold:
zt+1 V, pi 0f t), 26 Tu.
P5 = V V H (x) 02, (), pr = -V V (xTr)
0 = arg max HI 1
3

---

Under review as a conference paper at ICLR 2020
For simplicity of notations, here we assume the batch size is 1. One can easily extend the theorem to minibatch training case by summing over the batch.
The theorem can be proved by KKT conditions (Boyd & Vandenberghe 2004). where the co-states can be seen as the Lagrangian dual variables.
Consider the conditions in PMP, one find the equations exactly the forward propagation of a neural net, and the p equations resemble the backward propagation process. The third condition states that the model parameters must maximize the Hamiltonian function. This motivates to iteratively compute forward and backward propagation, and solve the Hamiltonian maximization to find the optimal control, which is exactly the Method of Successive Approximations (Algorithm I In practice, we usually add regularizer terms that penalize great changes in the maximization step to prevent drastic steps that cause divergence. For the connection between MSA and back-propagation- based gradient descent algorithms, see the appendix of Li & Hao 2018).
Algorithm The Method of Successive Approximations
Initialize 0 [0]--. ... ) set k 0;
repeat
Compute the states (forward propagation); T++1 V H Pit1, 0g. t), t 0, ......" T 1; Pet1, Compute the co-states (backward propagation): pt V Petp 0f t), T -I,..., 0,
with initial Pr V (xTr)
For each = 0, T solve the maximization 0f+11 = arg maxp H (It, pt-1,
Set + 1;
until Converge;
The advantages of training by MSA compared with gradient descent algorithms has been discussed in (Li et al. 2017). among which the most significant feature is that the optimization steps differ- ent layers are decoupled. Concretely, after computing the states and co-states p, the optimization step on layer is only searching for parameters O.. This not only suggests that the optimization pro- cess can be accelerated by parallelization, but also allows us to utilize the features of the problem. The parameter space greatly reduced compared with the original intractable optimization prob- lem, and hence the optimization is much more easier. This allows us to add constraints that ensure robustness of the model.
4.2 ROBUST CONSTRAINTS
Consider layer in the form of f((z) 0,z, where leave the activation individual layer with no parameters for simplicity, we can derive the following optimization problem for Hamiltonian maximization:
max Pc+1 (0,z)) - 0 B1 0,112.
subj. to p(0.) 1,
where a ee|6,112 is the L2 norm regularizer (weight decay), and 8 is the initial parameter (i.e., 8f in the algorithm). The last term keeps the training process from drastic steps that cause divergence. The constraint, as illustrated in section 3 is the stable condition for discrete systems. makes the optimization quite difficult if we directly add the constraints in gradient descent based algorithms, but the decoupled optimization in MSA allows us to do so.
With regard the constraint of parameter's spectral radius, simple method is apply special forms of matrices for parameters. e.g. anti-symmetric matrices. For continuous deep models, the only constraint is Theorem [ i.e., (e(A;((:)) S O. Anti-symmetric matrices have only imaginary eigenvalues, and hence we can replace 0, with 0, of 71, where is a small positive constant.
For general forms of parameters, one can prove the following transformation.
Theorem 4. One sufficient condition of p(A)
< is
I
A
AT
I
2 0,
where A 2z B denotes A - B is positive semi-defiitt.
4

---

Under review as a conference paper at ICLR 2020
Table 1: Results of robust training CIFARIO.
Method
Vanilla
Ady Training
Ours
Clean data
80.73%
79.61%
74.49%
FGSM (Goodfellow et al. 2015
2.34%
77.45%
49.32%
PGD-I0 (adry et al. (017))
0.02%
46.67%
36.33%
C& W (Carlini & Wagner 2017a)
0.01%
21.15%
16.80%
Proof.
Recall that p(A) S || = Vmax(AA (ATA), A), we have
I AT
A I 2
All2 $ A
Hence can replace p(0.) < with positive semi-definite condition, and we turn the Hamilto- nian maximization into a new optimization problem, where the target function is quadratic and the constraint is a semi-definite condition. This can be reduced problem /Vandenberghe & Boyd 1998). which is special case of convex optimization, and thus can be solved efficiently by, c.g., interior point methods (Helmberg et al. 1970) in polynomial time.
semi-definite programming (SDP)
Here summarize method. For given neural network, we use MSA to train the model, i.e., iteratively computing the states (forward propagation) and co-states (backward propagation), and solving the optimization for each layer. Instead of directly maximizing the Hamiltonian, add positive semi-definite constraint to the optimization problem, which leads to a stable control of the dynamics.
EXPERIMENTS
5.1 EXPERIMENT SETUP
To evaluate the effectiveness of method, conduct experiments CIFAR10. We trained the network on clean data, with adversarial training (PGD-10) and with robust training (our method), respectively. We used FGSM (Goodfellow et al. 2015), PGD-10 (Madry al. 2017) and C&W
CCrllini & Wagner 2017a) to attack the network.
Due to the limitation of TensorFlow, used simple interior point method with gradient descent to solve SDP. The network model was an 18-layer residual network (He et al. 2015). with 8 residual blocks. We the perturbation size
0.1 for both FGSM and PGD. For C&W, we used the Lo metric. We trained the model for 150 epochs with batch size of 200. The learning rate was set to be 10-2 initially, and was divided by 5 at epoch 30, 60 and 100. The regularizer term constant was set to be 10-3.
5.2 RESULTS
The results can be seen in [able[]] The accuracy of robust models on clean data is lower than vanilla model's that robust training and generalization is more difficult and requires more data (Schmid! et al. 2018).
Our method improves model's adversarial robustness, compared with the vanilla model. Figure displays the cigenvalues of the last fully-com collneccddee layer's parameter. The complex of eigen- values (spectral radius) of the model trained by our method are effectively bounded below which satisfies the robust constraint parameters in section 4.2 while eigenvalues of natural training randomly distributed in the complex plane.
Our method is not effective traditional adversarial training method. However, it mainly has the following advantages: (a) The training process doesn't require large numbers of gradient prop- agation, which consumes much time in adversarial training. In our experiment, adversarial training

---

Under review as a conference paper at ICLR 2020
0.33
2.2
-004
026
Figure 1: Eigenvalues of the last fully-connected layer for robust training (left) and vanilla training (right).
spends about 10 times GPU time as much as our method. (b) The decoupled training process allows us different hyperparameters and training methods for different layers, which more maneu- verable for large scale training. We can further control the behavior of different layers in adversarial settings. (c) Lyapunov stability provides framework for analyzing adversarial robustmess of deep models, which may lead to theoretical analysis of adversarial samples in future work.
6 DISCUSSION AND FUTURE WORK
Motivated by the dynamical system view of neural networks, this work bridges adversarial robust- ness of deep neural models with Lyapunov stability of dynamical systems, and also propose method that uses a stable optimal control algorithm to train neural networks to improve the adversar- ial robustness of deep neural models. Though the result didn't surpass STOA defense methods, the stable control view of training neural nets points out another direction towards adversarially robust models.
For future work, on the one hand, mathematical analysis on Lyapunov stability of neural models may studied provide theoretical understanding of adversarial robustness. On the other hand, popular platforms for deep learning, e.g., TensorFlow, PyTorch, didn't provide frameworks for op- timal control. We will obtain better results if specific algorithms for SDP are applied solve the optimization problem.
REFERENCES
N. Akhtar, J. Liu, and A. Mian. Defense against universal ad adversarial perturbations. arXiv preprint arXiv: hit.:1711.09999 2017.
Uri M. Ascher and Linda R. Petzold. Computer Methods for Ordinary Differential Equations and Differential-Algebre Equations. Society for Industrial and Applied Mathematics, Philadelphia, PA, USA, 1st edition, 1998. ISBN 0898714125,
V.G. Boltyanskii, R.V. Gamkrelidze, and L.S. Pontryagin. The theory of optimal processes. the maximum principle. Translations. Series 2. American Mathematical Sociery., 18:1, 10 1960.
Lon Bottou. Large-scale machine learning with stochastic gradient descent. Proc. of COMPSTAT, 01 2010. doi: 01.10079978-3-7908 7908-2604-3_16.
Stephen Boyd and Lieven Vandenberghe. Convex Optimization. Cambridge University Press, 03 2004. ISBN 0-521-83378-7.
Nicholas Carlini and David Wagner. Towards evaluating the robustness of neural networks. arXiv preprint 1608. 046444, 05 2017a.
Nicholas Carlini and David Wagner. Adversarial examples are not easily detected: Bypassing ten detection methods. arXiv preprint arXiv: 1705 2017b.

---

Under review as a conference paper at ICLR 2020
Weinan E. A proposal on machine learning via dynamical systems. Communications in Mathematics and Statistics, 5:1-11, 02 2017. doi: 10.1007/44030-011-000332
David E. Rumelhart, Geoffrey E. Hinton, and Ronald J. Williams. Learning representations by back propagating errors. Nature, 123:533-536, 10 1986. doi: 10. 10.1008/323533.0.
T. Gebhart and P. Schrater. Adversary detection in neural networks via persistent homology. arXiv preprint arXiv:/711.10055 2017.
Ian Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. arXiv preprint arXiv:1412.6657,, 2015.
S. Gu and L. Rigazio. Towards deep neural network architectures robust adversarial examples. arXiv preprint arXiv:1412.5068 2015.
Eldad Haber and Lars Ruthotto. Stable architectures for deep neural networks. Inverse Problems, 34, 05 2017. doi: 10. 10.1088//1661--422/aaa999
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. arXiv preprint 151.v:152 2015.
Christoph Helmberg, Franz Rendl, Robert Vanderbei, and Henry Wolkowicz. An interior pointt method for semidefinite programming. SIAM Journal on Optinization, 6. 02 1970. doi: 10.1137/ 0806020.
Morris Hirsch, Stephen Smale, and Robert L Devaney. Differential Equations, Dynamical Sys- tems and an Introduction to Chaos, volume 14. Academic Press, 2004. doi: 10.1016/ C2009-0-61100-0.
Andrew lyas, Shibani Santurkar, Dimitris Tsipras, Logan Engstrom, Brandon Tran, and Aleksander Madry. Adversarial examples are not bugs, they are features. arXiv preprint S.KIv19955002111 05 2019.
I.A. Krylov and R.L. Chernous'ke ko. On the method of successive approximations for solution of optimal control problems. USSR Computational Mathematics and Mathematical Physics, 2: 1371 1382, 12 1963. doi: 10. 101 00.100660041-555530 600004415553333390355377
A. Kurakin, I. Goodfellow, and S. Bengio. Adversarial examples in the physical world. arXiv preprint 1607.0000..02555 2016.
Yann LeCun, Y Bengio, and Geoffrey Hinton. Deep learning. Nature, 521:436-44, 05 2015. doi: 10.1038/nature11553
Hyeungill Lee, Sungyeob Han, and Jungwoo Lee. Generative adversarial trainer: Defense adver- sarial perturbations with gan. arXiv preprint 105.03375, 05 2017.
Qianxiao Li and Shuji Hao. An optimal control approach to deep learning and applications to weight neural networks. arXiv preprint / 03 2018
Qianxiao Li, Long Chen. Cheng Tai, and E Weinan. Maximum principle based algorithms for deep learning. Journal of Machine Learning Research, 18, 10 2017.
Yiping Lu, Aoxiao Zhong, Quanzheng Li, and Bin Dong. Beyond finite layer neural net- works: Bridging deep architectures and numerical differential equations. arXiv preprint arXiv:1710.1012. 2017.
Chunchuan Lyu, Kaizhu Huang, and Hai-Ning Liang. A unified gradient regularization family for adversarial examplsssilexxmmplssss arXiv preprint arXiv: 151.066511. 2015.
Alcksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. arXiv preprint arXiv:1706.0600 06088, 06 2017.
A. Nayebi and S. Ganguli. Biologically inspired protection of deep networks from adversarial at- tacks. arXiv preprint urXivv/703.000 2017.
7

---

Under review as a conference paper at ICLR 2020
Nicolas Paperno, Patrick McDaniel, Somesh Jha, Matt Fredrikson, Z. Berkay Celik, and Ananthram Swami. The limitations of deep learning in adversarial settings. arXiv preprint 2016a.
Nicolas Papernot, Patrick McDaniel, Xi Wu, Somesh Jha, and Ananthram Swami. Distilla- tion as a defense to adversarial perturbations against deep neural networks. arXiv preprint arXiv:1606.04435. 2016b.
Andrew Ross and Finale Doshi Velez. Improving the adversarial robustness and interpretability of deep neural networks by regularizing their input gradients. arXiv preprint 1.1004471..09444. 2017.
Sebastian Ruder. An overview of gradient descent optimization algorithms. arXiv preprint (609.004044447 2016.
Ludwig Schmidt, Shibani Santurkar, Dimitris Tsipras, Kunal Talwar, and Aleksander Madry. Ad- versarially robust generalization requires more data. arXiv preprint arXiv:1804. 11285, 2018.
Adi Shamir, Itay Safran, Eyal Ronen, and Orr Dunkelman. A simple explanation for the existence of adversarial examples with small hamming distance. arXiv preprint arXiv: 1901. 10861, 01 2019.
Ke Sun, Zhanxing Zhu, and Zhouchen Lin. Enhancing the robustness of deep neural networks by boundary conditional gan. arXiv preprint arXiv:1902.110299 2019.
Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus. Intriguing properties of neural networks. arXiv preprint arXiv:1312.6199 2013.
Lieven Vandenberghe and Stephen Boyd. Semidefinite programming. SIAM Review, 38, 10 1998. doi: 10.1137/1038003.
Weilin Xu, David Evans, and Yanjun Qi. Feature squeezing: Detecting adversarial examples in deep neural networks. arXiv preprint arXiv: 155, 01 2018.
Dinghuai Zhang, Zhang Tianyuan, Yiping Lu, Zhanxing Zhu, and Bin Dong. You only propagate once: Accelerating adversarial training via maximal principle. arXiv preprint 1r0Xiv:905.000 05 2019.
8


Reviews:
[{"review_id": "rkg4GwQTYH", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The paper contributes to the robust training of neural networks as follows:\n  1) The paper uses the theoretical view of a neural network as a discretized ODE to develop a robust control theory aimed at training the network while enforcing robustness;\n  2) Such an objective is achieved by introducing Lyaponov stability and practically implemented through the method of successive approximations;\n  3) Empirical evaluation demonstrate that the newly introduced method performs as well as the SOTA in terms of defensive training.\n\nThe paper is well written and proposes a well motivated and theoretically original strategy to robustly train neural networks against adversarial examples.\nThe strength of the paper is definitively in its theoretical section, it would be really great to see an empirical improvement improvement on the SOTA.\nHowever, I do not believe the paper should be penalized for only matching other algorithm as it relies on a tractable and principled theoretical analysis.", "extra_scores": {"experience_assessment": "I have read many papers in this area.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper at least twice and used my best judgement in assessing the paper.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I assessed the sensibility of the derivations and theory."}, "extra_text": {}}, {"review_id": "S1luKEChtB", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 1, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Summary:\nThe goal of this paper is to train neural networks (NNs) in a way to be robust to adversarial attacks. The authors formulate training a NN as finding an optimal controller for a discrete dynamical system. This formulation allows them to use an optimal control algorithm, called method of successive approximations (MSA), to train a NN. The authors then show how constraints can be added to this optimization problem in order to make the trained NN more robust. They show that the resulted constraint optimization problem can be formulated as a semi-definite programming and provide some experimental results. \n\nComments:\n- Although the problem studied in the paper is important and the approach is interesting, it seems the paper has been written in rush and in my opinion is not ready for publication. The writing is not good. The introduction and related work sections are incomplete and not very informative. It is not clear what has been done before and what is the contribution of this paper. The main technique/algorithm of the paper has not been explained clearly that someone can easily understand and implement it. The experimental results are not convincing. \n- There are strong claims in the paper such as \"experiments show that our method effectively improves deep model's adversarial robustness\", this is too strong, given the quality of the experiments of the paper. Or \"the constraint optimization problem can be formulated as a semi-definite programming (SDP) problem and hence can be solved efficiently\", to the best of my knowledge, SDP solvers are limited to small problems and cannot solve the large problems efficiently. \n- The area of making NNs robust to attacks is a very active area and there are many attacks and solutions out there, which require more comprehensive empirical studies of any new method. I do not see this in the paper. \n- Overall, I think this paper requires a major revision in order to be evaluated better and to be more useful for the community. ", "extra_scores": {"experience_assessment": "I have read many papers in this area.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper at least twice and used my best judgement in assessing the paper.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I assessed the sensibility of the derivations and theory."}, "extra_text": {}}, {"review_id": "HygB_6lkqB", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 1, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Neural Networks are vulnerable to adversarial perturbations. This paper proposes a method that based on optimal control theory that uses semidefinite-programming. This is a quite popular topic in Adversarial training recently, there has been a few works in that line. There are almost no experiments in this paper. There are several typos in the paper and writing of this paper requires more work. There are several typos in this paper, for example STOA, should be SOTA (in the Section 6.) In its current state, this paper looks very rushed.\n\n\nAs Yiping Lu pointed out, the PMP statement in this paper is also wrong. At this current stage, unfortunately this paper doesn’t meet the standards of ICLR. I would recommend the authors to go over the paper carefully and resubmit to a different venue.\n\n\n", "extra_scores": {"experience_assessment": "I have read many papers in this area.", "review_assessment:_thoroughness_in_paper_reading": "I made a quick assessment of this paper.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I assessed the sensibility of the derivations and theory."}, "extra_text": {}}]

Author rebuttal:

Meta-review: The authors propose a framework for improving the robustness of neural networks to adversarial perturbations via optimal control techniques (Lyapunov Stability and the Pontryagin Maximum Principle, in particular). By considering a continuous-time limit of the training process, the authors use the PMP to derive udpate rules for the neural network weights that would result in a robust network. While the approach is interesting, the paper has some serious deficiencies that make it unacceptable for publication in its current form:

1. Quality of empirical evaluation: The authors only report final numbers on CIFAR-10 for a fixed set of adversarial attacks. It has been observed repeatedly in the adversarial robustness literature that adversarial evaluation of neural networks has to be done carefully to guard against possible underestimation of the worst-case attack. In particular, the specific details of the adversarial attacks used (number of steps, number of initializations, performance under larger perturbation radii) that are necessary to trust the results are not given (see https://arxiv.org/pdf/1902.06705.pdf for example).

2. Unclear novelty: The authors do not sufficiently explain the novelty in their approach relative to prior work (particular prior work that has used optimal control ideas in this context).

3. Computational cost: The authors do not give sufficient details to judge the computational overhead of their method to judge how much more expensive it is to train with their approach relative to standard or adversarial training.

While one reviewer voted for a weak accept, the other reviewers were in consensus on rejection. The authors did not respond during the rebuttal phase and hence the reviews were unchanged.

In summary, I vote for rejection. However, I think this paper has potentially interesting ideas that should be carefully developed and evaluated in a future revision.
Decision: Reject