Venue: ICLR 2020 Conference

Reviews:

final_score: rating: 1: Reject
review_content: Neural Networks are vulnerable to adversarial perturbations. This paper proposes a method that based on optimal control theory that uses semidefinite-programming. This is a quite popular topic in Adversarial training recently, there has been a few works in that line. There are almost no experiments in this paper. There are several typos in the paper and writing of this paper requires more work. There are several typos in this paper, for example STOA, should be SOTA (in the Section 6.) In its current state, this paper looks very rushed.


As Yiping Lu pointed out, the PMP statement in this paper is also wrong. At this current stage, unfortunately this paper doesn’t meet the standards of ICLR. I would recommend the authors to go over the paper carefully and resubmit to a different venue.



review_title: Official Blind Review #2

final_score: rating: 6: Weak Accept
review_content: The paper contributes to the robust training of neural networks as follows:
  1) The paper uses the theoretical view of a neural network as a discretized ODE to develop a robust control theory aimed at training the network while enforcing robustness;
  2) Such an objective is achieved by introducing Lyaponov stability and practically implemented through the method of successive approximations;
  3) Empirical evaluation demonstrate that the newly introduced method performs as well as the SOTA in terms of defensive training.

The paper is well written and proposes a well motivated and theoretically original strategy to robustly train neural networks against adversarial examples.
The strength of the paper is definitively in its theoretical section, it would be really great to see an empirical improvement improvement on the SOTA.
However, I do not believe the paper should be penalized for only matching other algorithm as it relies on a tractable and principled theoretical analysis.
review_title: Official Blind Review #1

final_score: rating: 1: Reject
review_content: Summary:
The goal of this paper is to train neural networks (NNs) in a way to be robust to adversarial attacks. The authors formulate training a NN as finding an optimal controller for a discrete dynamical system. This formulation allows them to use an optimal control algorithm, called method of successive approximations (MSA), to train a NN. The authors then show how constraints can be added to this optimization problem in order to make the trained NN more robust. They show that the resulted constraint optimization problem can be formulated as a semi-definite programming and provide some experimental results. 

Comments:
- Although the problem studied in the paper is important and the approach is interesting, it seems the paper has been written in rush and in my opinion is not ready for publication. The writing is not good. The introduction and related work sections are incomplete and not very informative. It is not clear what has been done before and what is the contribution of this paper. The main technique/algorithm of the paper has not been explained clearly that someone can easily understand and implement it. The experimental results are not convincing. 
- There are strong claims in the paper such as "experiments show that our method effectively improves deep model's adversarial robustness", this is too strong, given the quality of the experiments of the paper. Or "the constraint optimization problem can be formulated as a semi-definite programming (SDP) problem and hence can be solved efficiently", to the best of my knowledge, SDP solvers are limited to small problems and cannot solve the large problems efficiently. 
- The area of making NNs robust to attacks is a very active area and there are many attacks and solutions out there, which require more comprehensive empirical studies of any new method. I do not see this in the paper. 
- Overall, I think this paper requires a major revision in order to be evaluated better and to be more useful for the community. 
review_title: Official Blind Review #3

Meta-review: The authors propose a framework for improving the robustness of neural networks to adversarial perturbations via optimal control techniques (Lyapunov Stability and the Pontryagin Maximum Principle, in particular). By considering a continuous-time limit of the training process, the authors use the PMP to derive udpate rules for the neural network weights that would result in a robust network. While the approach is interesting, the paper has some serious deficiencies that make it unacceptable for publication in its current form:

1. Quality of empirical evaluation: The authors only report final numbers on CIFAR-10 for a fixed set of adversarial attacks. It has been observed repeatedly in the adversarial robustness literature that adversarial evaluation of neural networks has to be done carefully to guard against possible underestimation of the worst-case attack. In particular, the specific details of the adversarial attacks used (number of steps, number of initializations, performance under larger perturbation radii) that are necessary to trust the results are not given (see https://arxiv.org/pdf/1902.06705.pdf for example).

2. Unclear novelty: The authors do not sufficiently explain the novelty in their approach relative to prior work (particular prior work that has used optimal control ideas in this context).

3. Computational cost: The authors do not give sufficient details to judge the computational overhead of their method to judge how much more expensive it is to train with their approach relative to standard or adversarial training.

While one reviewer voted for a weak accept, the other reviewers were in consensus on rejection. The authors did not respond during the rebuttal phase and hence the reviews were unchanged.

In summary, I vote for rejection. However, I think this paper has potentially interesting ideas that should be carefully developed and evaluated in a future revision.
Decision: Reject