Venue: ICLR 2020 Conference

Reviews:

final_score: rating: 3: Weak Reject
review_content: The submission follows a recent line of work that formulates adversarial training (https://arxiv.org/abs/1412.6572) as a differentiable game and attempts to reduce the computational complexity of solving the inner maximization, corresponding to finding (an) adversarial example(s) for a given parameter setting. The authors propose a way to anneal the truncation of the inner iteration, and investigate a quantity to represent the suboptimality of the truncation.

Strengths:
- (From Table 3), the method is competitive with adversarial training (PGD; Madry et al. (2017)) as well as an improved method for adversarial robustness (FOSC; Wang et al. (2019)), while requiring half the computation time.
- The writing is very clear.

Weaknesses:
- Significance: The technique AMATA itself, is a  heuristic to gradually increase the length of the inner loop optimization in adversarial training. It is unrelated to the optimal control formulation.
- Novelty: Much of the motivation from the perspective of an optimal control formulation of adversarial training is similar to https://arxiv.org/abs/1803.01299, including the PMP and successive approximation components; Eq. (13) of the submission is the continuous version of Theorem 2 in this paper.
- Some experimental details are missing.
- Minor clarity issues (see below).

Questions for the authors:
- Could the authors better contrast their work in reference to prior work (https://arxiv.org/abs/1803.01299)? 
- Could the authors clarify if they use random restarts for the PGD attacks? How was the number of iterations for the PGD attack (40) selected? Why was the iteration count of the PGD attack not varied?

Clarity issues:
- Please provide references for the optimal control formulation in 2.2.
- Some references to variables in the text are not made precise by identifying the variable in question (e.g.: "a set of optimal parameter choices"; "requires information on the entire training interval").
- The function of the hyperparameters \alpha, \gamma, K are not well discussed in Section 2.4.
- Reduce plot sizes and include more experimental details in the main text.
review_title: Official Blind Review #1

final_score: rating: 6: Weak Accept
review_content: Summary: Deep learning algorithms are known to be prone to adversarial examples. This paper proposes a simple modification for adversarial training in order to improve the robustness of the algorithms. The adversarial training of the neural networks involve two steps: the outer loop where the loss is being minimized, the inner loop where the adversarial examples are being found by using PGD. The inner PGD optimization makes training more expensive, because for every gradient step, algorithm needs to perform several steps of PGD. This paper proposes a simple modification to the PGD that is being used in the inner loop. The proposed modification involves the increasing the number of adversarial training steps and decreasing the adversarial training step size gradually as the training proceeds. Then, they analyze the modifications from the theory of optimal control. Their experiments show that their method can achieve similar robustness to the other existing approaches with less computations.

Pros: 
 - Interesting topic, important subject and a very simple approach.
 - Significant improvements in terms of training times.

Cons: 
- The math notation is a bit cumbersome, lots of undefined variables and functions used without properly giving enough background to the readers. This field is still new not every reader might be familiar with the optimal control theory or the common notation that is being used there. Please try to explain the equations and theorems more carefully.
- Experiments are only on small-scale toy-datasets like CIFAR10 and MNIST.

Complete Assessment: I like the proposed approach, in this paper. It is a fairly simple approach and seems to work well in practice, at least on the tasks that the authors have tried. The writing, I think still needs some work, some of the math notation is mostly not properly explained, in Sections 2.2 and 2.3. Overall, I am not sure how much those two sections are actually contributing to the paper.

Question: Did you try using a different annealing mechanisms, besides linear decaying, such as an exponential one?
review_title: Official Blind Review #2

final_score: rating: 6: Weak Accept
review_content: This paper proposed an annealing mechanism for PGD adversarial training in Madry et al. This mechanism gradually reduces the step size and increases the number of iterations of PGD maximization. The authors claim that the proposed method can achieve comparable performance as the original PGD adversarial training with less time. 

First I think this mechanism itself's contribution is kind of incremental because this technique is well under the PGD adversarial training framework. In fact, there are existing works trying to accelerate adversarial training, for example [1]. I think the authors may need to compare their proposed method with [1] if possible.

I like the optimal control formulation in 2.2 and 2.3. More discussion on (8) would be welcomed. In particularly, I think the authors implicitly define the training cost as the sum of iterations in all PGD attacks and turn it into an integral. More explanation here would help. Also, it seems that the reason why \alpha=\frac{\tau}{K^t} is not fully explained in the paper. The authors choose to fix the total step length of PGD. Is this a heuristics?

In the experimental part, I think it would be better to also report the number of SGD steps on \theta to get a sense of convergence. I am not sure the acceleration is due to fewer PGD steps at each t, or fewer training epoch T. 

[1] Zhang, Dinghuai, et al. "You Only Propagate Once: Accelerating Adversarial Training via Maximal Principle." NeurIPS 2019.
review_title: Official Blind Review #3

Meta-review: The paper proposes a modification for adversarial training in order to improve the robustness of the algorithm by developing an annealing mechanism for PGD adversarial training. This mechanism gradually reduces the step size and increases the number of iterations of PGD maximization. One reviewer found the paper to be clear and competitive with existing work, but raised concerns of novelty and significance. Another reviewer noted the significant improvements in training times but had concerns about small scale datasets. The final reviewer liked the optimal control formulation, and requested further details. The authors provided detailed answers and responses to the reviews, although some of these concerns remain. The paper has improved over the course of the review, but due to a large number of stronger papers, was not accepted at this time.
Decision: Reject