Venue: ICLR 2021 Conference

Reviews:

final_score: rating: 7: Good paper, accept
review_content: Keeping the performance of deep neural networks against data perturbations is an important and open problem. The authors propose an optimal control-based approach by taking dynamical systems perspective. The proposed method sounds intuitive and efficient. Authors supply theoretical analysis and (small) experimental evaluation. Overall, I believe paper is a good. However, I would like to get some points clarified:
a)	Authors used manifold assumption (which is a reasonable assumption for many problems) to define running loss (eq 3). (If I am not mistaken) They choose a quadratic loss to have a tractable optimization problem. However, under these assumptions, one may choose many different losses. Would you please comment on the form of the loss and its impact to the method?
b)	Let’s assume dynamical systems perspective is a right perspective for analysing deep neural networks (to be honest I don’t have any criticism about this). To use control theoretic tools, one needs to comment on controllability and observability of the controlled system. I suspect these mentioned properties are a function of the neural network architecture or do authors think the proposed method (as shown in Figure 1) makes each and every deep neural network architecture controllable and/or observable? I would like to hear authors perspective on these issues.
c)	As I mentioned before, the empirical study is quite small, and I didn’t see any baseline (do I miss something here). Do authors consider extending their empirical study and compare their method with some baselines.

I would like to emphasize one more time that, I am positive about the paper. However, I would like to note that I am not expert in the field and I am open to change my view in both direction.



review_title: Increasing robustness via optimal control

final_score: rating: 7: Good paper, accept
review_content: ### Summary
This study develops a closed-loop control strategy to improve robustness of neural networks to adversarial attacks. The study is technically sound and the empirical results on classification tasks are convincing.


### Quality

The paper is technically sound and the claims are appropriately backed by empirical evaluation. However, I would recommend the authors to discuss a bit more the additional computational cost of running the closed-loop method.


### Clarity

The manuscript is clearly written and provides enough information for an expert reader to understand all the steps to reproduce the results.


### Originality

The novelty of the study resides in the development of a closed-loop control method for increasing the robustness of neural networks. The strategy is devised to scale to the typical high-dimensional nature of neural network activations.


### Significance of the work

The results suggest that the developed approach is a solid step towards developing robust neural networks.


### Some typos:

-instead of "cause different data distribution deviating", "cause data distributions to deviate";

-instead of "The resulting control policy [...] make it", "The resulting control policy [...] makes it";

-instead of "the embedding are effective", "the embeddings are effective";

-instead of "the perturbed states in Fig.2 [...] has", "the perturbed states in Fig.2 [...] have";

-instead of "to obtain all the intermediate hidden states [...] and accumulates", "to obtain all the intermediate hidden states [...] and to accumulate";

-issue with reference "E. 2017".
review_title: Technically sound study and convincing empirical results

final_score: rating: 6: Marginally above acceptance threshold
review_content: The paper builds on recent revival in control-theoretic approaches to deep neural networks by proposing an adaptive controller that projects intermediate representations in the network to their "manifolds" and consequently makes the neural network robust to input perturbations. 

Pros:

+ Active controller-based projection of intermediate features is an interesting idea and the utility of Pontryagin's maximum principle to address the challenge of high dimensionality of the state (features) is a good observation.

Cons:

- The manifold-based defense has been shown to be broken previously. For e.g. please see section 4 of https://arxiv.org/pdf/1712.09196.pdf . Manifold/GAN/VAE based defenses can be easily broken by just attacking the projection network and the original network together. The paper considers manifold based defense at all layers using an active controller doing the projection. While comparison with pixeldefense is useful, it would be good to launch an attack similar to the reference above and then observe the effectiveness of the approach. The arguments in the paper are not sufficient to convince the reviewer that this defense is practical. The paper is severely lacking in comparison with existing defense approaches. The state of the art for the used dataset in the paper is significantly better than the effectiveness of the presented approach. For e.g. see https://github.com/MadryLab/robustness 

In summary, the paper is a good effort to exploit the use of active controllers in deep learning. But firstly, the use of manifold-based projection as a cost function is itself a non-robust defense against adversarial examples. Second, the experimental evaluation in the paper is significantly lacking and does not meet the standards of a venue such as ICLR. The reviewer will strongly recommend reviewing the advices in https://arxiv.org/abs/1902.06705 on this topic. At this point, the paper is interesting but it needs significant development and is not yet ready for publication. 

Questions for the author:

1. How critical is Pontryagin’s Maximum Principle to the presented approach? What prevents one from using projection to a lower dimensional embedding space followed by a state space control method? In particular, if one is building on the manifold assumption, then isn't it reasonable to not worry about high dimensionality for designing the controller too?

2. Is it realistic to assume the "input perturbation to be a random vector" in Section 5 for theoretical analysis when we are considering adversarial attacks such as PGD, CW? If not, then isn't theorem 1 not relevant to the primary topic of the paper?

------ After author's response:

* The response of authors identifies the problem of using running loss in projected space. While one can try to get around it by projecting the loss function as well but that would be a convoluted way to solve the problem, and in any case, not a strong criticism of the presented approach. 

* The updated document has generalized the derivation to take general perturbations into account. 

* Updates Tables 1-3 resolve empirical analysis concerns of the reviewer. 

With these improvements, the reviewer is happy to recommend acceptance of the paper. 
review_title: Interesting approach .. 

final_score: rating: 7: Good paper, accept
review_content: Strength:
1. This paper first introduced layer-wised projection from the poisoned data to the clean data.
2. The results show improvement of the robustness over the baseline on different types of attacks.

Weakness:
1. The statement of the closed-loop control is a little bit ambitious. The overall methods are the layer-wise projection from the poisoned data to the clean data manifold. Normally, in the closed-loop, we will use the control signal $u$ to control the original data instead of the next layer data. For example, the final balance stage should be $u=g(x+f(x+u))=0$. So closed-loop should be at least multi-steps within one layer. For different layers, the closed-loop control will have different control signals, since the dimension/ distribution between layers is much different. So this method is only a one-step layer-wise projection. The $x$ between layers cannot be viewed as the same sample to be controlled. I would recommend the author to change the statement from closed-loop into the layer-wise projection for a better suit. Also, this method is still a feed-forward network, not a "loop" control. I do think this method is interested, just a little ambitious. This could be a useful extension of the resnet-based network, since the control $u$ can be viewed as a complicated version of residuals.
2. The experiment is weak for only comparing with one baseline. Also, can the author provide which baseline model that the author is comparing with? I cannot find it in the text. I would appreciate it.
3. It's unclear to me about the training of $\mathcal{E}(x)$, will this requires extra data to train? What's the running speed of this "closed-loop" method compared with others?

Some tiny comments:
1. A trivial comparison would be to train an autoencoder for each layer of $x$ and only use the decoded results to pass through the network. This in principle learns the data manifold and provides the projection to this manifold. 
2. Table 2, the dataset name is not aligned in the center.
3. Table 3 should be more self-explainable. It's a little confusing in the current form.
4. Have the author tried multi-steps in a single layer or constraint $x_t$ to be in the same space? 



----- post rebuttal -----

The authors addressed most of my concerns and the revision is better than before. 
I would like to increase my score and would recommend an acceptance.
review_title: This paper argues to proposed a closed-loop control in the robustness training to achieve good results on different types of attack. To me, this statement is ambitious to be called closed-loop control, but the overall structure is meaningful and interesting.

Meta-review: Thank you for your submission to ICLR.  The reviewers and I are in agreement that the work presents some interesting connections between closed-loop control and stabilization of activations to an observed manifold.  Specifically, the idea of using optimal control dynamic programming techniques to compute optimal adjustments to ensure control on this manifold is an interesting one and may have other implications within deep networks.

Although the reviewers were convinced by the experiments on robustness, I remain a bit skeptical here.  The results show that while the method marginally improves robustness to small-epsilon perturbations, the models are still quite non-robust against the size perturbations frequently used in assessing adversarial robustness (e.g., to eps=8 perturbations on CIFAR10, where the best approach gets ~11% accuracy against PGD attacks).  It doesn't really matter how well a defense works against sub-optimal attacks: if PGD is able to decrease its accuracy this much, clearly the model is not very robust (and it seems upon reading that only 20-step PGD, with no restarts, was used as an attack, which is a fairly weak variant of PGD).  Furthermore, the approach didn't improve much upon PGD-based adversarial training when combined with it, either, overall suggesting that the impact on robustness is somewhat minor, and needs to be evaluated quite a bit more.

While I don't believe these concerns are substantive enough to override the beliefs of all reviewers, I think that the authors could do a much better job of evaluating the actual robustness of these models (following the advice of https://arxiv.org/abs/1902.06705).  And if the resulting metrics are not as strong as hoped for, then it would be good to evaluate other possible benefits of the approach (perhaps to random distribution shift? it seems a much more likely situation for there to be real gains?).  Thus, while I believe the paper has some interesting ideas, I think the authors should probably tone down some of the current claims of improving adversarial robustness unless they can provide a much more thorough evaluation.
Decision: Accept (Poster)