Venue: ICLR 2021 Conference

Reviews:

final_score: rating: 5: Marginally below acceptance threshold
review_content: The authors exploit the piecewise linear nature of ReLU neural networks to design a new regularizer that improves the robustness of the neural network. It can be viewed as a alternative to the regularizer proposed in Croce et al. (2018) -- the current regularizer uses the analytic center, whereas the previous work (named MMR) guarantees that are straightforward consequences of the geometric interpretation. The experimental setup is similar to the one in Croce et al. (2018), which compares the result of different adversarial defense methods on MNIST, F-MNIST, and CIFAR10 on small shallow networks. These computational results generally slightly outperform MMR on MNIST.

The paper is polished and well-written, and the use of the analytic center is reasonably motivated via the geometry of the neural network.

However, the paper has a few major drawbacks that need to be addressed before acceptance.

(1) Novelty: The paper can be seen as a twist on Croce et al., working off some of the same intuition and using very similar experiments. The theoretical results are a simple consequence of using the analytic center. Hence, this work can be viewed as incremental.

(2) Experimental Results: Given that the set of regularization approaches has significantly grown since the prior paper was published, the authors should at least compare their methods with some of these. This is especially important given that the results are not much better than MMR (and worse in some cases).

(3) Computational Considerations: The computation effort needed to incorporate this regularizer should be discussed explicitly so the reader can understand the trade-offs between incorporating this regularizer vs. other defense methods.

Side note:
The authors should update the citation for Croce et al. to refer to the conference version of the paper. 
review_title: An slight improvement over an existing regularization method for neural network robustness, but more convincing experiments needed.

final_score: rating: 6: Marginally above acceptance threshold
review_content: The paper present some regularization schemes for ReLU networks, based in geometrical properties (polytopes and analytical centers), aiming at robustifying networks against adversarial examples. The idea is to train the network such that the partition of the input space contains many less linear regions, and such that in each region, training points are gathered around the analytical center (and thus far from boundaries), making more difficult the attack task. 
Doing so the authors can  give certificates for robustness.  The proposed approach appears to be competitive in terms of bounds and do so with less hyper-parameters than other methods. 

The idea of using regularization terms to make networks more robust makes sense, the particular proposed regularizers have some quite easy to catch motivation. 
The presented results match state-of-the-art, however I have some (maybe naive) questions : 
-> why TE are missing sometimes? 
-> how can LB>UB (CNN l_inf) ? I would have say that max(LB)=min(UB)=true minimal distortion ? 
Fig 3 is unreadable which is a pity since I think it could be some relevant information, its interpretation is quite brief too. 

Overall I think the presented work is interesting and quite well executed. Theoretical part seems ok, but I'm not familiar with some geometric notions so I might have missed things. 
I tend to accept the paper but I'm waiting for more explanations on the experimental results. 
review_title: some questions on the experimental results

final_score: rating: 4: Ok but not good enough - rejection
review_content: As a reader and a reviewer, I am not unable to follow the main theoretical point in the paper. The proof of Theorem 1 is not properly explained with ambiguous reference to other articles. 

More specifically, regarding the proof of Theorem 1 in Appendix A.2, the authors do not provide enough details nor explanation for the second part of the statement. Merely writing something like 'follow from papers xxx' without any details makes the proof unreadable.

I think as a basic principle, the authors should consider readers' convenience and provide a self-contained proof and intuitive explanation of the claims. At least they should specify what results from outside are used in their theoretical analysis.

The careless treatment of theory makes the manuscript unreadable, and thus the present form is not qualify for an ICLR submission.

review_title: detailed theoretical analysis missing

Meta-review: The paper presents a novel regularization scheme for showing provable robustness guarantees for the class of ReLU networks.
The reviews of this paper were mixed but leaning more to reject. Even though geometric approaches to adversarial robustness are appealing there are too many issues left open (see below for detailed comments) so that the paper is not above the bar for ICLR.

Detailed comments:
 - incremental resp. small theoretical contribution: the bounds in Theorem 1 are worse than what has been derived in Croce et al (2019), which is admitted only in the Experiments section but this should directly be discussed after Theorem 1, or are based on other's work Moosavi-Dezfooli et al(2019)
 - the proofs are still not very clear even after the update e.g. it is not clear to me what is meant with the distance to the decision boundary d_D(x) as this quantity cannot be computed (this is the whole point here)
 - the experimental results are in most of the cases worse than prior work (KW and MMR). The authors argue that their results hold for both threat models but then the l_infty/l_2 robustness of the other models should have been provided to make the point.
Moreover, if multiple norm ball threat models is the point, then the authors should compare to the follow up work of Croce et al at ICLR 2020 which explicitly optimizes all l_p-balls. 
- the point with gradient obfuscation of MMR does not make sense to me as the gap between upper and lower bounds on the robust error is quite small for MMR in most cases. Gradient obfuscation would mean that the PGD attack fails and thus the lower bound would have to be close to zero. 
- Figure 1c) looks like the Figure from the book of Boyd and Vandenberghe - if this is the case then it has to be correctly referenced in the caption

Decision: Reject