Venue: ICLR 2020 Conference

Reviews:

final_score: rating: 6: Weak Accept
review_content: Summary:
The author proposed MMR regularization for the provable robustness of union of l-1 and l-infty balls, which is robust to any l-p norm for p>=1. 

Strengths:
1. The paper is well organized. 
2. The theoretical part is completed and experiments are done on MNIST, Fashion-MNIST, GTS, CIFAR-10. 
3. The proposed method presents a significant improvement over SOTA. 

Weakness:
1. The author should provide more empirical analysis on the MMR regularization, like how it changes during the training process. 
2. The captions of the table and figure are too small. 

review_title: Official Blind Review #3

final_score: rating: 8: Accept
review_content: Summary of the paper's contributions:

This paper proves a result on the l_p robustness (p \neq 1, \infty) of a piecewise affine classifier in terms of its l_1 and l_\infty robustness. This result is based on the insight that a guarantee for l_1 and l_\infty robustness of a piecewise affine classifier also guarantees robustness within the convex hull of the union of the l_1 and l_\infty balls. The paper then proposes a regularization scheme called MMR-Universal for a ReLU neural network that simultaneously trains for l_1 and l_\infty robustness. This scheme is based on maximizing the linear regions of the network as in Croce et al., AISTATS 2019. Using the main result of the paper, it is implied that the proposed regularization scheme also enforces l_p robustness for any p \geq 1.

Main comments: (1) The paper provides an interesting result that guarantees l_p robustness for a piecewise affine classifier based on only l_1 and l_\infty robustness. (2) The experiments show that the proposed regularization scheme is indeed effective in simultaneously guaranteeing robustness with respect to all l_p balls.

Detailed comments:

- The proposed regularization scheme does not come with any explicit robustness guarantee. Is it possible to show that a model that minimizes the regularized loss is guaranteed to be l_p robust for some radius? In Appendix C.3, it is mentioned that the best values for \gamma_p were empirically found to be 1-2 times the desired \epsilon_p robustness. Could this be formalized in theory?

- From Figure 2, it seems that there is an optimal ratio of l_1 to l_\infty robustness for which the l_2 guarantee is maximized. It would be interesting to see what this optimal ratio is for maximizing the l_p guarantee, as a function of dimension d and p.

- In the experiments, the MMR-Universal scheme is compared with MMR+AT-l_2 and MMR+AT-l_\infty. It would be interesting to compare it with MMR-l_1+AT-l_\infty or with MMR-l_\infty+AT-l_1.

- In eq(1), is c intended to be the true label at x?

- In Figure 2, the red curves are missing in the first and third plots.
review_title: Official Blind Review #2

final_score: rating: 6: Weak Accept
review_content: Overview:

The paper is dedicated to developing a regularization scheme for the provably robust model. The author proposes the MMR-Universal regularizer for ReLU based networks. It enforces l1 and l infinity robustness and leads to be provably robust with any lp norm attack for p larger than and equal to one.

Strength Bullets:

1. Using convex hull to enforce robustness, it is very reasonable and straightforward which highly aligns our intuition.
2. The author provides detailed and conniving derivations and proof. And the results in table 2 do achieve the state-of-the-art provable robustness.
3. It is the first robust regularizer that is able to provide non-trivial robustness guarantees for multiple lp balls. The lp balls don't contain any other. It also is one of the most straightforward methods among all potential similar methods.

Weakness Bullets:

1. I am very curious about the landscape or decision boundary analysis and visualization. For the author's MMR-Universal regularization, it should give the model a very "good" decision boundary which has clear marginal between any two categories. In my opinion, it is necessary to evidence to convince more readers.
2. Try a few more classical CNN architectures. 

Recommendation:

Due to the logical derivations and supportive experiment results, this is a weak accept.
review_title: Official Blind Review #1

Meta-review: This paper extends the degree to which ReLU networks can be provably resistant to a broader class of adversarial attacks using a MMR-Universal regularization scheme.  In particular, the first provably robust model in terms of lp norm perturbations is developed, where robustness holds with respect to *any* p greater than or equal to one (as opposed to prior work that may only apply to specific lp-norm perturbations).

While I support accepting this paper based on the strong reviews and significant technical contribution, one potential drawback is the lack of empirical tests with a broader cohort of representative CNN architectures (as pointed out by R1).  In this regard, the rebuttal promises that additional experiments with larger models will be added in the future to the final version, but obviously such results cannot be used to evaluate performance at this time.
Decision: Accept (Poster)