Paper:
We study adversarial robustness of neural networks from a margin maximization perspective, where margins are defined as the distances from inputs to a classifier's decision boundary. Our study shows that maximizing margins can be achieved by minimizing the adversarial loss on the decision boundary at the "shortest successful perturbation", demonstrating a close connection between adversarial losses and the margins. We propose Max-Margin Adversarial (MMA) training to directly maximize the margins to achieve adversarial robustness. Instead of adversarial training with a fixed , MMA offers an improvement by enabling adaptive selection of the "correct" as the margin individually for each datapoint. In addition, we rigorously analyze adversarial training with the perspective of margin maximization, and provide an alternative interpretation for adversarial training, maximizing either a lower or an upper bound of the margins. Our experiments empirically confirm our theory and demonstrate MMA training's efficacy on the MNIST and CIFAR10 datasets w.r.t. ∞ and 2 robustness. Despite their impressive performance on various learning tasks, neural networks have been shown to be vulnerable to adversarial perturbations (Szegedy et al., 2013; Biggio et al., 2013). An artificially constructed imperceptible perturbation can cause a significant drop in the prediction accuracy of an otherwise accurate network. The level of distortion is measured by the magnitude of the perturbations (e.g. in ∞ or 2 norms), i.e. the distance from the original input to the perturbed input. Figure 1 shows an example, where the classifier changes its prediction from panda to bucket when the input is perturbed from the blue sample point to the red one.
Figure 1 also shows the natural connection between adversarial robustness and the margins of the data points, where the margin is defined as the distance from a data point to the classifier's decision boundary. Intuitively, the margin of a data point is the minimum distance that x has to be perturbed to change the classifier's prediction. Thus, the larger the margin is, the farther the distance from the input to the decision boundary of the classifier is, the more robust the classifier is w.r.t. this input.
Although naturally connected to adversarial robustness, "directly" maximizing margins has not yet been thoroughly studied in the adversarial robustness literature. Instead, the method of minimax adversarial training (Madry et al., 2017; Huang et al., 2015)  is arguably the most common defense to adversarial perturbations due to its effectiveness and simplicity. Adversarial training attempts to minimize the maximum loss within a fixed sized neighborhood about the training data using projected gradient descent (PGD). Despite advancements made in recent years (Hendrycks et al.,  2019; Zhang et al., 2019a; Shafahi et al., 2019; Zhang et al., 2019b; Stanforth et al., 2019; Carmon  et al., 2019), adversarial training still suffers from a fundamental problem, the perturbation length has to be set and is fixed throughout the training process. In general, the setting of is arbitrary, based on assumptions on whether perturbations within the defined ball are "imperceptible" or not. Recent work (Guo et al., 2018; Sharma et al., 2019)  has demonstrated that these assumptions do not consistently hold true, commonly used settings assumed to only allow imperceptible perturbations 1 Under review as a conference paper at ICLR 2020 in fact do not. If is set too small, the resulting models lack robustness, if too large, the resulting models lack in accuracy. Moreover, individual data points may have different intrinsic robustness, the variation in ambiguity in collected data is highly diverse, and fixing one for all data points across the whole training procedure is likely suboptimal.
Instead of further improving adversarial training with a fixed perturbation magnitude, we revisit adversarial robustness from the margin perspective, and propose Max-Margin Adversarial (MMA) training, a practical algorithm for direct input margin maximization. By directly maximizing margins calculated for each data point, MMA training allows for optimizing the "current robustness" of the data, the "correct" at this point in training for each sample individually, instead of robustness w.r.t. a predefined magnitude.
While it is intuitive that one can achieve the greatest possible robustness by maximizing the margin of a classifier, this maximization has technical difficulties. In Section 2, we overcome these difficulties and show that margin maximization can be achieved by minimizing a classification loss w.r.t. model parameters, at the "shortest successful perturbation". This makes gradient descent viable for margin maximization, despite the fact that model parameters are entangled in the constraints.
We further analyze adversarial training (Madry et al., 2017; Huang et al., 2015)  from the perspective of margin maximization in Section 3. We show that, for each training example, adversarial training with fixed perturbation length is maximizing a lower (or upper) bound of the margin, if is smaller (or larger) than the margin of that training point. As such, MMA training improves adversarial training, in the sense that it selects the "correct" , the margin value for each example.
Finally in Section 4, we test and compare MMA training with adversarial training on MNIST and CIFAR10 w.r.t. ∞ and 2 robustness. Our method achieves higher robustness accuracies on average under a variety of perturbation magnitudes, which echoes its goal of maximizing the average margin. Moreover, MMA training automatically balances accuracy vs robustness while being insensitive to its hyperparameter setting, which contrasts sharply with the sensitivity of standard adversarial training to its fixed perturbation magnitude. MMA trained models not only match the performance of the best adversarially trained models with carefully chosen training under different scenarios, it also matches the performance of ensembles of adversarially trained models.
In this paper, we focus our theoretical efforts on the formulation for directly maximizing the input space margin, and understanding the standard adversarial training method from a margin maximization perspective. We focus our empirical efforts on thoroughly examining our MMA training algorithm, comparing with adversarial training with a fixed perturbation magnitude.

Review:
Summary: 
This paper proposes an adaptive margin-based adversarial training (eg. MMA) approach to train robust DNNs by maximizing the shortest margin of inputs to the decision boundary. Theoretical analyses have been provided to understand the connection between robust optimization and margin maximization. The main difference between the proposed approach to standard adversarial training is the adaptive selection of the perturbation bound \epsilon. This makes adversarial training with large perturbation possible, which was previously unachievable by standard adversarial training (Madry et al.) Empirical results match the theoretical analysis.
Pros:
1. The margin maximization idea has been well-explained, both intuitively and theoretically.
2. Interesting theoretical analyses and understandings of robust optimization from the margin perspective.
3. Clear advantage of MMA over standard adversarial training under large perturbations.
8. The proposed PGDLS is very interesting, actually quite good and much simpler, without extra computational cost. A similar idea was discussed in paper [3], where they gradually increase the convergence quality of training adversarial examples, and show the convergence guarantee  of "dynamic training".
9. The gradient-free SPSA helps confirm the improvements of MMA under large perturbations are not a side effect of gradient masking.
Cons:
1. The idea of "shortest successful perturbation" appears like a type of weak training attack, looking for minimum perturbations to just cross the classification boundary, like deepfool [1] or confidence 0 CW-L2 attack [2]. 
2. The margin d_\theta in Equation (1)/(2)/... defined on which norm? L_\infty or L2 norm? I assume it's the infinity norm. In Theorem 2.1, the \delta^{*} = argmin ||\delta||, is a norm? Looks like a mistake. 
3. The minimum margin \delta^{*} is a bit confusing, is it used in maximization or just in the outer minimization? The last paragraph of page 3, L(\theta, \delta) or L(\delta, \theta), consistency check?
4. Why do we need the "gradients of margins to model parameters" analysis from Proposition 2.1 to remark 2.2? Given the \delta^{*} found in the inner maximization (eg. attacking) process (step 1), minimizing the loss over this \delta^{*}  seems quite a straightforward step 2. Why don't go directly from Theorem 2.1 to Proposition 2.4, since the extensions from LM loss to SLM and CE loss via Proposition 2.3 -> Proposition 2.4., just proves that the standard classification loss CE can already maximize the margin given \delta^{*}? 
5. Section 4 Experiments. The experimental settings are not clear, and are not standard. What CIFAR10-\ell_{\infty} means: is it the CW-L2 attack, used for training, or for testing? How the test attacks were generated, the m and N, are confusing: for each test image, you have 260 samples for CIFAR10 (which means 260*10K in total), or just 260 in total (this is far less than a typical setting causing inaccurate results)? How are the d_max determined, and what are their relationship to standard \epsilon? How the m models were trained?
6. Fairness of the comparison. Since MMA changes \epsilon, how to fairly compare the robustness to standard epsilon bounded adversarial training is not discussed. Is it fair to compare MMA-3.0 vs PGD-2.5, since they have different epsilon? Why robustness was not tested against strong, unrestricted attacks like CW-L2 [2], and report the average L2 perturbations required to completely break the robustly trained model (and show MMA-trained models enforce large perturbations to succeed)?
7. Significance of the results. Normally, \epsilon_{infty} > 16/255 will cause perceptual difference. Under 16/255, PGD-8/16, PGDLS-8/16 are still the best. At this level, it is quite a surprise that MMA does not improve robustness, although it does increase clean accuracy. This means the theoretical analysis only stand under certain circumstances. I don't think the optimal \epsilon < margin can explain this, as it does not make sense to me the margin can be larger than 16/255. On the other hand, I thought the theoretical parts were discussing the  ROBUSTNESS, not the CLEAN ACCURACY? But it turns out the MMA benefits a lot the clean accuracy?  Why do we need robustness against large \infty perturbations, this definitely deserves more discussion, as when perturbation goes large, the L2 attack (eg. CW-L2) makes more sense than PGD-\infty.
[1] Moosavi-Dezfooli, Seyed-Mohsen, Alhussein Fawzi, and Pascal Frossard. "Deepfool: a simple and accurate method to fool deep neural networks." Proceedings of the IEEE conference on computer vision and pattern recognition. 2016.
[2] Carlini, Nicholas, and David Wagner. "Towards evaluating the robustness of neural networks." 2017 IEEE Symposium on Security and Privacy (SP). IEEE, 2017.
[3] Wang, Yisen, et al. "On the Convergence and Robustness of Adversarial Training." International Conference on Machine Learning. 2019.
============
My rating stays the same after reading through all the responses. I appreciate the authors' clarification on the notations and experimental settings. My 8/9 are positive points. My major concern is still the effectiveness of the proposed approach, and fairness of the comparison.  It seems that MMA only works when the perturbation is large, which often larger than the \epsilon used to train baseline adversarial training methods such as Trades. The authors seem have misunderstood my request for CWL2 results, I was just suggesting that the average L2 perturbation of CWL2 attack can be used as a fair test measure for robustness, instead of the AvgRobAcc used in the paper, and the susceptible comparison between  MMA-12 vs PGD-8, or MMA-32 vs Trades.