Rethinking Softmax Cross-Entropy Loss for Adversarial Robustness
Authors: Tianyu Pang
Kun Xu
Yinpeng Dong
Chao Du
Ning Chen
Jun Zhu
Venue:  (2020)
Keywords: Trustworthy Machine Learning
Adversarial Robustness
Training Objective
Sample Density

Abstract: Previous work shows that adversarially robust generalization requires larger sample complexity, and the same dataset, e.g., CIFAR-10, which enables good standard accuracy may not suffice to train robust models. Since collecting new training data could be costly, we focus on better utilizing the given data by inducing the regions with high sample density in the feature space, which could lead to locally sufficient samples for robust learning. We first formally show that the softmax cross-entropy (SCE) loss and its variants convey inappropriate supervisory signals, which encourage the learned feature points to spread over the space sparsely in training. This inspires us to propose the Max-Mahalanobis center (MMC) loss to explicitly induce dense feature regions in order to benefit robustness. Namely, the MMC loss encourages the model to concentrate on learning ordered and compact representations, which gather around the preset optimal centers for different classes. We empirically demonstrate that applying the MMC loss can significantly improve robustness even under strong adaptive attacks, while keeping state-of-the-art accuracy on clean inputs with little extra computation compared to the SCE loss.

Published as a conference paper at ICLR 2020
RETHINKING SOFTMAX CROSS-ENTROPY Loss FOR ADVERSARIAL ROBUSTNESS
Tianyu Pang, Kun Xu, Yinpeng Dong, Chao Du, Ning Chen, Jun Zhu" Dept. of Comp. Sci. & Tech., BNRist Center, Institute for AI, Tsinghua University; RealAI 1pty17, (ayk1, 6, d du-ei 4 @ma 1...111.............. wtw.tgghh..aeeet. .en,.cn, [gghen, desaj / wawaangghuuuu cm
ABSTRACT
Previous work shows that adversarially robust generalization requires larger sample complexity, and the same dataset, c.g., CIFAR-10, which enables good standard accuracy may not suffice to train robust models. Since collecting new training data could be costly, we focus better utilizing the given data by inducing the regions with high sample density in the feature space, which could lead to locally sufficient samples for robust learning. We first formally show that the softmax cross-entropy (SCE) loss and its variants convey inappropriate supervisory signals, which encourage the learned feature points to spread over the space sparsely in training. This inspires us to propose the Mlax-Maallanobis center (MMC) loss to explicitly induce dense feature regions order benefit robustness. Namely, the MMC loss encourages the model to concentrate on learning ordered and compact representations, which gather around the preset optimal centers for different elasses. We empirically demonstrate that applying the MMC loss can significantly improve robustness even under strong adaptive attacks, while keeping high accuracy on clean inputs comparable to the SCE loss with little extra computation.
INTRODUCTION
The deep neural networks (DNNs) trained by the softmax cross-entropy (SCE) loss have achieved sfate-of-the-art performance various tasks (Goodfellow et al., 2016). However, in terms of robustness, the SCE loss is not sufficient to lead to satisfactory performance of the trained models. It has been widely recognized that the DNNs trained by the SCE loss vulnerable to adversarial attacks (Carlini & Wagner, 20;7a; Goodfellow et al., 2015; Kurakin et al., 2017; Moosavi-Dezzoon al., 2016; Papernot al., 2016). where human imperceptible perturbations can be crafted to fool high-performance network. To improve adversarial robustness of classifiers, various kinds of defenses have been proposed, but many of them are quickly shown to ineffective to the adaptive attacks, which are adapted to the specific details of the proposed defenses (Athalye et al., 2018).
Besides, the methods on verification and training provably robust networks have been proposed (Dvi- jotham et al., 201 Sa;b; Hein & Andriushchenko, 2017; Wong & Kolter, 2018). While these methods are exciting. the verification process often slow and not scalable. Among the previously proposed defenses, the adversarial training (AT) methods can achieve state-of- thethe-rrt robustness under dif- ferent adversarial settings (Madry et al., 2018; 8; Zhang et al., 2019b). These methods either directly impose the AT mechanism on the SCE loss or add additional regularizers. Although the AT methods are relatively strong, they could sacrifice accuracy on clean inputs and are computationally expen- sive (Xie et al., 2019). Due to the computational obstruction, many recent efforts have been devoted to proposing faster verification methods (Wong et al., 2018; Xiao et al., 2019) and accelerating AT procedures (Shafahi et al., 2019; Zhang et al., 2019a). However, the problem still remains.
Schmidt et al. (2018) show that the sample complexity of robust learning can be significantly larger than that of standard learning. Given the difficulty of training robust classifiers in practice, they further postulate that the difficulty could stem from the insufficiency of training samples in the commonly used datasets, e.g., CIFAR-10 (Krizhevsky & Hinton, 2009). Recent work intends to solve this problem by utilizing extra unlabeled data (Carmon et al., 2019; Stanforth et al., 2019),
"Corresponding author.

---

Published as a conference paper at ICLR 2020
while we focus the complementary strategy to exploit the labeled data in hand more efficiently. Note that although the samples in the input space are unchangeable, we could instead manipulate the local sample distribution, i.e., sample density in the feature space via appropriate training objectives. Intuitively, by inducing high-density feature regions, there would be locally sufficient samples to train robust classifiers and return reliable predictions (Schmidt et al., 2018).
Similar to our attempt to induce high-density regions in the feature space, previous work has been proposed to improve intra-class compactness. Contrastive loss (Sun et al., 2014) and triplet loss (Schroff et al.. 2015) are two classical objectives for this purpose, but the training iterations will dramatically grow to construct image pairs or triplets, which results in slow convergence and in- stability. The center loss (Wen et al., 2016) avoids the pair-wise or triplet-wise compu- tation by minimizing the squared distance between the features and the corresponding class centers. However, since the class centers are updated w.r.t. the learned features during training, the center loss has to be jointly used with the SCE loss to seek for a trade-off between inter-class dispersion and intra-class compactness. Therefore, the center loss cannot concentrate on inducing strong intra-class compactness to construct high-density regions and consequently could not lead to reliable robustness, as shown in our experiments.
Eay C [C, C, 1 (modiam comple deadiy
Lun K4, C, - songle drasity
E. 1:1.11 11.
Lag C (C2 f uc) imediams cample deadits
Dhigh deadity
SCE
MMC
Pryflond suree conte la Caruc
AC
Figure I: Intuitive illusion of how training data moves and how sample den- sity varies in a two-dimensional Feature space during the training procedure.
In this paper, we first formally analyze the sample density distribution induced by the SCE loss and its other variants (Pang et al., 2018: Wan et al., 2018) in Sec. 3.2, which demonstrates that these previously proposed objectives convey unexpected supervisory signals on the training points, which make the learned features tend to spread over the space sparsely. This undesirable behavior mainly roots from applying the softmax function in training, which makes the loss function only depend on the relative relation among logits and cannot directly supervise on the learned representations.
We further propose novel training objective which can explicitly induce high-density regions in the feature space and learn more structured representations. To achieve this, we propose the Max- Mahalanobis center (MMC) loss (detailed in Eq. (8)) the substitute of the SCE loss. Specifically, in the MMC loss, we first preset untrainable class centers with optimal inter-class dispersion in the feature space according Pang et al. (2018), then we encourage the features gather around the centers by minimizing the squared distance similar with the center loss. The MMC loss can explicitly control the inter-class dispersion by single hyperparameter, and further concentrate intra-class compactness in the training procedure to induce high-density regions, as intuitively shown in Fig. 1. Behind the simple formula, the MMC loss elegantly combines the favorable merits of the previous methods, which leads to a considerable improvement on the adversarial robustness.
improving
In experiments, we follow the suggestion by Carlini et al. (2019) that we test under different threat models and attacks, including the adaptive attacks (Athalye et al., 2018) on MNIST, CIFAR- 10, and CIFAR- 100 (Krizhevsky Hinton, 2009; LeCun et al., 1998). The results demonstrate that our method can lead to reliable robustness of the trained models with little extra computation, while maintaining high clean accuracy with faster convergence rates compared to the SCE loss and its variants. When combined with the existing defense mechanisms, e.g., the AT methods (Madry et al., 2018), the trained models can be further enhanced under the attacks different from the one used to craft adversarial examples for training.
PRELIMINARIES
This section first provides the notations, then introduces the adversarial attacks and threat models.
2.1 NOTATIONS
In this paper, we use the lowercases to denote variables and the uppercases to denote mappings. Let L be the number of classes, we define the softmax function softmax(h). : R' + R as softmax(h); exp(h:)/Ek,, i E [7.]. where [T.] = {I...., I.) and h is termed logit.
i
2

---

Published as a conference paper at ICLR 2020
A deep neural network (DNN) learns non-lineat mapping from the input E Rp to the feature 2 = (z) E R One common training objective for DNNs is the softmax cross-entropy (SCE) loss:
= -1y log [softmaxx W b]],
(I)
for single input-label pair (z. y). where ly is the one-hot encoding of and the logarithm is defined as element-wis wise. Here W and are the weight matrix and bias vector of the SCE loss, respectively.
2.2 ADVERSARIAL ATTACKS AND THREAT MODELS
Previous work has shown that adversarial examples can be casily crafted to fool DNNs (Biggio et al., 2013; Nguyen al., 2015; Szegedy al., 2014). A large amount of attacking methods generating adversarial examples have been introduced in recent years (Carlini & Wagner, 20172 7a; Chen et al., 2017; Dong al., 2018; Goodfellow et al., 2015; Ilyas al., 2018; Kurakin et al., 2017; Madry et al., 2018; Moosavi-Dezfodd et al., 2016; Papernot et al., 2016; Uesato et al., 2018). Given the space limit, we try perform comprehensive evaluation by considering five different threat models and choosing representative attacks for each threat model following Carlini et al. (2019):
White-box loc distortion attack: We apply the projected gradient descent (PGD) (Madry al., 2018) method, which is efficient and widely studied in previous work (Pang et al., 2019).
White-box l2 distortion attack: We apply the C&W (Carlini & Wagner, 2017a) method, which has a binary search mechanism on its parameters to find the minimal l2 distortion for a successful attack.
Black-box transfer- aftack: We use the momentum iterative method (MIM) (Dong al.. 2018) that is effective on boosting adversarial transferability (Kurakin et al., 2018).
Black-box gradient-free attack: We choose SPSA (Uesato al., 2018) since has broken many previously proposed defenses. It can still perform well even when the loss is difficult to optimize.
General-purpose attack: We also evaluate the general robustness of models when adding Gaussian noise (Gilmer et al., 2019) or random rotation (Engstrom et al., 2019) on the input images.
Furthermore, to exclude the false robustness caused by, e-g., gradient mask (Athalye et al., 2018), we modify the above attacking methods to be adaptive attacks (Carlini & Wagner, 2017b; Carlini et al., 2019; Herley & Van Oorschot, 2017) when evaluating on the robustness of our method. The adaptive attacks are much more powerful than the non-adaptive ones, as detailed in Sec. 4.2.
3 METHODOLOGY
Various theoretical explanations have been developed for adversarial examples (Fawzi et al., 2016; 2018; Ilyas et al., 2019; Papernot et al., 2018). In particular, Schmidt et al. (2018) show that training robust classifiers requires significantly larger sample complexity compared to that training standard ones, and they further postulate that the difficulty of training robust classifiers stems from, at least partly, the insufficiency of training samples in the common datasets. Recent efforts propose alternatives to benefit training with extra unlabeled data (Carmon al., 2019: Stanforth et al., 2019), while we explore the complementary way to better use the labeled training samples for robust learning.
Although given sample is fixed in the input space, we can instead manipulate the local sample distribution. i.e., sample density in the feature space, via designing appropriate training objectives. Intuitively, by inducing high-density regions in the feature space, can be expected to have locally sufficient samples to train robust models that are able to return reliable predictions. In this section, we first formally define the notion of sample density in the feature space. Then we provide theoretical analyses of the sample density induced by the SCE loss and its variants. Finally, we propose our new Max-N Mahalanobis center (MMC) loss and demonstrate its superiority compared to previous losses.
3.1 SAMPLE DENSITY IN THE FEATURE SPACE
Given a training dataset D with N input-label pairs, and the feature mapping Z trained by the objective [(Z(z), y) this dataset, we define the sample density nearby the feature point = = (x))
3

---

Published as a conference paper at ICLR 2020
following the similar definition in physics (Jackson, 1999)
AN
(I) = (o)(A
(2)
Here Vol(.) denotes the volume of the input set, A is a small neighbourhood containing the feature point 2, and AN (D) AB| the number of training points in AB, where Z (P) is the set of all mapped features for the inputs in D. Note that the mapped feature z is still of the label y.
In the training procedure, the feature distribution directly induced by the training loss L. where minimizing the loss value is the only supervisory signal for the feature points to move (Goodfellow al., 2016). This means that the sample density varies mainly along the orthogonal direction w.r.t. the loss contours, while the density along certain contour could be approximately considered as the same. For example, in the right panel of Fig. 1, the sample density induced by our MMC loss (detailed in Sec. 3.3) changes mainly along the radial direction, i.e., the directions of red arrows, where the loss contours are dashed concentric circles. Therefore, supposing [(z, y) = C. we choose AB = {z E IR (Z( y) E [C, C' + ACJJ, where AC > 0 is small value. Then Vol(A is the volume between the loss contours of C' and C' + AC for label y in the feature space.
3.2 THE SAMPLE DENSITY INDUCED BY THE GENERALIZED SCE LOSS
Generalized SCE loss. To better understand how the SCE loss and its variants (Pang et al., 2018; Wan et al., 2018) affect the sample density of features, we first generalize the definition in Eq. (1) as:
Cg-SCE (Z(x,, (g-sce(((x),)) y) = -1, log [softmax(h)].
where the logit h = (2) E R is a general transformation of the feature 2, for example, = W 2 in the SCE loss. We call this family of losses the generalized SCE (g-SCE) loss. Wan et al. (2018) propose the large-margin Gaussian Mixture (L-GM) loss, where h2 = (2 (4:) Ex (4;] mm. under the assumption that the learned features 2 distribute as mixture of Gaussian. Here μ4, and E, are extra trainable means and covariance matrices respectively, m is the margin, and 6,,y is the indicator function. Pang et al. (2018) propose the Max-Mahalas linear discriminant analysis (MMLDA) loss, where h1 = -1 || pi 2 under the similar mixture of Gaussian assumption, but the main difference is that μf are not trainable, but calculated before training with optimal inter-class dispersion. These two losses both fall into the family of the g-SCE loss with quadratic logits:
h = - pu: Ex u4) + B,,
where B, are the bias variables. Besides, note that for the SCE loss, there is
exp(W, = 4bbb))
exp( -11 2
1 W. 113 + b, + 113)
softmax(W= + b); Eier exp W, 2 2 bi) ElE(y) exp( (-1--= 1 Wil2 + by
According to Eq. (4), the SCE loss can also be regraded as a special case of the g-SCE loss with quadratic logits, where μ4; W. B, b |W,12 and Σ, I are identity matrices. Therefore, later when we refer to the g-SČE loss, we assume that the logits are quadratic as in Eq. (4) by default.
The contours of the g-SCE loss. To provide a formal representation of the sample density induced by the g-SCE loss, we first derive the formula of the contours, i.e., the closed-form solution of f.g-sc (zx) v) C' in the space of z, where C E (0, +00) is given constant, Let C. exp(C) (1, too), from Eq. (3), we can represent the contours as the solution of
log 1+ Eisy exp(hi)) =C - hy = log exp(hy)
Σ exp(h)) - log((C. -1. LY
(5)
The function in Eq. (5) does not provide an intuitive closed-form solution for the contours, since the existence of the term log Eivy exp(tu) However, note that this term belongs to the family of Log-Sum-Exp (LSE) function, which is a smooth approximation to the maximum function (Nesterov, 2005; Nielsen Sun, 2016). Therefore, we can locally approximate the function in Eq. (5) with
hy hg log(Ce 1),
(6)
4

---

Published as a conference paper at ICLR 2020
(4) g-SCE
(b) MMC
o0, loss value
Joes value
loss value
McG loss Valuee""
Ma Vallee
6 loss value:0
00, loss value oo
c0, loss value 0
00. loss value
The cuse: 02 > 0z
The cuse: 6 o'g
(Preferred by models since lower loss values)
Moving diecctonss
g-SCE loss. Reasonably learned features for a elassification task should distribute clusters, counter-intuiiive that the feature points tend to move to infinity pursue lower loss values when applying the g SCE loss. In contrast, MMC induces models to learn more siructured and onderly features.
Figure 2: Intuitive illustration on inherent limitations
where y arg 1 hy. the following text, we apply colored characters with tilde like I to better
visually distinguish them. According to Eq. (6), we can define (2.)(2) = log[exp(hg -,,) + 1] as the local approximation of the g-SCE loss nearby the feature point z, and substitute the neighborhood AB by ABy = (z E R (z) [C, C + AC]]. For simplicity, we assume scaled identity covariance matrix in Eq. (4), ie., Σg = J,I, where o; > 0 are scalars. Through simple derivations (detailed in Appendix A.I), we show that if σy * oy. the solution of (2 = C is a (d 1)- dimensional hypersphere with the center My.0 -dy= (d - gglgg); otherwise if og dg. the hypersphere-shape contour will degenerate
hyperplane.
The induced sample density. Since the approximation Eq. (6) depends on the specifie y and 7.
we define the training subset PA, = {(x,)) y) E Dly = k, ğ = k} and N, = [Da.al- Intuitively, DE,E includes the data with the true label of class k; while the highest prediction returned by the classifier is class k among other classes. Then we can derive the approximated sample density in the feature space induced by the g-SCE loss, as stated in the following theorem:
Theorem 1. (Proof in Appendix A.IJ Given (x, y) E D Z(x) and Cg.scE(2, u) = C, if there are Σ oxI, Σg ogl, and os # oi then the sample density nearby the feature point based on the approximation in Eq. (6) is
NE.K (C)
ORORIIAK Miel3
B B
SD(2) x
1-11 and B
+
(7)
B B
loR(C.-1)
(x))0))2
ok 0b
Ox o6-
where for the input-label pair in Dr.jo there is LgSSCE ~ (c.
Limitations of the g-SCE loss. Based Theorem 1 and the approximation in Eq. (6), let C* log(1 + exp( (o - ox))) and C = exp(C*), such that B log(C"--) = According 0x-0z
to Appendix A. if 0& > og. then C* will act as tight lower bound for C, i.e. the solution set of < C* is empty. This will make the training procedure tend to avoid this case since the loss C' cannot be further minimized to zero, which will introduce unnecessary biases on the returned predictions. On the other hand, if o < dg. C could be minimized to zero. However, when C -+ 0, the sample density will also tend zero since there B log(C.-1) - oo, which means the feature point will be encouraged to go further and further from the hypersphere center M only to make the loss value C' be lower, as intuitively illustrated in Fig. 2(a).
This counter-intuitive behavior mainly roots from applying the softmax function in training. Namely, the softmax normalization makes the loss value only depend on the relative relation among logits. This causes indirect and unexpected supervisory signals on the learned features, such that the points with low loss values tend to spread over the space sparsely. Fortunately, in practice, the feature point will not really move infinity, since the existence of batch normalization layers (loffe & Szegedy, 2015), and the squared radius from the center M increases as O log C') when minimizing the loss C. These theoretical conclusions are consistent with the empirical observations on the two wwo-dimensional features in previous work (cf. Fig. 1 in Wan et al. (2018)).
5

---

Published as a conference paper at ICLR 2020
Another limitation of the g-SCE loss is that the sample density is proportional to N which is on average N/L.². For example, there are around 1.3 million training data in ImageNet (Deng et al., 2009), but with a large number of classes L 1, 000, there are averagely less than two samples in each D k. These limitations inspire us to design the new training loss as in Sec 3.3.
Remark 1. of og (e.g., as in the SCE loss), the features with loss values in [C, C+ AC) will be encouraged to locate between two hyperplane contours without further supervision, and consequently there will not be explicit supervision on the sample density as shown in the left panel of Fig. 1.
Remark 2. Except for the g-SCE loss, Wen al. (2016) propose the center loss in order to improve the intra-class compactness of learned features, formulated as LCeeer (2(x), ), y) = -ll= the center By is updated based on mini-batch of learned features with label y in each training iteration. The center loss has to be jointly used with the SCE loss as Esce + XCComer. since simply supervise the DNNs with the center loss term will cause the learned features and centers to degrade to zeros (Wen et al., 2016). This makes it difficult to derive a closed-fie form formula for the induced sample density. Besides, the center loss method cannot concentrate on improving intra-class compactness, since it has to seek for trade-off between inter-class dispersion and intra-class compactness.
11. Here
3.3 MAX-MAHALLNOBEE CENTER LOSS
Inspired by the above analyses, we propose the Max-Mahalanobis center (MMC) loss to explicitly learn more structured representations and induce high-density regions in the feature space. The MMC loss is defined in regression form without the softmax function as
LMMC (Z()) v) = = 11
(8)
Here n=- = {14 hE(L) are the centers of the Max-Mahalanobis distribution (MMD) (Pang et al., 2018). The MMD is mixture of Gaussian distribution with identity covariance matrix and preset centers ja", where || ll2 = CMM for any / E [L]. and CN is hyperparameter. These MMD centers are invari- able during training. which are crafted according to the criterion: μ arg min, p maxiş (M): mz)) Intuitively, this criterion is to maximize the minimal angle between any two centers, which can provide optimal inter-class dispersion as shown in Pang al. (2018). In Appendix B. we provide the generation algorithm for μ in MMC. We derive the sample density induced by the MMC loss in the feature space, as stated Theorem 2. Similar to the previously introduced notations, here we define the subset Dr- = {(x, E Dly k} and N₂ [Da].
Theorem 2. (Proof in Appendix A.2) Given (æ, y) E Dr. 2 = Z(x) and LxMc(2, y) = C, the sample density nearby the feature point z
NO N (C)
(ID &
0²21
where for the input-label pair in Dr, there is LMMC ps(c).
According to Theorem 2, there are several attractive merits of the MMC loss, as described below.
Inducing higher sample density. Compared Theorem 1, the sample density induced by MMC is proportional to N& rather than N & where N₂ is on average N/L. It facilitates producing higher sample density. Furthermore, when the loss value C' is minimized to zero, the sample density will exponentially increase according to Eq. (9), as illustrated in Fig. 2(b). The right panel of Fig. also provides an intuitive insight on this property of the MMC loss: since the loss value C' is proportional to the squared distance from the preset center μ the feature points with lower loss values are certain to locate in smaller volume around the center. Consequently, the feature points of the same class are encouraged to gather around the corresponding center, such that for each sample, there will be locally enough data in its neighborhood for robust learning (Schmidt et al., 2018). The MMC loss value also becomes a reliable metric of the uncertainty on returned predictions.
Better exploiting model capacity. Behind the simple formula, the MMC loss can explicitly monitor inter-class dispersion by the hyperparameter CAM- while enabling the network to concentrate on minimizing intra-class compactness in training. Instead of repeatedly searching for internal trade- off in training as the center loss, the monotonicity of the supervisory signals induced by MMC can better exploit model capacity and also lead faster convergence, empirically shown in Fig. 3(a).

---

Published as a conference paper at ICLR 2020
70-
A fiee-n
60
Text crtor rate (%)
Iter10
40
30
Xy
10
50
20
Truining time (hours)
30
* Clean isput
Xtten-40
A
4 Adaptive untargeted ottock
Uenker
40
2 Adaptive targeted attack
3000-113
S0ME-100
LAU
ttee-50
Non-odapiioo tongeted
(a) Test erroe rates w.r. training time
(h) Visualization in the learned feature space of MNIST
Figure 3: (a) Test error rates on clean images w.rt training time on CIFAR-I0. Here AT refers to 10-steps targeted PGD adversarial training. ie.. Al (b) Two-dimensional visualization the attacks on trained MMC networks the feature space MNIST, For
there 0.3 with step size of 0.01. The total number of iteration steps is 50, where lter indicates the perturbed fealures at with iteration step.
Avoiding the degradation problem. The MMC loss can naturally avoid the degradation problem encountered in Wen et al. (2016) when the center loss is not jointly used with the SCE loss, since the preset centers 1 for MMC are untrainable. In the test phase, the network trained by MMC can still return a normalized prediction with the softmax function. More details about the empirical superiorities the MMC loss over other previous losses are demonstrated in Sec. 4.
Remark 3. In Appendix B.2, we discuss why the squared-error form in Eq. (8) is preferred compared to, e.g., the absolute form or the Huber form in the adversarial setting. We further introduce flexible variants of the MMC loss in Appendix B.3, which can better adapt various tasks.
Remark 4. Pang et al. (2018) propose a Max- Maahllnoois linear discriminant analysis (MMLDA) method, which assumes the features to distribute an MMD. Due to the Gaussian mixture assumption, the training loss for the MMLDA method is obtained by the Bayes' theorem as
exp(
2
11
exp(2 T (4)) EiE(2) exp(= (xsp(=TT) 14))
LMMLDA (Z(D))((())))))))))))))))) - llog
log
(10)
E [eer)] exp(
p
2
Note that there is Σ, 1/ in Eq. (4) for the MMLDA loss, similar with the SCE loss. Thus the MMLDA method cannot explicitly supervise on the sample density and induce high-density regions in the feature space, analyzed in Sec. 3.2. Compared to the MMLDA method, the MMC loss introduces extra supervision on intra-class compactness, which facilitates better robustness.
EXPERIMENTS
In this section, we empirically demonstrate several attractive merits of applying the MMC loss. We experiment on the widely used MNIST, CIFAR-10, and CIFAR- 100 datasets (Krizhevsky & Hinton, 2009; LeCun al., 1998). The main baselines for the MMC loss are SCE (He et al., 2016), Center loss (Wen et al., 2016), MMLDA (Pang et al., 2018), and L-GM (Wan et al., 2018). The codes are provided in https:/ i thub. ccom/P233
anooii a- Tra iring.
4.1 PERFORMANCE ON THE CLEAN INPUTS
The network architecture applied is ResNet-32 with five core layer blocks (He al., 2016). Here we use MMC-10 to indicate the MMC loss with CMM 10, where CMм is assigned based on the cross-validation results in Pang et al. (2018). The hyperparameters for the center loss, L-GM loss and the MMLDA method all follow the settings in the original papers (Pang et al., 2018; Wan et al., 2018; Wen et al., 2016). The pixel values are scaled to the interval [0, 1). For each training loss with or without the AT mechanism, we apply the momentum SGD (Qian, 1999) optimizer with the initial learning rate of 0.01, and train for 40 epochs on MNIST, 200 epochs on CIFAR- 10 and CIFAR- The learning rate decays with factor of 0.1 100 and 150 epochs, respectively.
7

---

Published as a conference paper at ICLR 2020
Table I: Classification sccuracy (8) on the while box udversarial examples crafted on the test set of CIFAR-10. The supersctipe far indicates targeted attacks, while un indicates untargeted attacks. subseripts indicate the number iteration stops when performing attacks. The results wazt the MMC loss reported under the adaplive versions of difterent altacks. The nctation represents accuracy less than 1%. The MMC (rand) is an ablation study where the class centers are uniformly sampled on the hypersphere.
Perturbation = 8/255
Perturbation e 16/255
Methods
Clean
PGD
PGD
PGD for
PG
PGD
PGD
PGD
PGD
SCE
92.9
S
3.7
S
3.6
S
2.9
S
2.6
Center loss
92.8
1
4.4
S
4.3
S
3.1
$
2.9
MMLDA
92.4
S
16.5
9.7
S 1
6.7
S
5.5
L-GM
92.5
37.6
19.8
8.9
4.9
26.0
11.0
2.5
2.8
MMC-10 (rand)
92.3
43.5
29.2
20.9
18.4
31.3
17.9
8.6
11.6
MMC-10
92.7
48.7
36.0
26.6
24.8
36.1
25.2
13.4
17.5
ATTO (SCE)
83.7
70.6
49.7
69.8
47.8
48.4
26.7
31.2
16.0
AT (MMC-10)
83.0
69.2
54.8
67.0
53.5
58.6
47.3
44.7
45.1
TT (SCE)
80.9
69.8
55,4
69.4
53.9
53.3
34.1
38.5
21.5
ATTS (MMC-10)
81.8
70.8
56.3
70.1
55.0
54.7
37.4
39.9
27.7
PGD E 8/256
MIM E 8/256
PGD
16/256
MIM
16/256
Subssitute Modill
5.44
0.61
5.05
8.03
5.61
0.21
0:2
0.01
5.14
0.11
0.73
XDCD
DAT
0:31
0.82
0:50
0.39
0.03
0.60
0.49
033
0.01
0.00
0.38
0.47
0.00
0.74
D66
0.0
8:57
8M
0.54
4:48
0.2
0.74
AT (SCE)
7.79
077
0999
677
0.74
8
0.37
0.41
8:45
0.34
0.4
1MM
281
0.00
0.00
0.70
0.5
2.08
0.22
0.50
0.50
0.00
230
0.5
0.50
0.22
0.1
2:5
0.59
042
0.11
0.25
2.44
0.53
AT (MMIC-10
SCE
MMMC-j
MMC lee
GrN...1 (p6)
sCt
MMC
MMC Ipe
(r
AT (MMM:
GMMC me
SCB
MMC.
AT (RCB)
AT 6m
At HAMM (80)
SCB
MBC.
104C-100
AT ArOMeri
MMMM (P6)
Target Model
Figure 4: Classification accuracy under the black-box transfer- hassee attacks on the test set of CIFAR-10. The substince model refers to the
adversarial examples, the torget model
that adversary actually intends fool. Here refers AT
When applying the AT mechanism (Madry et al., 2018), the adversarial examples for training crafted by 10-steps targeted or untargeted PGD with = 8/ /55.. In Fig. 3(a), we provide the curves of the test error rate w.r.t. the training time. Note that the MMC loss induces faster convergence rate and requires little extra computation compared to the SCE loss and its variants, while keeping comparable performance on the clean images. In comparison, implementing the AT mechanism computationally expensive in training and will sacrifice the accuracy on the clean images.
4.2 ADAPTIVE ATTACKS FOR THE MMC LOSS
As stated in Athalye al. (2018), only applying the existing attacks with default hyperparameters is not sufficient to claim reliable robustness. Thus, we apply the adaptive versions of existing attacks when evading the networks trained by the MMC loss (detailed in Appendix B.4). For instance, the non-adaptive objectives for PGD are variants of the SCE loss (Madry et al., 2018), while the adaptive objectives are - (2, y) and LMMC y/) the untargeted and targeted modes for PGD, respectively. Here y/ is the target label. To verify that the adaptive attacks are more effective than the non-adanaa adaptive ones, we modify the network architecture with a -iodimensionnl feature layer and visualize the PGD attacking procedure in Fig. 3(b). The two panels separately correspond to two randomly selected clean inputs indicated by black stars. The ten colored clusters each panel consist class.
of the features of all the 10,000 test samples in MNIST, where each color corresponds We can see that the adaptive attacks are indeed much more efficient than the non-adaptive one.
4.3 PERFORMANCE UNDER THE WHITE-BOR ATTACKS
We first investigate the white-bon box lcc distortion setting using the PGD attack, and report the results in Table 1. According to Carlini et al. (2019), we evaluate under different combinations of the attacking parameters: the perturbation e, iteration steps, and the attack mode, i.e., targeted untargeted.
8

---

Published as a conference paper at ICLR 2020
Table 2: Experiments on CIFAR-10. Part I: Averaged la distortion of the white-box adversarial examples crafted by C&W with 1,000 itcration stops. Part HI: Classification accuracy (%) under the block-box SPSA attack. Part III: Classification accuracy (%) under general transoomations. The standaard deviation for the Caussian noise U,05, the degree range 3:30" for random rotation.
Part
Part (c [==80 8 3/55)
Part II (c= 16/255)
Part III
Methods
C
C&W
SPSA Ur
SPSA10
SPSA 10
SPSA10 An
Noisc
Rotation
SCE
0.12
0.07
12.3
1.2
5.1
S
52.0
83.5
Center loss
0.13
0.07
21.2
6.0
10.6
2.0
55.4
84.9
MMLDA
0.17
0.10
25.6
13.2
11.3
5.7
57.9
84.8
L-GM
0.23
0.12
61.9
45.9
46.1
28.2
59.2
82.4
MMC-10
0.34
0.17
69.5
56.9
57.2
41.5
69.3
87.2
ATT (SCE)
1.19
0.63
81.1
67.8
77.9
59.4
82.2
76.0
ATTS (MMC-10)
1.91
0.85
79.1
69.2
74.5
62.7
83.5
75.2
ATT0 (SCE)
1.26
0.68
78.8
67.0
73.7
60.3
78.9
73.7
AT (MMC-10)
1.55
0.73
80.4
69.6
74.6
62.4
80.3
75.8
Following the setting in Madry al. (2018). we choose the perturbation 8/255 and 16/255, with the step size be 2/255. We have also run PGD-101 and PGD-200 attacks, and find that the accuracy converges compared to PGD-50. each PGD experiment, we ran several times with different random restarts to guarantee the reliability of the reported results.
Ablation study. To investigate the effect on robustness induced by high sample density in MMC, we substitute uniformly sampled center set (Liu et al., 2018; Duan et al., 2019), i.e., n²¹ {ur he(L) for the MM center set μ=,, and name the resulted method as "MMC-10 (rand)" as shown Table I. CMm. but μ is no longer the solution of the min-max problem in Sec. 3.3,
There is also ||xT
From the results in Table 1, we can see that higher sample density alone in "MMC-10 (rand)" can already lead to much better robustness than other baseline methods while using the optimal center set ju" as in "MMC-10" can further improve performance. When combining with the AT mechanism, the trained models have better performance under the attacks different from the one used to craft adversarial examples for training, e.g, PGD with c = 16/255.
under the adaptive attacks,
Then investigate the white-box l2 distortion setting. We apply the C&W attack, where it has binary search mechanism to find the minimal distortion to successfully mislead the classifier under the untargeted mode, lead the classifier to predict the target label in the targeted mode. Following the suggestion in Carlini & Wagner (201 7a), we set the binary search steps to be 9 with the initial constant e 0.01. The iteration steps for each value of are to be 1,000 with the learning rate of 0.005. In the Part I of Table 2, we report the minimal distortions found by the C&W attack. As expected, requires much larger distortions to successfully evade the networks trained by MMC.
4.4 PERFORMANCE UNDER THE BLACK- BOX ATTACKS
As suggested in Carlini al. (2019), providing evidence of being robust against the black-box attacks is critical to claim reliable ro- bustness. We first perform the transfer-based attacks using PGD and MIM. Since the targeted attacks usually have poor transferabil- ity (Kurakin al., 2018), we only focus on the untargeted mode in this case, and the results are shown in Fig. 4. We further perform the gradient-free attacks using the SPSA method and report the results in the Part II of Table 2. To perform numerical approximations on gradients in SPSA, we set the batch size to 128, the learning rate is 0.01, and the step size of the finite difference is 8 0.01, suggested by Uesato et al. (2018). We also evaluate under stronger SPSA attacks with batch size to be 4096 and 8192 in Table 3, where the 8/255. With larger batch sizes, we can find that the accuracy under the black-bor box SPSA attacks converges to it under the white-box PGD attacks. These results indicate that training with the MMC loss also leads to
Accuracy (%) MMC- under SPSA with different hatch sizes.
CIFAR-10
Batch
SPSA un
SPSA
128
57.0
69.0
4096
41.0
52.0
8192
37.0
49.0

---

Published as a conference paper at ICLR 2020
Tablle 4: Experiments on CIEAR-100. Part I: Classdication accuracy (1) on the clean test samples. Part II: Classfication accuracy (58 under the white-box PGD attacks and block-box SPSA attack. attacks are adaptive for MMC. Hore the batch size SPSA 128. Part III: Averaged distortion of the white-box adversarial examples crafted by C&W with 1,000 iteration steps and binary search epochs.
Part I
Part PGD
8/255)
Part
Methods
Clean
PGD
SPSA tar
SPSA
C&
C&w
SCE
72.9
1
8.0
14.0
1.9
0.16
0.047
Center
72.8
S 1
10.2
14.7
2.3
0.18
0,048
MMLDA
72.2
1
13.9
18.5
5.6
0.21
0.050
L-GM
71.3
15.8
15.3
22.8
7.6
0.31
0.063
MMC-10
71.9
23.9
23.4
33.4
15.8
0.37
0.085
robustness under the black-box attacks, which verifies that our method can induce reliable robustness, rather than the false one caused by, e.g., gradient mask (Athalye et al., 2018).
4.5 PERFORMANCE UNDER THE GENERAL-PPRPONS ATTACKS
To show that our method is generally robust, we further test under the general-purpose attacks (Carlini al., 2019). We apply the Gaussian noise (Fawzi al., 2016; Gilmer al., 2019) and rotation transformation (Engstrom al., 2019), which are not included in the data augmentation for training. The results are given in the Part III of Table 2. Note that the AT methods are less robust to simple transformations like rotation, as also observed in previous work (Engstrom al., 2019). In comparison, the models trained by the MMC loss are still robust to these easy-to-apply attacks.
4.6 EXPERIMENTS ON CIFAR- 100
In Table 4 and Table 5, provide the results CIFAR- 100 under the white-box PGD and C&W attacks, and the black-box gradient-free SPSA attack. The hyperparameter setting for each attack the same it CIFAR-IC Compared previous defense strategies which also evaluate on CIFAR-100 100 (Pang et al., 2019; Mustafa et al., 2019), MMC can improve robustness more significantly. while keeping better performance on the clean inputs. Compared to the results on CIFAR- 10, the averaged distortion of C& W CIFAR- 100 is larger for successful targeted attack and is much smaller for a successful untargeted attack. This because when only the number of classes increases, e.g., from 10 to 100, it is casier to achieve coarse untargeted attack, but harder to make a subtle targeted attack. Note that in Table 5, we also train on the ResNet- model with eighteen core block layers except for the ResNet-32 model. The results show that MMC can further benefit from deep network architectures and better exploit model capacity to improve robustmess. Similar properties are also observed in previous work when applying the AT methods (Madry et al., 2018). In contrast, as shown in Table 5, the models trained by SCE are comparably sensitive to adversarial perturbations for different architectures, which demonstrates that SCE cannot take full advantage of the model capacity to improve robustness. This verifies that MMC provides effective robustness promoting mechanism like the AT methods, with much less computational cost.
CONCLUSION
In this paper, we formally demonstrate that applying the softmax function in training could potentially lead unexpected supervisory signals. To solve this problem, propose the MMC loss to learn more structured representations and induce high-density regions in the feature space. In our experiments, empirically demonstrate several favorable merits of our method: (i) Lead to reliable robustness even under strong adaptive attacks in different threat models; (ii) Keep high performance on clean inputs comparable to SCE; (iii) Introduce little extra computation compared to the SCE loss; (iv) Compatible with the existing defense mechanisms, e.g., the AT methods. Our analyses in this paper also provide useful insights for future work on designing new objectives beyond the SCE framework.
10

---

Published as a conference paper at ICLR 2020
ACKNOWLEDGEMENTS
This work was supported by the National Key Research and Development Program of China (No. 2017YTA0700904), NSFC Projects (Nos. 61620106010, U19B2034, 01811461), Beijing NSF Project (No. L172037), Beijing Academy of Artificial Intelligence (BAAI) Tsinghua-Huawei Joint Research Program. grant from Tsinghua Institute for Guo Qiang, Tiangong Institute for Intelligent Computing, the JP Morgan Faculty Research Program and the NVIDIA NVAIL Program with GPU/DGX Acceleration.
REFERENCES
Anish Athalye, Nicholas Carlini, and David Wagner. Obfuscated gradients give a false sense of security: Circumventing defenses to adversarial examples. In International Conference on Machine Learning (ICML), 2018.
Battista Biggio, Igino Corona, Davide Maiorca, Blaine Nelson, Nedim Šindić, Pavel Laskov, Giorgio Giacinto, and Fabio Roli. Evasion attacks against machine learning at test time. In Joint European conference machine learning and knowledge discovery in databases, pp. 387-402. Springer, 2013.
Nicholas Carlini and David Wagner. Towards evaluating the robustness of neural networks. In IEEE Symposium Security and Privacy (S&P), 2017a.
Nicholas Carlini and David Wagner. Adversarial examples are easily detected: Bypassing ten detection methods. In ACM Workshop on Artificial Intelligence and Security (AlSec), 2017b.
Nicholas Carlini, Anish Athalye, Nicolas Papernot, Wieland Brendel, Jonas Rauber, Dimitris Tsipras, Ian Goodfellow, Aleksander Madry, and Alexey Kurakin. On evaluating adversarial robustness. arXiv preprint arXiv: 190..00900.00055. 2019.
Yair Carmon, Aditi Raghunathan, Ludwig Schmidt, Percy Liang, and John C Duchi. Unlabeled data improves adversarial robustness. In Advances in Neural Information Processing Systems (NeurIPS), 2019.
Pin-Yu Yu Chen, Huan Zhang, Yash Sharma, Jinfeng Yi, and Cho-Jui Hsieh. Zoo: Zeroth order optimization based black-box attacks to deep neural networks without training substitute models. In Proceedings of the 10th ACM Workshop on Artificial Intelligence and Security (AlSec), pp. 15-26. ACM, 2017.
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 248-255, IEEE, 2009.
Jiankang Deng, Jia Guo, Niannan Xue, and Stefanos Zafeiriou. Areface: Additive angular margin loss for deep face recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 4690-4699, 2019.
Yinpeng Dong, Fangzhou Liao, Tianyu Pang. Hang Su, Jun Zhu, Xiaolin Hu, and Jianguo Li. Boosting adversarial attacks with momentum. In Proceedings of the IEEE Conference Computer Vision and Pattern Recognition (CVPR), 2018.
Yueqi Duan, Jiwen Lu, and Jie Zhou. Uniformface: Learning deep equidistributed representation for face recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3415-3424, 2019.
Krishnamurthy Dvijotham, Sven Gowal, Robert Stanforth, Relja Arandjelovic, Brendan O'Donoghue, Jonathan Uesato, and Pushmeet Kohli. Training verified learners with learned verifiers. arXiv preprint arXiv: 1805. 10265, 2018a.
Krishnamurthy Dvijotham, Robert Stanforth, Sven Gowal, Timothy Mann, and Pushmeet Kohli. A dual approach to scalable verification of deep networks. In Annual Conference on Uncertainty in Artificial Intelligence (UAI), 2018b.
11

---

Published as a conference paper at ICLR 2020
Logan Engstrom, Brandon Tran, Dimitris Tsipras, Ludwig Schmidt, and Aleksander Madry. A rotation and a translation suffice: Fooling cnns with simple transformations. In International Conference Machine Learning (ICML), 2019.
Alhussein Fawzi, Seyed-Mohsen Moosavi-Dezfooli, and Pascal Frossard. Robustness of classi- fiers: from adversarial to random noise. In Advances in Neural Information Processing Systems (NeurIPS), pp. 1632-1640, 2016.
Alhussein Fawzi, Hamza Fawzi, and Omar Fawzi. Adversarial vulnerability for any classifier. In Advances in Neural Information Processing Systems (NeurIPS), 2018.
Jerome Friedman, Trevor Hastie, and Robert Tibshirani. The elements of statistical learning, volume 1. Springer series in statistics New York, 2001.
Justin Gilmer, Nicolas Ford, Nicolas Carlini, and Ekin Cubuk. Adversarial examples are a natural consequence of test error in noise. In International Conference on Machine Learning (ICML), 2019.
lan Goodfellow, Yoshua Bengio, and Aaron Courville. Deep Learning. MIT Press, 2016. http: //www. www.deepleerninggook. org.
lan J Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. In International Conference on Learning Representations (ICLR), 2015.
Yandong Guo and Lei Zhang. One-shot face recognition by promoting underrepresented classes.
arXiv preprint arXiv: 7055055744 2017.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Identity mappings in deep residual networks. In European Conference on Computer Vision (ECCV), pp. 630-645. Springer, 2016.
Matthias Hein and Maksym Andriushchenko. Formal guarantees on the robustness of a classifier against adversarial manipulation. In Advances in Neural Information Processing Systems (NeurIPS), pp. 2266-2276, 2017.
Cormac Herley and Paul C Van Oorschot. Sok: Science, security and the clusive goal of security scientific pursuit. In 2017 IEEE Symposium on Security and Privacy (S&P), pp. 99-120. IEEE, 2017.
Andrew Ilyas, Logan Engstrom, Anish Athalye, and Jessy Lin. Black-box adversarial attacks with limited queries and information. In International Conference on Machine Learning (ICML), 2018.
Andrew Ilyas, Shibani Santurkar, Dimitris Tsipras, Logan Engstrom, Brandon Anish Athalye, Tran, and Aleksander Madry. Adversarial examples are not bugs, they are features. In Advances in Neural Information Processing Systems (NeurIPS), 2019.
Sergey loffe and Christian Szegedy. Batch normalization: Accelerating deep network training by reducing internal covariate shift. In International Conference on Machine Learning (ICML), pp. 448-456, 2015.
John David Jackson. Classical electrodynamics. American Journal of Physics, 1999.
Harini Kannan, Alexey Kurakin, and Ian Goodfellow. Adversarial logit pairing. arXiv preprint arXiv:1803.06373, 2018.
Alex Krizhevsky and Geoffrey Hinton. Learning multiple layers of features from tiny images. Technical report, Citeseer, 2009.
Alexey Kurakin, Ian Goodfellow, and Samy Bengio. Adversarial examples in the physical world. In The International Conference on Learning Representations (ICLR) Workshops, 2017.
Alexey Kurakin, Ian Goodfellow, Samy Bengio, Yinpeng Dong, Fangzhou Liao, Ming Liang, Tianyu Pang, Jun Zhu, Xiaolin Hu, Cihang Xie, et al. Adversarial attacks and defences competition. arXiv preprint arXiv 180::880000000 00097, 2018.
12

---

Published as a conference paper at ICLR 2020
Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner. Gradient- learning applied to document recognition. Proceedings of the IEEE, 86(11):2278-235 2324, 1998.
Xuezhi Liang, Xiaobo Wang, Zhen Lei, Shengeai Liao, and Stan Li. Soft-margin softmax for deep classification. In International Conference on Neural Information Processing (ICNIP), pp- 413-421. Springer, 2017.
Weiyang Liu, Yandong Wen, Zhiding Yu, and Meng Yang. Large-margin softmax loss for convolu- tional neural networks. In International Conference on Machine Learning (ICML), 2016.
Weiyang Liu, Yandong Wen, Zhiding Yu, Ming Li, Bhiksha Raj, and Le Song. Sphereface: Deep hypersphere embedding for face recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), pp. 212-220, 2017.
Weiyang Liu, Rongmei Lin, Zhen Liu, Lixin Liu, Zhiding Yu, Bo Dai, and Le Song. Learning towards minimum hyperspherical energy. In Advances in Neural Information Processing Systems (NeurIPS), pp. 6222-6222 2223,, 2018.
Pavel Loskot and Norman C. Beaulieu. On monotonicity of the hypersphere volume and area. Journal of Geometry, 2007.
Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. International Conference on Learning Representations (ICLR), 2018.
Seyed- Seyed-Mohsen Moosavi-Dezfooli: Alhussein Fawzi, and Pascal Frossard. Deepfool: a simple and accurate method to fool deep neural networks. In Proceedings of the IEEE Conference Computer Vision and Pattern Recognition (CVPR), pp. 2574 2016.
Aamir Mustafa, Salman Khan, Munawar Hayat, Roland Goecke, Jianging Shen, and Ling Shao. Adversarial defense by restricting the hidden space of deep neural networks. In International Conference Computer Vision (ICCV), 2019.
Yu Nesterov, Smooth minimization of non-smooth functions. Mathematical programming. 103(1): 127-152, 2005.
Anh Nguyen, Jason Yosinski, and Jeff Clune. Deep neural networks are easily fooled: High confidence predictions for unrecognizable images. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 427-436, 2015.
Frank Nielsen and Ke Sun. Guaranteed bounds the kullback-leibler divergence of univariate mixtures. IEEE Signal Processing Letters, 23(11))1543-1555 2016.
Tianyu Pang, Chao Du, and Jun Zhu. Max- maxalanaassssss linear discriminant analysis networks. In International Conference on Machine Learning (ICML), 2018
Tianyu Pang, Kun Xu, Chao Du, Ning Chen, and Jun Zhu. Improving adversarial robustness via promoting ensemble diversity. In International Conference on Machine Learning (ICML), 2019.
Nicolas Papernot, Patrick McDaniel, Somesh Jha, Matt Fredrikson, Z Berkay Celik, and Ananthram Swami. The limitations of deep learning in adversarial settings. In IEEE European Symposium Security and Privacy (EuroS&P), pp. 372-387. IEEE, 2016.
Nicolas Papernot, Patrick McDaniel, Arunesh Sinha, and Michael Wellman. Towards the science of security and privacy in machine learning. In European Symposium on Security and Privacy (EuroS&P), 2018.
Xianbiao Qi and Lei Zhang. Face recognition via centralized coordinate learning, arXiv preprint 1001v:1801.050 2018.
Ning Qian. On the momentum term in gradient descent learning algorithms. Neural networks, 12(1): 145-151, 1999.
13

---

Published as a conference paper at ICLR 2020
Ludwig Schmidt, Shibani Santurkar, Dimitris Tsipras, Kunal Talwar, and Aleksander Madry. Adver- sarially robust generalization requires more data. In Advances in Neural Information Processing Systems (NeurIPS), pp. 5019-5031, 2018.
Florian Schroff, Dmitry Kalenichenko, and James Philbin. Facenet: A unified embedding for face recognition and clustering. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 815-823, 2015.
Ali Shafahi, Mahyar Najibi, Amin Ghiasi, Zheng Xu, John Dickerson, Christoph Studer, Larry S Davis, Gavin Taylor, and Tom Goldstein. Adversarial training for free! In Advances in Neural Information Processing Systems (NeurIPS), 2019.
Robert Stanforth, Alhussein Fawzi, Pushmeet Kohli, et al. Are labels required for improving adversarial robustness? In Advances in Neural Information Processing Systems (NeurIPS). 2019.
Yi Sun, Yuheng Chen, Xiaogang Wang, and Xiaoou Tang. Deep learning face representation by joint verification. In Advances in Neural Information Processing Systems (NeurIPS), pp- 2014.
Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, lan Goodfellow, and Rob Fergus. Intriguing properties of neural networks. International Conference on Learning Representations (ICLR), 2014.
Jonathan Uesato, Brendan O' O'Donoghue, Aaron van den Oord, and Pushmeet Kohli. Adversarial risk and the dangers of evaluating against weak attacks. In International Conference on Machine Learning (ICML), 2018
Weitao Wan, Yuanyi Zhong, Tianpeng Li, and Jiansheng Chen. Rethinking feature distribution for loss functions in image classification. In Proceedings of the IEEE Conference on Computer Vision and Puttern Recognition (CVPR), pp. 9117-9126, 2018.
Feng Wang, Xiang Xiang, Jian Cheng, and Alan Loddon Yuille. Normface: 12 hypersphere embedding for face verification. In Proceedings of the 25th ACM international conference Multimedia (ACM MM), pp. 1041-1049. ACM, 2017.
Hao Wang, Yitong Wang, Zheng Zhou, Xing Ji, Dihong Gong, Jingchao Zhou, Zhifeng Li, and Wei Liu. Cosface: Large margin cosine loss for deep face recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1265-5274, 2018.
Yandong Wen, Kaipeng Zhang, Zhifeng Li, and Yu Qiao. A discriminative feature learning approach for deep face recognition. In European Conference on Computer Vision (ECCV), pp. 499-515. Springer, 2016.
Eric Wong and Zico Kolter. Provable defenses against adversarial examples via the convex outer adversarial polytope. In International Conference on Machine Learning (ICML), pp. 5283-5292, 2018.
Eric Wong, Frank Schmidt, Jan Hendrik Metzen, and J Zico Kolter. Scaling provable adversarial defenses. In Advances in Neural Information Processing Systems (NeurIPS), pp. 8400-8409, 2018.
Kai Y Xiao, Vincent Tjeng, Nur Muhammad Shafiullah, and Aleksander Madry. Training for faster adversarial robustness verification via inducing relu stability. In International Conference Learning Representations (ICLR), 2019.
Cihang Xie, Yuxin Wu, Laurens van der Maaten, Alan Yuille, and Kaiming He. Feature denoising for improving adversarial robustness. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2019.
Dinghuai Zhang, Tianyuan Zhang, Yiping Lu, Zhanxing Zhu, and Bin Dong. You only propagate once: Accelerating adversarial training via maximal principle. In Advances in Neural Information Processing Systems (NeurIPS), 2019a.
Hongyang Zhang, Yaodong Yu, Jiantao Jiao, Eric P Xing, Laurent El Ghaoui, and Michael Jordan. Theoretically principled trade-off between robustness and accuracy. In International Conference Machine Learning (ICMI.), 2019b.
14

---

Published as a conference paper at ICLR 2020
A
PROOF
In this section, we provide the proof of the theorems proposed the paper.
A.1 PROOF OF THEOREM 1
According to the definition of sample density
AN (I)
(B)
separately calculate AN and Vol (B). Since fg-SCE (s,ū(()) for the data points in D2 k, recall that AB = {z E Rd CA-SCE E [C,C + AC]], then there is
AN |Z k AB
(11)
~Ng,e' (.. AC.
Now we calculate Vol (B) by approximating it with Vol(A We first derive the solution of Ly.g = C. For simplicity, we assume scaled identity covariance matrix, i.e,, E, J,I, where 0; > are scalars. Then Vi,j E [L]. e is any constant, o; * o, the solution of h; - h3 is (d-1) (1-1))diimenional hypersphere embedded in the d-dimensional space of the feature
where Mij- J.H; Dip--oyH4 Br 0,0y1lu - M (0i-o,)
B, B- (12)
M 1 = Bi,
aj-oj
0 Jj
0: Ji-Oj
Note that each value of corresponds specific contour, where M.j and B can be regraded as constant w.r.t. When B, < (o, - -0, -,) the solution set becomes empty. Specially, if
0==0, ; a, the hypersphere-shape contour will degenerate [[]uul1 || 12
hyperplane: 2 2 14y) + o "B, B, c)]. For example, for the SCE loss, the solution of the contour
".WW W3) by by + For more general E, the conclusions similar, e.g., the solution in Eq. (12) will become hyperellipse. Now easy to show that the solution of Ly.5 C when
y k. A is the hypersphere:
log(Ce- 11
==
M BR,E
(13)
σk - og
According to the formula of the hypersphere surface area (Loskot & Beaulieu, 2007), the volume of AB B is
2T $
log(C. - 1) B K.
Vol( QMAB (
AC,
(14)
v.a)
T(
where I'()) is the gamma function. Finally approximate the sample density
AN
SD(2)
A BB
No. P (C)
(15)
B + log(C. Co-)
ou O5
A.2 PROOF OF THBBOREM 2
Similar to the proof of Theorem 1, there is
AN = (Dx)) AB
(16)
N2 pa (CC) AC.
Unlike for the g g-SCE, exactly calculate Vol( AB) for the MMC loss. Note that the solution of LMMC = C is the hypersphere:
|| 17 2C.
(17)
15

---

Published as a conference paper at ICLR 2020
21
33
44
M1
f2
112
L = 2
L =
L = 4
Figure Intuitive illustration the Max Mahalanobis in the of 3,4.
According to the formula of the hypersphere surface area (Loskot & Beaulieu, 2007), we have
VoI(AB) 2 TiC
AC,
(18)
(
where r(-) is the gamma function. Finally obtain the sample density
AN
SID(2) AB
(19)
NO PM (C) &
C 7
L
B TECHNICAL DETAILS
In this section, provide more technical details applied in paper. Most of our experiments are conducted on the NVIDIA DGX-1 server with eight Tesla P100 GPUs.
B.1 GENERATION ALGORITHM FOR THE MAX-MAHALANORIS CENTERS
We give the generation algorithm for crafting the Max-Mahalando Centers in Algorithm 1, proposed by Pang et al. (2018). Note that there are two minor differences from the originally proposed algorithm. First is that in Pang al. (2018) they use C' = | g. while Second is that we denote the feature E Rdd, while they denote 2 E RR. The Max-Mahalanobis centers generated in the low-dimensional cases quite intuitive and comprehensible Fig. 5. For examples, when segment; when 3, they are the three vertexes of equilateral triangle; when L they are the four vertexes of a regular tetrahedron.
use CMM ||
shown in 2, the Max-Mahalanobis centers are the two vertexes of a line
Algorithm 1
McentesssMMenterss
Input: The constant Cмм. the dimension of vectors d and the number of classes L. (L S d 1) Initialization: Let the mean vectors be μ 61 and ne Og, i # Here 61 and 02 separately denote the first unit basis vector and the zero vector in Rdd for i = 2 to L do for to i do
(J) = - -[1 +1+ (U) PG) (L 1)1/11)))) 1)]
end for
(i) /1 - ||m 12
end for
for l: = to L, do
15 Cмм 2
end for
Return: The optimal mean vectors 14, E [Z].
16

---

Published as a conference paper at ICLR 2020
B.2 WHY THE SQUARED-ERRON FORM PREFERRED
In the feature space, penalizing the distance between the features and the prefixed centers can be regarded as a regression problem. In the MMC loss, we apply the squared-error form as || - m Other substitutes could be the absolute form || M5lla or the Huber form. As stated in Friedman et al. (2001), the absolute form and the Huber form are more resistant to the noisy data (outliers) or the misspecification of the class labels, especially in the data mining applications. However, in the classification tasks that we focus on in this paper, the training data is clean and reliable. Thus the squared-erro form can lead high accuracy with faster convergence rate compared other forms. Furthermore, in the adversarial setting, the adversarial examples have similar properties as the outliers. When we apply the AT mechanism in the training procedure, we expect the classifiers to pay more attention to the adversarial examples, i.e., the outliers. Note that this goal is the opposite of it in the data mining applications, where outliers are intended to be ignored. Therefore, due to the sensitivity to the outliers, the squared-error form can better collaborate with the AT mechanism to improve robustness.
Besides, the MMC loss can naturally perform stronger AT mechanism without additional regularizer term. Specifically, let x be the clean input, x= be the adversarial example crafted based on z, then in the adversarial logit pairing (ALP) method (Kannan et al., 2018), there is an extra regularizer except for SCE as:
[|=(x)) =(xx))12
(20)
When adding x² as an extra training point for MMC, then the MMC loss will minimize [|=(z))-1) 16 12 ||=(==)) My 11, which is an upper bound for 12(x) - >(x"))| Thus performing naive adversarial training (Goodfellow et al., 2015; Madry et al., 2018) with MMC is equivalent to performing stronger adversarial training variants like ALP. As analyzed above, the squared-error form in the MMC loss can accelerate the convergence of the AT mechanism, since the objective is sensitive to the crafted adversarial examples.
B.3 VARIANTS OF THE MMC LOSS
In the MMC loss, we encourage the features to gather around the preset Max-Mahalanobis (MM) centers n² = {uî heeL)) which leads many attractive properties. However, this 'hard' supervision, which induces quite an orderly feature distribution may beyond the reach of the model capability, espe- cially when the classification tasks themselves are already challenging to learn, e.g., ImageNet (Deng et al., 2009). Therefore, we propose potential variants of the MMC loss that could probably solve the problem and make our method more adaptable. We leave the experimental investigations as future work.
Note that the MMC loss can be regarded as minimizing the negative log likelihood (NLL) of log ((=ly)), where the conditional feature distribution is modeled as zly N( I). As described above, this distribution model may not be easy to learn by the DNNs in some cases. Thus, construct softer model: 2|8, My N(H,, and Hy ~ N(pc, aT), where > 0 is a scalar. Here we give the feature center Hy a prior distribution, while the prior is centered at μī- Intuitively, we relax the constraint that the features have to gather around pzz Instead, we encourage the features to gather around a substitute My. while Byy should be in the vicinity of jrz In the training, we minimize the joint NLL of log(((( ((2,, Py (y)) = - log( P(2yy, Hy)) - log( ((H))), which is equivalent to minimize the what we call clastic Max-Mahalanobis center (EMC) loss as:
1 LFMC (Z(x), v) 2 = My 122
1
||n 11
(21)
2a:
Here μ = {mr hel are simply extra trainable parameters, the prior variance a is hyperparameter. When a - 0, the EMC loss degenerates to the MMC loss. Note that although μi are all on the hypersphere {z E |||||| }.M). the support sets of μ4 are the entire feature space Rd.
Further improvement can be made w.r.t. the MM centers μ², An implicit assumption behind the generation process of μ² is that any two classes are mutually independent. This assumption could be approximately true for MNIST and CIFAR-10, 10, but for more complex datasets, e.g., CIFAR-100 or ImageNet, this assumption may not be appropriate since there are structures in the relation among classes. These structures can usually be visualized by a tree. To solve this problem, we introduce the hierarchical Max- Max-Mahalanobis (HM) centers y.
[oft) ( ]E([)) which adaptively craft the centers
17

---

Published as a conference paper at ICLR 2020
Dedsion boundary
My
badsnanbbuncary
Proficed fecture centers in Latic
Leumed leature oertain Lapu:
Moving directions advenarial examples
Auxillary lines
?
Ads
Pry
Figure 6: Intuilive demonsiration of the atincking mechamisns under different aduptive objectives. Here y is the original label, B arg yn y the label of the nearest other decision region w.r.. the feature and yo the target label targeted attacks.
according to the tree structure. Specifically, we first assign a virtual center (i.e., the origin) to the root node. For any child node n. in the tree, we denote its parent node as np. and the number of its brother nodes as Tu.. We locally generate set of MM centers as (e( = Generate (**, d, Le). where is the depth of the child node Re. C* is constant with smaller values for larger s. Then assign the virtual center to each child node of ny from Ung + p.(+.Te), i.e., shifted set of crafted MM centers, where May is the virtual center assigned to np. If the child node n, is leaf node, i.e., correspond to a class label 1, then there is fu = Un.- For example, the CIFAR- 100 dataset, there are 20 superclasses, with 5 classes in each superclass. We first craft 20 MM centers as (1) = GenerateMIMcenttess((C d, 20) and 5 MM centers as (2(2.5) = Generate C²2, 5), where C² < Ct. Note that (2255) could be different for each superclass, e.g., by a rotation transformation. Then if the label I is the j-th class in the i-th superclass, there is p/ p
(1,20) + (2,5)
B.4 ADAPTIVE OBJECTIVES AND THE INDUCED ATTACKING MECHANISMS
We apply the adaptive versions of existing attacks when evading the networks trained by the MMC loss. We separately design two adaptive adversarial objectives LAda to minimize under the untargeted mode: A = - [MMc(2, y): A = (MMC(2, ÿ) (2Mc y), and under the targeted mode:
ctar.1 [MMM(2, vt); c [MMc(=, Ve ) D--MMc(2 y). where yı is the targeted label, is generally the highest predicted label except for as defined in Sec. 3.2. These objectives refer previous work
by Carlini & Wagner (2017a:b). Specifically, the adaptive objectives Clar,1 and L are used the
PGD, MIM and SPSA attacks, while the objectives clar.2 and c are used in the C&W attacks.
In Fig. 6, we demonstrate the attacking mechanisms induced by different adaptive adversarial objectives. Note that we only focus on the gradients and ignore the specific method which implements the attack. Different adaptive objectives are preferred under different adversarial goals. For examples, when decreasing the confidence of the true label is the goal, L is the optimal choice; in order to mislead the classifier to predict an untrue label or the target label, _um.2 and C are the optimal Ada
choices, respectively. Sometimes there are additional detectors, then the adversarial examples generated by Clar.1 could be assigned to the target label with high confidence by the classifiers.
B.5 RELATED WORK IN THE FACE RECOGNITION AREA
There are many previous work in the face recognition area that focus on angular margin- softmax (AMS) losses (Liu et al., 2016; 2017; Liang et al., 2017; Wang et al., 2018; Deng et al., 2019). They mainly exploit three basic operations: weight normalization (WN), feature normalization (FN), and angular margin (AN). It has been empirically shown that WN can benefit the cases with unbalanced data (Guo& & Zhang, 2017); FN can encourage the models to focus more on hard examples (Wang et al.. 2017); AN can induce larger inter-class margins and lead to better generalization in different facial tasks (Wang et al., 2018; Deng et al., 2019). However, there are two critical differences between MMC loss and these AMS losses:
18

---

Published as a conference paper at ICLR 2020
Table S: Classification sccuracy (96 on the wilte--bo adversarial examples craittd on the test set of CIFAR-10 10 and CIEAR-100. The results whr MMC loss are reported under the adaptive versions of different attacks. MMC can better exploit deep architoctures, while SCE cannot.
Perturbation 8/255
Perturbation
16/255
Methods
Cle.
PGD10
PGD
PGD 30
PGD
PGD10
PGD 10
PGD
far 30
PGD HR
CIFAR-10
SCE (Res.32)
93.6
S
3.7
S
3.6
S
2.7
S
MMC (Res.32)
92.7
48.7
36.0
26.6
24.8
36.1
25.2
13,4
17.5
SCE (Res. 100)
94.7
3.0
S 1
2,9
S
2.1
S 1
2.0
MMC (Res.110)
93.6
54.7
46.0
34.4
31.4
41.0
30.7
16.2
21.6
CIFA 100
SCE (Res.32)
72.3
7.8
S 1
7.4
S 1
4.8
S 1
4.7
MMC (Res.32)
71.9
23.9
23.4
15.1
16.4
16.7
SCE (Res. (0)
74.8
<
7.5
S 1
7.3
S
4.7
S 1
4.5
MMC (Res.110)
73.2
34.6
22.4
23.7
16.5
24.1
14.9
13.9
10.5
Difference one: The inter-class margin
The AMS losses induce the inter-class margins mainly by encouraging the intra-class compactness, while the weights explicitly forced to have large margins (Qi & Zhang, 2018).
The MMC loss simultancously fixes the class centers to optimally dispersed and encour- ages the intra-class distribution be compact. Note that both of the mechanisms induce inter-class margins, which can finally lead to larger inter-class margins compared to the AMS losses.
Difference two: The normalization
The AMS losses use both WN and FN to exploit the angular metric, which makes the normalized features distribute hyperspheres. The good properties of the AMS losses at the cost of abandoning the radial degree of freedom, which may reduce the capability of models.
In the MMC loss, there is only WN the class centers, i.e., ||g||| Cıм, and leave the degree of freedom in the radial direction for the features to keep model capacity. However, natural penalty term the
note that the MMC loss || 14 ll (I feature norm, which encourage ||æ||2 to be far from CMM- This prevents models from increasing feature norms for easy examples and ignoring hard examples, just similar to the effect caused by FN but more flexible.
CMM)²
19


Reviews:
[{"review_id": "Sye48n16FB", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The paper compares between SCE loss,  large-margin Gaussian Mixture (L-GM) loss and proposes the Max-Mahalanobis center (MMC) loss as an alternative to explicitly learn more structured representations and induce high-density regions in the feature space. Overall the paper is well written, with sufficient theoretical reasoning and experiments. However, the reviewer has the following concerns and questions,\nThe theoretical analysis depends largely on the Gaussian assumption and argues that when the loss is distributed as Gaussian, it seems to be not even a fair comparison since assuming L_{MMC} is gaussian is totally different from assuming L_{g-SCE} is Gaussian. Also in practice it is hard to justify whether certain loss function really behaves like a Gaussian distribution, which makes the application of the theorem more limited. In fact, if the samples are concentrated (which can be common in practice), is the proposed method still able to induce high density sample region?\nThe experiments give very competitive results for MMC loss. It would also be interesting to see if implementing other defenses or do an adversarial training would still make MMC loss much better than other loss (at least from the AT example, it seems that MMC does not perform uniformly better than SCE as before).\nAre the experiment results sensitive to the choice of parameters C_MM and L?\n\nI have read the author responses and I think they are quite solid. I have updated my score. \n", "extra_scores": {"experience_assessment": "I have read many papers in this area.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper at least twice and used my best judgement in assessing the paper.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I assessed the sensibility of the derivations and theory."}, "extra_text": {}}, {"review_id": "HJxtbjQUcS", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper first shows some potential issues of softmax loss (i.e., cross-entropy loss with softmax function) and then propose the Max-Mahalanobis center (MMC) loss to encourge the intra-class compactness for better adversarial robustness.\n\nThe MMC loss is essentially minimizing the distance between the feature and the pre-fixed class center. Different from center loss, these centers are determined by minimizing the maximum inner product between any two class centers. Since the norm of these class centers are normalized to a constant. It is equivalent to angles. This acutally reminds me of a number of works in angular margin-based softmax loss. Just to name a few:\n\n[1] Large-Margin Softmax Loss for Convolutional Neural Networks, ICML 2016\n[2] SphereFace: Deep Hypersphere Embedding for Face Recognition, CVPR 2017\n[3] Soft-margin softmax for deep classification, ICNIP 2017\n[4] CosFace: Large Margin Cosine Loss for Deep Face Recognition, CVPR 2018\n[5] ArcFace: Additive Angular Margin Loss for Deep Face Recognition, CVPR 2019\n\nI think these works are closely related to what the authors aim to do, and therefore they should be discussed methodologically and compared empirically.\n\nBesides that, I think it is also worth conducting an ablation study for how to determine these class centers. This paper considers to minimize the maximum inner product. There are a few papers listed below that explicitly discusses how to make the class centers uniformly spaced. The authors may consider to compare these methods for determining the class centers. \n\n[1] Learning towards Minimum Hyperspherical Energy, NeurIPS 2018 \n[2] UniformFace: Learning Deep Equidistributed Representation for Face Recognition, CVPR 2019\n\nFor the experiments, the MMC loss indeed shows some advantages over the softmax loss. I am basically convinced by the experiments, although it can further strengthen the paper if the authors can conduct some evaluations on large-scale datasets like ImageNet.\n\nI appreciate the authors provide many theoretical justifications, which is inspiring. Intuitively speaking, I can understand that shrinking the feature space (i.e., make feature distribution more compact) can improve the adversarial robustness. As a result, I think this paper is naturally motivated and is also theoretically sound. The experiments can be further improved.", "extra_scores": {"experience_assessment": "I have read many papers in this area.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper at least twice and used my best judgement in assessing the paper.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I assessed the sensibility of the derivations and theory."}, "extra_text": {}}, {"review_id": "HJg7WGbhYB", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "[Post rebuttal: The rebuttal addresses most of my concerns. I have updated my score.]\n\nThis paper points out some limitation of the traditional softmax cross-entropy (SCE) loss, e.g., causing indirect and unexpected supervisory signals on the learned features so that the points with low loss values tend to spread over the space sparsely. To remedy this, the authors then proposes the max-mahalanobis center (MMC) loss. By analyzing the sample density, it is proved that the MMC loss has a sample density proportional to the number of data samples in each class. This means MMC loss induces higher feature densities than the SCE loss, thus is expected to be more robust under adverssarial attacks.\n\nI find the paper fairly well written in general, and the arguments are well supported by the development of the theoretical results. There are, however, several questions raised by the reviewer.\n\nOne question is that by looking at the definition (8), it seems the MMC loss essentially tries to move data from different classes apart. This seems to be quite similar to the max margin loss. I would expect the authors differentiate the differences between these two. Consequently, I would also like to see empirical comparisons between these two losses.\n\nAnother question is even if you have Theorem 2, the transfer from Theorem 2 to robustness does not seem direct, e.g., how do you guarantee when the density form in eq.9 is better than that in eq.7 for robustness?\n\nThen when looking back at the sample density definition in eq. (2). I wonder how the Vol is defined to guarantee eq.2 is a valid density function? e.g., to guarantee the integration equals to 1.\n\nIn terms of experiments, since it is claimed that the proposed loss adds little computation cost compared to the SCE loss, I think it is better to include running time comparison in the results.\n\nThe last two lines in page 8, what are unseen attacks? From the experiments, it seems that it means the PGD with different steps. I think the authors  should be careful to use the term, because some literature use the term *different attacks* to refer to attacks with different *norms*.\n\nSecond line below Table 5: it is said the MMC loss keeps state-of-the-art performance on clean data. However, by looking at Table 4, the state-of-the-art is obvious SCE. Why did you say that?\n\nOthers:\n1. S_k,\\tilde{k}^2 is undefined.\n2. How do you solve the minmax problem below eq.8? I guess you use the algorithm in Appendix B.1? Please clarify.\n", "extra_scores": {"experience_assessment": "I have published one or two papers in this area.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper at least twice and used my best judgement in assessing the paper.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I assessed the sensibility of the derivations and theory."}, "extra_text": {}}, {"review_id": "ByeZZ-gc5B", "reviewer": "AnonReviewer4", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper proposes an alternative loss function for classification models that they claim leads to improved adversarial robustness even under strong adaptive attacks. It also attempts to analyze how the softmax cross-entropy loss discourages robustness by considering the problem in terms of local density in the pre-logit feature space.\n\nAlthough as an emergency reviewer I haven't had time for a thorough verification, the overall idea seems sound, as do their theoretical and experimental results. I appreciate the careful analysis of the sample densities induced by each method; the N/L^2 vs. N/L result is especially nice. They also seem to follow best practices for evaluating adversarial defenses. Overall I do feel like the research topic of developing alternate loss functions and identifying pathologies with current popular loss functions is important and maybe insufficiently studied / publicized, so I think publishing this paper would be helpful for the field.\n\nA few questions:\n(1) If the MMC loss makes your final models so much more robust to small perturbations, why is there still such a large clean accuracy drop when combining your method with adversarial training in Figure 3a? I would have hoped that the tradeoff would have been less extreme. If you started adversarial training midway through the training process, or used a smaller perturbation size, would the tradeoff still be as large as with SCE models?\n(2) It makes sense that the optimal method of choosing MM centers would be to place them at the vertices of simplexes when #dims = #classes, but if there was some other way of avoiding the degradation problem from Wen et al. 2016, would it ever make sense (especially when #dims < #classes) to allow some automatic slackening of the MM centers (i.e. allow them to move from their original positions, but at a heavy cost), in order to permit classes that are similar (e.g. different breeds of imagenet dogs) to cluster more closely together, or perhaps to allow for classes with different levels of dispersion? Something feels suboptimal about forcing class centers to be equidistant and identical. This isn't intended as a major criticism but it might be an interesting direction for future work.", "extra_scores": {"experience_assessment": "I have published one or two papers in this area.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_thoroughness_in_paper_reading": "I made a quick assessment of this paper.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I did not assess the derivations or theory."}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper proposes an alternative loss function, the max-mahalanobis center loss, that is claimed to improve adversarial robustness. 

In terms of quality, the reviewers commented on the convincing experiments and theoretical results, and were happy to see the sample density analysis. 

In terms of clarity, the reviewers commented that the paper is well-written. 

The problem of adversarial robustness is relevant to the ICLR community, and the proposed approach is a novel and significant contribution in this area. 

The authors have also convincingly answered the questions of the authors and even provided new theoretical and experimental results in their final upload. 
Decision: Accept (Poster)