Title: ROBUST DETERMINANTAL GENERATIVE CLASSIFIER FOR NOISY LABELS AND ADVERSARIAL ATTACKS

Abstract: Large-scale datasets may contain significant proportions of noisy (incorrect) class labels, and it is well-known that modern deep neural networks poorly generalize from such noisy training datasets. In this paper, we propose a novel inference method, Deep Determinantal Generative Classifier (DDGC), which can obtain a more robust decision boundary under any softmax neural classifier pre-trained on noisy datasets. Our main idea is inducing a generative classifier on top of hidden feature spaces of the discriminative deep model. By estimating the parameters of generative classifier using the minimum covariance determinant estimator, we significantly improve the classification accuracy, with neither re-training of the deep model nor changing its architectures. In particular, we show that DDGC not only generalizes well from noisy labels, but also is robust against adversarial perturbations due to its large margin property. Finally, we propose the ensemble version of DDGC to improve its performance, by investigating the layer-wise characteristics of generative classifier. Our extensive experimental results demonstrate the superiority of DDGC given different learning models optimized by various training techniques to handle noisy labels or adversarial samples. For instance, on CIFAR-10 dataset containing 45% noisy training labels, we improve the test accuracy of a deep model optimized by the state-of-the-art noise-handling training method from 33.34% to 43.02%.

1. INTRODUCTION:

Deep neural networks (DNNs) are known to generalize well when they are trained on large-scale datasets with clean label annotations. For example, DNNs have achieved state-of-the-art performance on many classification tasks, e.g., speech recognition (Amodei et al., 2016) , object detection (Girshick, 2015) and image classification (He et al., 2016) . However, as the scale of the training dataset increases, it becomes infeasible to obtain all class labels from domain experts. A common practice is collecting the class labels from data mining on social media (Mahajan et al., 2018) and web data (Krause et al., 2016) . However, they may contain missing/noisy (incorrect) labels, and recent studies have shown that modern deep architectures may generalize poorly from the noisy datasets (Zhang et al., 2017; Arpit et al., 2017) . For example, in Figure 1 (a), the test set accuracy (black line) of DenseNet-100 model (Huang & Liu, 2017) trained on the CIFAR-10 dataset (Krizhevsky & Hinton, 2009) significantly decreases as the noise fraction increases.

To overcome the poor generalization issue of DNNs against noisy labels, many training strategies have been investigated in the literature. Reed et al. (2014) proposed a bootstrapping method which trains deep models with new labels generated by a convex combination of the raw (noisy) labels and their predictions, and Ma et al. (2018b) improved the bootstrapping method by utilizing the dimensionality of subspaces during training. Patrini et al. (2017) modified the loss and posterior distribution to eliminate the influence of noisy labels, and Hendrycks et al. (2018) improved such a loss correction method by utilizing the information from data with true class labels. Goldberger & Ben-Reuven (2017) added an additional softmax layer to model a noise transition matrix. Finally, training DNNs on selected samples also has been studied (Jiang et al., 2018; Ren et al., 2018; Malach & Shalev-Shwartz, 2017; Han et al., 2018) . However, adopting such training methods might incur expensive back-and-forth costs, e.g., additional time and hyperparameter tuning. This motivates our approach of developing a more plausible inference method which can be applied to any pre-trained Softmax Generative (sample estimator on noisy labels) Generative (MCD estimator on noisy labels) Generative (MCD estimator + ensemble on noisy) Generative (sample esitmator on clean labels) [ideal] Test set accuracy (%) deep model. Nevertheless, our direction is complementary to the prior works, where one can also combine them for even better performance (see Table 3 and 4 in Section 3).

Contributions. It has been observed that DNNs can learn meaningful feature patterns shared by multiple training examples even for datasets with noisy labels (Arpit et al., 2017) . We also found that an induced generative classifier (Ng & Jordan, 2002) under linear discriminant analysis (LDA) assumption (with naive estimations on sample mean and covariance) built upon the hidden feature space can outperform the softmax classifier (orange line in Figure 1 (a)). In Figure 1 (b), the hidden features from test samples projected in a 2-dimensional space using t-SNE (Maaten & Hinton, 2008) are illustrated. Here, one can observe that all ten classes are well-separated in the embedding space, even though the model is trained under a noisy dataset. More importantly, Figure 1 (c) plotting noisy training samples implies that outliers induce the class-wise multi-modal distributions in the feature space. Therefore, an LDA-like generative classifier assuming the class-wise unimodal distribution might be more robust, as a discriminative classifier is easier to be overfitted by outliers. This motivates our goal to induce a generative classifier on the pre-trained hidden features of DNNs.

To this end, we propose the so-called deep determinantal generative classifier (DDGC), based on the minimum covariance determinant (MCD) (Rousseeuw, 1984; Rousseeuw & Driessen, 1999) estimation on its parameters. While a naive sample estimator can be highly influenced by outliers, MCD estimator can improve the robustness by removing them. Note that MCD is known to have a near-optimal breakdown point (Hampel, 1971) of almost 50% in most situations, i.e., the number of outliers should be larger than that of normal samples to fool it. We further provide a new theoretical support on the larger margin property of DDGC such that not only does it generalize well from noisy labels (Durrant & Kabán, 2010) , but also improves the robustness against adversarial perturbations (Pang et al., 2018) . In addition, we observe that DNNs tend to have similar hidden features, regardless of whether they are trained with clean or noisy labels at early layers (Morcos et al., 2018) , and DDGC built from low-level features can be often more effective. Under the observation, we finally propose an ensemble version of DDGC to incorporate all effects of low and high layers.

We demonstrate the effectiveness of DDGC using modern neural architectures, such as DenseNet (Huang & Liu, 2017) and ResNet (He et al., 2016) trained for image classification tasks including CIFAR (Krizhevsky & Hinton, 2009) and SVHN (Netzer et al., 2011) . First, our methods (green and blue lines in Figure 1 (a)) significantly outperform the softmax classifier, although they use the same feature representations trained by the noisy dataset. For example, we improve the test accuracy of DenseNet on CIFAR-10 datasets with 60% noisy labels from 53.34% to 74.72%. We also demonstrate that DDGC can be used to further improve various prior training methods (Reed et al., 2014; Patrini et al., 2017; Ma et al., 2018b; Han et al., 2018; Jiang et al., 2018; Malach & Shalev-Shwartz, 2017) which are specialized to handle the noisy environment. Finally, DDGC is shown to be robust against various adversarial attacks (Goodfellow et al., 2015; Moosavi Dezfooli et al., 2016; Carlini & Wagner, 2017) .

2. ROBUST INFERENCE VIA GENERATIVE CLASSIFIER:

In this section, we propose a novel inference method which obtains a robust posterior distribution from any softmax neural classifier pre-trained on datasets with noisy labels. Our idea is inducing the generative classifier using fixed features from any pre-trained model. We show the advantages of our method in terms of high breakdown points (Hampel, 1971) , generalization error (Durrant & Kabán, 2010) and adversarial robustness (Pang et al., 2018) . We also investigate the layer-wise characteristics of generative classifiers, and introduce an ensemble of them to improve its performance.

We emphasize again that we study inference methods, not requiring any modification on pre-trained deep models. Hence, the proposed methods are easily applicable on top of any pre-trained modern neural classifiers.

2.1. GENERATIVE CLASSIFIER AND MCD ESTIMATOR:

Let x be an input and y ∈ {1, • • • , C} be its class label. Without loss of generality, suppose that a pre-trained softmax neural classifier is given:

P (y = c|x) = exp(w c f (x)+bc) c exp(w c f (x)+b c )

, where w c and b c are the weight and the bias of the softmax classifier for class c, and f (•) ∈ R d denotes the output of the penultimate layer of DNNs. Then, without any modification on the pre-trained softmax neural classifier, we induce a generative classifier by assuming the class-conditional distribution follows the multivariate Gaussian distribution. In particular, we define C Gaussian distributions with a tied covariance Σ, i.e., linear discriminant analysis (LDA) (Fisher, 1936) , and a Bernoulli distribution for the class prior:

P (f (x)|y = c) = N (f (x)|µ c , Σ) , P (y = c) = β c

, where µ c is the mean of multivariate Gaussian distribution and β c is the normalized prior for class c. We provide an analytic justification on the LDA (i.e., tied covariance) assumption in Appendix A. Then, based on the Bayesian rule, we induce a new posterior different from the softmax one as follows:

P (y = c|f (x)) = P (y = c) P (f (x)|y = c) c P (y = c ) P (f (x)|y = c ) = exp µ c Σ -1 f (x) -1 2 µ c Σ -1 µ c + log β c c exp µ c Σ -1 f (x) -1 2 µ c Σ -1 µ c + log β c .

To estimate the parameters of the generative classifier, one can compute the sample class mean and covariance of training samples X N = {(x 1 , y 1 ), . . . , (x N , y N )}:

EQUATION

where N c is the number of training samples labeled to be class c. We remark that inducing a generative classifier (e.g., a mixture of Gaussian) on pre-trained deep models was studied for various purposes, e.g., speech recognition (Hermansky et al., 2000) and novelty detection (Lee et al., 2018) . However, the setting of noisy training labels was not investigated under existing approaches on this line. To handle this, we design a more advanced generative classifier as stated in below.

One can expect that the naive sample estimator (1) can be highly influenced by outliers (i.e., training samples with noisy labels). In order to improve the robustness, we propose the so-called deep determinantal generative classifier (DDGC), which utilizes the minimum covariance determinant (MCD) estimator (Rousseeuw & Driessen, 1999) to estimate its parameters. For each class c, the main idea of MCD is finding a subset X Kc for which the determinant of the corresponding sample covariance is minimized:

EQUATION

where X Nc is the set of training samples labeled to be class c, Σ c is the sample covariance of X Kc and 0 < K c < N c is a hyperparameter. Then, only using the samples in c X Kc , it estimates the parameters, i.e., µ c , Σ, β c , of the generative classifier, by following (1). Such a new estimator can be more robust by removing the outliers which might be widely scattered in datasets. The robustness of MCD estimator has been justified in the literature: it is known to have near-optimal breakdown points (Hampel, 1971) , i.e., the smallest fraction of data points that need to be replaced by arbitrary values (i.e., outliers) to fool the estimator completely. Formally, denote Y M as a set obtained by replacing M data points of set Y by some arbitrary values. Then, for a multivariate mean estimator µ = µ(Y) from Y, the breakdown point is defined as follows (see Appendix A for more detailed explanations including the breakdown point of covariance estimator):

ε * (µ, Y) = 1 |Y| min M ∈ {1, • • • , |Y|} : sup Y M µ(Y) -µ(Y M ) = ∞ .

While the breakdown point of the naive sample estimator is 0%, the MCD estimator for the generative classifier under LDA assumption is known to attain its breakdown value of min c et al., 1991) . Inspired by this fact, we choose the default value of K c in (2) by (N c + d + 1)/2 . We also establish the following theoretical support that the MCD-based generative classifier, i.e., DDGC, can have smaller errors on parameter estimations and produce a larger margin, compared to the naive sample estimator, under some assumptions for its analytic tractability. Theorem 1. Assume the followings:

(Nc-d+1)/2 Nc ≈ 50% (Lopuhaa

(A1) The distribution of hidden features is P (f (x)|y = c) = N f (x)|µ c , σ 2 I (i.e., the conditional Gaussian distribution) and that of outliers is an arbitrary distribution with mean µ out and covariance matrix σ 2 out I, where I ∈ R d×d is the identity matrix. (A2) µ out = 0, and 1 C c µ c µ T c is a diagonal matrix.

(A3) All classes have the same number of samples (i.e., N c = N C ), the same fraction δ out < 1 of outliers, and the sample fraction δ mcd = Kc Nc < 1 of samples selected by MCD estimator.

(A4) The outliers are widely scattered such that σ 2 < σ 2 out .

(A5) The number of outliers is not too large such that δ out < 1 -δ mcd and δ mcd > d Nc .

Let µ, Σ and μ, Σ be the outputs of the MCD and sample estimators, respectively. Then, µ, μ, Σ, Σ converge almost surely to their expectation as N → ∞, and it holds that

EQUATION

EQUATION

for all c, c , where

B( µ c , µ c , Σ, Σ) := exp -1 8 [( µc-µ c ) T Σ -1 ( µc-µ c )] 2 ( µc-µ c ) T Σ -1 Σ Σ -1 ( µc-µ c ) .

The proof of the above theorem is given in Appendix F, where it is built upon the fact that the determinants can be expressed as the d-th degree polynomial of outlier ratio. We note that one might enforce the assumptions of the diagonal covariance matrices in A1 and the zero-mean/uncorrelated properties of class mean in A2 to hold under an affine translation of hidden features. In addition, the assumption in A5 holds when N c is large enough. Nevertheless, we think most assumptions of Theorem 1 are not necessary to claim the superiority of DDGC (they are rather from a limitation of our proof techniques) and it is an interesting future direction to explore to relax them.

First, (5) implies that the MCD estimator induces a larger margin and improves the robustness to adversarial attacks, since Pang et al. (2018) showed that the margin of a generative classifier corresponds to the Mahalanobis distance between class-conditional distributions and the optimal robustness to adversarial samples is achieved by the maximum margin. More importantly, (3) and (4) together imply that a MCD-based classifier can generalize better since the generalization error of a generative classifier is known to be bounded as follows (Durrant & Kabán, 2010) :

P x y * = arg max y P µc, Σ (y|x) ≤ c c =c B( µ c , µ c , Σ, Σ) + D c µ c -µ c 1 ,

for some constant D > 0.

Algorithm 1 (Rousseeuw & Driessen, 1999) Approximating MCD for a single Gaussian.

Input: X Nc = {x i : i = 1, • • • , N c } and the maximum number of iterations I max .

Uniformly sample initial subset X Kc ⊂ X Nc , where 

|X Kc | = (N c + d + 1)/2 . Compute µ c = 1 |X Kc | x∈X Kc f (x), Σ c = 1 |X Kc | x∈X Kc (f (x) -µ c ) (f (x) -µ c ) . for i = 1 to I max do Compute the Mahalanobis distance: α(x) = (f (x) -µ c ) Σ -1 c (f (x) -µ c ) , ∀x ∈ X Nc . Update X Kc such that it includes (N c + d +

2.2. APPROXIMATION ALGORITHM FOR MCD:

Even though the MCD estimator has several advantages, the optimization (2) is computationally intractable (i.e., NP-hard) to solve (Bernholt, 2006) . To handle this issue, we aim for computing its approximate solution, following a similar idea to that by (Hubert & Van Driessen, 2004) . We design two step scheme as follows: (a) obtain the mean and covariance, i.e., µ c , Σ c , using Algorithm 1 for each class c, and (b) compute the tied covariance by Σ = c Kc Σc c Kc . In other words, we apply the MCD estimator for each class, and combine the individual covariances into a single one due to the tied covariance assumption of LDA. Even though finding the optimal solution of MCD estimator under a single Gaussian distribution is still intractable, Algorithm 1 can produce a local optimal solution since it monotonically decreases the determinant under any random initial subset (Rousseeuw & Driessen, 1999) . We choose I max = 2 in our experiments as the additional iterations would not improve the results significantly.

2.3. ENSEMBLE OF GENERATIVE CLASSIFIERS:

To further improve the performance of our method, we consider the ensemble of generative classifiers not only from the penultimate features but also from other low-level features in DNNs. Formally, given training data, we extract -th hidden features of DNNs, denoted by f (x) ∈ R d , and compute the corresponding parameters of a generative classifier (i.e., µ ,c and Σ ) using the (approximated version of) MCD estimator. Then, the final posterior distribution is obtained by the sum of all posterior distributions of generative classifiers:

α P (y = c|f (x)) = α exp µ ,c Σ -1 f (x) -0.5 µ ,c Σ -1 µ ,c + log β c c exp µ ,c Σ -1 f (x) -0.5 µ ,c Σ -1 µ ,c + log β c

, where α is an ensemble weight at -th layer. In our experiments, we choose the weight of each layer by optimizing negative log-likelihood (NLL) loss over the validation set. One can expect that this natural scheme can bring an extra gain in improving the performance due to ensemble effects.

To confirm that the proposed ensemble approach is indeed effective, we measure the classification accuracy of the generative classifier from different basic blocks of ResNet-34 (He et al., 2016) trained on CIFAR-10 dataset (Krizhevsky & Hinton, 2009) with various noise fractions, where the corresponding results on DenseNet models (Huang & Liu, 2017 ) can be found in Appendix C. For computational efficiency, the dimensions of the intermediate features are reduced using average pooling (see Section 3 for more details). First, we found that the performances of the generative classifiers from low-level features are more stable, while the accuracy of generative classifier from penultimate layer significantly decreases as the noisy fraction increases as shown in Figure 2 (a). We expect that this is because the dimension (i.e., number of channels) of low-level features is usually smaller than that of high-level features. Since the breakdown point of MCD is inversely proportional to the feature dimension, the generative classifiers from low-level features can be more robust. This also coincides with the prior observation in the literature (Morcos et al., 2018) that DNNs tend to have similar hidden features at early layers, regardless of whether they train clean or noisy labels.

More importantly, we found that generative classifiers from low-level features are more robust to strong adversarial attacks like CW (Carlini & Wagner, 2017) , as shown in Figure 2 (b). Therefore, we utilize the low-level generative classifiers as well to improve the generalization from noisy labels and robustness to adversarial attacks simultaneously.

3. EXPERIMENTAL RESULTS:

In this section, we demonstrate the effectiveness of the proposed method using deep convolutional neural networks including DenseNet (Huang & Liu, 2017) and ResNet (He et al., 2016) on various vision datasets: CIFAR (Krizhevsky & Hinton, 2009) and SVHN (Netzer et al., 2011) . Due to the space limit, we provide the more detailed experimental setups and results in Appendix B.

3.1. GENERALIZATION FROM NOISY LABELS:

Setup. First, we evaluate the effectiveness of DDGC using deep models trained on datasets with noisy labels. We train DenseNet-100 and ResNet-34 for classifying CIFAR-10, CIFAR-100 and SVHN datasets. Similar to (Ma et al., 2018b; Han et al., 2018) , we consider two types of noisy labels: corrupting a label to other class uniformly at random (uniform) and corrupting a label only to a specific class (flip). For ensembles of generative classifiers, we induce the generative classifiers from every end of basic block of DenseNet (or ResNet), where ensemble weights of each layer are tuned on a separate validation set, which consists of 500 images (i.e., only ≤ 1% of the number of training samples) with clean labels.1 Similar to (Lee et al., 2018) , the size of feature maps on each convolutional layers is reduced by average pooling for computational efficiency: F × H × W → F × 1, where F is the number of channels and H × W is the spatial dimension.

Verification of contributions from each technique. We first evaluate the performance of generative classifiers with various assumptions: identity covariance (Euclidean) and tied covariance (LDA).

In the case of identity covariance, we also apply a robust estimator called the least trimmed square (LTS) estimator (Rousseeuw, 1984) which finds a K-subset with smallest error and computes the sample mean from it, i.e., min µ K i=1 ( x i -µ 2 2 ). As shown in Figure 2 (c), the generative classifiers with LDA assumption (blue and purple bars) generalize better than the generative classifiers with identity covariance (orange and green bars) well from noisy labels. Table 1 validates the contributions of the proposed techniques by incrementally applying our techniques to see the improvement from adding each component one by one. One can note that the generative classifier on features extracted from the penultimate layer outperforms the softmax classifier without the MCD estimator or ensemble method, while it still provides a comparable classification accuracy when the model is trained on clean dataset (i.e., noise = 0%). On top of that, by utilizing the MCD estimator, the classification accuracy is further improved compared to that employs only the naive sample estimator. This implies that the proposed method can improve the performance without any information (Reed et al., 2014) , forward/backward (Patrini et al., 2017) , and D2L (Ma et al., 2018b) . It uses ResNet-44 and only considers the uniform noise. The second setup considers the following methods training multiple networks, i.e., an ensemble of classifiers or a meta-learning model: Decoupling (Malach & Shalev-Shwartz, 2017) , MentorNet (Jiang et al., 2018) and Co-teaching (Han et al., 2018) . It uses a 9-layer CNN architecture, and considers the CIFAR-10 and CIFAR-100 datasets with uniform and flip noise types. Table 3 and 4 

3.2. ROBUSTNESS AGAINST ADVERSARIAL ATTACKS:

Setup. We also evaluate if the proposed method can improve the robustness on adversarial attacks (Szegedy et al., 2014) . It is well-known that the adversarial (visually imperceptible) perturbation to clean inputs can induce the DNNs to make incorrect predictions at test time. This undesirable property of DNNs has raised major security concerns. To verify that DDGC can improve the robustness to adversarial attacks, we train DenseNet-100 and ResNet-34 for classifying CIFAR-10, CIFAR-100 and SVHN datasets, and generate the adversarial samples using FGSM (Goodfellow et al., 2015) , DeepFool (Moosavi Dezfooli et al., 2016) and CW (Carlini & Wagner, 2017) detailed explanations can be found in Appendix B. We consider the two types of adversarial attacks: generating the adversarial samples using a network, and then measuring their accuracy using the same (white-box) or another network (black-box). For all experiments, the adversarial samples are generated by targeting a softmax classifier or our generative classifier (see Appendix B for more details).

Robustness against adversarial attacks. 

4. CONCLUSION:

We propose a new inference method, easily applicable to any softmax neural classifier pre-trained on datasets with noisy labels. Our main idea is defining the generative classifier on top of fixed features from the pre-trained model. Such "deep generative classifiers" have been largely dismissed for fullysupervised classification settings as they are often substantially outperformed by discriminative deep classifiers (e.g., softmax classifiers). In contrast to this common belief, we show that it is possible to formulate a simple generative classifier that is more robust without sacrificing the discriminative performance. We expect that our work would bring a refreshing angle for other related tasks, e.g., adaptive attacks, memorization (Zhang et al., 2017) and interpretability (Morcos et al., 2018) .

A PRELIMINARIES:

Gaussian discriminant analysis. In this section, we describe the basic concepts of the discriminative and generative classifier (Ng & Jordan, 2002) . Formally, denote the random variable of the input and label as x and y = {1, • • • , C}, respectively. For the classification task, the discriminative classifier directly defines a posterior distribution P (y|x), i.e., learning a direct mapping between input x and label y. A popular model for discriminative classifier is softmax classifier which defines the posterior distribution as follows:

P (y = c|x) = exp(w c x+bc) c exp(w c x+b c )

, where w c and b c are weights and bias for a class c, respectively. In contrast to the discriminative classifier, the generative classifier defines the class conditional distribution P (x|y) and class prior P (y) in order to indirectly define the posterior distribution by specifying the joint distribution P (x, y) = P (y) P (x|y).

Gaussian discriminant analysis (GDA) is a popular method to define the generative classifier by assuming that the class conditional distribution follows the multivariate Gaussian distribution and the class prior follows Bernoulli distribution:

P (x|y = c) = N (x|µ c , Σ c ) , P (y = c) = βc c β c

, where µ c and Σ c are the mean and covariance of multivariate Gaussian distribution, and β c is the unnormalized prior for class c. This classifier has been studied in various machine learning areas (e.g., semi-supervised learning (Lasserre et al., 2006) and incremental learning (Lee et al., 2018) ).

In this paper, we focus on the special case of GDA, also known as the linear discriminant analysis (LDA). In addition to Gaussian assumption, LDA further assumes that all classes share the same covariance matrix, i.e., Σ c = Σ. Since the quadratic term is canceled out with this assumption, the posterior distribution of generative classifier can be represented as follows:

P (y = c|x) = P (y = c) P (x|y = c) c P (y = c ) P (x|y = c ) = exp µ c Σ -1 x -1 2 µ c Σ -1 µ c + log β c c exp µ c Σ -1 x -1 2 µ c Σ -1 µ c + log β c .

One can note that the above form of posterior distribution is equivalent to the softmax classifier by considering µ c Σ -1 and -1 2 µ c Σ -1 µ c + log β c as its weight and bias, respectively. This implies that x might be fitted in Gaussian distribution during training a softmax classifier.

Breakdown points. The robustness of MCD estimator can be explained by the fact that it has high breakdown points (Hampel, 1971) . Specifically, the breakdown point of an estimator measures the smallest fraction of observations that need to be replaced by arbitrary values to carry the estimate beyond all bounds. Formally, denote Y M as a set obtained by replacing M data points of set Y by some arbitrary values. Then, for a multivariate mean estimator µ = µ(Y) from Y, the breakdown point is defined as follows (see Appendix A for more detailed explanations including the breakdown point of covariance estimator):

ε * (µ, Y) = 1 |Y| min M ∈ {1, • • • , |Y|} : sup Y M µ(Y) -µ(Y M ) = ∞ .

For a multivariate estimator of covariance Σ, we have

ε * (Σ, Y) = 1 |Y| min{M ∈ {1, • • • , |Y|} : sup M max i {| log λ i (Σ(Y)) -log λ i (Σ(Y M ))|}},

where the k-th largest eigenvalue of a general n × n matrix is denoted by

λ k (Σ), k = 1, • • • , n such that λ 1 (Σ) ≤ λ 2 (Σ) ≤ • • • ≤ λ n (Σ)

. This implies that we consider a covariance estimator to be broken whenever any of the eigenvalues can become arbitrary large or arbitrary close to 0.

B EXPERIMENTAL SETUP:

In this section, we describe detailed explanation about all the experiments described in Section 3.

Detailed model architecture and datasets. We consider two state-of-the-art neural network architectures: DenseNet (Huang & Liu, 2017) and ResNet (He et al., 2016) . For DenseNet, our model follows the same setup as in Huang & Liu (2017) : 100 layers, growth rate k = 12 and dropout rate 0. Also, we use ResNet with 34 and 44 layers, filters = 64 and dropout rate 0. 5 The softmax classifier is used, and each model is trained by minimizing the cross-entropy loss. We train DenseNet and ResNet for classifying CIFAR-10 (or 100) and SVHN datasets: the former consists of 50,000 training and 10,000 test images with 10 (or 100) image classes, and the latter consists of 73,257 training and 26,032 test images with 10 digits. 6 By following the experimental setup of Ma et al. (2018b) , All networks were trained using SGD with momentum 0.9, weight decay 10 -4 and an initial learning rate of 0.1. The learning rate is divided by 10 after epochs 40 and 80 for CIFAR-10/SVHN (120 epochs in total), and after epochs 80, 120 and 160 for CIFAR-100 (200 epochs in total). For our method, we extract the hidden features at {34, 46, 56, 67, 79, 89, 99}-th layers and {3, 7, 9, 11, 13, 15, 17, 19, 21, 23, 25, 27, 29, 31 , 33}-th layers for DenseNet and ResNet, respectively.

Training method for noisy label learning. We consider the following training methods for noisy label learning: (e) D2L (Ma et al., 2018b) : Training with new labels generated by a convex combination of the noisy labels and their predictions, where its weights are chosen by utilizing the Local Intrinsic Dimensionality (LID). (Ma et al., 2018a) .

(f) Decoupling (Malach & Shalev-Shwartz, 2017) : Updating the parameters only using the samples which have different prediction from two classifier.

(g) MentorNet (Jiang et al., 2018) : An extra teacher network is pre-trained and then used to select clean samples for its student network.

(h) Co-teaching (Han et al., 2018) : A simple ensemble method where each network selects its small-loss training data and back propagates the training data selected by its peer network.

(i) Cross-entropy: the conventional approach of training with cross-entropy loss.

Adversarial attacks. In this paper, we consider the following attack methods: fast gradient sign method (FGSM) (Goodfellow et al., 2015) , DeepFool (Moosavi Dezfooli et al., 2016) and Carlini-Wagner (CW) (Carlini & Wagner, 2017) . The FGSM directly perturbs normal input in the direction of the loss gradient. Formally, non-targeted adversarial samples are constructed as

x adv = x + ε F GSM sign ( x (y * , P (y|x))) ,

where ε F GSM is a magnitude of noise, y * is the prediction of classifier and is a loss function to measure the distance between the prediction and the ground truth. DeepFool works by finding the closest adversarial samples with geometric formulas. CW is an optimization-based method which arguably the most effective method. Formally, non-targeted adversarial samples are constructed as arg min

x adv λd(x, x adv ) -(y * , P (y|x adv )),

where λ is penalty parameter and d(•, •) is a metric to quantify the distance between an original image and its adversarial counterpart. However, compared to FGSM, this method is much slower in practice. For all experiments, L 2 distance is used as a constraint. We used the library from (Guo et al., 2018) for generating adversarial samples.7 

For all experiments, the adversarial samples are generated by targeting a softmax classifier or our generative classifier. Specifically, we generate the adversarial samples by attacking the ensemble of generative classifiers:

α P (y = c|f (x)) =

Under review as a conference paper at ICLR 2019 

α exp( µ ,c Σ -1 f (x)-0.5 µ ,c Σ -1 µ ,c +log βc) c exp µ ,c Σ -1 f (x)-0.5 µ ,c Σ -1 µ ,c +log β c

. Here, we remark that a margin loss on the logit layer of each generative classifier is used in the case of CW attacks8 . Due to time complexity, we only attack a generative classifier from a final layer in the case of DeepFool. Table 7 and 8 report the statistics of adversarial attacks including the L ∞ mean perturbation and classification accuracy on adversarial attacks. 

C LAYER-WISE CHARACTERISTICS OF GENERATIVE CLASSIFIERS:

Figure 3 shows the classification accuracy of the generative classifiers from different basic blocks of ResNet-34 (He et al., 2016) and DenseNet-101 (Huang & Liu, 2017) . One can note that the generative classifiers from DenseNet and ResNet have different patterns due to the architecture design. In the case of DenseNet, we found that it produces meaning features after 20-th basic blocks. However, we remark that the performances of the generative classifiers from low-level features (from 20 to 40-th layers) of DenseNet are still more robust to adversarial attacks and noisy labels. Because of that, the ensemble of generative classifiers on DenseNet also improves the generalization from noisy labels and robustness to adversarial attacks.

D FINE-TUNING SOFTMAX LAYER USING VALIDATION SET:

In this paper, we utilize a validation set, which consists of 500 images (i.e., only ≤ 1% of the number of training samples) with clean labels. For fair comparisons, one might also suggest fine-tuning the softmax classifier (with fixed features) using the validation improve the classification accuracy. We indeed measure the classification accuracy of softmax classifier after fine-tuning in Table 9 . One can note that the proposed method still outperforms softmax classifiers. Since 1% data is not enough compared to the number of softmax classifiers, fine-tuning can decrease the performance in the case of softmax classifier, while DDGC is still working well (it only trains the ensemble weights).

E MORE EXPERIMENTAL RESULTS ON ADVERSARIAL ATTACKS:

In this section, we provide more experimental results on adversarial attacks. First, 

F PROOF OF THEOREM 1:

In this section, we present a proof of Theorem 1, which consists of three statements: the limit of estimated error ratio (3), estimation error ratio (4) and mahalanobis distance (5). We prove each statement one by one as stated in below. For convenience, we skip to mention the Continuous Mapping Theorem9 and the number of training samples N goes to infinity for all convergences in the proof.

F.1 PROOF OF THE LIMIT OF ESTIMATION ERROR (4):

We start with the following lemma, which shows the convergences of sample and MCD estimators with single Gaussian distribution as the number of training samples N goes to infinity.

Lemma 1. Suppose we have N number of d-dimensional training samples X N = {x 1 , • • • , x N } and X N contains outlier samples with the fixed fraction δ out < 1. We assume the outlier samples are from an arbitrary distribution P out with zero mean and finite covariance matrix σ 2 out I, and the other samples are from a multivariate Gaussian distribution P data with mean µ and covariance matrix σ 2 I. Let μ and Σ be the mean and covariance matrix of sample estimator, and let µ and Σ be the mean and covariance matrix of MCD estimator which selects samples from X N with the fixed fraction d N < δ mcd < 1 to optimize its objective (2). Then the mean and covariance matrix of sample estimator converge almost surely to below as N → ∞ :

μ a.s. → (1 -δ out ) µ, Σ a.s. → (1 -δ out ) σ 2 + δ out σ 2 out I + δ out (1 -δ out ) µµ T . In addition, if δ mcd ≤ 1 -δ out and σ 2 < σ 2

out , the mean and covariance matrix of MCD estimator converge almost surely to below as N → ∞ :

µ a.s. → µ, Σ a.s.

→ σ 2 I.

A proof of the above lemma is given in appendix F.4, where it is built upon the fact that the determinant of covariance matrix with some assumptions can be expressed as the d-th degree polynomial of outlier ratio. 

µ c a.s. → µ c , μc a.s. → (1 -δ out ) µ c , which implies that µ c -µ c 1 a.s. → 0, µ c -μc 1 a.s. → δ out µ c 1 .

This completes the proof of the limit of estimation error (4).

F.2 PROOF OF THE LIMIT OF MAHALANOBIS DISTANCE RATIO (5):

From the assumptions A1, all class covariance matrices are the same, i.e., Σ c = σ 2 I. Then tied covariance matrices Σ and Σ are given by gathering Σc and Σ c on each class c respectively:

EQUATION

From the tied covariance matrices (6) and Lemma 1, one can induce their convergences and limits as follow: → σ 2 I.

EQUATION

Since the assumption A2 gives a diagonal matrix D = 1 C c µ c µ T c , the limit of covariance matrix of sample estimator (7) and its inverse are also diagonal matrices. Then the limit of inverse matrix Σ-1 is given as follows:

EQUATION

Since the limit of inverse matrix Σ-1 (8) is a diagonal matrix, the limit of the mahalanobis distance between μc and μc is given as follows:

(μ c -μc ) T Σ-1 (μ c -μc ) = tr Σ-1 (μ c -μc )(μ c -μc ) T a.s.

EQUATION

where D = diag (D 1 , ..., D d ). By the assumption A4, the denominator term of ( 9) is greater than σ 2 for all i, i.e.,

(1 -δ out ) σ 2 + δ out σ 2 out + δ out (1 -δ out ) D i ≥ σ 2 for ∀ i. Then the limit of mahalanobis distance between mean of sample estimators (9) has the following upper bound:

EQUATION

)

One can also induce the limit of the mahalanobis distance between mean of MCD estimators µ c and µ c as follows:

EQUATION

)

From the limits of mahalanobis distance ( 10) and ( 11), the limit of mahalnobis distance ratio ( 5) is induced as follows:

(μ c -μc ) T Σ-1 (μ c -μc ) ( µ c -µ c ) T Σ -1 ( µ c -µ c ) a.s. → i (1-δout) 2 (µci-µ c i ) 2 (1-δout)σ 2 +δoutσ 2 out +δout(1-δout)Di (µi-µc) T (µi-µc) σ 2 ≤ (1 -δ out )

2 (µi-µc) T (µi-μc) σ 2 (µi-µc) T (µi-µc)

σ 2 = (1 -δ out ) 2 ≤ 1.

This completes the proof of the limit of mahalnobis distance ratio (5).

F.3 PROOF OF THE LIMIT OF ESTIMATED ERROR RATIO (3):

Remind that the estimated error B( µ c , µ c , Σ, Σ) is defined as follows:

EQUATION

By the property of trace operator, the inner term of estimated error ( 12) is equal to

EQUATION

)

From the right hand side of (13), the negative logarithms of estimated error (12) of sample and MCD estimators converge as follows:

-8 log(B(μ c , μc , Σ, Σ)) = tr Σ-1 (μ c -μc )(μ c -μc ) T 2 tr Σ-1 σ 2 I Σ-1 (μ c -μc )(μ c -μc ) T a.s.

→:

(1 -δ out ) The relation between the limits of ( 14) and ( 15) can be induced by the Cauchy-Schwarz inequality,

i (µ ci -µ c i ) 2 i µ ci -µ c i (1 -δ out ) σ 2 + δ out σ 2 out + δ out (1 -δ out ) D i 2 ≥ i (µ ci -µ c i ) 2 (1 -δ out ) σ 2 + δ out σ 2 out + δ out (1 -δ out ) D i 2 .

The above inequality implies

(µ c -µ c ) T (µ c -µ c ) ≥ i (µci-µ c i ) 2 (1-δout)σ 2 +δoutσ 2 out +δout(1-δout)Di 2 i (µci-µ c i ) 2 ((1-δout)σ 2 +δoutσ 2 out +δout(1-δout)Di) 2

.

From the fact that (1 -δ out ) 2 ≤ 1, the limits of ( 14) and ( 15) hold the following inequality: -8 log(B( µ c , µ c , Σ, Σ)) denote the limits of the negative logarithms of estimated error (12) of sample and MCD estimators, respectively. From ( 14), ( 15) and ( 16), it holds that This implies that the limit of estimated error ratio ( 3) is smaller than one, i.e., This completes the proof of Theorem 1.

(µ c -µ c ) T (µ c -µ c ) σ 2 ≥ (1 -δ out ) 2 σ 2 i (µci-µ c i ) 2 (1-δout)σ 2

F.4 PROOF OF LEMMA 1:

In this part, we present a proof of Lemma 1. We show the almost surely convergences of sample and MCD estimators as the number of training samples N goes to infinity.

Proof of the convergence of sample estimator. First of all, the set of training samples X N = {x 1 , • • • , x N } contains outlier samples with the fixed fraction δ out . So, X N is from a mixture distribution P mix = (1 -δ out ) P data +δ out P out . Then mean and covariance matrix of sample estimator, μ and Σ, estimate mean µ mix and covariance matrix Σ mix of the mixture distribution P mix , respectively. One can induce µ mix and Σ mix directly as follow:

EQUATION

)

Since P mix has the finite covariance matrix, i.e., Σ mix < ∞, one can apply the the Strong Law of Large Numbers10 to the sample estimator of the mixture distribution P mix . Then the mean and covariance matrix of sample estimator converge almost surely to the mean and covariance matrix of P mix , respectively:

μ a.s. → µ mix , Σ a.s. → Σ mix .

This completes the proof of the convergence of sample estimator.

Proof of the convergence of MCD estimator. Consider a collection E q of subsets X K,q ⊂ X N with the size K (= δ mcd N ), and each subset X K,q contains the outlier samples with the fraction q ∈ [0, 1]. Then X K,q ∈ E q is from a mixture distribution P q = (1 -q) P data + qP out . One can induce that the mean µ q and covariance matrix Σ q of the mixture distribution P q as (17):

µ q = (1 -q)µ, Σ q = (1 -q)σ 2 I + qσ 2 out I + q(1 -q)µµ T .

(18)

:

For fair comparisons, one might also suggest fine-tuning the softmax classifier (with fixed features) using the validation data to improve the performance. However, the 1% data is often not enough for the purpose as the number of parameters of softmax classifier is too large. We indeed report the performance of the fine-tuned softmax classifier on Table9in Appendix D.

We used a reference implementation ofMa et al. (2018b): https://github.com/xingjunm/ dimensionality-driven-learning.

We used a reference implementation: https://github.com/bhanML/Co-teaching

In the second setup, we only apply DDGC to a model pre-trained by Co-teaching because it outperforms other training methods.

ResNet architecture is available at https://github.com/kuangliu/pytorch-cifar.

We do not use the extra SVHN dataset for training.

The code is available at https://github.com/facebookresearch/adversarial_image_ defenses.

Similar toCarlini & Wagner (2017), we use 2000 random samples.

P. Billingsley, Convergence of Probability Measures, JohnWiley & Sons, 1999

W. Feller, An Introduction to Probability Theory and Its Applications, John Wiley & Sons, 1968

Figures and Tables:

Figure 1: Experimental results under DenseNet-100 model and CIFAR-10 dataset. (a) Test accuracy comparison when the labels of a given proportion of training samples are flipped to other labels uniformly at random. Visualization of features on the penultimate layer using t-SNE from (b) test samples (same colors indicate same classes) and (c) training samples when the noise fraction is 20%.

Figure 2: Experimental results under ResNet-34 model and CIFAR-10 dataset. (a) Test accuracy of generative classifiers computed at different basic blocks. (b) Test accuracy on various adversarial attacks when the model is trained on clean dataset. (c) Test accuracy of generative classifiers from penultimate features under various assumptions: identity covariance and tied covariance (LDA).

report the classification accuracy of softmax classifier and the ensemble version of DDGC, for the first and secon setups, respectively. They show that DDGC consistently outperforms the softmax inference under various training methods and noise types.4

Hard bootstrapping (Reed et al., 2014): Training with new labels generated by a convex combination (the hard version) of the noisy labels and their predicted labels. (b) Soft bootstrapping (Reed et al., 2014): Training with new labels generated by a convex combination (the soft version) of the noisy labels and their predictions. (c) Backward (Patrini et al., 2017): Training via loss correction by multiplying the crossentropy loss by a noise-aware correction matrix. (d) Forward (Patrini et al., 2017): Training with label correction by multiplying the network prediction by a noise-aware correction matrix.

Figure 3: Layer-wise characteristics of generative classifiers from (a)/(b) ResNet-34 and (c)/(d) DenseNet-100 trained on the CIFAR-10 dataset.

µ c i ) 2 ((1-δout)σ 2 +δoutσ 2 out +δout(1-δout)Di) 2 , (14) -8 log(B( µ c , µ c , Σ, Σ)) a.s. → tr (σ 2 I) -1 (µ c -µ c )(µ c -µ c ) T 2 tr ((σ 2 I) -1 σ 2 I(σ 2 I) -1 (µ c -µ c )(µ c -µ c ) T ) = (µ c -µ c ) T (µ c -µ c ) σ 2 . (15)

µ c i ) 2 ((1-δout)σ 2 +δoutσ 2 out +δout(1-δout)Di) 2 (B(μ c , μc , Σ, Σ)) and lim N →∞

c , µ c , Σ, Σ) ≤ lim N →∞ B(μ c , μc , Σ, Σ).

c , µ c , Σ, Σ) B(μ c , μc , Σ, Σ) ≤ 1.

1)/2 samples with smallest distance α(x).Compute sample means and covariance, i.e., µ c , Σ c , using new subset X Kc . Exit the loop if the determinant of covariance matrix is not decreasing anymore. end for Return µ c and Σ c

Table content: <table><tr><td/><td>100</td><td/><td/><td/><td/><td/><td>100</td><td/><td/><td/><td/><td/><td>100</td></tr><tr><td/><td/><td/><td/><td/><td/><td/><td/><td/><td>Clean data</td><td/><td/><td/></tr><tr><td/><td>80</td><td/><td/><td/><td/><td/><td>80</td><td/><td>FGSM CW DeepFool</td><td/><td/><td/><td>90</td></tr><tr><td>Test set accuracy (%)</td><td>40 60</td><td/><td/><td/><td/><td>Test set accuracy (%)</td><td>40 60</td><td/><td/><td/><td/><td/><td>Test set accuracy (%)</td><td>50 60 70 80</td></tr><tr><td/><td>20</td><td/><td/><td/><td/><td/><td>20</td><td/><td/><td/><td/><td/><td>Softmax Generative (identity) + sample</td></tr><tr><td/><td/><td/><td>Clean</td><td>Uniform (40%)</td><td/><td/><td/><td/><td/><td/><td/><td/><td>40</td><td>Generative (identity) + LTS</td></tr><tr><td/><td>0</td><td>2</td><td colspan="2">Uniform (20%) Index of basic block Uniform (60%) 4 6 8 10 12</td><td>14</td><td/><td>0</td><td>2</td><td>4</td><td>Index of basic block 6 8 10</td><td>12</td><td>14</td><td>30</td><td>Generative (LDA) + sample [Lee et al., 2018] Noise fraction Generative (LDA) + MCD [DDGC] 0% 20% 40%</td><td>60%</td></tr><tr><td colspan="6">(a) Generalization of generative</td><td colspan="7">(b) Robustness of generative</td><td>(c) Generative classifiers under</td></tr><tr><td colspan="5">classifiers from noisy labels</td><td/><td colspan="7">classifiers to adversarial attacks</td><td>various assumptions</td></tr></table>

Effects of an ensemble method. We use the CIFAR-10 dataset with various uniform noise fractions. All values are percentages and the best results are highlighted in bold if the gain is bigger than 1%. 87.25 60.92 / 66.08 83.52 / 91.67 80.24 / 87.25 57.63 / 62.19 86.92 / 89.50 Uniform (40%) 61.85 / 80.01 44.08 / 59.72 72.89 / 87.16 68.61 / 81.04 45.08 / 53.98 81.91 / 85.71 Uniform (60%) 35.02 / 71.06 23.43 / 48.85 61.23 / 80.52 53.34 / 74.72 35.83 / 45.27 71.18 / 77.67 Flip (20%) 79.83 / 87.60 64.64 / 71.42 85.49 / 93.00 78.52 / 88.88 65.41 / 68.24 95.04 / 94.86 Flip (40%) 58.21 / 77.23 46.32 / 63.87 65.88 / 87.96 60.15 / 85.91 47.91 / 64.73 88.83 / 93.57

Table content: <table><tr><td>Model</td><td colspan="2">Inference method</td><td colspan="2">Ensemble Noise = 0%</td><td>20%</td><td>40%</td><td>60%</td></tr><tr><td/><td colspan="2">Softmax</td><td>-</td><td>94.42</td><td colspan="3">80.24 68.61 53.34</td></tr><tr><td>DenseNet</td><td colspan="2">Generative + sample</td><td>-</td><td>94.31 93.73</td><td colspan="3">85.08 74.72 59.49 87.23 80.01 69.17</td></tr><tr><td/><td colspan="2">Generative + MCD (DDGC)</td><td>-</td><td>94.37 93.49</td><td colspan="3">85.96 78.34 66.16 87.25 81.04 74.72</td></tr><tr><td/><td colspan="2">Softmax</td><td>-</td><td>95.01</td><td colspan="3">79.28 61.85 35.02</td></tr><tr><td>ResNet</td><td colspan="2">Generative + sample</td><td>-</td><td>94.98 94.98</td><td colspan="3">81.61 64.60 40.63 87.23 78.40 61.94</td></tr><tr><td/><td colspan="2">Generative + MCD (DDGC)</td><td>-</td><td>94.73 94.32</td><td colspan="3">83.04 68.04 42.74 87.25 80.01 71.06</td></tr><tr><td/><td/><td>ResNet</td><td/><td/><td colspan="2">DenseNet</td></tr><tr><td>Noise type (%)</td><td>CIFAR-10</td><td>CIFAR-100</td><td>SVHN</td><td>CIFAR-10</td><td colspan="2">CIFAR-100</td><td>SVHN</td></tr><tr><td/><td/><td>Softmax / DDGC</td><td/><td/><td colspan="2">Softmax / DDGC</td></tr><tr><td>Clean</td><td colspan="7">95.01 / 94.32 77.51 / 76.55 95.96 / 96.09 94.42 / 93.49 76.41 / 73.65 96.59 / 96.18</td></tr><tr><td colspan="2">Uniform (20%) 79.28 /</td><td/><td/><td/><td/><td/></tr></table>



Table content: <table/>

attacks, where the Test accuracy (%) of ResNet trained on various training methods which utilize a single classifier. We use the ensemble version of DDGC, and the best results are highlighted in bold if the gain is bigger than 1%.

Table content: <table><tr><td>Dataset</td><td>Training method</td><td>Clean</td><td colspan="5">Uniform (20%) Uniform (40%) Uniform (60%) Softmax / DDGC</td></tr><tr><td/><td>Cross-entropy</td><td>94.34 / 94.29</td><td>81.95 / 86.80</td><td colspan="2">63.84 / 78.75</td><td colspan="2">62.45 / 69.06</td></tr><tr><td/><td colspan="2">Bootstrap (hard) 94.56 / 94.49</td><td>82.90 / 87.26</td><td colspan="2">75.97 / 81.81</td><td colspan="2">72.91 / 75.61</td></tr><tr><td>CIFAR-10</td><td>Bootstrap (soft) Forward</td><td>94.46 / 94.31 94.53 / 94.39</td><td>80.29 / 85.20 85.80 / 87.66</td><td colspan="2">65.22 / 77.86 77.95 / 81.24</td><td colspan="2">58.55 / 69.71 72.56 / 74.92</td></tr><tr><td/><td>Backward</td><td>94.39 / 94.45</td><td>77.44 / 81.38</td><td colspan="2">62.83 / 72.76</td><td colspan="2">56.64 / 65.95</td></tr><tr><td/><td>D2L</td><td>94.55 / 94.38</td><td>88.89 / 89.00</td><td colspan="2">86.68 / 87.03</td><td colspan="2">76.83 / 78.08</td></tr><tr><td/><td>Cross-entropy</td><td>76.31 / 75.87</td><td>61.11 / 66.33</td><td colspan="2">45.08 / 58.68</td><td colspan="2">34.97 / 44.96</td></tr><tr><td/><td colspan="2">Bootstrap (hard) 75.65 / 75.49</td><td>61.61 / 65.14</td><td colspan="2">51.27 / 57.61</td><td colspan="2">39.04 / 47.24</td></tr><tr><td>CIFAR-100</td><td>Bootstrap (soft) Forward</td><td>76.40 / 75.88 75.84 / 75.43</td><td>60.28 / 65.54 63.73 / 67.46</td><td colspan="2">47.66 / 57.97 53.03 / 60.52</td><td colspan="2">34.68 / 44.84 41.28 / 47.88</td></tr><tr><td/><td>Backward</td><td>76.75 / 76.28</td><td>56.24 / 62.13</td><td colspan="2">37.70 / 51.82</td><td colspan="2">23.55 / 39.16</td></tr><tr><td/><td>D2L</td><td>76.13 / 75.57</td><td>71.90 / 72.07</td><td colspan="2">63.61 / 64.67</td><td/><td>9.51 / 39.42</td></tr><tr><td/><td>Cross-entropy</td><td>96.38 / 96.15</td><td>83.45 / 91.46</td><td colspan="2">60.86 / 83.40</td><td colspan="2">38.29 / 68.71</td></tr><tr><td/><td colspan="2">Bootstrap (hard) 96.40 / 96.26</td><td>83.43 / 92.24</td><td colspan="2">74.25 / 88.04</td><td colspan="2">66.51 / 82.03</td></tr><tr><td>SVHN</td><td>Bootstrap (soft) Forward</td><td>96.51 / 96.09 96.36 / 96.24</td><td>86.43 / 91.25 88.21 / 92.47</td><td colspan="2">58.22 / 83.90 80.35 / 88.55</td><td colspan="2">44.52 / 73.11 82.16 / 87.56</td></tr><tr><td/><td>Backward</td><td>96.43 / 96.31</td><td>87.00 / 87.31</td><td colspan="2">72.02 / 78.65</td><td colspan="2">50.54 / 71.06</td></tr><tr><td/><td>D2L</td><td>96.49 / 96.37</td><td>92.31 / 93.58</td><td colspan="2">94.46 / 94.68</td><td colspan="2">92.87 / 93.25</td></tr><tr><td>Dataset</td><td colspan="6">Noise type (%) Cross-entropy Decoupling MentorNet Co-teaching</td><td>Co-teaching + DDGC</td></tr><tr><td/><td>Flip (45%)</td><td>49.50</td><td>48.80</td><td>58.14</td><td>71.17</td><td/><td>70.50</td></tr><tr><td>CIFAR-10</td><td>Uniform (50%)</td><td>48.87</td><td>51.49</td><td>71.10</td><td>74.12</td><td/><td>76.26</td></tr><tr><td/><td>Uniform (20%)</td><td>76.25</td><td>80.44</td><td>80.76</td><td>82.13</td><td/><td>84.49</td></tr><tr><td/><td>Flip (45%)</td><td>31.99</td><td>26.05</td><td>31.60</td><td>33.34</td><td/><td>43.02</td></tr><tr><td>CIFAR-100</td><td>Uniform (50%)</td><td>25.21</td><td>25.80</td><td>39.00</td><td>41.49</td><td/><td>44.81</td></tr><tr><td/><td>Uniform (20%)</td><td>47.55</td><td>44.52</td><td>52.13</td><td>54.27</td><td/><td>57.74</td></tr></table>

Test accuracy (%) of 9-layer CNN trained on various training methods which utilize an ensemble of classifiers or meta-learning model. We use the ensemble version of DDGC and best results are highlighted in bold if the gain is bigger than 1%.

Table content: <table/>

Table5reports the classification accuracy of ResNet-34 on black-box adversarial attacks, and more results on white-box adversarial attacks and DenseNet-100 can be found in Appendix E. It shows that DDGC significantly improves the robustness against adversarial attacks when the training datasets contain noisy labels. We found that this is because the generative classifiers from low-level features are not utilized well, i.e., the trained weights (using validation) would be nearly zero for lower layers, when a training dataset only contains clean labels. Namely, the generative classifiers from low-level features are very robust (see Figure2(b)), since Test accuracy (%) of ResNet on black-box adversarial attacks. We use the ensemble version of DDGC, and the best results are highlighted in bold if the gain is bigger than 1%.

Table content: <table><tr><td>Dataset</td><td>Target</td><td>Adversarial attacks</td><td>Clean</td><td colspan="3">Uniform (20%) Uniform (40%) Uniform (60%) Softmax / DDGC</td></tr><tr><td/><td/><td>FGSM</td><td>73.67 / 73.19</td><td>63.19 / 73.24</td><td>47.80 / 68.68</td><td>30.34 / 58.42</td></tr><tr><td/><td>Softmax</td><td>DeepFool</td><td>77.88 / 77.74</td><td>75.27 / 84.39</td><td>56.99 / 78.35</td><td>36.47 / 70.91</td></tr><tr><td>CIFAR-10</td><td/><td>CW FGSM</td><td>36.36 / 36.86 94.62 / 94.38</td><td>65.45 / 76.26 75.11 / 82.78</td><td>50.68 / 73.59 48.06 / 68.86</td><td>33.08 / 67.81 28.84 / 57.40</td></tr><tr><td/><td>DDGC</td><td>DeepFool</td><td>70.23 / 70.07</td><td>73.93 / 83.35</td><td>56.23 / 77.66</td><td>36.46 / 70.75</td></tr><tr><td/><td/><td>CW</td><td>13.00 / 12.50</td><td>28.40 / 28.40</td><td>17.20 / 19.40</td><td>12.70 / 10.70</td></tr><tr><td/><td/><td>FGSM</td><td>50.20 / 46.97</td><td>36.16 / 41.95</td><td>26.34 / 39.82</td><td>19.19 / 40.68</td></tr><tr><td/><td>Softmax</td><td>DeepFool</td><td>46.63 / 42.44</td><td>47.13 / 53.71</td><td>36.18 / 51.98</td><td>23.10 / 45.65</td></tr><tr><td>CIFAR-100</td><td/><td>CW FGSM</td><td>35.66 / 33.50 73.07 / 70.63</td><td>38.57 / 45.63 45.36 / 49.39</td><td>32.21 / 47.61 28.97 / 41.31</td><td>22.52 / 44.74 19.63 / 39.55</td></tr><tr><td/><td>DDGC</td><td>DeepFool</td><td>44.86 / 39.40</td><td>42.59 / 49.63</td><td>32.59 / 48.27</td><td>21.90 / 44.72</td></tr><tr><td/><td/><td>CW</td><td>26.81 / 24.13</td><td>15.54 / 17.04</td><td>10.59 / 17.95</td><td>8.81 / 22.90</td></tr><tr><td/><td/><td>FGSM</td><td>61.94 / 62.21</td><td>56.26 / 60.25</td><td>39.93 / 52.20</td><td>31.93 / 42.76</td></tr><tr><td/><td>Softmax</td><td>DeepFool</td><td>78.01 / 78.54</td><td>83.55 / 89.51</td><td>69.46 / 85.05</td><td>61.15 / 79.35</td></tr><tr><td>SVHN</td><td/><td>CW FGSM</td><td>57.27 / 58.11 96.13 / 96.12</td><td>76.21 / 83.85 51.40 / 54.54</td><td>60.64 / 79.56 35.82 / 47.31</td><td>51.73 / 73.83 29.29 / 40.21</td></tr><tr><td/><td>DDGC</td><td>DeepFool</td><td>69.90 / 70.00</td><td>81.40 / 88.63</td><td>67.90 / 83.68</td><td>59.36 / 77.81</td></tr><tr><td/><td/><td>CW</td><td>36.10 / 36.80</td><td>46.30 / 51.20</td><td>34.30 / 36.70</td><td>30.00 / 33.80</td></tr><tr><td>Dataset</td><td>Target</td><td>Adversarial attacks</td><td>Clean</td><td colspan="3">Uniform (20%) Uniform (40%) Uniform (60%) Softmax / DDGC</td></tr><tr><td/><td/><td>FGSM</td><td>91.63 / 91.59</td><td>79.01 / 84.54</td><td>62.47 / 78.45</td><td>37.71 / 66.96</td></tr><tr><td/><td>Softmax</td><td>DeepFool</td><td>93.72 / 93.34</td><td>80.95 / 85.83</td><td>63.79 / 79.29</td><td>38.23 / 68.04</td></tr><tr><td>CIFAR-10</td><td/><td>CW FGSM</td><td>90.00 / 89.71 94.81 / 94.55</td><td>79.38 / 84.70 80.82 / 86.08</td><td>62.92 / 78.49 62.22 / 78.62</td><td>37.78 / 67.51 37.22 / 66.81</td></tr><tr><td/><td>DDGC</td><td>DeepFool</td><td>93.06 / 92.65</td><td>80.66 / 85.64</td><td>63.72 / 79.23</td><td>38.32 / 68.01</td></tr><tr><td/><td/><td>CW</td><td>86.10 / 85.50</td><td>71.80 / 79.00</td><td>53.40 / 65.30</td><td>30.70 / 50.30</td></tr><tr><td/><td/><td>FGSM</td><td>63.67 / 62.74</td><td>51.39 / 58.88</td><td>35.78 / 52.85</td><td>22.13 / 44.86</td></tr><tr><td/><td>Softmax</td><td>DeepFool</td><td>60.92 / 59.29</td><td>53.34 / 60.94</td><td>37.10 / 54.90</td><td>23.47 / 46.35</td></tr><tr><td>CIFAR-100</td><td/><td>CW FGSM</td><td>59.74 / 58.88 70.17 / 69.08</td><td>51.67 / 59.14 54.25 / 61.23</td><td>36.27 / 54.00 36.81 / 52.98</td><td>23.24 / 46.21 21.97 / 44.22</td></tr><tr><td/><td>DDGC</td><td>DeepFool</td><td>59.81 / 57.90</td><td>52.72 / 61.27</td><td>36.90 / 54.86</td><td>23.81 / 46.45</td></tr><tr><td/><td/><td>CW</td><td>57.00 / 55.50</td><td>39.27 / 47.31</td><td>25.22 / 41.72</td><td>15.27 / 35.50</td></tr><tr><td/><td/><td>FGSM</td><td>68.00 / 68.34</td><td>68.08 / 72.33</td><td>51.56 / 62.60</td><td>39.01 / 54.11</td></tr><tr><td/><td>Softmax</td><td>DeepFool</td><td>85.68 / 85.77</td><td>87.98 / 91.63</td><td>74.69 / 86.48</td><td>65.64 / 81.32</td></tr><tr><td>SVHN</td><td/><td>CW FGSM</td><td>77.48 / 77.70 96.36 / 96.27</td><td>85.66 / 90.02 64.40 / 67.44</td><td>71.77 / 84.95 47.48 / 58.00</td><td>62.67 / 79.95 35.90 / 51.89</td></tr><tr><td/><td>DDGC</td><td>DeepFool</td><td>79.54 / 79.36</td><td>85.59 / 89.90</td><td>71.95 / 84.81</td><td>63.36 / 79.86</td></tr><tr><td/><td/><td>CW</td><td>62.90 / 62.40</td><td>70.40 / 74.10</td><td>56.30 / 67.60</td><td>44.20 / 58.40</td></tr></table>

Test accuracy (%) of ResNet optimized by adversarial training. We use the ensemble version of DDGC, and best results are highlighted in bold if the gain is bigger than 1%.

Table content: <table/>

The L ∞ mean perturbation and classification accuracy of ResNet-34 on clean and adversarial samples.

Table content: <table><tr><td>Dataset</td><td>Target</td><td>Data type</td><td colspan="2">Clean L ∞ Acc.</td><td colspan="5">Uniform (20%) Uniform (40%) Uniform (60%) L ∞ Acc. L ∞ Acc. L ∞ Acc.</td></tr><tr><td/><td/><td>Clean</td><td>0</td><td>95.01</td><td>0</td><td>79.28</td><td>0</td><td>61.85</td><td>0</td><td>35.02</td></tr><tr><td/><td>Softmax</td><td colspan="4">FGSM DeepFool 0.34 0.36 0.06 0.05 56.90 0.05</td><td>25.26 0.04</td><td>0.05 0.03</td><td>23.71 0.00</td><td>0.05 0.02</td><td>20.88 0.00</td></tr><tr><td>CIFAR-10</td><td/><td>CW Clean</td><td colspan="3">0.09 0.02 0.04 0 94.32 0</td><td>0.57 87.25</td><td>0.02 0</td><td>0.40 80.01</td><td>0.01 0</td><td>0.27 71.06</td></tr><tr><td/><td>DDGC</td><td colspan="4">FGSM DeepFool 0.44 5.25 0.09 0.05 94.59 0.05</td><td>78.40 66.26</td><td>0.05 0.04</td><td>59.33 71.77</td><td>0.05 0.02</td><td>52.87 68.76</td></tr><tr><td/><td/><td>CW</td><td colspan="3">0.21 0.00 0.27</td><td>0.00</td><td>0.27</td><td>0.10</td><td>0.23</td><td>0.20</td></tr><tr><td/><td/><td>Clean</td><td>0</td><td>77.51</td><td>0</td><td>60.92</td><td>0</td><td>44.08</td><td>0</td><td>23.43</td></tr><tr><td/><td>Softmax</td><td colspan="4">FGSM DeepFool 0.33 0.27 0.12 0.05 31.78 0.05</td><td>17.95 0.08</td><td>0.05 0.06</td><td>13.20 0.05</td><td>0.05 0.04</td><td>10.11 0.01</td></tr><tr><td>CIFAR-100</td><td/><td>CW Clean</td><td colspan="3">0.06 0.05 0.03 0 76.55 0</td><td>0.23 66.08</td><td>0.02 0</td><td>0.13 59.72</td><td>0.01 0</td><td>0.01 48.85</td></tr><tr><td/><td>DDGC</td><td colspan="4">FGSM DeepFool 0.32 8.50 0.14 0.05 72.44 0.05</td><td>42.25 28.40</td><td>0.05 0.09</td><td>34.06 30.90</td><td>0.05 0.07</td><td>31.31 35.86</td></tr><tr><td/><td/><td>CW</td><td colspan="3">0.07 0.04 0.08</td><td>0.00</td><td>0.08</td><td>0.32</td><td>0.09</td><td>0.77</td></tr><tr><td/><td/><td>Clean</td><td>0</td><td>95.96</td><td>0</td><td>83.52</td><td>0</td><td>72.89</td><td>0</td><td>61.23</td></tr><tr><td/><td>Softmax</td><td colspan="4">FGSM DeepFool 0.52 1.30 0.09 0.20 54.96 0.20</td><td>42.58 0.14</td><td>0.20 0.06</td><td>24.61 0.13</td><td>0.20 0.05</td><td>24.42 0.17</td></tr><tr><td>SVHN</td><td/><td>CW Clean</td><td colspan="3">0.14 0.11 0.05 0 96.09 0</td><td>0.16 91.67</td><td>0.04 0</td><td>0.01 87.16</td><td>0.04 0</td><td>0.03 80.52</td></tr><tr><td/><td>DDGC</td><td colspan="4">FGSM DeepFool 0.77 17.16 0.10 0.20 96.15 0.20</td><td>47.26 71.75</td><td>0.20 0.07</td><td>41.23 80.08</td><td>0.20 0.05</td><td>35.46 75.00</td></tr><tr><td/><td/><td>CW</td><td colspan="3">0.27 0.00 0.26</td><td>1.00</td><td>0.25</td><td>0.30</td><td>0.29</td><td>0.50</td></tr><tr><td>Dataset</td><td>Target</td><td>Data type</td><td colspan="2">Clean L ∞ Acc.</td><td colspan="5">Uniform (20%) Uniform (40%) Uniform (60%) L ∞ Acc. L ∞ Acc. L ∞ Acc.</td></tr><tr><td/><td/><td>Clean</td><td>0</td><td>94.42</td><td>0</td><td>80.24</td><td>0</td><td>68.61</td><td>0</td><td>53.34</td></tr><tr><td/><td>Softmax</td><td colspan="4">FGSM DeepFool 0.14 0.20 0.03 0.05 32.50 0.05</td><td>22.84 0.05</td><td>0.05 0.03</td><td>22.49 0.10</td><td>0.05 0.02</td><td>20.90 0.01</td></tr><tr><td>CIFAR-10</td><td/><td>CW Clean</td><td colspan="3">0.06 0.08 0.03 0 93.49 0</td><td>0.39 87.25</td><td>0.03 0</td><td>0.27 81.04</td><td>0.02 0</td><td>0.18 74.72</td></tr><tr><td/><td>DDGC</td><td colspan="4">FGSM DeepFool 0.14 30.45 0.05 0.05 83.88 0.05</td><td>62.69 70.09</td><td>0.05 0.03</td><td>36.75 71.50</td><td>0.05 0.02</td><td>44.16 67.50</td></tr><tr><td/><td/><td>CW</td><td colspan="3">0.12 0.00 0.11</td><td>0.04</td><td>0.12</td><td>0.09</td><td>0.23</td><td>0.54</td></tr><tr><td/><td/><td>Clean</td><td>0</td><td>76.41</td><td>0</td><td>57.63</td><td>0</td><td>45.08</td><td>0</td><td>35.83</td></tr><tr><td/><td>Softmax</td><td colspan="4">FGSM DeepFool 0.12 0.10 0.05 0.05 18.73 0.05</td><td>12.74 0.00</td><td>0.05 0.03</td><td>10.55 0.01</td><td>0.05 0.02</td><td>9.07 0.01</td></tr><tr><td>CIFAR-100</td><td/><td>CW Clean</td><td colspan="3">0.03 0.22 0.02 0 73.65 0</td><td>0.20 62.19</td><td>0.02 0</td><td>0.20 53.98</td><td>0.01 0</td><td>0.12 45.27</td></tr><tr><td/><td>DDGC</td><td colspan="4">FGSM DeepFool 0.12 17.40 0.06 0.05 52.14 0.05</td><td>34.36 33.63</td><td>0.05 0.04</td><td>24.24 39.95</td><td>0.05 0.03</td><td>22.79 37.36</td></tr><tr><td/><td/><td>CW</td><td colspan="3">0.07 0.18 0.08</td><td>1.04</td><td>0.07</td><td>2.31</td><td>0.06</td><td>3.86</td></tr><tr><td/><td/><td>Clean</td><td>0</td><td>96.59</td><td>0</td><td>86.92</td><td>0</td><td>81.91</td><td>0</td><td>71.18</td></tr><tr><td/><td>Softmax</td><td colspan="4">FGSM DeepFool 0.26 3.04 0.22 0.20 51.18 0.20</td><td>46.64 4.21</td><td>0.20 0.20</td><td>40.21 2.14</td><td>0.20 0.18</td><td>36.62 2.11</td></tr><tr><td>SVHN</td><td/><td>CW Clean</td><td colspan="3">0.12 0.15 0.12 0 96.18 0</td><td>0.31 89.50</td><td>0.10 0</td><td>0.16 85.71</td><td>0.10 0</td><td>0.09 77.67</td></tr><tr><td/><td>DDGC</td><td colspan="4">FGSM DeepFool 0.26 50.00 0.23 0.20 90.74 0.20</td><td>54.87 58.59</td><td>0.20 0.20</td><td>41.91 66.81</td><td>0.20 0.14</td><td>42.91 63.68</td></tr><tr><td/><td/><td>CW</td><td colspan="3">0.25 0.00 0.33</td><td>1.59</td><td>0.39</td><td>4.63</td><td>0.46</td><td>11.22</td></tr></table>

The L ∞ mean perturbation and classification accuracy (%) of DenseNet-100 on clean and adversarial samples.

Table content: <table/>

Comparison with softmax classifier fine-tuned with validation data. All values are percentages and the best results are highlighted in bold if gain is bigger than 1%.

Table content: <table><tr><td/><td/><td/><td/><td/><td colspan="3">Noisy Dataset</td><td/></tr><tr><td>Model</td><td>Inference Method</td><td/><td>CIFAR-10</td><td/><td/><td>CIFAR-100</td><td/><td/><td>SVHN</td></tr><tr><td/><td/><td>20%</td><td>40%</td><td>60%</td><td>20%</td><td>40%</td><td>60%</td><td>20%</td><td>40%</td><td>60%</td></tr><tr><td/><td>Softmax</td><td colspan="8">79.28 61.85 35.02 60.92 44.08 23.43 83.52 72.89 61.23</td></tr><tr><td>ResNet</td><td colspan="9">Softmax + fine-tuning (val) Generative + MCD 83.04 68.04 42.74 63.20 51.71 33.96 89.60 78.88 66.96 80.48 69.42 54.16 51.15 35.17 20.41 85.10 81.93 75.06</td></tr><tr><td/><td>Generative + MCD + ensemble (val)</td><td colspan="8">87.25 80.01 71.06 66.08 59.72 48.85 91.67 87.16 80.52</td></tr><tr><td/><td>Softmax</td><td colspan="8">80.24 68.61 53.34 57.63 45.08 35.83 86.92 81.91 71.18</td></tr><tr><td>DenseNet</td><td colspan="9">Softmax + fine-tuning (val) Generative + MCD 85.96 78.34 66.16 58.22 48.54 37.44 88.64 84.08 74.47 82.47 74.38 63.25 48.69 35.97 27.73 86.83 76.39 68.13</td></tr><tr><td/><td>Generative + MCD + ensemble (val)</td><td colspan="8">87.25 81.04 74.72 62.19 53.98 45.27 89.50 85.71 77.67</td></tr></table>

Table 10 and 11 show the classification accuracy on white-box adversarial attacks when we train ResNets using the Test accuracy (%) of ResNet on white-box adversarial attacks. We use the ensemble version of DDGC, and the best results are highlighted in bold if gain is bigger than 1%.

Table content: <table><tr><td>Dataset</td><td>Attack type</td><td>Adversarial attacks</td><td>Clean</td><td colspan="3">Uniform (20%) Uniform (40%) Uniform (60%) Softmax / DDGC</td></tr><tr><td/><td/><td>FGSM</td><td>56.90 / 57.51</td><td>25.26 / 57.11</td><td>23.71 / 59.32</td><td>20.88 / 54.97</td></tr><tr><td/><td>Softmax</td><td>DeepFool</td><td>0.36 / 18.48</td><td>0.04 / 71.14</td><td>0.00 / 73.21</td><td>0.00 / 68.92</td></tr><tr><td>CIFAR-10</td><td/><td>CW FGSM</td><td>0.02 / 1.16 94.74 / 94.59</td><td>0.57 / 52.39 70.68 / 78.40</td><td>0.40 / 61.14 25.54 / 59.33</td><td>0.27 / 64.29 14.95 / 52.87</td></tr><tr><td/><td>DDGC</td><td>DeepFool</td><td>11.13 / 5.25</td><td>4.71 / 66.26</td><td>3.20 / 71.77</td><td>4.17 / 68.76</td></tr><tr><td/><td/><td>CW</td><td>0.00 / 0.00</td><td>0.20 / 0.00</td><td>0.20 / 0.10</td><td>0.20 / 0.20</td></tr><tr><td/><td/><td>FGSM</td><td>31.78 / 31.94</td><td>17.95 / 30.23</td><td>13.20 / 30.81</td><td>10.11 / 35.62</td></tr><tr><td/><td>Softmax</td><td>DeepFool</td><td>0.27 / 9.84</td><td>0.08 / 37.07</td><td>0.05 / 42.15</td><td>0.01 / 40.82</td></tr><tr><td>CIFAR-100</td><td/><td>CW FGSM</td><td>0.05 / 5.07 74.51 / 72.44</td><td>0.23 / 27.95 35.41 / 42.25</td><td>0.13 / 35.32 23.68 / 34.06</td><td>0.01 / 39.90 12.08 / 31.31</td></tr><tr><td/><td>DDGC</td><td>DeepFool</td><td>4.09 / 8.50</td><td>4.72 / 28.40</td><td>3.54 / 30.90</td><td>1.63 / 35.86</td></tr><tr><td/><td/><td>CW</td><td>0.95 / 0.04</td><td>0.86 / 0.00</td><td>0.31 / 0.32</td><td>0.22 / 0.77</td></tr><tr><td/><td/><td>FGSM</td><td>54.96 / 56.46</td><td>42.58 / 54.26</td><td>24.61 / 46.80</td><td>24.42 / 40.92</td></tr><tr><td/><td>Softmax</td><td>DeepFool</td><td>1.30 / 45.40</td><td>0.14 / 79.29</td><td>0.13 / 81.59</td><td>0.17 / 75.87</td></tr><tr><td>SVHN</td><td/><td>CW FGSM</td><td>0.11 / 13.39 96.19 / 96.15</td><td>0.16 / 56.58 40.82 / 47.26</td><td>0.01 / 68.22 19.80 / 41.23</td><td>0.03 / 61.93 18.13 / 35.46</td></tr><tr><td/><td>DDGC</td><td>DeepFool</td><td>13.00 / 17.16</td><td>17.58 / 71.75</td><td>19.66 / 80.08</td><td>27.83 / 75.00</td></tr><tr><td/><td/><td>CW</td><td>0.10 / 0.00</td><td>0.70 / 1.00</td><td>0.40 / 0.30</td><td>0.10 / 0.50</td></tr><tr><td>Dataset</td><td>Target</td><td>Adversarial attacks</td><td>Clean</td><td colspan="3">Uniform (20%) Uniform (40%) Uniform (60%) Softmax / DDGC</td></tr><tr><td/><td/><td>FGSM</td><td>74.72 / 74.53</td><td>35.21 / 64.06</td><td>24.64 / 67.93</td><td>20.70 / 59.87</td></tr><tr><td/><td>Softmax</td><td>DeepFool</td><td>1.09 / 14.45</td><td>0.04 / 61.54</td><td>0.05 / 69.22</td><td>0.00 / 61.40</td></tr><tr><td>CIFAR-10</td><td/><td>CW FGSM</td><td>0.05 / 1.31 94.68 / 94.44</td><td>0.00 / 40.81 59.01 / 73.32</td><td>0.00 / 59.86 31.60 / 67.96</td><td>0.00 / 57.22 12.83 / 55.54</td></tr><tr><td/><td>DDGC</td><td>DeepFool</td><td>9.41 / 10.77</td><td>9.77 / 44.86</td><td>9.68 / 66.50</td><td>9.36 / 60.40</td></tr><tr><td/><td/><td>CW</td><td>0.14 / 0.04</td><td>0.04 / 0.00</td><td>0.18 / 0.04</td><td>0.31 / 0.00</td></tr><tr><td/><td/><td>FGSM</td><td>37.73 / 42.57</td><td>24.04 / 47.00</td><td>16.52 / 46.46</td><td>11.52 / 41.62</td></tr><tr><td/><td>Softmax</td><td>DeepFool</td><td>0.13 / 10.40</td><td>0.04 / 36.09</td><td>0.00 / 42.95</td><td>0.00 / 41.45</td></tr><tr><td>CIFAR-100</td><td/><td>CW FGSM</td><td>0.00 / 9.31 62.60 / 62.88</td><td>0.04 / 34.81 40.98 / 51.68</td><td>0.00 / 44.04 25.35 / 43.29</td><td>0.00 / 41.54 10.92 / 36.39</td></tr><tr><td/><td>DDGC</td><td>DeepFool</td><td>2.81 / 6.22</td><td>3.45 / 21.36</td><td>2.31 / 27.36</td><td>1.86 / 34.13</td></tr><tr><td/><td/><td>CW</td><td>1.18 / 0.00</td><td>0.81 / 0.00</td><td>0.45 / 0.04</td><td>0.27 / 0.04</td></tr><tr><td/><td/><td>FGSM</td><td>57.86 / 60.31</td><td>44.25 / 66.10</td><td>30.17 / 60.10</td><td>27.59 / 56.45</td></tr><tr><td/><td>Softmax</td><td>DeepFool</td><td>2.54 / 35.72</td><td>0.18 / 77.50</td><td>0.13 / 79.63</td><td>0.00 / 75.40</td></tr><tr><td>SVHN</td><td/><td>CW FGSM</td><td>0.50 / 19.40 96.42 / 96.35</td><td>0.04 / 63.68 50.07 / 61.63</td><td>0.04 / 69.13 22.10 / 52.05</td><td>0.00 / 66.90 18.79 / 51.15</td></tr><tr><td/><td>DDGC</td><td>DeepFool</td><td>15.81 / 21.81</td><td>20.95 / 73.31</td><td>18.18 / 78.09</td><td>33.36 / 76.27</td></tr><tr><td/><td/><td>CW</td><td>0.27 / 0.63</td><td>0.77 / 0.90</td><td>0.36 / 0.54</td><td>1.18 / 1.40</td></tr></table>

Test accuracy (%) of ResNet optimized by adversarial training on CIFAR-10. We test the white-box adversarial attacks. We use the ensemble version of DDGC, and best results are highlighted in bold if the gain is bigger than 1%. cross-entropy loss and adversarial training, respectively. We also report the classification accuracy of DenseNet on white-box and black-box adversarial attacks in Table12and 13, respectively.

Table content: <table><tr><td>Dataset</td><td>Target</td><td>Adversarial attacks</td><td>Clean</td><td colspan="3">Uniform (20%) Uniform (40%) Uniform (60%) Softmax / DDGC</td></tr><tr><td/><td/><td>FGSM</td><td>32.50 / 38.00</td><td>22.84 / 56.05</td><td>22.49 / 46.51</td><td>20.90 / 52.63</td></tr><tr><td/><td>Softmax</td><td>DeepFool</td><td>0.20 / 36.09</td><td>0.05 / 76.94</td><td>0.10 / 75.03</td><td>0.01 / 68.83</td></tr><tr><td>CIFAR-10</td><td/><td>CW FGSM</td><td>0.08 / 4.27 83.92 / 83.88</td><td>0.39 / 55.15 53.33 / 62.69</td><td>0.27 / 48.83 20.51 / 36.75</td><td>0.18 / 54.83 13.97 / 44.16</td></tr><tr><td/><td>DDGC</td><td>DeepFool</td><td>30.81 / 30.45</td><td>6.95 / 70.09</td><td>12.22 / 71.50</td><td>13.68 / 67.50</td></tr><tr><td/><td/><td>CW</td><td>0.00 / 0.00</td><td>0.13 / 0.04</td><td>0.22 / 0.09</td><td>0.63 / 0.54</td></tr><tr><td/><td/><td>FGSM</td><td>18.73 / 22.65</td><td>12.74 / 27.18</td><td>10.55 / 22.36</td><td>9.07 / 22.09</td></tr><tr><td/><td>Softmax</td><td>DeepFool</td><td>0.10 / 19.31</td><td>0.00 / 40.09</td><td>0.01 / 42.10</td><td>0.01 / 37.46</td></tr><tr><td>CIFAR-100</td><td/><td>CW FGSM</td><td>0.22 / 12.87 54.25 / 52.14</td><td>0.20 / 30.14 30.14 / 34.26</td><td>0.20 / 32.20 19.67 / 24.24</td><td>0.12 / 31.46 12.46 / 22.79</td></tr><tr><td/><td>DDGC</td><td>DeepFool</td><td>15.27 / 17.40</td><td>8.77 / 33.63</td><td>6.45 / 39.95</td><td>7.90 / 37.36</td></tr><tr><td/><td/><td>CW</td><td>2.45 / 0.18</td><td>1.09 / 1.04</td><td>0.90 / 2.31</td><td>1.00 / 3.86</td></tr><tr><td/><td/><td>FGSM</td><td>51.18 / 53.75</td><td>46.64 / 50.98</td><td>40.21 / 46.60</td><td>36.62 / 40.05</td></tr><tr><td/><td>Softmax</td><td>DeepFool</td><td>3.04 / 66.30</td><td>4.21 / 71.28</td><td>2.14 / 71.97</td><td>2.11 / 62.26</td></tr><tr><td>SVHN</td><td/><td>CW FGSM</td><td>0.15 / 27.01 90.39 / 90.74</td><td>0.31 / 36.48 52.14 / 54.87</td><td>0.16 / 33.89 36.31 / 41.91</td><td>0.09 / 26.86 41.18 / 42.91</td></tr><tr><td/><td>DDGC</td><td>DeepFool</td><td>41.63 / 50.00</td><td>26.40 / 58.59</td><td>36.27 / 66.81</td><td>52.54 / 63.68</td></tr><tr><td/><td/><td>CW</td><td>1.45 / 0.00</td><td>1.40 / 1.59</td><td>3.31 / 4.63</td><td>11.36 / 11.22</td></tr></table>

Test accuracy (%) of DenseNet on white-box adversarial attacks. We use the ensemble version of DDGC, and the best results are highlighted in bold if gain is bigger than 1%.

Table content: <table><tr><td>Dataset</td><td>Target</td><td>Adversarial attacks</td><td>Clean</td><td colspan="3">Uniform (20%) Uniform (40%) Uniform (60%) Softmax / DDGC</td></tr><tr><td/><td/><td>FGSM</td><td>62.96 / 64.66</td><td>60.63 / 70.19</td><td>47.77 / 55.11</td><td>40.07 / 62.02</td></tr><tr><td/><td>Softmax</td><td>DeepFool</td><td>87.68 / 87.78</td><td>76.48 / 84.54</td><td>67.93 / 80.38</td><td>51.31 / 73.57</td></tr><tr><td>CIFAR-10</td><td/><td>CW FGSM</td><td>52.12 / 54.92 87.69 / 87.82</td><td>63.48 / 74.23 63.25 / 70.94</td><td>55.41 / 67.45 44.47 / 51.43</td><td>45.03 / 69.03 38.14 / 59.76</td></tr><tr><td/><td>DDGC</td><td>DeepFool</td><td>88.31 / 88.36</td><td>75.81 / 84.31</td><td>66.04 / 79.86</td><td>52.36 / 73.68</td></tr><tr><td/><td/><td>CW</td><td>24.31 / 24.22</td><td>26.77 / 31.04</td><td>19.68 / 19.45</td><td>15.36 / 15.81</td></tr><tr><td/><td/><td>FGSM</td><td>48.84 / 45.83</td><td>35.07 / 39.32</td><td>24.60 / 27.43</td><td>21.64 / 28.72</td></tr><tr><td/><td>Softmax</td><td>DeepFool</td><td>66.87 / 62.53</td><td>53.00 / 56.05</td><td>43.09 / 48.18</td><td>32.42/ 41.71</td></tr><tr><td>CIFAR-100</td><td/><td>CW FGSM</td><td>55.74 / 53.09 61.21 / 57.29</td><td>44.70 / 49.49 39.52 / 42.20</td><td>36.49 / 41.53 26.35 / 28.31</td><td>29.67 / 38.97 22.82 / 29.00</td></tr><tr><td/><td>DDGC</td><td>DeepFool</td><td>68.04 / 64.95</td><td>52.04 / 55.86</td><td>43.27 / 48.36</td><td>33.59 / 42.31</td></tr><tr><td/><td/><td>CW</td><td>37.86 / 34.09</td><td>23.31 / 26.81</td><td>18.63 / 20.81</td><td>17.54 / 24.68</td></tr><tr><td/><td/><td>FGSM</td><td>58.08 / 59.10</td><td>53.76 / 56.76</td><td>47.58 / 52.08</td><td>46.82 / 47.91</td></tr><tr><td/><td>Softmax</td><td>DeepFool</td><td>89.59 / 90.13</td><td>86.60 / 88.25</td><td>82.60 / 85.95</td><td>81.63 / 82.81</td></tr><tr><td>SVHN</td><td/><td>CW FGSM</td><td>66.35 / 68.61 89.87 / 90.23</td><td>68.27 / 71.23 52.57 / 55.92</td><td>60.25 / 66.23 43.59 / 47.60</td><td>64.19 / 66.20 46.66 / 47.91</td></tr><tr><td/><td>DDGC</td><td>DeepFool</td><td>87.40 / 88.04</td><td>81.18 / 83.95</td><td>79.90 / 83.72</td><td>78.81 / 80.31</td></tr><tr><td/><td/><td>CW</td><td>40.59 / 41.50</td><td>39.68 / 41.18</td><td>38.59 / 39.77</td><td>45.45 / 46.63</td></tr></table>

Test accuracy (%) of DenseNet on black-box adversarial attacks. We use the ensemble version of DDGC, and the best results are highlighted in bold if gain is bigger than 1%.

Table content: <table/>

Lemma 1 states the convergences of sample and MCD estimators on single multivariate Gaussian distribution. One can extend it to C number of multivariate Gaussian distributions, which have the class mean µ c and class covariance matrix Σ c on each class label c ∈ {1, ..., C}, by the assumptions A1 ∼ A5. Then one can induce the class mean of MCD and sample estimators converge almost surely as follows:

Table content: <table/>

Peer review (iclr2019):
 The paper proposes a new method for robustifying a pre-trained model improving its decision boundaries. The goal is to defend the model from mistakes in training labels and to be more robust to adversarial examples at test time. The main idea is to train a LDA on top of the last-layer, or many layers in its ensemble version, making use of a small set of clean labels after training the main model. Additionally, robustness to outliers is achieved by the minimum covariance determinant estimator for the LDA covariance matrix. While I find this idea interesting and of potential practical use, I have concerns about novelty and the experimental results and overall I recommend rejection. == Method At a high level, the idea of imposing a mixture of gaussian structure in the feature space of a deep neural network classifier is not new. See for example [A, B]. In particular, [B] performs experiments on adversarial examples. Moreover, in spite of the authors writing that their goal is “completely different” from [Lee at al 18a, Ma et al 18a], I found the two cited papers having a similar intent and approach to the problem, but a comparison is completely missing. Without a proper comparison (formal and experimental) with these lines of work, the paper is incomplete. Theorem 1 well supports the proposed method and it is well explained. I did not check the proofs in appendix. Regarding the presentation, I found odd having some experimental results (page 5) before the Section on experience even have started. == Experiments The authors did not comment on the computational overhead of training their LDA. But I assume it is very cheap compared to training e.g. the ResNet, correct? I also did not find an explanation of which version backward/forward losses [Patrini et al. 17] is used in the experiments: are the noise transition matrices estimated on the data or assumed to be known (for fair comparison, I would do the former). I disagree on the importance of the numbers reported on the abstract: DenseNet on Cifar10 with 60% goes from 53.34 to 74.72. This is the improvement with the weakest possible baseline, i.e. no method to defend for noise! Looking at Table 3, which is on ResNets, I will make this point clear. Noise 60% on CIFAR10, DDGC improves 60.05-> 71.38, while (hard) bootstrap and forward do better. Even more, it seems that forward does always better than DDGC with noise 60% on every dataset. Therefore, I don’t find interesting to report how DDGC improve upon “no baseline”, because known methods do even better. Yet, it is interesting --- and I find this to be a contribution of the paper --- that DDGC can be used in combination with prior work to boost performance even further. A missing empirical analysis is on class-conditional noise (see for example Patrini et al. 17 for a definition). An additional column on the table showing that the algorithm can also work in this case would improve the confidence that the proposed method is useful in practice. Uniform noise is the least realistic assumption for label noise. Regarding the experiments on adversarial examples, I am not convinced of their relevance at all. There are now dozens of defence methods that work (partially) for improving robustness. I don’t think it is of any practical use to show that a new algorithm (such at DDGD) provide some defence compared to no defence. A proper baseline should have been compared. One more unclear but important point: is Table 3 obtained by white-box attacks on the Resnet/Denset but oblivious of the MCD? Is so, I don’t think such an experiment tells the whole story: as the the MCD would arguably also be deployed for classification, the attacker would also target it. Additionally, the authors state “we remark that accessing the parameters of the generative classifiers […] is not a mild assumption since the information about training data is required to compute them”. I don’t follow this argument: this is just part of the classifier. White box attacks are by definition performed with the knowledge of the model, what is the difference here? Table 8 rises some concerns. I appreciate the idea of testing full white-box adversarial attacks here. But I don’t understand how it is possible that DDGC is more robust, with higher adversarial test accuracy, than in Table 3. [A] Wen, Yandong, et al. "A discriminative feature learning approach for deep face recognition." European Conference on Computer Vision. Springer, Cham, 2016. [B] Wan, Weitao, et al. "Rethinking feature distribution for loss functions in image classification." Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2018.