Simultaneous Classification and Out-of-Distribution Detection Using Deep Neural Networks
Authors: Aristotelis-Angelos Papadopoulos
Nazim Shaikh
Jiamian Wang
Mohammad Reza Rajati
Venue:  (2020)
Keywords: Out-of-Distribution Detection
OOD detection
Outlier Exposure
Classification
Open-World Classification
Anomaly Detection
Novelty Detection
Calibration
Neural Networks

Abstract: Deep neural networks have achieved great success in classiﬁcation tasks during the last years. However, one major problem to the path towards artiﬁcial intelligence is the inability of neural networks to accurately detect samples from novel class distributions and therefore, most of the existent classiﬁcation algorithms assume that all classes are known prior to the training stage. In this work, we propose a methodology for training a neural network that allows it to efﬁciently detect out-of-distribution (OOD) examples without compromising much of its classiﬁcation accuracy on the test examples from known classes. Based on the Outlier Exposure (OE) technique, we propose a novel loss function that achieves state-of-the-art results in out-of-distribution detection with OE both on image and text classiﬁcation tasks. Additionally, the way this method was constructed makes it suitable for training any classiﬁcation algorithm that is based on Maximum Likelihood methods.

Under review as a conference paper at ICLR 2020
SIMULTANEOUS CLASSIFICATION AND OUT-OF-DISTRIBUTION DETECTION USING DEEP NEURAL NETWORKS
Anonymous authors Paper under double-bling review
ABSTRACT
Deep neural networks have achieved great success in classification tasks during the last years. However, one major problem to the path towards artificial intelli- gence is the inability of neural networks accurately detect samples from novel class distributions and therefore, most of the existent classification algorithms as- sume that all classes are known prior the training stage. In this work, we propose a methodology for training a neural network that allows it to efficiently detect out- od-distribution (OOD) examples without compromising much of its classification accuracy on the test examples from known classes. Based on the Outlier Exposure (OE) technique, we propose novel loss function that achieves sfate-of-the-an re- sults in out-of diitti--iittitttinnn detection with OE both on image and text classification tasks. Additionally, we experimentally show that the combination of our method with the Mahalanobis distance-based classifier achieves sate-of-the-art results in the OOD detection task.
INTRODUCTION
Modern neural networks have recently achieved superior results in classification problems (Krizhevsky et al. 2012 He et al., 2016). However, most of the classification algorithms proposed so far make the assumption that data generated from all the class conditional distributions are avail- able during training time i.e., they make the closed-wood world assumption. an open world environment (Bendale & Boult 2013). where examples from novel class distributions might appear during test time, it is necessary to build classifiers that are able to detect OOD examples while having high classification accuracy on known class distributions.
is generally known that deep neural networks can make predictions for out-of-distribution (OOD) examples with high confidence (Nguyen et al.) 2015). High confidence predictions undesirable since they consist a symptom of overfitting (Szegedy et al. (0015). They also make the calibration of neural networks difficult. Guuo al. (2017) 7) observed that modern neural networks miscalibrated by experimentally showing that the average confidence of deep neural networks is usually much higher than their accuracy.
A simple yet effective method to address the problem of the inability of neural networks to detect OOD examples is train them that they make highly uncertain predictions for examples gener- ated by novel class distributions. In order to achieve that, [ee et al. (2018a) defined a loss function based on the Kullback-Leible (KL) divergence metric minimize the distance between the output distribution given by softmax and the uniform distribution for samples generated by a GAN (Good- fellow al. 2014]. Using similar loss function, Hendrycks et (2019) showed that the technique of Outlier Exposure (OE) that draws anomalies from real and diverse dataset can outperform the GAN framework for OOD detection.
Using the OE technique, our main contribution is threefold:
We propose a novel loss function consisting of two regularization terms. The first regular- ization term minimizes the I norm between the output distribution given by softmax and the uniform distribution which constitutes a distance metric between the two distributions (Deza & Deza 2009). The second regularization term minimizes the Fuclidean distance

---

Under review as a conference paper at ICLR 2020
between the training accuracy of a DNN and its average confidence in its predictions on the training set.
We experimentally show that the proposed loss function outperforms the previous work of of[Hendyycks et al. (2019) and achieves state-of-the-art results in OOD detection with OE both on image and text classification tasks.
We experimentally show that our proposed method can be combined with the Mahalanobis distance- based classifier (Lee et al.) 2018b). The combination of the two methods outper- forms the original Mahalanobis method in all of the experiments and the best of our knowledge, achieves state-of-the-ar results in the OOD detection task.
2 RELATED WORK
Yu et al. ) used the GAN framework (Goodfellow et al.] 2014) to generate negative instances of seen classes by finding data points that are close to the training instances but are classified as fake by the discriminator. Then, they used those samples in order to train SVM classifiers to de- tect examples from unseen classes. Similarly, Kliger & Fleishman (2018) used a multi-class GAN framework in order to produce generator that generates mixture of nominal data and novel data and a discriminator that performs simultaneous classification and novelty detection.
Hendrycks & Gimpel (017)) proposed baseline for detecting misclassified and out-of-distibution examples based on their observation that the prediction probability of out-of-distribution examples tends to be lower than the prediction probability for correct examples. Recently, Corbière et al. (2019) also studied the problem of detecting overconfident incorrect predictions. A single-parameter variant of Platt scaling (Plati 1999), temperature scaling, was proposed by Guo et al. (2017) for cal- ibration of modern neural networks. For image data, based on the idea of Henndyyks & Gimpel (2017). Liang et al. (2018) observed that simultaneous use of temperature scaling and small pertur- bations at the input can push the softmax scores of in- and out-of-distribune from each other, making the diiti--dittiiuution images distinguishable. [Lee al. (2018a) generated GAN examples and forced the neural network to have lower confidence in predicting their classes. Hendrycks et al. (2019) substituted the GAN samples with a real and diverse dataset using the tech- nique of OE. Similar works (Malinin & Gales 2018 Bevandić al. 2018) also force the model to make uncertain predictions for OOD examples. Using an ensemble of classifiers, Lakshminaaaaaaaa it al. (2017) showed that their method was able to express higher uncertainty in OOD examples. Liu et al. 2018) provided theoretical guarantees for detecting OOD examples under the assumption that an upper bound of the fraction of OOD examples is available.
images further apart
Under the assumption that the pre-trained features of a softmax neural classifier can be fitted well by class-conditional Gaussian distribution, [Lee et al.) (2018b) defined confidence score using the Mahalanobis distance that can efficiently detect abnormal test samples. As also mentioned by Lee al. (2018b), Euclidean distance can also be used but with less efficiency. We prefer to call these methods Distance- Dassence-Based Post-Traing sast-rainnng (DBPT) methods for OOD detection.
3 SIMULTANHOUS CLASSIFICATION AND OUT-OF-DISTRIBUTTU DETECTION
We consider the multi-class classification problem under the open-world assumption (Bendale & Boult 2015). where samples from some classes are not available during training. Our task is to design deep neural network classifiers that can achieve high accuracy on examples generated by a learned probability distribution called D... while at the same time, they can effectively detect exam- ples generated by a different probability distribution called Dout during the test phase. The examples generated by Dr, are called in-distribution while the examples generated by Dout called out- of-distribution (OOD). Adopting the idea of Outlier Exposure (OE) proposed by [Hendrycks et al. (2019), we train the neural network using training examples sampled from D,n and D Dur- ing the test phase, we evaluate the OOD detection capability of the neural network using examples sampled from Drest, where DOE and Dtest are disjoint.
Lee et al. 8a) and Hendryeks et al. (2019) used the KL divergence metric in order to minimize the distance between the output distribution produced by softmax for the OOD examples and the
2

---

Under review as a conference paper at ICLR 2020
uniform distribution. In our work, choose minimize the I norm between the two distributions which has shown great success in machine learning applications.
Viewing the knowledge of model the class conditional distribution it produces over outputs given an input (Hinton et al., 2013), the entropy of this conditional distribution can used as a regularization method that penalizes confident predictions of neural network (Pereyra al. 2017). In our approach, instead of penalizing the confident predictions of posterior probabilities yielded by neural network, we force make predictions for examples generated by Din with an average confidence close to its training accuracy. In such a manner, not only do we make the neural network avoid making overconfident predictions, but we also take into consideration its calibration (Guo et al. 2017).
Let us consider classification model that can be represented by a parametrized function fe. where 0 stands for the vector of parameters in fe. Without loss of generality, assume that the cross entropy loss function is used during training. We propose the following constrained optimization problem for finding 0:
minimize 0
e
subject to
ExnDin
max kl..n
Air
ea1
K" (z)())
DOE
1..., Li=1 e²s
where Loe is the cross entropy loss function and K the number of classes available in Dan- Even though the constrained optimization problem (I) can be used for training various classification models, for clarity tation of the example (.( in the feature space produced by the last layer of the deep neural network (DNN) and let Aer be the training accuracy of the DNN. Observe that the optimization problem (I) minimizes the cross entropy loss function subject two additional constraints. The first constraint forces the average maximum prediction probabilities calculated by the softmax layer towards the training accuracy of the DNN for examples sampled from Din. while the second constraint forees the maximum probability calculated by the softmax layer towards K for all examples sampled from the probability distribution DOF. In other words, the first constraint makes the DNN predict exam- ples from known classes with an average confidence close to its training accuracy, while the second constraint forces the DNN be highly uncertain for examples of classes it has never before by producing a uniform distribution at the output for examples sampled from the probability distri- bution DOE It is also worth noting that the first constraint of (1) uses the training accuracy of the neural network Air which is not available in general. To handle this issue, one can train neural network by only minimizing the cross entropy loss function for few number of epochs in order to calculate Air and then fine-tune it using
limit our discussion to deep neural networks. Let denote the vector represen-
Because solving the nonconvex constrained optimization problem described by ① is extremely difficult, let us introduce Lagrange multipliers (Boyd & Vandenberghe 2004) and convert it into the following unconstrained optimization problem:
minimize Bres))-o..Coo((f(o)..a A Aer Atrr-Ban ExwDin 0
F1
max l=I,..,K
K
Σ
K j==
(2)
Σ
A2
max l=
K
Σj E
DOD
put
where is worth mentioning that in 2 we used only one Lagrange multiplier for the second of constraints ( instead of using one for each constraint order to avoid introducing large number of hyperparameters to our loss function. This modification is a special case where we consider the Lagrange multiplier Ag to be common for each individual constraint involving a different (1) DOE Note also that according to the original Lagrangian theory, one should
3

---

Under review as a conference paper at ICLR 2020
optimize the objective function of (2) both with respect to 0,Ar and A but it commonly happens in machine learning applications, we approximate the original problem by calculating appropriate values for X and 22 through validation technique (Hastie et al. (2011).
After converting the constrained optimization problem (T into an unconstrained optimization prob- lem as described by 2 it is possible that at each training epoch, the maximum prediction probability produced by softmax for each example drawn from D changes, introducing difficulties in mak- ing the DNN produce a uniform distribution the output for those examples. For instance, assume that we have a K-class classifier with K = 3 and at epoch i,, the maximum prediction probability produced by softmax for an example x²()) ~ Dour corresponds to the second class. Then, the last term of (2) will push the prediction probability of example xx(9) for the second class towards while concurrently increasing the prediction probabilities for either the first class or the third class or both. At the next epoch 12+1, it is possible that the prediction probability for either the first class or the third class becomes the maximum among the three and hence, the last term of (2) will push that one towards by possibly increasing again the prediction probability for the second class. It becomes obvious that this process introduces difficulties in making the DNN produce a uniform distribution at the output for examples sampled from Dou- However, this issue can be resolved by concurrently pushing all the prediction probabilities produced by the softmax layer for examples drawn from DoE towards K'
Additionally, in order to prevent the second and the third term of 2 from taking negative values during training, let us convert ( into the following:
2
e
minimize (.aw)=D...(Cer((foo A1 -RR-DD
!=I...,K K
max
K
e
1==
(3)
K
e
+ t
K
K
E
DOR out 1=1
j=1
The second term of the the loss function described by (3) minimizes the squared distance between the training accuracy of the DNN and the average confidence in its predictions for examples drawn from Din. Additionally, the third term of (3) minimizes the l norm between the uniform distribution and the distribution produced by the softmax layer for the examples drawn from DOE
While converting the unconstrained optimization problem 22 into ( one could use several combi- nations of norms to minimize. However, we found that minimizing the squared distance between the training accuracy of the DNN and the average confidence in its predictions for examples drawn from Dan and the I norm between the uniform distribution and the distribution produced by the softmax layer for the examples drawn from DOE works best. This is because l norm uniformly attracts all the prediction probabilities produced by softmax to the desired value 1. better contributing to producing a uniform distribution at the output of the DNN for the examples drawn from DOE On the other hand, minimizing the squared distance between the training accuracy of the DNN and the average confidence in its predictions for examples drawn from Din emphasizes more on attracting the maximum softmax probabilities that are further away from the average confidence of the DNN, making the neural network better detect in- and out-of-distribution examples at the low softmax probability levels.
4 EXPERIMENTS
During the experiments, we observed that we start training the DNN with a relatively high value of >ı, the learning process might slow down since constantly force the neural network to make predictions with an average confidence close to its training accuracy. Therefore, is recommended to split the training of the algorithm into two stages where in the first stage, train the DNN using only the cross entropy loss function until reaches the desired level of accuracy Ayr and then using fixed Acr, we fine-tune it using the combined loss function given by 3
4

---

Under review as a conference paper at ICLR 2020
4.1 COMPARISON WITH STATE- OFATE-OF-THE-AR OE
The experimental setting is as follows. We draw samples from Din and we train the DNN until reaches the desired level of accuracy Agr. Then, drawing samples from D we fine-tune it using the combined loss function given by (3). During the test phase, we evaluate the OOD detection capability of the DNN using examples from Des which is disjoint from D We demonstrate the effectiveness of our method in both image and text classification tasks by comparing it with the previous OOD detection with OE method proposed by Henndrykk et al. (2019). A part of our experiments was based the publicly available code of Hendrycks al. (2019).
4.1.1 EVALUATION METRICS
Our OOD detection method belongs to the class of Maximum Softmax Probability (MSP) detectors (Hendrycks & Gimpel 2017) and therefore, we adopt the evaluation metrics used in Hendrycks ct al. (2019). Defining the OOD examples the positive class and the in-distribution examples as the negative class, the performance metrics associated with OOD detection are the following:
False Positive Rate at N% True Positive Rate (FPRN): This performance metric (Baintas et al. 2016; Kumar et al., (016) measures the capability of an OOD detector when the maximum soft- max probability threshold is set to predefined value. More specifically, assuming N% of OOD examples need to be detected during the test phase, we calculate a threshold the softmax prob- ability space and given that threshold, distribution examples that are incorrectly classified as OOD.
measure the false positive rate, i.e. the ratio of in-
Area Under the Receiver Operating Characteristic curve (AUROC): In the out-of-distribue de- tection task, the ROC curve (Davis & Goadrich, 2006) summarizes the performance of an OOD detection method for varying threshold values.
Area Under the Precision-Recall curve (AUPR): The AUPR (Manning & Schütze 1999) is an important measure when there exists class dlas--imbalance between OOD and in-distribution amples in a dataset. As in Hendrycks et al.] (2019), in our experiments, the ratio of OOD and in-distribution test examples is 1:5.
4.1.2 IMAGE CLASSIFICATION EXPERIMENTS
Results. The results of the image classification experiments shown in Table [ In Figure I as an example, we plot the histogram of softmax probabilities using CIFAR-10 as Din and Places365 as Diest. The detailed description of the image datasets
diar10 places365
Number of acamotes
8000
used in the image OOD detection experiments is presented in Ap-
pendix[A.]
400
Network Architecture and Training Details. Similar to Hendrycks et al. (019)), for CIFAR and CIFAR 100 experi- ments, we used 40-2 wide residual networks (WRNs) proposed by Zagoruyko & Komodakis (2016). We initially trained the WRN for 100 epochs using cosine learning rate (Loshchilov & Hutter 2017) with an initial value 0.1, a dropout rate of 0.3 and a batch size of 128. As in Hendryykk et al. (2019), we also used Nes- terov momentum and 12 weight regularization with a decay fac- tor of 0.0005. For CIFAR 10, we fine-tuned the network for 15
09
pmitaiitiess
Sumber of exampla.
800
dfar10 places365
820
400
epochs minimizing the loss function given by (3) using a learning rate of 0.001, while for the CIFAR 100 the corresponding number of epochs was 20. For the SVHN experiments, we trained 16-4 WRNs using a learning rate of 0.01, dropout rate of 0.4 and a batch size of 128. We then fine-tuned the network for 5 epochs using a learning rate of 0.001. During fine-tuning, the 80 Million Tiny Images dataset was used as DOE The values of the hyperpa- rameters A1 and A2 were chosen in the range [0.03. 0.09] using a separate validation dataset Deel similar to Hendryccks al. (2019). We note that Drat and Deest are disjoint. "The data used for vali- dation are presented in Appendix A.3
0.4 0.
3.0
Figure 1: Histogram of soft- max probabilities with CIFAR- 10 as Din and Places365 as out (1,000 samples from each dataset). Top: MSP base- line detector. Bottom: MSP de- tector fine-tuned with 3
5

---

Under review as a conference paper at ICLR 2020
FPR95
AUROC1
AUPR
Din
+OE
OURS
+OE
OURS
+OE
OURS
SVHN
0.10
0.03
99.98
99.99
99.83
99.55
CIFAR-10
9.50
6.56
97.81
98.40
90.48
93.08
CIFAR-100 100
38.50
28.89
87.89
91.80
58.15
71.50
Table I: Image OOD example detection for the maximum softmax probability (MSP) baseline de- tector after fine-tuning with OF (Hendrycks et al.] 2019) versus fine-tuning with our proposed loss function given by 3 All results percentages and averaged over 10 and over OOD datasets. Detailed experimental results are in Appendix A.1
Contribution of each regularization term. To demonstrate the effect of each regularization term of the loss function described by 3 in the OOD detection task, classification experiments which are presented in Table 2 For these experiments, we incrementally added each regularization term to the loss function described by (3) and we measured its effect both in the OOD detection evaluation metrics as well as in the accuracy of the DNN on the test images of D Din- The results of these experiments validate that the combination of the two regularization terms of (3) not only improves the OOD detection performance of the DNN but also improves its accuracy on the test examples of Din compared to the case where A = 0. Table 2 also demonstrates that our method can significantly improve the OOD detection performance of the DNN compared to the case where only the cross coss-eyy loss is minimized at the expense only an insignificant degradation in the test accuracy of the DNN examples generated by Din-
some additional image
Din
A1
A2
FPR95
AUROCT
AUPR
Test Accuracy
34.94
89.27
59.16
94.65
CIFAR-10
8.87
96.72
77.65
92.72
/
<
6.56
98.40
93.08
93.86
62.66
73.11
30.05
75.73
CIFAR-100
/
26.75
91.59
68.27
71.29
28.89
91.80
71.50
73.14
Table Contribution of each regularization term of (3) on the OOD detection performance and the test accuracy of the DNN. Results are averaged over 10 runs and over OOD datasets.
4.1.3 TEXT CLASSIFICATION EXPERIMENTS
Results.
The results of the text classification experiments are shown Table 3 The detailed
description of the text datasets used in the NLP OOD detection experiments is presented in Ap- pendix B.I
FPR90
AUROC1
AUPR
Din
+OE
OURS
+OB
OURS
+0E
OURS
20 Newsgroups
4.86
0.63
97.71
99.18
91.91
97.02
TREC
0.78
0.75
99.28
99.32
97.64
97.52
SST 27.33
17.91
89.27
93.79
59.23
74.10
Table 3: NLP OOD example detection for the maximum softmax probability (MSP) baseline de- tector after fine-tuning with OE (Hendrycks al. 2019) versus fine-tuning with our proposed loss function given by 33). All results are percentages and averaged over 10 runs and over 10 OOD datasets. Detailed experimental results are in Appendix B.3
Network Architecture and Training Details. For all text classification experiments, similar to Hendrycks al. (2019), train 2-layer GRUs (Cho al. 2014) for epochs with learning rate 0.01 and a batch size of 64 and then we tee-tune them for epochs using the loss function given by During fine-tuning, the WikiText-2 dataset was used as DOE The values of the hyperparameters

---

Under review as a conference paper at ICLR 2020
A1 and λ were chosen in the range (0.04, 0.1] using separate validation dataset as described in Appendix B.2
4.2
COMBINATION OF OE AND DBPT METHOSSS FOR OOD DETECTION
Lee al. (018b) proposed DBPT method for OOD detection that be applied to any pre- trained softmax neural classifier. Under the assumption that the pre-trained features of DNN can be fitted well by class-conditional Gaussian distribution, they defined the confidence score using the Mahalanobis distance with respect to the closest class- conditsondii probability distribution, where its parameters are chosen as empirical class means and tied empirical covariance of training samples (Lee et al. 2018b). To further distinguish in- and out-of out-of-distribution examples, they proposed two additional techniques. In the first technique, they added a small perturbation before processing each input example to increase the confidence score of their method. In the second technique, they proposed feature ensemble method in order to obtain better calibrated score. The feature ensemble method extracts all the hidden features of the DNN and computes their empirical class mean and tied covariances. Subsequently, it calculates the Mahalanobis Distance-basee confidence score for each layer and finally calculates the weighted average of these scores by training logistic regression detector using validation samples in order to calculate the weight of each layer at the final confidence score.
Since the Mahalanobis distance-based classifier proposed by Lee et al. (0188) is a post-training method, it can be combined with our proposed loss function described by ( More specifically, in our experiments, we initially trained a DNN using the standard cross entropy loss function and then we fine-tuned it with the proposed loss function given by (3). After fine-tuning, we applied the Mabalanobis distance- based classifier and we compared the obtained results against the results presented in Lee et al. (2018b). The simulation experiments on image classification tasks show that the combination of our method which belongs to the OE "family" of methods and the Mahalanobis distance- based classifier which belongs the "family" of DBPT methods achieves state-of-the-an -frt results in the OOD detection task. A part of our experiments was based on the publicly available code of Lee et al. 2018b).
4.2.1 EVALUATION METRICS
To demonstrate the adaptability of our method, in these experiments, we adopt the OOD detection evaluation metrics used in Lee et al. (0188).
True Negative Rate at N% True Positive Rate (TNRN): This performance metric measures the capability of an OOD detector to detect true negative examples when the true positive rate is set to 95%.
In the out-of- distribbutioon de- tection task, the ROC curve (Davis & Goadrich, 2006) summarizes the performance of an OOD detection method for varying threshold values.
Area Under the Receiver Operating Characteristic curve (AUROC):
Detection Accuracy (DAcc): As also mentioned in Lee al. (0188)) this evaluation metric corresponds to the maximum classification probability over all possible thresholds e:
1 minn sn (q(x) < (xx is from Din) + Dout(g(x)) > P( is from Dout)},
where q(æ) is confidence score. P( is from Dmn) = P(x is from Dow).
Similar to Lee (2018)), we assume that
4.2.2 EXPERIMENTAL SETUP
To demonstrate the adaptability and the effectiveness of our method, we adopt the experimental setup Le al, (2018) 8b). We train ResNet (He et al. (2166) with 34 layers using CIFAR- 10, CIFAR-100 and SVHN datasets as Din- For the CIFAR experiments, SVHN, TinyImageNet (a sample 10,000 images drawn from the ImageNet dataset) and LSUN are used Diest For the SVHN experiments, 10, TinylmageNet and L.SUN are used as Diest Both TinyImageNet and LSUN images are downsampled to 32 32.
Similar to Lee et al. (0188), for the Mahalanobis distance-based classifier, we train the ResNet model for 200 epochs with batch size 128 by minimizing the cross entropy loss using the SGD
7

---

Under review as a conference paper at ICLR 2020
TNR95
AUROCT
DAcef
D.n.
Diest
Mahal.
OURS Mahal.
Mahal.
OURS+Mahal.
Mahal.
OURS+Vahal.
SVHN
96.4
97.3
99.1
99.2
95.8
96.3
CIFAR-I0
TinylmageNet
97.1
98.8
99.5
99.6
96.3
97.3
LSUN
98.9
99.7
99.7
99.8
97.7
98.5
SVHN
91.9
93.0
98.4
98.7
93.7
94.2
CIFAR-100
TityImageNet
90.9
92.3
98.2
98.3
93.3
93.9
LSUN
90.9
95.6
98.2
98.6
93.5
95.4
CIFAR-TO
98.4
99,9
99.9
96.9
99.2
SVHN
TinyImageNet
99.9
100.0
100.0
99.1
99.9
LSUN
99.9
100.0
99.9
100.0
99.5
100.0
Table 4: Comparison between the Mahalanobis distance- based classifier (Lee et al. 2018b) and the combination of our proposed method with the Mahalanobis method. The hyper-parameterr are tuned using a validation dataset of in- and out-of-distributtin data similar to Lee et al. (2018b). Additional training details for our method are presented in Appendix
algorithm with momentum 0.9. The learning rate starts at 0.1 and is dropped by a factor of 10 at 50% and 75% of the training progress, respectively. Subsequently, we compute the Mahalanobis distance- based confidence score using both the input pre-processing and the feature ensemble techniques. The syper-parameter
that need to be tuned are the magnitude of the noise added at each test input example as well as the layer indexes for feature ensemble. Similar to Lee et al. (2018b), both of them are tuned using separate validation dataset consisting of both in- and out-o--distributiin data.
Since the Mahalanobis distance-based classifier belongs to the "family" of DBPT methods for OOD detection tasks, it can be combined with proposed method. More specifically. we initially train the ResNet model with 34 layers for 200 epochs using exactly the same training details as mentioned above. Subsequently, we fine-tune the network with the proposed loss function described by (3) using the 80 Million Tiny Images as DOE- During fine-tur theettunngg we use the SGD algorithm with momentum 0.9 and cosine learning rate (Loshchilov & Hutter] (017) with initial value 0.001 using a batch size of 128 for data sampled from Din and a batch size of 256 for data sampled from DOE. For CIFAR-IO and 100 experiments, we fine-tuned the network for 30 and 20 epochs respectively, while for SVHN the corresponding number of epochs was 5. The values of the hyper- parameters A and X2 were chosen using a separate validation dataset consisting of both in- and out-of-distribution images similar to
Lee et al. (20T8b). The results are shown in Table 4
Discussion. The results in Table 4 demonstrate the effectiveness of our method when combined with the Mahalanobis distance-based classifier since it outperforms the original version of the Ma- halanobis method proposed by [ee et al. (2018b) in all of the experiments. This result validates the contribution of our technique further, since it does not only achieve state-of-the results in OOD detection with OE, but it can be additionally combined with DBPT methods like the Maha- lanobis distance-based classifier to achieve state-of-the-aar results in the OOD detection task. The superior performance of our method when combined with the Mahalanobis distance-basee classifier can be justified by the fact that the latter extracts the learned features from the layer(s) of the DNN and it subsequently uses those features to define confidence score based on the Mahalanobis dis- tance. The simulation results presented in Table and Table sshwed that our method can teach the DNN learn feature representations that can further distinguish in- and out-of distribution data and therefore, the combination of the two methods improves the OOD detection capability of a DNN.
and
5 CONCLUSION
In this paper, we proposed a method for simultancous classification and out-of-distribttion detection. The proposed loss function includes two regularization terms where the first minimizes the norm between the output distribution of the softmax layer of DNN and the uniform distribution, while the second minimizes the Euclidean distance between the training accuracy of a DNN and its average confidence in its predictions on the training set. Experimental results showed that the proposed loss function achieves state-of-the-ar results in OOD detection with OE (Hendrycks et al. 2019) in both image and text classification tasks. Additionally. we experimentally showed that our method can be combined with DBPT methods for OOD detection like the Mahalanobis distance- classifier (Lee al. 2018b) and achieves state-of-the-arr results in the OOD detection task.
8

---

Under review as a conference paper at ICLR 2020
REFERENCES
Vassilcios Baintas, Edgar Riba, Daniel Ponsa, and Krystian Mikolajczyk. Learning local feature descriptors with triplets and shallow convolutional neural networks. In British Machine Vision Conference (BMVC), 2016.
Abhijit Bendale and Terrance Boult. Towards open world recognition. JEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2015.
Petra Bevandić, Ivan Krešo, Marin Oršić, and Siniša Segvić. Discriminative out-of-distribution detection for semantic segmentation. arXiv preprint arXiv: 1808.07703, 2018.
Samuel R. Bowman, Gabor Angeli, Christopher Potts, and Christopher Manning. A large anno- tated corpus for learning natural language inference. In Proceedings of the 2015 Conference Empirical Methods Natural Language Processing (EMNLP), 2015.
Stephen Boyd and Lieven Vandenberghe. Convex Optimization. Cambridge University Press, 2004.
Kyunghyun Cho, Bart van Merrienboer, Çaglar Gülçehre, Dzmitry Bahdanau, Fethi Bougares, Hol- ger Schwenk, and Yoshua Bengio. Learning phrase representations using rnn encoder-decoder for statistical machine translation. In Empirical Methods in Natural Language Processing (EMNLP), 2014.
Mircea Cimpoi, Subhransu Maji, lasonas Kokkinos, Sammy Mohamed, and Andrea Vedaldi. De- scribing textures in the wild. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2014.
Charles Corbière, Nicolas Thome, Avner Bar-Hen, Matthieu Cord, and Patrick Pérez. Addressing Failure Prediction by Learning Model Confidence. arXiv preprint arXiv: 1910.04851, 2019.
Jesse Davis and Mark Goadrich. The relationship between precision-recall and roc curves. In International Conference on Machine Learning (ICML). 2006.
Michel Marie Deza and Elena Deza. Encyclopedia of Distances. Springer Berlin Heidelberg, 2009.
Desmond Elliott, Stella Frank, Khalil Sima' an, and Lucia Specia. Multi30K: Multilingual English- German Image Descriptions. arXiv preprint arXiv: 1605.00459 2016.
Ian J. Goodfellow, Jean Pouget-Abadie, Mchdi Mirza, Bing Xu, David Warde-Farles Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial nets. International Conference Neural Information Processing Systems (NIPS), pp. 2672-2680, 2014.
Chuan Guo, Geoff Pleiss, Yu Sun, and Kilian Q. Weinberger. On calibration of modern neural networks. In International Conference Machine Learning (ICML), 2017.
Trevor Hastie, Robert Tibshirani, and Jerome Friedman. The Elements of Statistical Learning. Springer. NY, USA, 2001.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In IEEE Conference Computer Vision and Pattern Recognition (CVPR), 2016.
Dan Hendrycks and Thomas G. Dietterich. Benchmarking Neural Network Robustness Common Corruptions and Surface Variations. arXiv preprint arXiv: 1807.018 2018
Dan Hendrycks and Kevin Gimpel. A baseline for detecting misclassified and -ittr--disttibttionn examples in neural networks. International Conference on Learning Representations (ICLR), 2017.
Dan Hendrycks, Mantas Mazeika, and Thomas Dietterich. Deep anomaly detection with outlier exposure. International Conference on Learning Representations (ICLR), 2019.
Geoffrey Hinton, Oriol Vinyals, and Jeff Dean. Distilling the knowledge neural network. arXiv preprint 1503.0/503.020 2015.

---

Under review as a conference paper at ICLR 2020
Mark Kliger and Shachar Fleishman. Novelty Detection with GAN. arXiv preprint arXiv: 1802.10560, 2018.
Alex Krizhevsky and Geoffrey Hinton. Learning multiple layers of features from tiny images. Tech- nical report, University of Toronto, 2009.
Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. Imagenet classification with deep convo- lutional neural networks. In Advances in Neural Information Processing Systems 25, pp. 1097- 1105. 2012.
Vijay Kumar, Gustavo Carneiro, and Ian Reid. Learning Local Image Descriptors with Deep Siamese and Triplet Convolutional Networks by Minimising Global Loss Functions. In JEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016.
Balaji Lakshminarayanan, Alexander Pritzel, and Charles Blundell. Simple and scalable predictive uncertainty estimation using deep ensembles. In International Conference on Neural Information Processing Systems, 2017.
Kimin Lee, Honglak Lee, Kibok Lee, and Jinwoo Shin. Training Confidence-calibrace Classifiers for Detecting Dut-off-Dstrrbuuc Samples. International Conference on Learning Representa- lions (ICLR), 2018a.
Kimin Lee, Kibok Lee, Honglak Lee, and Jinwoo Shin. A simple unified framework for detect- ing out-of-distribution samples and adversarial attacks. In International Conference on Neural Information Processing Systems, 2018b.
Shiyu Liang, Yixuan Li, and R. Srikant. Enhancing The Reliability Out-of-diisributioo Image Detection in Neural Networks. International Conference Learning Representations (ICLR), 2018.
Si Liu, Risheek Garrepalli, Thomas G. Dietterich, Alan Fern, and Dan Hendrycks. Open Category Detection with PAC Guarantees. arXiv preprint arXiv: 1808,00529, 2018.
I. Loshchilov and F. Hutter. Sgdr: Stochastic gradient descent with warm restarts, In International Conference on Learning Representations (ICLR), 2017.
Andrey Malinin and Mark Gales. Predictive uncertainty estimation via prior networks. In Neural Information Processing Systems, 2018.
Christopher D. Manning and Hinrich Schütze. Foundations of Statistical Natural Language Pro- cessing. MIT Press, 1999.
Yuval Netzer, Tao Wang, Adam Coates, Alessandro Bissacco, Bo Wu, and Andrew Ng. Reading digits in natural images with unsupervised feature learning. In NIPS Workshop on Deep Learning and Unsupervised Feature Learning 201 1, 2011.
Anh Mai Nguyen, Jason Yosinski, and Jeff Clune. Deep neural networks are easily fooled: High confidence predictions for unrecognizable images. In IEEE Conference Computer Vision and Pattern Recognition (CVPR), pp. 427-436, 2015.
Gabriel Pereyra, George Tucker, Jan Chorowski, Lukasz Kaiser, and Geoffrey E. Hinton. Regular- izing neural networks by penalizing confident output distributions. International Conferece on Learming Representations (ICLR), 2017.
Platt. Probabilistic outputs for support vector machines and comparison to regularized likelihood methods. In Advances in Large Margin Classifiers, 1999.
Richard Socher, Alex Perelygin, Jean Y. Wu, Jason Chuang, Christopher D. Manning, Andrew Y. Ng, and Christopher Potts. Recursive deep models for semantic compositionality over sentiment treebank. In Conference on empirical methods in natural language processing (EMNLP), 2013.
Christian Szegedy, Vincent Vanhoucke, Sergey loffe, Jonathon Shlens, and Zbigniew Wojna. Re- thinking the inception architecture for computer vision. arXiv preprint arXiv: 1512.00567, 2015.
10

---

Under review as a conference paper at ICLR 2020
Antonio Torralba, Rob Fergus, and William T. Freeman. 80 million tiny images: A large data set for nonparametric object and scene recognition. JEEE Transactions on Pattern Analysis and Machine Intelligence, 30(11), 2008.
Fisher Yu, Yinda Zhang, Shuran Song, Ari Seff, and Jianxiong Xiao. Lsun: Construction of large-scale image dataset using deep learning with humans in the loop. arXiv preprint arXiv:1506.00335. 2015.
Yang Yu, Wei-Yang Qu Nan Li, and Zimin Guo. Open-Category Classification by Adversarial Sample Generation. arXiv preprint arXiv: 1705.08722, 2017.
Sergey Zagoruyko and Nikos Komodakis. Wide residual networks. In British Machine Vision Conference (BMVC), 2016.
B. Zhou, A. Lapedriza, A. Khosla, A. Oliva, and A. Torralba. Places: A 10 million image database for scene recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 40(06); 1452-1464, 2018.

---

Under review as a conference paper at ICLR 2020
A EXPANDED IMAGE OOD DETECTION RESULTS AND DATASETS USED FOR COMPARISON WITH STATE-OF-THE-AA IN OE
A.1
IMAGE OOD DETECTION RESULTS
FPR95
AUROC1
AUPR
Din
Diest
+OE
OURS
+OE
OURS
+OE
OURS
out
Gaussian
0.0
0.0
100.
100.
100.
99.4
Bernulli
0.0
0.0
100.
100.
100.
99.2
Blobs
0.0
0.0
100.
100.
100.
99.6
SVHN
Icons-50
0.3
0.1
99.8
99.9
99.2
99.5
Textures
0,2
0.1
100.
100.
99.7
99.6
Places365
0.1
0.0
100.
100.
99.9
99.7
LSUN
0.1
0.0
100.
100.
99.9
99.7
CIFAR-10
0.1
0.0
100.
100.
99.9
99.7
Mean
0.10
0.03
99.98
99.99
99.83
99.55
Gaussian
0.7
0.7
99.6
99.8
94.3
99.0
Rademacher
0.5
1.1
99.8
99.6
97.4
97.6
Blobs
0.6
1.5
99.8
99.1
98.9
91.7
CIFAR-10
Textures
12.2
4.0
97.7
98.9
91.0
95.0
SVHN
4.8
1.4
98.4
99.6
89.4
97.9
Places365
17.3
13.3
96.2
96.9
87.3
89.5
LSUN
12.1
6.7
97.6
98.4
89.4
91.9
CIFAR-106
100
28.0
23.8
93.3
94.9
76.2
82.0
Mean
9.50
6.56
97.81
98.40
90.48
93.08
Gaussian
12.1
0.7
95.7
99.7
71.1
97.2
Rademacher
17.1
0.7
93.0
99.7
56.9
96.2
CIFAR 100
Blobs
12.1
1.3
97.2
99.6
86.2
96.3
Textures
54.4
50.1
84.8
87.8
56.3
61.5
SVHN
42.9
16.7
86.9
94.9
52.9
74.1
Places365
49.8
47.8
86.5
88.1
57.9
58.5
LSUN
57.5
56.6
83,4
85.9
51.4
53.0
CIFAR-10
62.1
57.2
75.7
78.7
32.6
35.2
Mean
38.50
28.89
87.89
91.80
58.15
71.50
Table 5: Image OOD example detection for the maximum softmax probability (MSP) baseline de-
tector after fine-tuning with OE (Hendrycks al.) (019)) versus fine-tuning with proposed loss function given by (3). All results are percentages and averaged over 10 runs. Values are rounded to the first decimal digit.
A.2 Din, DOE AND Diesst FOR IMAGE EXPERIMENTS
SVHN: The Street View House Number (SVHN) dataset (Netzer al. (011) consists of 32 32 color images out of which 604,388 are used for training and 26,032 are used for testing. The dataset has 10 classes and
collected from real Google Street View images. Similar to Hendrycks et al.
2019), we rescale the pixels of the images to be in [0, 1].
CIFAR 10: This dataset (Krizhevsky & Hinton 2009) contains 10 classes and consists of 60,000 32 32 color images of which 50,000 belong the training and 10,000 belong the test set.
Before training, we standardize the images per channel similar to Hendrycks et al. (2019).
CIFAR 100: This dataset (Krizhevsky & Hinton 2009) consists of 20 distinct superclasses each of which contains 5 different classes giving
total of 100 classes. The total number of images in the dataset are 60,000 and we use the standard 50, 50,000/10,000 train/test split. Before training, we standardize the images per channel similar Heendyykks al. (2019).
80 Million Tiny Images: The 80 Million Tiny Images dataset (Torralb et al. 2008 exclusively used in our experiments in order to represent D consists of 32 x 32 color images collected from the Internet. Similar to Hendrycks al. (2019), in order make sure that DOE and Dteat
12

---

Under review as a conference paper at ICLR 2020
disjoint, removed all the images of the dataset that appear CIFAR 10 and CIFAR 100 datasets. Places365: Places365 dataset introduced by Zhou al. (2018) was exclusively used in our experi- ments in order to represent Dour. It consists of millions of photographs of scenes.
Gaussian:
synthetic image dataset created by i.id. sampling from an isotropic Gaussian distri-
bution.
Bernoulli: A synthetic image dataset created by sampling from Bernoulli distribution.
Blobs: A synthetic dataset of images with definite edges.
Icons-50: This dataset intoduced by Hendryykk & Dietterich (2018) consists of 10,000 images belonging to 50 classes icons. As part of preprocessing, we removed the class "Number" order to make it disjoint from the SVHN dataset.
Textures: This dataset contains 5,640 textural images (Cimpoi et al.
2014
LSUN: It consists of around million large-scale images of scenes Yu et al. 2015).
Rademacher: A synthetic image dataset created by sampling from a symmetric Rademacher distri- bution.
A.3 VALIDATION DATA FOR IMAGE EXPERIMENTS
Uniform Noise: A synthetic image dataset where each pixel sampled from u(00, 1] 1[-1, depending on the input space of the classifier.
Arithmetic Mean: A synthetic image dataset created by randomly sampling a pair of in-distribution images and subsequently taking their pixelwise arithmetic mean.
Geometric Mean: A synthetic image dataset created by randomly sampling pair of in-distribution images and subsequently taking their pixelwise geometric mean.
Jigsaw: A synthetic image dataset created by partitioning an image sampled from Dan into 16 equally sized patches and by subsequently permuting those patches.
Speckle Noised: A synthetic image dataset created by applying speckle noise to images sampled from Din-
Inverted Images: A synthetic image dataset created by shifting and reordering the color channels of images sampled from Din-
RGB Ghosted: A synthetic image dataset created by inverting the color channels of images sampled from Din-
B EXPANDED TEXT OOD DETECTION RESULTS AND DATASETS USED FOR COMPARISON WITH STATE-OF-THE-ARY IN OE
B.1 Din. DOE AND Drest FOR NLP EXPERIMENTS
This dataset contains 20 different newsgroups, each corresponding to a specific topic. It contains around 19,000 examples and we used the standard 60/40 train/test split. TREC: A question classification dataset containing around 6,000 examples from 50 different classes. Similar to Hendrrycks al. (2019), we used 500 examples for the test phase and the rest for training.
20 Newsgroups:
SST: The Stanford Sentiment Treebank (Socher et al. 2013 is binary classification dataset for sentiment prediction of movie reviews containing around 10,000 examples.
WikiText-22: This dataset contains over million articles from Wikipedia and is exclusively used as DOE our experiments. We used the same preprocessing as Hendrycks et al. (2019) in order to have a valid comparison.
SNLI: The Stanford Natural Language Inference (SNLI) corpus is collection of 570,000 human-
written English sentence pairs (Bowman et al.) 2013).
IMDB: A sentiment classification dataset containing movies reviews.
Multi30K: A dataset of English and German deseriptions of images (Elliott et 2016). For our experiments, only the English descriptions were used.
WMT16: A dataset used for machine translation tasks. For our experiments, only the English part of the test set was used.
Yelp: A dataset containing reviews of users for businesses Yelp.
13

---

Under review as a conference paper at ICLR 2020
EWT: The English Web Treebank (EWT) consists of different datasets: weblogs (EWTT-W W), news- groups (EWT-N), emails (EWT-E), reviews (EWT-R) and questions-answers (EWT-A).
B.2 VALIDATION DATA FOR NLP EXPERIMENTS
The validation dataset Deat used for the NLP OOD detection experiments was constructed fol- lows. For each Din dataset used, we used the rest two in n--disribution datasets as D For instance, during the experiments where 20 Newsgroups represented Din. we used TREC and SST as D making sure that out and Deeat are disjoint.
B.3 TEXT OOD DETECTION RESULTS
FPR90
AUROCT
AUPR
Din.
Drest
+OE
OURS
+OE
OURS
+OE
OURS
SNLI
12.5
2.1
95.1
97.1
86.3
93.0
IMDB
18.6
2.5
93.5
98.2
74.5
92.9
20 Newssrroups
Multi30K
3.2
0.1
97.3
99.4
93.7
98.6
WMT16
2.0
0.2
98.8
99.8
96.1
99.4
Yelp
3.9
0.4
97.8
99.6
87.9
97.9
EWT-A
1.2
0.2
99.2
99.8
97.3
98.4
EWT-E
1.4
0.1
99.2
99.9
97.2
98.9
EWT-N
1.8
0.5
98.7
99.2
95.7
94.5
EWT-R
1.7
0.1
98.9
99.4
96.6
98.3
EWT-W
2.4
0.1
98.5
99.4
93.8
98.3
Mean
4.86
0.63
97.71
99.18
91.91
97.02
SNLI
4.2
0.8
98.1
99.1
91.6
94.9
IMDB
0.6
0.6
99.4
98.9
97.8
97.1
Multi3KK
0.3
0.2
99.7
99.9
99.0
99.6
TREC
WMT16
0.2
0.2
99.8
99.9
99.4
99.6
Yelp
0.4
0.8
99.7
99.1
96.1
92.9
EWT-A
0.9
4.0
97.7
98.0
96.1
95.6
EWT-E
0.4
0.3
99.5
99.2
99.1
98.1
EWT-N
0.3
0.2
99.6
99.9
99.2
99.6
EWT-R
0.4
0.2
99.5
99.6
98.8
98.9
EWT-W
0.2
0.2
99.7
99.6
99.4
98.9
Mean
0.78
0.75
99.28
99.32
97.64
97.52
SNLI
33.4
7.4
86.8
95.8
$2.0
76.4
IMDB
32.6
10.8
85.9
95.8
51.5
77.6
Multi30K
33.0
5.1
88.3
97.9
58.9
86.9
SST
WMT16
17.1
3.6
92.9
98.3
68.8
88.1
Yelp
15.6
92.7
95.2
60.0
81.1
EWT-A
33.6
21.4
87.2
92.7
53.8
70.8
EWT-E
26.5
22,6
90.4
92.4
63.7
67.7
EWT-N
27.2
19.2
90.1
93.6
62.0
67.4
EWT-R
41.4
36.7
85.6
88.1
54.7
62.5
EWT-W
17.2
36.7
92.8
88.1
66.9
62.5
Mean
27.33
17.91
89.27
93.79
59.23
74.10
Table 6: NLP OOD example detection for the maximum softmax probability (MSP) baseline de-
tector after fine-tuning with OE (Hendrycks et al. 2019) versus fine-tuning with our proposed loss function given by (3). All results are percentages and the result of 10 runs. Values rounded to the first decimal digit.
14

---

Under review as a conference paper at ICLR 2020
C
TRAINING DETAILS FOR THE EXPERIMENTAL RESULTS FOR COMPARISON WITH MAHALANOBIS DISTANCE- -BASDD CLASSIFIER
During fine-tuning with proposed loss function given by (3 used the training details pre- sented in Table 7 The values of the hyper-parameters A and X2 were chosen using a separate validation dataset consisting of both in- and out-of-distribttion images similar to Lee al. (018b).
Dm
A
Ag
Epochs
Test Accuracy(D;n)
CIFAR-I0 10
0.15
0.15
30
94.60
CIFAR-IO0 100
0.09
0.07
75.73
SVHN
0.07
0.03
5
96.87
Table Additional training details for the experimental results in Table
15


Reviews:
[{"review_id": "rklj1r55Yr", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 1, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This work proposes a new loss function to train the network with Outlier Exposure(OE) [1] which leads to better OOD detection compared to simple loss function that uses KL divergence as the regularizer for OOD detection. The new loss function is the cross entropy plus two more regularizers which are : 1) Average ECE (Expected Calibration Error) function to calibrate the model and 2) absolute difference of the network output to $1/K$ where $K$ is the number of tasks. The second regularizer keeps the softmax output of the network uniform for the OE samples. They show adding these new regularizers to the cross-entropy loss function will improve the Out-distribution detection capability of networks more than OE method proposed in [1] and the baseline proposed in [2].\n\n\nPros:\nThe paper is written clearly and the motivation of designed loss functions are explained well.\n\nCons:\n1- The level of contributions is limited.  \n\n2- The variety of comparison is not enough. The authors did not show how the approach is working in compared to the other OOD methods like ODIN[3] and the proposed method in [4].\n\n3- The experiments are not supporting the idea. First, the paper claims that the KL is not a good regularizer for OOD detection as it is not a distance metric. But there is no experiment or justification in the paper that supports why this claim is true. Then the second contribution claims that the calibration term that is added to the loss function improves the OOD detection as well as calibration in the network, but the experiments are not designed to show the impact of  each regularizer term separately in improving the OOD detection rate.  Figure 2 also does not depict any significant conclusion. It only shows that the new loss function makes the network more calibrated than the naive network. This phenomenon was reported before in [1]. It would be better if the paper investigated the relation between the calibration and OOD detection by designing more specific experiments for calibration section.   \n\nOverall, I think the paper should be rejected as the contributions are limited and are not aligned with the experiments.  \n\nReferences\n[1]Hendrycks, Dan, Mantas Mazeika, and Thomas G. Dietterich. \"Deep Anomaly Detection with Outlier Exposure.\" arXiv preprint arXiv:1812.04606 (2018).\n\n[2] A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networks, ICLR2016.\n\n[3] Liang, Shiyu, Yixuan Li, and Rayadurgam Srikant. \"Enhancing the reliability of out-of-distribution image detection in neural networks.\" arXiv preprint arXiv:1706.02690 (2017).\n\n[4] Lee, Kimin, et al. \"A simple unified framework for detecting out-of-distribution samples and adversarial attacks.\" Advances in Neural Information Processing Systems. 2018.", "extra_scores": {"experience_assessment": "I have read many papers in this area.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I did not assess the derivations or theory.", "review_assessment:_checking_correctness_of_experiments": "I carefully checked the experiments.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper at least twice and used my best judgement in assessing the paper."}, "extra_text": {}}, {"review_id": "SkeRf5OTYS", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The paper considers the problem of out-of-distribution detection in the context of image and text classification with deep neural networks. The proposed method is based on [1], where cross-entropy between a uniform distribution and the predictive distribution is maximized on the out-of-distribution data during training. The authors propose two simple modifications to the objective. The first modification is to replace cross-entropy between predictive and uniform distributions with an l1-norm. The second modification is to add a separate loss term that encourages the average confidence on training data to be close to the training accuracy. The authors show that these modifications improve results compared to [1] on image and text classification.\n\nThere are a few concerns I have for this paper. The main issue is that I am not sure if the level of novelty is sufficient for ICLR. The contributions of the paper consist of a new loss term and a modification of the other loss term in OE [1]. At the same time, the paper achieves an improvement over OE consistently on all the considered problems. Given the limited novelty, experiments aimed at understanding the proposed modification would strengthen the paper. Right now I am leaning towards rejecting the paper, but if authors add more insight into why the proposed modifications help, or provide a strong rebuttal, I may update my score.\n\nI discuss the other less general concerns below.\n1. I believe the presentation of the method in Section 3 is suboptimal. The authors start with presenting a constrained minimization problem, then convert it to a problem with Lagrange multipliers, then modify the problem in an ad hoc way (adding a square and a norm without any mathematical reason), to get the standard form of loss with regularizers. The presentation would be much cleaner, and wouldn’t lose anything if the authors directly presented the loss with regularizers. Furthermore, in the Lagrange multiplier view the Lagrange multipliers are not hyper-parameters, they are dual variables, and the optimization problem shouldn’t be just with respect to theta, we need to find a stationary point with respect to both lambda and theta.\n\n2. The motivation for changing the distance measure between the predictive distribution on outlier data and the uniform distribution is unclear. The authors state multiple times that KL is not a distance metric, but it isn’t clear why this is important. KL is commonly used as a measure of distance between distributions. One could also use symmetrized KL in order to get a distance metric that is similar to KL. I am not opposed to just using l1-norm because it performs better, but if the switch of distance measures is listed as one of the two main methodological contributions, I believe more insight needs to be provided for why it helps.\n\n3. The motivation for the other loss term which is enforcing calibration of uncertainty on train data is also not very clear. At least in image classification, strong networks typically achieve perfect accuracy (or close to that) on the train set, and then the proposed loss term would basically push the predictive confidence on all training data to 1, which is already enforced by the standard cross-entropy loss. Does outlier exposure prevent the classifier from getting close to 100% accuracy on train? What is the train accuracy for the experiments on CIFAR-10, CIFAR-100 and SVHN?\n\n4.  While the authors report accuracy of out-of-distribution detection in the experiments,  they don’t report the accuracy of the actual classifier on in-distribution data. Is this accuracy similar for the proposed method and OE? Is the accuracy also similar for the proposed method and baseline for the experiment in section 4.4?\n\n5. The method is only being compared to the OE method of [1]. Why is this comparison important, and are there other methods that the authors could compare against? \n\n[1] Deep Anomaly Detection with Outlier Exposure. Dan Hendrycks, Mantas Mazeika, Thomas Dietterich", "extra_scores": {"experience_assessment": "I have read many papers in this area.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper at least twice and used my best judgement in assessing the paper.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_checking_correctness_of_derivations_and_theory": "N/A"}, "extra_text": {}}, {"review_id": "S1xPO7UtFS", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper proposes to tackle the problems of out-of-distribution (OOD) detection and model calibration by adapting the loss function of the Outlier Exposure (OE) technique [1]. In OE, model softmax outputs are encouraged to be uniform for OOD samples, which is enforced through the use of a KL divergence loss function. The first proposed modification in this paper is to replace KL divergence term with an L1 penalty. The second change is the addition of an L2 penalty between the maximum softmax probability and the model accuracy. Experimental results demonstrate that adding these two components increases performance over OE on standard OOD benchmarks for both vision and text domains, and also improves model calibration.\n\nAlthough this paper presents some good quantitative results, I tend towards rejection in its current state. This is mainly due to the limited comparison to alternative methods, and the lack of ablation study. If these were addressed I would consider increasing my score.\n\nThings to improve the paper:\n1) Currently, one of the most commonly used benchmark methods for OOD detection is the Mahalanobis distance based confidence score (MD) [2], which, as far as I am aware, is state-of-the-art among published works. The authors claim that they do not compare to this work because it is a post-training method, and, presumably, the techniques should be doubly effective when combined. However, we do not have any proof that this is actually the case. Therefore, I think it is important to verify that the two techniques are indeed compatible, and if not, then direct comparison with MD would still be necessary.\n\n2) In the case of confidence calibration, there is no comparison made with other calibration techniques, such as temperature scaling [3]. I think it would be good to included these for reference.\n\n3) Since two distinct components are being added to the loss function, I think it is important to include an ablation study to identify how much each component contributes to improvements in OOD detection and confidence calibration.\n\n\nMinor things to improve the paper that did not impact the score:\n4) With regards to the confidence calibration loss, there is similar work by [4] which also optimizes the output of the model to make sure confidence predictions are close to the true accuracy. It may be worth citing if you think it is relevant.\n\n\nAdditional questions:\n5) How are lambda1 and lambda2 tuned? I could not find this information in the paper.\n\n6) How sensitive is model performance to the setting of the lambda hyperparameters? It would be nice to see a plot of lambda versus the OOD detection metrics.\n\n7) Have you evaluated how this method performs at detecting adversarial attacks? I do not think the paper will suffer without these results, but they are certainly relevant and of interest to practitioners in this area.\n\n\nReferences:\n[1] Hendrycks, Dan, Mantas Mazeika, and Thomas G. Dietterich. \"Deep anomaly detection with outlier exposure.\" ICLR (2018).\n\n[2] Lee, Kimin, Kibok Lee, Honglak Lee, and Jinwoo Shin. \"A simple unified framework for detecting out-of-distribution samples and adversarial attacks.\" NeurIPS, (2018).\n\n[3] Guo, Chuan, Geoff Pleiss, Yu Sun, and Kilian Q. Weinberger. \"On calibration of modern neural networks.\" In Proceedings of the 34th International Conference on Machine Learning-Volume 70, pp. 1321-1330. JMLR. org, 2017.\n\n[4] Corbière, Charles, Nicolas Thome, Avner Bar-Hen, Matthieu Cord, and Patrick Pérez. \"Addressing Failure Prediction by Learning Model Confidence.\" NeurIPS (2019).\n\n\n### Post-Rebuttal Comments ###\nI would like to thank the authors for their hard work during the rebuttal period. I think the current version of the paper is much improved over the previous version. The choice to remove the claims about calibration definitely improves the focus of the paper. The addition of the  Mahalanobis distance experiments and the ablation study also significantly strengthen the paper. However, as the other reviewers have pointed out, the novelty of the paper is quite limited since the majority of the gains come from simply swapping the KL-divergence penalty with an L1 penalty. Despite simplicity, this single change yields a significant improvement in performance for the OE algorithm, which is noteworthy. As a result, I will increase my score from a weak reject (3) to a very, very weak accept (more like a 5 than a 6). ", "extra_scores": {"experience_assessment": "I have published one or two papers in this area.", "review_assessment:_checking_correctness_of_experiments": "I carefully checked the experiments.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper at least twice and used my best judgement in assessing the paper.", "review_assessment:_checking_correctness_of_derivations_and_theory": "N/A"}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper proposes a method for out-of-distribution (OOD) detection for neural network classifiers.

The reviewers raised several concerns about novelty, choice of baselines and the experimental evaluation. While the author rebuttal addressed some of these concerns, I think the paper is still not ready for acceptance as is. 

I encourage the authors to revise the paper and resubmit to a different venue.
Decision: Reject