Out-of-Distribution Detection Using Layerwise Uncertainty in Deep Neural Networks
Authors: Hirono Okamoto
Masahiro Suzuki
Yutaka Matsuo
Venue:  (2020)
Keywords: out-of-distribution
uncertainty

Abstract: In this paper, we tackle the problem of detecting samples that are not drawn from the training distribution, i.e., out-of-distribution (OOD) samples, in classification. Many previous studies have attempted to solve this problem by regarding samples with low classification confidence as OOD examples using deep neural networks (DNNs). However, on difficult datasets or models with low classification ability, these methods incorrectly regard in-distribution samples close to the decision boundary as OOD samples. This problem arises because their approaches use only the features close to the output layer and disregard the uncertainty of the features. Therefore, we propose a method that extracts the uncertainties of features in each layer of DNNs using a reparameterization trick and combines them. In experiments, our method outperforms the existing methods by a large margin, achieving state-of-the-art detection performance on several datasets and classification models. For example, our method increases the AUROC score of prior work (83.8%) to 99.8% in DenseNet on the CIFAR-100 and Tiny-ImageNet datasets.

Under review as a conference paper at ICLR 2020
OUT-OF-DISTTRBUTIOO DETECTION USING LAYER- WISE UNCERTAINTY IN DEEP NEURAL NETWORKS
Anonymous authors Paper under double-blind review
ABSTRACT
In this paper, tackle the problem of detecting samples that are not drawn from the training distribution, i.e., out-of-distribution (OOD) samples, in classification. Many previous studies have attempted to solve this problem by regarding samples with low classification confidence as OOD examples using deep neural networks (DNNs). However, on difficult datasets or models with low classification abil- ity. these methods incorrectly regard in n--disriibttion samples close to the decision boundary as OOD samples. This problem arises because their approaches use only the features close to the output layer and disregard the uncertainty of the features. Therefore, we propose a method that extracts the uncertainties features in each layer of DNNs using reparameterization trick and combines them. In experi- ments, our method outperforms the existing methods by a large margin, achieving state-of-te ofate-of-the-att detection performance several datasets and classification mod- els. For example, our method increases the AUROC score of prior work (83.8%) 99.8% in DenseNet on the CIFAR-100 and Tiny-ImageNet datasets.
INTRODUCTION
Deep neural networks (DNNs) have achieved high performance in many classification tasks such as image classification (Krizhevsky et al. 2012, Simonyan & Zisserman 2014). object detection (Lin et al. 2017: Redmon & Farhadi] 2018 and speech recognition (Hinton et al. 2012 Hannun et al. 2014 However, DNNs tend make high confidence predictions even for samples that are not drawn from the training distribution, i.e., out-of-distribution (OOD) samples (Hendrycks & Gimpel, 2016). Such errors can be harmful to medical diagnosis and automated driving. Because it is not generally possible to control the test data distribution in real-world applications, OOD samples are inevitably included in this distribution. Therefore, detecting OOD samples is important for ensuring the safety of artificial intelligence system (Amodei al. 2016).
There have been many previous studies (Hendrycks & Gimpel 2016 Liang et al. 2017 Lee et al. 2017 Devies & Taylor 2018 Lee et [018|]0088]] Hendrycks et al. (0188) that have attempted to solve this problem by regarding samples that are difficult to classify or samples with low classification confidence OOD examples using DNNs. Their approaches work well and they are computation- ally efficient. The limitation of these studies is that, when using difficult datasets or models with low classification ability, the confidence of inputs will be low, even if the inputs are in-distribution samples. Therefore, these methods incorrectly regard such in-distribution samples as OOD samples, which results their poor detection performance (Malinin & Gales 2018). as shown in Figure 1]
One cause of the abovementioned problem is that their approaches use only the features close to the output layer and the features are strongly related the classification accuracy. Therefore, we use not only the features close to the output layer but also the features close to the input layer. We hypothesize that the uncertainties of the features close to the input layer are the uncertainties of the feature extraction and are effective for detecting OOD samples. For example, when using convolutional neural networks (CNNs), the filters of the convolutional layer close to the input layer extract features such edges that useful for in-distribution classification. In other words, in- distribution samples possess more features that convolutional filters react to than OOD samples. Therefore, the uncertainties of the features will be larger when the inputs in-distribution samples.
Another cause of the abovementioned problem is that their approaches disregard the uncertainty of the features close to the output layer. We hypothesize that the uncertainties of the latent features close

---

Under review as a conference paper at ICLR 2020
Baseline (Hendrycks & Gimpel 2016)
UFEL (ours)
22
35-
00D
20-
degree of uncertainty
0.7
0.6
0
max softmax probability
Figure I: Comparison of existing and proposed methods. We visualized scatter plots of the outputs of the penultimate layer of CNN that can estimate the uncertainties of latent features using the SVHN dataset (Netzer et al. 2011). We used only classes 0, I, and 2 for the training data. Classes 0, 1, 2, and OOD, indicated by red, yellow, blue, and black, respectively, dation data. We plot the contour of the maximum output of the softmax layer of the model. Left: Because the image of "204" includes the digits "2" and "0," the maximum value of the softmax output decreases because the model does not know to which class the image belongs. Right: The sizes of points in the seatter plots indicate the value of the combined uncertainties of features. We can classify the image of "204" as an in-distribution image according to the value of the combined uncertainties.
used for the vali-
to the output layer are the uncertainties of classification and are also effective for detecting OOD samples. For example, in-distribution samples embedded in the feature space close to the output layer to classify samples. In contrast, OOD samples have no fixed regions for embedding. Therefore, the uncertainties of the features of OOD samples will be larger than those of in-distribution samples.
Based on the hypotheses, we propose a method that extracts the Uncertainties of Features in Each Layer (UFEL) and combines them for detecting OOD samples. Each uncertainty is easily estimated after training the discriminative model by computing the mean and the variance of their features using a reparameterization trick such as the variational autoencoder (Kingma & Welling, 2013) and variational information bottleneck (Alemi et al.) 2016 2018). Our proposal is agnostic to the model architecture and can be easily combined with any regular architecture with minimum modifications.
We visualize the maximum values of output probability and the combined uncertainties of the latent features in the feature space of the penultimate layer in Figure I] The combined uncertainties of the features discriminate the in in-distribution and OOD images that are difficult to classify. For example, although the images that are surrounded by the red line are in-distribution samples, they have low maximum softmax probabilities and could be regarded as OOD samples in prior work. Meanwhile, their uncertainties are smaller than those of OOD samples and they are regarded as in-distribution samples in our method.
In experiments, we validate the hypothesis demonstrating that each uncertainty effective for de- tecting OOD examples. We also demonstrate that UFEL can obtain state-of-the-a performance in several datasets including CIFAR-100, which difficult to classify, and models including LeNet5 with low classification ability. Moreover, UFEL is robust to hyperparameters such as the number of in-distribution classes and the validation dataset.
RELATED WORK
Methods based on the classification confidence Hendrycks & Gimpel (2016) proposed the base-
line method to detect OOD samples without the need to further re-train and change the structure of the model. They define low-maximum softmax probabilities as indicating the low confidence of
in-distribution examples and detect OOD samples using the softmax outputs of pre-trained deep
classifier. Building on this work, many models have recently been proposed. [iang et al. (2017) proposed ODIN, calibration technique that uses temperature scaling (Guo al. 2017 in the
2

---

Under review as a conference paper at ICLR 2020
f1
f 42
f 43
f $
Conv Block
Conv Block
FC Block
4r
Softmax
X
z
2
Figure 2: Network structure of UFEL when using DenseNet. Black arrow: Extracting the variance of latent features using the reparameterization trick. Blue arrow: Combining these features.
softmax function and adds small controlled perturbations to the inputs widen the gap between in- distribution and OOD features, which improves the performance of the baseline method. Likewise, Lee et al. (2018); Vries & Taylor/ (018); et al. (2017); Hendrycks et al. (2018) also extended the baseline method. Like Hendrycks & Gimpel (2016), we use the feature of maximum softmax probability as one of our features.
Methods based on the uncertainty Malinin & Gales (0018) attempted to solve the problem of classifying in-distribution samples close to the decision boundary as OOD samples by distinguishing between data uncertainty and distributional uncertainry. Data uncertainty, aleatoric uncertainty [Keddall & Gal 2017). is irreducible uncertainty such as class overlap. whereas distributional un- certainty arises because of the mismatch between training and testing distributions. They argue that the value distributional uncertainty depends on the difference in the Dirichlet distribution of the categorical parameter. Further, they estimate the parameter of the Dirichlet distribution using DNN and train the model with in-distribution and OOD datasets. The motivation for our work is similar that of Malinin & Gales (2018). our work, the distribution of the logit of the categorical pa- rameters is modeled as a Gaussian distribution, which enables us to train the model without an OOD dataset. Furthermore, we estimate the parameters of the Gaussian distribution of latent features close to the input layer.
3 PROPOSED METHOD
this section, we present UFEL, which extracts the uncertainties features each layer and combines them for detecting OOD samples. First, in Hendrycks & Gimpel (2016), as one our features. Second, we also use the distribution of the categorical parameter, in Malinin & Gales (2018), using the uncertainty of logits. Furthermore, we use the uncertainty of the feature extraction extracted from the latent space close to the input layer because they will not be relevant the classification accuracy. We probabilistically model the values of these features, estimate their uncertainties, and combine them.
the maximum of the softmax output, as
Let X' be an input, y E y {1,.... , be its label, and (1,-..,,4 L} be the index of the block layers. The objective function of normal deep classification is as follows:
J Ex.yr-((..)) ( (f( y)],
(I)
where p(r, y) is the empirical data distribution, L is cross entropy loss function, and fo a DNN. We the following notation So før foz .... So, as shown in Figure
To extract the uncertainties of features in each layer, we model the /th block layer's output 2 Gaussian whose parameters depend on the (-1th block layer's output z!-1 as follows: p(z1|z/-1)- (22',, (2'-1),
(. f (a/-1)). where fo, the ith block layer, which outputs both the mean μ and covariance matrix Σ. this paper, we use a diagonal covariance matrix reduce the model param- eters. We use the reparameterization trick (Kingma & Welling 2013) to write z' = p¹ - a'e, where n' = f (2/-1), σi =f (x (z1-1), and e is the Gaussian noise. Then, our objective function as
follows:
Tu Bey.ppleehhec
(2)
3

---

Under review as a conference paper at ICLR 2020
where 20 Because of the reparameterization trick, the loss gradient is backpropagated directly through our model, and we can train our model like the regular classification models in Equation
Next, explain the two methods of combining the features extracted in each layer. In the first method, we sum the uncertainties of each value of the features in cach layer and linearly combine them. Because the feature maps of convolutional block layer are three dimensional, each element computed as z/yx = Myys + ijk.C Moreover, because the output of a fully connected layer is one dimensional, each element is formed of nu + ofe. We use weighted summation of the scale of each feature and the maximum value the softmax scores as final feature d.n as follows:
Eije Lagk E
(if the /th block is Cony)
(LR(Z) Σ No (x) A max((=)), d 1=1
(otherwise)
We choose the parameter X' by training logistic regression (LR) using in-distribution and OOD validation samples. In the second method, we combine the features directly and nonlinearly using CNN as follows:
(x) = (NN (x). m)))
We train the CNN parameter 9 with in-distribution and OOD validation samples using binary crossentropy loss. The detailed structures of the CNN are given in Table A.3 We use the values of these feature d(xx) to test the performance of detecting OOD samples.
EXPERIMENTAL SETUP
In this section, we present the details of the experiments, which includes the datasets, metrics, com- parison methods, and models. Because of space limitations, more details are given in Appendix A
Datasets used several standard datasets for detecting OOD samples and classifying in- distribution samples. The SVHN, CIFAR-IO. 10, and CIFAR-100 datasets were used as in-distribution datasets, whereas Tiny ImageNet (TIM), LSUN, iSUN, Gaussian noise, and uniform noise were used OOD datasets. These data were also used in Liang al. (2017); Veies & Taylor (2018). We applicd standard augmentation (cropping and flipping) in all experiments. We used 5,000 validation images split from each training dataset and chose the parameter that obtain the best accuracy in the validation dataset. We also used 68,257 training images from the SVHN dataset and 45,000 training images from the CIFAR-I0 and CIFAR-100 datasets. All the hyperparameters of ODIN and UFEL were tuned on a separate validation set, which consists of 100 OOD images from the test dataset and 1,000 images from the in-distribution validation set. We tuned the parameters of the CNN in Equation 4 Jusing 50 validation training images taken from the 100 validation images. The best parameters were chosen by validating the performance using the rest of 50 validation images. Finally, we tested the models with a test dataset that consisted of 10,000 in-distribution images and 9,900 OOD images.
Evaluation metrics and the classification of in-distribution samples. We used TNR at 95% TPR, AUROC, AUPR, and accuracy (ACC), which were also used in Lee al. (2017 2018
We used several standard metrics for testing the detection of OOD samples
Comparison method We compare UFEL with the baseline (Hendrycks & Gimpel 2016) and ODIN (Liang al. 2017) methods. For the baseline method, we used max; p(y k|x) as the detec- tion metric. For ODIN, we used the same detection metric and calibrated it using temperature scaling and small perturbations the input. The temperature parameter T E {1, 10, 100, 1000) and the per- turbation parameter E {0, 0.001, 0.005, 0.01, 0.05, 0.1} were chosen using the in-distribution and OOD validation datasets.
Model training details We adopted LeNet5 (LeCun et al. 1998 and two state-of-the-art models, WideResNet (He et al. 2016) and DenseNet (Huang et al. 2017). in this experiment. In all experi- ments, used the same model and conditions compare UFEL with existing methods. Only the structure used to extract the variance parameters differs. For LeNet5, we increased the number of channels of the original LeNet5 improve accuracy. See Table A.3 for model details. We inserted
4

---

Under review as a conference paper at ICLR 2020
Table 1: Results for the OOD detection test data for image classification when the in-distribution
dataset is CIFAR-100 and the OOD dataset is iSUN. We trained Dense-Be BC for each method under
the same conditions. All results are averaged over three runs one standard deviation. All values
are percentages, and the best results are indicated in bold.
Method
ACC
TNR at TPR 95%
AUROC
AUPR-In
AUPR-Out
Baseline
71.1
10.5
61.723.4
63.4=6.5
57.1
ODIN
43.4
85.1+5.77
86.6-5.8
82.9-6.00
max P
71.610.4
12.3E1.7
63.914.1
67.8=4.9
59.2#2.9
var 1
79.7+3.8
93.9+0099
92.8=0.8
94.7+00.9
var 2
35.2++0.0
79.4+2.9
79.8=2.0
77.5+4.6
var 3
14.7
90.8+2.5
92.4-2.1
87.413.4
UFEL (LR)
93.1
98.610.1
98.7=0.1
98.5+0.1
UFEL (CNN)
99.3
99.8+0.1
99.8=0.1
99.
the reparamcterization trick to the second convolutional layer and the softmax layer. LeNet5 was trained using the Adam (Kingma & Bal 2014) optimizer for 10 epochs and the learning rate was set to 5e-4. Both DenseNet and WideResNet were trained using stochastic gradient descent, with a Nes- terov momentum of 0.9. We inserted the reparameterization trick to the first convolutional block, used WideResNet
the second convolutional block, and the softmax layer. For WideResNet, with a depth of 40 and width of 4 (WRN-40-4), which was trained for 50 epochs. The learning rate was initialized 0.1 and reduced by a factor of 10x after the 40th epoch. For DenseNet, we used a DenseNet with depth L = 100 (Dense-BC), growth rate of 12, and drop rate of O. DenseNet-BC was trained for 200 epochs with batches of 64 images, and weight decay of le-4 for the CIFAR-10 I0 and CIFAR-100 datasets. It was trained for 10 epochs for the SVHN dataset. The learning rate was initialized 0.1 and reduced by factor of 10x after the 150th epoch.
5 EXPERIMENTAL RESULTS
In this section, demonstrate the performance of UFEL by conducting five experiments. In the first experiment, we show that UFEL performs better than the baseline (Hendrycks & Gimpel| (016)) and ODIN (Liang et al. 2017) methods on several datasets and models. In the second experiment, we confirm that the features of UFEL have almost no relationship with the ACC. In the third experiment, we demonstrate that UFEL has a strong ability detect OOD data, even if the number of classes of in-distribution data is small. In the fourth experiment, we confirm that UFEL is robust to the number of OOD samples, and in the fifth experiment, we test the performance of UFEL on unseen OOD datasets. The objective of these experiments is to show the uncertainties of the features obtained in each CNN layer distinguish the in-distribution and OOD data. Moreover, we obtain state-of-the- performance for OOD sample detection by combining these features.
Detecting OOD samples on several datasets and models In this experiment, we evaluate the performance of OOD detection using Equation 3 and [Equation 4 In this study, var I used to denote of Puma max is max(="). UFEL (LR) denotes di in Equation and UFEL (CNN) denotes dONN in Equation 4] We measured the detection performance using DenseNet trained on CIFAR- 100 when the iSUN dataset is used to provide the OOD images. Table I shows that var I and var are strong features that, by themselves, can outperform ODIN. This indicates that the uncertainties of the feature extraction and classification are effective for detecting OOD samples. Moreover, the combination of these features yields sfate-of-the-art performance.
In Table 2, we demonstrate that UFEL outperforms the baseline and ODIN methods on several datasets and models. Furthermore, UFEL is also slightly superior to them with respect to in- distribution accuracy, which indicates that our model is robust to noise because of the reparame- terization trick. Here, we do not report ODIN accuracy because the model of ODIN is the same as that of the baseline. We conducted this experiment three times and used the average of the results. We used the CIFAR-10, CIFAR-100, and SVHN datasets the in-distribution datasets and the other datasets as the OOD samples. Note that our UFEL outperformed the baseline and ODIN methods by a large margin, especially when using CIFAR-100, which is difficult classify, or LeNet5 which
5

---

Under review as a conference paper at ICLR 2020
Table 2: Results for the OOD detection test data for various situations, All results averaged over three runs, All values are percentages. and the best results are indicated in bold.
In-dist (model)
OOD
TNR at TPR 95%
AUROC
Baseline / UFEL (ACC)
Baseline / ODIN UFEL (LR) / UFEL (CNN)
TIM
43.8 68.1 95.7 99.1
89.7 93.0 / 98.8 99.8
SVHN
LSUN
42.3 66.0 / 98.4 99.8
89.2 /9.7 99.6 99.9
(LeNet5) 90.7 / 91.2
iSUN
46.8 / 72.0 97.7 99.3
90.6 94.1 / 19.4 99.8
Gaussian
99.9 /100.0/110.0/110000
99.9 100000/100.0/1000
Uniform
98.7 /5.000000/10000//0000.
99.6 99.6/99.9 /10..0/10000
TIM
72.4784.7799 49.499999999
95.8797.17 99.87999999 99.9
SVHN (Dense-BC) 95.8 / 95.9
LSUN
67.6 80.5 99.9 9 99.9
25.0/9644 1999 /100.0
iSUN
7 99.8 99.9
95.9 97.3 99.9 /100.0
Gaussian
/ /2.4 99.8 /100.0
97.2 /9.3/9..// / 19.77 /100.0
Uniform
90.2/97.7
/100.0/100.0
98.2 99.1 19.99 /100.0
TIM
7.57 8.6 /92.3
58.8/59.77 83.7 97.7
CIFAR-10 (LeNet5) 72.6 / 73.1
LSUN
5.71 6.4 62.6 98.0
56.9 58.4 90.9 19.5
iSUN
6.1/ 7.2 53.9 /95.7
57.2 / 86.0 98.99
Gaussian
80.3 48.3 /100.0
93.3 / /9.6 89.9 /100.0
Uniform
48.7
88.5 /100.0
94.9/94.4/ 97.5 /100.0
TIM
25.8 /93.2 99.4
84.6/95.2 98.5 99.88
CIFAR-10 (WRN-40-4) 88.8 / 89.6
LSUN
/ 96.9 /100.0
87.5 / 19.3 /100.0
iSUN
30.3 94.1 99.9
87.1 /97.0/ 98.6 /100.0
Gaussian
36.2/90.1 /100.0/100.0
90.4 / /9.2//8///// 99.9 /100.0
Uniform
58.0 96.2 /100.0/100.0
94.7 99.0 / 99.8 /100.0
TIM
40.0785.00 90.67 96.4
90.5797.4 98.1 99.2
CIFAR-I0 (Dense-BC)
LSUN
/ 93.9 99.4
92.2 / 98.5 / 98.8 99.8
iSUN
46.4 91.22 93.6 99.7
92.1 / 19.4 98.8/ 99.9
92.1/92.4
Gaussian
37.2 97.3 /100.0/100.0
92.6 / 99.2 /100.0/100.0
Uniform
53.8 /100.0/100.0
94.4 / 99.7 /10.00//0//
TIM
1.67 16/514/ 88.7
45.9/ 01577855.8 96.4
CIFAR-100
LSUN
0.6/ 0.6 64.6 97.3
42.0 42.0 92.0 99.3
(LeNet5) 38.8 39.6
iSUN
0.97 56.9 96.2
43.3 / /1.3/333//// 88.1 98.9
Gaussian
0.0% 0.0/ /100.0
42.6/ 47.9/ /9.11 /100.0
Uniform
0.2/ 99.2 /100.0
50.2 50.2/56.5/ 97.6 /100.0
TIM
9.0/ 88.6/ 99.1
63.2 81.5 97.6/ 99.8
CIFAR-100
LSUN
9.4/38.1 / /94.2/ 99.9
65.2 65.2/85.2/ 99.7 /100.0
(WRN-40-4) 63.9 / 66.0
iSUN
8.4/32.9 89.9 99.6
63.8 / 83.0 97.8 99.9
Gaussian
6.8/ /100.0/100.0
60.2 89.0/ 99.6 /100.0
Uniform
4.8/34.1 92.6 /100.0
63.5 87.0 97.7 /100.0
TIM
11.5 87.9 98.9
61.5 83.8 97.3 99.8
CIFAR-100 (Dense-BC) 71.1 /7.6
LSUN
11.3 46.// 95.5 99.9
62.4 86.2 99.0 /100.0
iSUN
10.5 43.4 93.1 99.3
61.7 85.1 99866 99.8
Gaussian
23.6 / 99.0 /100.0/100.0
84.7 / 99.4 /100.0/10000
Uniform
32.7 98.0 /100.0/100.0
89.5 / 99.3 /100.0/100.0
has lower classification ability. This result indicates that UFEL is less affected by in-distribution accuracy, and can discriminate between actual OOD samples and class-overlapping samples from in-distribution samples.
Relationship between performance of detecting OOD samples and in-distribution accuracy In this experiment, we show that the features of our method are not related to the in-distribution accuracy. We used CIFAR-10 as the nn-distribution dataset and TIM as the OOD dataset. We trained DenseNet-BC for nine epochs and tested the performance at each epoch. As shown in Figure each variance (var () less related to the accuracy than the baseline and ODIN methods. The var 1 of the feature close the input layer has the highest ability to detect OOD samples in this experiment.

---

Under review as a conference paper at ICLR 2020
Baseline
var
VBT
AUROC 0op
Accuracy on In in-distribution dataset (CIFARIO)
Figure 3: Plot of ACC (x-axis) and AUROC (y-axis). The number on the plot indicates the number of training epochs. We used CIFAR-10 (as in-distribution), TIM (as OOD), and the DenseNet-BC model. This graph shows that the AUROC of UFEL is less related to ACC than those of the baseline and ODIN.
Out of distributin: TIM
Out of distribution: LSUN
Out of distr bution ISUN
1.30
353
0.90
0.00
AUROC
0.83
Baseline
0.80
0.00
max P
373
0.70
0.70
UFEL (UR)
6.0-
the number of classes
number SVIIN classes
the number of SVNN classes
Figure 4: Plot of AUROC (y-axis) when changing the number of in-distribution dataset classes (x- axis). We used SVHN as in-distribution dataset, TIM, LSUN, and iSUN as OOD datasets, and the LeNet5 model. All plots were averaged over three runs and the error bar indicates one standard deviation.
These results also indicate that we can discriminate in-distribution and OOD examples when using dataset that is difficult to classify.
Detecting OOD samples while changing the number of in-distribution classes In this periment, we show that UFEL is robust to the number of class labels. We used SVHN as in- distribution dataset and changed the number of in-distribution classes in training as {0.1). (0.1.2).... (0.1.2.....)). We also used TIM, LSUN and iSUN datasets as OOD samples, and LeNet5 as a model. We compared the proposed method with the baseline and ODIN methods, as shown in Figure 4 This graph shows the AUROC score of each model when changing the number of training data classes. As this graph shows. UFEL outperforms other methods all cases and is robust to the number of in-distribution classes, whereas the performance of ODIN drops as the number of class labels decreases. These results suggest that UFEL is effective for small datasets because the number of samples can be decreased to one fifth of the original number when there are two in-distribution classes and the cost of label annotation is reduced.
Detecting OOD samples while changing the number of OOD samples this experiment, we present the performance of UFEL while changing the number of OOD validation examples. All the hyperparameters of ODIN and UFEL were tuned on a separate validation set, which consists of 30, 50, and 100 OOD images in the test dataset and 1,000 images from the in-distribution validation set. As shown Figure 5] although UFEL (CNN) outperforms other methods including UFEL (LR) in most cases, performs worse than ODIN in part of the results because some tuning for OOD samples is needed. Meanwhile, UFEL (LR) outperforms prior methods constantly because the number of hyperparameters is small and tuning samples are almost unneeded.
7

---

Under review as a conference paper at ICLR 2020
number validation
number O0D valiation examples:
validation
OTARL2R 2000 20 CrAttitbree craigneemeetee CTFCC C700033000000
Baselin
L00
ODIN
0.00
UFEL (CNN)
0.93
1-0
100
0.-
31-
14-
1003
9325
1.5
00
08
Figure 5: Plot of AUROC (y-axis) when changing the OOD dataset (x-axis). We used CIFAR-10 and CIFAR-100 as the in-distribution dataset. All plots are averaged over three runs and the error bar indicates one standard deviation.
Generalization to unseen OOD dataset Because OOD validation samples might not be available
in practice, used only uniform noise as the validation OOD dataset and tested the ability of model to detect another OOD dataset. We added binary classification as a comparison method. This method was trained using an in-distribution dataset (positive) and uniform noise (negative). Table 3) shows that UFEL outperforms prior work in all cases and generalize well. Table 3 also indicates that the binary classification method does not generalize well because cannot distin- guish in-distribution dataset and OOD datasets TIM, LSUN, and iSUN, although it can distinguish Gaussian noise, which is similar to uniform noise.
Table 3: AUROC score for OOD detection test set data. All results are averaged over three runs. All values are percentages and the best results are indicated in bold.
AUROC
WRN-40-4
Dense-BC
In-dist
OOD
Binary Baseline ODIN
UFEL (LR) (CNN)
CIFAR-I0
TIM
4 84.6 94.6 98.0 98.7
71.2 90.50 96.6 97.8
LSUN
/ 96.7 98.5 99.3
75.1 02119922//9955 97.1 97.9
iSUN
/2.1/87.1 96.6 98.0 99.3
75.4/92.1/9744 / 98.4
Gaussian
100.0/ 90.4 98.2 99.9 /100.0
100.0/ 92.6 99.2 99.9 /100.0
CIFAR-100
TIM
46.3
63.2 63.3//3./// / 97.1 98.1
67.0 / /4.55/77.//9 94.1 99.4
LSUN
4 65.2 / 85.2 / 98.0 98.8
73.4 / 62.4 79.3 /94.5 94.7
iSUN
46.9 / 63.8 / 83.0 97.5 98.7
70.0 / 61.7 78.2 94.7 95.1
Gaussian
100.0/ 60.2 89.0 99.4 /100.0
100.0/ 84.7 / 99.3 /100.0/100.0
6 CONCLUSION AND FUTURE WORK
In this paper, we demonstrated that the uncertainties of features extracted in each hidden layer are important for detecting OOD samples. We combined these uncertainties to obtain state-of-the-art OOD detection performance on several models and datasets. The approach proposed this paper has the potential to increase the safety of many classification systems by improving their ability to detect OOD samples. In future work, our model could be used in an unsupervised model by training it to minimize reconstruction error, which would avoid the need to use in-distribution labels to detect OOD samples. Furthermore, although we compared our model with ODIN, UFEL will perform better if combine UFEL with ODIN because they orthogonal methods.
8

---

Under review as a conference paper at ICLR 2020
REFERENCES
Alexander A Alemi, lan Fischer, Joshua Dillon, and Kevin Murphy. Deep variational information bottleneck. arXiv preprint arXiv:1612.00410, 2016.
Alexander A Alemi, lan Fischer, and Joshua V Dillon. Uncertainty in the variational information bottleneck. arXiv preprint arXiv: 00906, 2018.
Dario Amodei, Chris Olah, Jacob Steinhardt, Paul Christiano, John Schulman, and Dan Mané. Con- crete problems in ai safety. arXiv preprint arXiv: 16ddv 1606.0666, 2016.
Jesse Davis and Mark Goadrich. The relationship between precision-recall and roc curves. In Proceedings of the 23rd international conference on Machine learning, pp. 233-2400 ACM, 2006.
Jia Deng. Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hi- erarchical image database. In 2009 JEEE conference on computer vision and pattern recognition, pp. 248-255, Icee, 2009.
Terrance DeVries and Graham W Taylor. Learning confidence for out-of-distribution detection in neural networks. arXiv preprint arXiv: 048660 2018.
Chuan Guo, Geoff Pleiss, Yu Sun, and Kilian Q Weinberger. On calibration of modern neural networks. In Proceedings of the 34th International Conference on Machine Learning-Volume 70, pp. 1321-1330. JMLR. org, 2017.
Awni Hannun, Carl Case, Jared Casper, Bryan Catanzaro, Greg Diamos, Erich Elsen, Ryan Prenger, Sanjeev Satheesh, Shubho Sengupta, Adam Coates, al. Deep speech: Scaling up end-to-end speech recognition. arXiv preprint arXiv:1412.5567, 2014.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp- 770-778, 2016.
Dan Hendrycks and Kevin Gimpel. A baseline for detecting misclassified and out-of-distribution examples in neural networks. arXiv preprint arXiv.1610.02136, 2016.
Dan Hendrycks, Mantas Mazeika, and Thomas Dietterich. Deep anomaly detection with outlier exposure. arXiv preprint arXiv: I8Ii::88..... 2.066006, 2018.
Geoffrey Hinton, Li Deng, Dong Yu, George Dahl, Abdel-rahman Mohamed, Navdeep Jaitly, An- drew Senior, Vincent Vanhoucke, Patrick Nguyen, Brian Kingsbury, et al. Deep neural networks for acoustic modeling in speech recognition. IEEE Signal processing magazine, 29, 2012.
Gao Huang, Zhuang Liu, Laurens Van Der Maaten, and Kilian Weinberger. Densely connected convolutional networks. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 4700-4708, 2017.
Alex Kendall and Yarin Gal. What uncertainties do we need bayesian deep learning for computer vision? In Advances in neural information processing systems, pp. 5574-5584, 2017.
Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv: 14/12698.. 2014.
Diederik Kingma and Max Welling. Auto-encoding variational bayes. arXiv:1312.61 14, 2013.
arXiv preprint
Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. Imagenet classification with deep convo- lutional neural networks. In Advances in neural information processing systems, pp. 1097-1105, 2012.
Alex Krizhevsky et al. Learning multiple layers of features from tiny images. Technical report, Citeseer, 2009.
Yann LeCun, Léon Bottou, Yoshua Bengio, Patrick Haffner, et al. Gradient-based learning applied to document recognition. Proceedings of the IEEE 86(1 (1111)2:27722222 1998.

---

Under review as a conference paper at ICLR 2020
Kimin Lee, Honglak Lee, Kibok Lee, and Jinwoo Shin. Training confidence-calibrate classifiers for detecting out-of of-to-ribtibutiion samples. arXiv preprint arXiv: 2017.
Kimin Lee, Kibok Lee, Honglak Lee, and Jinwoo Shin. simple unified framework for detecting out-of-distribution samples and adversarial attacks. In Advances in Neural Information Process- ing Systems, pp. 7167-7177, 2018.
Shiyu Liang, Yixuan Li, and Srikant. Enhancing the reliability out-of-distribution image detec- tion in neural networks. arXiv preprint (al:::700.26900 2017.
Tsung-Y Yi Lin, Priya Goyal, Ross Girshick, Kaiming He, and Piotr Dollár. Focal loss for dense object detection. In Proceedings of the IEEE international conference computer vision, pp- 2980-2988, 2017.
Andrey Malinin and Mark Gales. Predictive uncertainty estimation via prior networks. In Advances in Neural Information Processing Systems, pp- 7047-7058, 2018.
Yuval Netzer, Tao Wang, Adam Coates, Alessandro Bissacco, Bo Wu, and Andrew Y Ng. Reading digits in natural images with unsupervised feature learning. 2011.
Joseph Redmon and Ali Farhadi. Yolov3: An incremental improvement. arXiv preprint arXiv:1804.02767 2018.
Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409 1556, 2014.
Pingmei Xu, Krista A Ehinger, Yinda Zhang, Adam Finkelstein, Sanjeev R Kulkarni, and Jianxiong Xiao. Turkergaze: Crowdsourcing saliency with webcam based eye tracking. arXiv preprint arXiv:1504.0675 2015.
Fisher Yu, Ari Seff, Yinda Zhang, Shuran Song, Thomas Funkhouser, and Jianxiong Xiao. Lsun: Construction of large-scale image dataset using deep learning with humans in the loop. arXiv preprint arXiv: 155500336635 2015.
A DETAILS OF EXPERIMENTAL SETTINGS
A.1 EVALUATION METRIC
We used several standard metrics for testing the detection of OOD samples and the classification of in-distribution samples.
TNR 95% TPR. Let TP, TN, FP, and FN denote the numbers of true positives, true negatives, false positives, and false negatives, respectively. We measure TNR = TN (FP+TN), when TPR TP/ (TP+FN) is 95%.
AUROC. The Area Under the Receiver Operating Characteristic curve (Davis & Goadrich 2006 is threshold-indeeeedennt metric. The ROC curve depicts the relationship between TPR and FPR.
AUPR. The Area under the Precision-Recall curve another threshold independent metric. The PR curve depicts the relationship between precision and recall. The metrics AUPR-In and AUPR- Out denote the area under the precision-recall curve in which in-distribution and OOD images are specified as positives, respectively.
ACC. We choose accuracy for measuring the classification ability of models when using in- distribution samples. We measure ACC = (TP+TN) / (TP FP TN FN).
A.2 DATASET
We used several standard datasets for detecting OOD samples and classifying in-distribution sam- ples.
10

---

Under review as a conference paper at ICLR 2020
CIFAR. The CIFAR dataset (Krizhevsky et al. 2009) contains 32 32 natural color images. The training set has 50,000 images and the test set has 10,000 images. CIFAR-10 has 10 classes, whilst CIFAR-100 has 100 classes.
SVHN. The Street View Housing Numbers (SVHN) dataset (Netzer et al. 2011) contains 32 x 32 color images of house numbers. The training set has 604,388 images and the test set has 26,032 images. SVHN has 10 classes comprising the digits 0-9.
TIM. The Tiny ImageNet dataset consists of subset of ImageNet images (Deng al. 2009). It contains 10,000 test images from 200 different classes. We downsampled the images to 32 x 32 pixels.
LSUN. The I Large-scale Scene UNderstanding (LSUN) dataset (Yu et al. 2015) has 10,000 test images of 10 different scenes. We downsampled the images 32 32 pixels.
iSUN. The iSUN (Xu et al. 2015) dataset consists of a subset of 8,925 SUN images. We downsam- pled the images 32 x 32 pixels.
Gaussian Noise. The Gaussian noise dataset consists of 10,000 random two-dimensional Gaussian noise images, where each value of every pixel is sampled from i.id Gaussian distribution with mean 0.5 and unit variance.
Uniform Noise. The uniform noise dataset consists of 10,000 images, where each value of every pixel is independently sampled from a uniform distribution on [0, 1].
A.3 MODEL STRUCTURE
Layers
In channels
Out channels
Ksize
Stride
Padding
Conv2d
64
5
ReL.U
MaxPool2d
Conv2d
64
128
5
ReLU
MaxPool2d
Linear
128*5R5
120
ReLU
Linear
120
84
ReLU
Linear
84
10
softmax
Table 4: Network architecture LeNet5.
Layers
In channels
Out channels
Ksize
Stride
Padding
Conv2d
128
64
2
BatchNorm2d
ReLU
MaxPool2d
2
Conv2d
64
1
2
BatchNorm2d
ReLU
MaxPool2d
2
Feature Combination
1#3#3
1#3#3 + y-dii +
Linear
1*3*3 y-dim
100
BatchNormid
ReLU
Linear
100
Sigmoid
Table 5: Network architecture of the feature-combining CNN for LeNet5.
11

---

Under review as a conference paper at ICLR 2020
CIFARIO
CIFAR100
SVHN
5
9
18
18
TIM
LSUN
iSUN
Gaussian Noise
Uniform Noise
Figure Visualization of original images.
Conv Layer
Layer
FC Layer
FC Block
d
d?
concal
Conv
Figure 7: CNN for feature combination.
12

---

Under review as a conference paper at ICLR 2020
Layers
In channels
Out channels
Ksize
Stride
Padding
Conv2d
108
150
2
BatchNorm2d
ReLU
MaxPool2d
2
Feature Combination
150
150#2
Conv2d
150*2
64
4
1
2
BatchNorm2d
ReLU
MaxPool2d
2
Conv2d
64
4
1
2
BatchNorm2d
ReLU
MaxPool2d
2
Feature Combination
1#2*2 1*2*2 +
1#2*2 + y_dim +
Linear
1024
BatchNormid
ReLU
Linear
1024
Sigmoid
Table 6: Network architecture of the feature- combbnig CNN for Dense-BC.
Layers
In channels
Out channels
Ksize
Stride
Padding
Conv2d
64
128
2
BatchNorm2d
ReLU
MaxPool2d
2
Feature
Combination
128
128*2
Conv2d
128*2
64
4
1
2
BatchNorm2d
ReLUU
MaxPool2d
2
Conv2d
64
4
2
BatchNorm2d
ReLU
MaxPool2d
2
Feature Combination
1 144*4
144%4 + dim +
Linear
1*4*4 + y dim +
1024
BdtchNorm
ReLU
Linear
1024
Sigmoid
Table 7: Network architecture of the feature-combining CNN for WRN-40-4
13


Reviews:
[{"review_id": "r1gOgic6FS", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "** post rebuttal start **\n\nAfter reading reviews and authors' response, I decided not to change my score.\nHowever, I feel that this paper is somewhat under-evaluated initially, so I hope the authors have an opportunity in another venue with their revision.\n\n\nDetailed comments:\n\n1.1. I recommend to add an algorithm box describing the learning scheme. It is not end-to-end learning, so it is hard to catch (and potentially, replicate) the learning part. I am also a bit skeptical about the convergence (with non-zero \\sigma), as Reviewer 2 has a concern about it.\n\n1.3. \"We hypothesized that the value of the uncertainty is different depending on whether the inputs are OOD or in-distribution inputs. The results of the ablation study listed in Table1 demonstrate that this hypothesis is true.\"\n2. \"In order to use the data uncertainty, we used the value of \\sigma.\"\n-> Table 1 proves that your proposal (playing with \\sigma) is effective, but it does not mean that \\sigma is the uncertainty which is only essential component for detecting OOD. I recommend the authors to validate their hypothesis, maybe by conducting more experiments to show that the role of \\mu and \\sigma is as expected. At least, if \\mu is proven to have no effect on OOD detection by some experiment, then it can be a clue.\n\n\nMinor comment: I hope ICLR papers are cited as ICLR papers at least in ICLR submissions, not arXiv preprint.. Alemi's paper is ICLR'17 paper, for example.\n\n** post rebuttal end **\n\n\n\n- Summary: This paper proposes to train an OOD detection model from a portion of modified latent vectors; more specifically, similar to VAE, they assume unimodal Gaussian distributed latent space at each layer and use the collection of standard deviation to train an OOD detector. Experimental results on several OOD benchmarks with different backbone networks show that their method outperforms ODIN (Liang et al., 2017).\n\n\n- Decision and supporting arguments:\nWeak reject.\n\n1. Though the idea of extracting uncertainty is interesting, but I think the motivation and explanation is not enough, so I couldn't find a rationale why we should do this. I have several questions that I couldn't find an answer in the submission, could you answer them?\n1.1. Are the classification loss and OOD detection loss optimized jointly?\n1.2. Is it reasonable to assume unimodal Gaussian distribution over all latent spaces without a carefully designed learning objective? More specifically, to make it learnable, don't you need a learning objective other than the conventional cross-entropy loss, e.g., \"Bayes by backprop\" proposed in the early work (Blundell, 2015)?\n1.3. Why only the standard deviation values are useful for the OOD detection performance? If they are really useful, how the standard deviation values are related to the OOD detection performance?\n\nBlundell et al. Weight Uncertainty in Neural Networks. In ICML, 2015.\n\n2. More ablation study is required to verify the effectiveness of their method. Again, I am not sure why \\mu and \\sigma should be split, and why \\mu should be discarded for the OOD detection part.\n\n3. The architecture design of CNN in Figure 7 also looks arbitrary.\n\n4. Comparison with more state-of-the-art methods is required. ODIN (Liang et al., 2017) is a powerful method but it is somewhat old and many recent works actually combine their method with ODIN for better performance. Why don't you compare the proposed method with the Mahalanobis distance-based classifier (Lee et al., 2018)? They also estimate the uncertainty by measuring the Mahalanobis distance on the feature spaces & combine them for better OOD detection.\n\n\n- Comments:\n1. I couldn't find any statement about the classification accuracy, does the proposed model have a good classification performance as well? Since {a half of the model capacity is spent to split \\mu and \\sigma} and {it should take account of uncertainty in the forward pass}, I am not sure it maintains a good classification performance, compared to the standard classification model with the same capacity.", "extra_scores": {"experience_assessment": "I have published in this field for several years.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper at least twice and used my best judgement in assessing the paper.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I assessed the sensibility of the derivations and theory."}, "extra_text": {}}, {"review_id": "HkxJSbehYS", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 1, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper tackles out-of-distribution samples detection via training VAE-like networks. The key idea is to inject learnable Gaussian noise to each layer across the network in the hope that the variance of the noise correlates well with the uncertainty of the input features. The network is trained to minimize the empirical loss subject to noise perturbation. The paper is well written, and the background is introduced clearly.\n \nAs I understand it, the goal of *out-of-distribution sample detection* is to train a deep network that simultaneously generalizes well and also be discriminative to outliers. However, it’s not clear to me why the proposed method server this purpose; empirical results are not convincing either. My major concerns are as follows:\n \nFirst of all, from my intuition, it would be much easier to train deterministic networks than their counterparts with randomness. Empirically, researchers also often observe near-zero training loss for large deterministic networks such as Dense-BC trained on simple CIFAR/SVHN datasets. Especially, in this case, the training goal is simply to map higher-dimensional inputs to lower-dimensional classification categories. That being said, one would expect the variances go to zero at convergence to achieve lower empirical loss in the case of no additional diversity (or uncertainty) promotion terms. \n \nIt is not clear to me how to avoid degenerate solutions at convergence \nwhile maintaining good testing performance with the proposed training strategy. \nFrom the empirical results, it also appears that all models reported might not be fully optimized? \nThe baseline results are significantly worse than those reported in previous work.  \nSpecifically, \nin table 1, the testing accuracy of Dense-BC trained on CIFAR-100 is only 71.6.\nIn table 2, the reported testing accuracy on CIFAR-10 using Dense-BC is 92.4.\n \nHowever, the results of DenseNet-BC (k=12, L=100, table 2) reported in the original paper are:\nCIFAR10  94.0  (also leave 5K examples as validation set)\nCIFAR100 75.9\n  \nMeanwhile, the reported accuracy of WRN-40-4 trained on CIFAR-10 and CIFAR-100 are 89.6 and 66.0, respectively. However, the corresponding baseline numbers in the original WRN paper are much higher,  \nCIFAR-10  95.03\nCIFAR-100 77.11 \n\nCould the authors comment on that?\n\nReferences:\nGao Huang, Zhuang Liu, Laurens van der Maaten, Kilian Q. Weinberger.\nDensely Connected Convolutional Networks\nhttps://arxiv.org/abs/1608.06993\n\nSergey Zagoruyko, Nikos Komodakis. \nWide Residual Networks.  \nhttps://arxiv.org/pdf/1605.07146.pdf\n", "extra_scores": {"experience_assessment": "I have read many papers in this area.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I carefully checked the derivations and theory.", "review_assessment:_checking_correctness_of_experiments": "I carefully checked the experiments.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper thoroughly."}, "extra_text": {}}, {"review_id": "HJg4ZAZ0tH", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 1, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The paper considers the problem of out-of-distribution (OOD) sample detection while solving a classification task. The authors tackle the problem of OOD detection with exploiting uncertainty while passing a test sample through the neural network. They treat outputs of (some) layers in a NN as random Gaussian-distributed variables and measure uncertainty as variance of these Gaussians. Then when uncertainty is high, OOD is detected.\n\nThe overall idea behind the paper could be interesting, but its realisation in the current form is questionable. \n\nThe paper seems totally misusing the reparametrisation trick and stochastic outputs of layers in NNs. Eq. (2) is not the objective of variational inference that seems to be required for stochastic outputs and the reparametrisation trick as presented before the equation. The objective misses the KL-divergence term! Without it what would stop a neural net to set sigmas to 0 and forget about the stochasticity altogether? Not to mention that the current objective is not mathematically justified.\nIf there is no mix and error in eq. (2) and the networks were trained using this loss (and based on provided code they were using this loss), my wild guess of explaining why this may give best results in the experiments is that the models were trained for surprisingly small number of epochs. Therefore, a hypothesis would be that this small number of epochs did not allow the networks to switch sigmas to 0. \n\nUntil the authors can clarify and justify the objective, I will vote for rejection only based on this ground.\n\nHowever, there are other issues in the paper as well. First of all, its clarity. It seems that the paper requires a lot of polishing.  The first paragraph of this review is based on my assumptions from the paper since I am not completely sure I understand it correctly. More about the clarity issues below\n\nFor strong evaluation, comparison with Malinin & Gales (2018) work seems to be important since it was the only work also using uncertainties for OOD detection in related work. Also related work section does not look like an exhaustive overview.\n\nSome of the detailed comments:\n1.\t“In other words, in-distribution samples possess more features that convolutional filters react to than OOD samples” – first of all, this sentence is not easy to parse. Secondly, it is unclear, why this should be true. If OOD samples are still natural images, they would contain edges just the same as in-distribution samples. That is the power of deep learning enabling transfer learning, that low-level features are the transferable across different data and tasks. Therefore, the claim that “Therefore, the uncertainties of the features will be larger when the inputs are in-distribution samples” requires more elaboration and arguments\n2.\tThe arguments of the next paragraph regarding uncertainty of deeper layers should be larger for OOD samples are not very convincing either.  It is either requires a definition what the authors mean here as uncertainty, or it is not necessarily true that absence of fixed regions for embeddings leads to higher uncertainty. \n3.\t3rd and 4th paragraphs in Introduction have too many repetitions of phrases between each other. Compare, e.g. the first sentences of the paragraphs or the last sentences. \n4.\t“One cause of the abovementioned problem is that their approaches” and similarly the next paragraph: “their approaches” stylistically sound wrong. It is appropriate in the previous paragraph since there is a link to “previous studies”. It seems that “these approaches” or “the existing approaches” would be a better choice for this and the next paragraph.\n5.\t“Each uncertainty is easily estimated after training the discriminative model by computing the mean and the variance of their features using a reparameterization trick” – conventional discriminative models do not estimate mean and variances of the features. The issue of estimating uncertainty is addressed by several special methods such as Bayesian variational methods used in the referred papers. Therefore, in order to use a reparametrisation trick one need to firstly choose a special class of models, which is not obvious from the text.  \n6.\t“Moreover, UFEL is robust to hyperparameters such as the number of in-distribution classes and the validation dataset.” – the size of the validation dataset? In any case neither size of the validation dataset nor the validation dataset itself are not hyperparameters (should not be hyperparameters for out-of-distribution detection). The number of classes can hardly be called a hyperparameter also. \n7.\t“depends on the difference in the Dirichlet distribution of the categorical parameter <…> In our work, the distribution of the logit of the categorical parameters” – what is/are this/these categorical parameter(s)?\n8.\t“Further, they estimate the parameter of the Dirichlet distribution using a DNN and train the model with in-distribution and OOD datasets” – this sentence may mislead to impression that the proposed method does not need OOD dataset for training, which does not seem to be the case, since \\lambda and \\theta are trained based on OOD samples\n9.\t“because they will not be relevant to the classification accuracy” – who are they?\n10.\t“and \\epsilon is the Gaussian noise” –> the standard Gaussian noise\n11.\t“where z^0 = x” – it seems this should be placed somewhere earlier when z^l is introduced since z^0 is not used in eq.(2) after which this text is placed\n12.\tIt is unclear how \\lambda^l and CNN \\theta are learnt\n13.\tIt is unclear how the values of features d(x) are used to detect OOD samples\n14.\t“comparison methods, and models” – not clear what models mean here\n15.\tMissing references to datasets in the main text. At least reference to Appendix A.2 is required\n16.\t“We used 5,000 validation images split from each training dataset and chose the parameter that can obtain” – which parameter? \n17.\t“All the hyperparameters of ODIN” – a reader does not know yet that ODIN is used for comparison\n18.\t“which consists of 100 OOD images from the test dataset and 1,000 images from the in-distribution validation set” – it is a bit confusing to call OOD dataset as a test dataset in this context\n19.\t“We tuned the parameters of the CNN in Equation 4 using 50 validation training images taken from the 100 validation images. The best parameters were chosen by validating the performance using the rest of 50 validation images.” – this is confusing. What parameters do the authors talk about in the second sentence if not the parameters of the CNN?\n20.\t“We used TNR at 95% TPR, AUROC, AUPR, and accuracy (ACC),” – Some elaboration is required, at least the reference to Appendix A.1. What is the changing threshold for AUROC and AUPR? Why AUPR-In and AUPR-Out are considered and only a single AUROC is considered. What is the positive class for AUROC?\n23.\t“For LeNet5, we increased the number of channels of the original LeNet5 to improve accuracy” – do the authors mean that they allowed RGB images as input rather than greyscale? If yes, this explicit explanation would be preferable \n24.\t“We inserted the reparameterization trick” – not the best word choice. Reparametrisation trick is a computational/implementation trick/method and it is hard to say that it can be inserted into a network. I believe what the authors mean is that they inserted mean/std outputs instead of point outputs. Conceptually, this means that the output of the corresponding layers is considered to be stochastic rather than deterministic. \nAlso, it is unclear when the authors say they insert it to the softmax layer. According to Section 3 the softmax layer is never considered to output means and stds.\n25.\tThe numbers of epochs for training NNs are very small for LeNet and WideResNet in the experiments. Did the models manage to converge during this short training?\n\nMinor:\n1.\t“These data were also used” -> “this data”", "extra_scores": {"experience_assessment": "I do not know much about this area.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper thoroughly.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I carefully checked the derivations and theory."}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper proposes a method for OOD detection which leverages the uncertainties associated with the features at the intermediate layers (and not just the output layer).

All the reviewers agreed that while this is an interesting direction, the paper requires more work before it can be accepted. In particular, the reviewers raised several concerns about other relevant baselines, some of the reported empirical results, and clarity of the explanation.

I encourage the authors to revise the draft based on the reviewers’ feedback and resubmit to a different venue.

Decision: Reject