Neural Network Out-of-Distribution Detection for Regression Tasks
Authors: Geoff Pleiss
Amauri Souza
Joseph Kim
Boyi Li
Kilian Q. Weinberger
Venue:  (2020)
Keywords: Out-of-distribution
deep learning
regression

Abstract: Neural network out-of-distribution (OOD) detection aims to identify when a model is unable to generalize to new inputs, either due to covariate shift or anomalous data. Most existing OOD methods only apply to classification tasks, as they assume a discrete set of possible predictions. In this paper, we propose a method for neural network OOD detection that can be applied to regression problems. We demonstrate that the hidden features for in-distribution data can be described by a highly concentrated, low dimensional distribution. Therefore, we can model these in-distribution features with an extremely simple generative model, such as a Gaussian mixture model (GMM) with 4 or fewer components. We demonstrate on several real-world benchmark data sets that GMM-based feature detection achieves state-of-the-art OOD detection results on several regression tasks. Moreover, this approach is simple to implement and computationally efficient.

Under review as a conference paper at ICLR 2020
NEURAL NETWORK OUT-OF-DISTRIBUTION DETECTION FOR REGRESSION TASKS
Anonymous authors Paper under double-blind review
ABSTRACT
Neural network out-of- distriibbbt (OOD) detection aims to identify when a model is unable to generalize to new inputs. either due to covariate shift or anoma- lous data. Most existing OOD methods only apply to classification tasks, as they assume a discrete set possible predictions. In this paper, we propose method for neural network OOD detection that can be applied to regression problems. We demonstrate that the hidden features for in-distribution data can be described by highly concentrated, low dimensional distribution. Therefore, we model these in-distribution features with an extremely simple generative model, such Gaussian mixture model (GMM) with 4 or fewer components. We demon- strate on several real-world benchmark data sets that GMM-based feature detec- tion achieves state-of-the-are OOD detection results several regression tasks. Moreover, this approach is simple to implement and computationally efficient.
INTRODUCTION
The success of deep neural networks in many domains (Krizhessky et al. 2012 Lample et al, 2016 Mnih et al. (016) is due to their ability to learn complex functions that generalize to new samples. However, this observed generalization only extends to data that are sufficiently similar to the training data. If the neural network encounters data that deviates from the distribution of training data, its predictions are likely to be erroneous or nonsensical (Guo et al. [017]] Diang et al.) 2012 Begoli et al., 2219). This may occur if the model is used in scenarios that experience covariate shift (Sugiyama et al. (007) or if the model encounters previously-unee -unseen categories of data (Yu et al.) 2017 Hassen & Chan 2018). Such scenarios are examples of outtt--distribution (OOD) inputs.
Ideally we would like for neural networks to adapt to such shifts in the data distribution (Amodei) al. 2016). In the absence of such adaptation, out-of-distribue
detection should be used to identify when model is unable to generalize to previously-unsed input. While there are sev- eral proposed methods for neural network OOD detection (Hendrycks & Gimpel| 2017 Liang et al.) 2018 Lee et al., 2018b). many of these methods rely architectural components specific classifi- cation neural networks. Consequentially, they cannot be applied to regression problems. Regression neural networks typically output only point prediction rather than predictive distribution, and thus the output does not indicate its uncertainty or reliability for a given input. This is illustrated in Figure I] which displays predictions from network trained to predict prices of middle-class houses in Kentucky. The network outputs house price prediction for any possible input even for out- of-distribution images like California mansion chair. These predictions fall within the normal range of possible prices, and therefore do not convey that the inputs are not valid for this model.
Because the predictions cannot identify OOD inputs, we must look for alternative signals. Previous approaches perform regression OOD detection through ensembles (Gal & Ghahramani [0166] Lak-
shminarayanan et 2017) or through an additional uncertainty prediction layer (Kendall & Gal] Malinin et al. 2017). In this paper, we instead turn to the space of hidden features. During training a neural network learns to extract relevant features about the training data and discards ir- relevant information. Whether not network generalizes extracted features from that sample. Networks do not generalize to out-of--distiibttin data because the distributional shift causes the network to extract the wrong information. For example, when the housing neural network is applied to the California mansion in Figure I, the network's 's features do not extract the relevant information that would indicate the true price of the house. (e.g. presence of
given test sample depends on the

---

Under review as a conference paper at ICLR 2020
Kentucky House (In-dist. Input)
Kentucky House (In-dist. input)
California Mansion (Out-of-dist. input)
Chair (Out-offdist. input)
Actual: $130,000
Actual: $125 ,000
Actual: $1 $1,4300000 ,,00
Actual: N/A Pred: $140,000
Pred: $132,000
Pred: $156,000
Pred: $270,000
OOD Score: 0.05
OOD Score: 0.11
OOD Score: 0.86
OOD Score: 0.99
Figure 1: A neural network trained to predict house prices in Kentucky from curb-side images. Although the algorithm generalizes well to subiin-distribution test images (left two houses), it has high error for nut-of-diitributton)) mansion from California and is forced to make nonsensical prediction for the armchair. Both latter cases are detected as out-of-sample with high OOD scores.
palm trees). In other words, the extracted features the California mansion will differ significantly from training data features and therefore the network does not generalize.
Based on this intuition, we investigate how to utilize the hidden features of regression neural net- works for OOD detection. In particular, make several contributions. First, we argue empirically and theoretically that we cannot identify OOD inputs simply using only the networks' predictions. However, then demonstrate that the features of in-distribution inputs lie an intrinsically low- dimensional portion of the feature space. It is unlikely that OOD inputs map to similar locations in feature space because of this low-dimensionality. We additionally show that, because of this low- dimensionality, it is possible to model the in-distribution features with simple generative models.
To evaluate our proposed approach, we develop new OOD detection tasks based on large-scale com- puter vision regression datasets. We evaluate the OOD detection performance of several generative models trained on in-distribution features. Surprisingly, we find that simple mixture of Gaussians often with no more than 2 components is better at OOD detection than more complex models such as variational autoencoders. Finally, we demonstrate that GMM models of in-distribution features are able to outperform other regression OOD methods across several benchmarks.
2 RELATED WORK
detection and the related problems of outlier detection (Hodge & Austin 2004
Chalapathy & Chawla (219)) and novelty detection (Pimentel et al. (2014) are well-studied statis- ties and machine learning. Arguably, the most straightforward method to these problems is genera- tively modeling the distribution inputs p(x using a parametric distribution (Chow) 1970 Eskin
2000 nonparametric density estimate (Kim & Scott 2012). If p(x) is small for given input, then it is likely -iatt---distiibution or an anomaly. Several recent works (Choi et al.) [018 Nalismick
al. 2019: Piihhorskyi et al., 2018) have suggested identifying OOD) inputs with deep generative models (Goodfellow et al. 2014; Kingma & Welling, [014 |eezndee et al., 22144) Van Den Oord (et al. 2016).. One challenge with these methods is that deep generative models cannot necessar- ily model large-scale images or other complex distributions (Hendrycks et al. 2019) and may be overconfident when modeling input data (Nalisnick al. 2019).
Rather than directly modeling the input distribution, the approach proposed in this paper operates on feature spaces using a mixture model to capture potentially anomalous inputs. This falls under the category of model model-dependent OOD detection methods, which are ideal when the input distribution is too complex to model with generative methods (e.g. for high resolution images) (Hendrycks al,, 2019). Several model-dependent methods have been proposed for classification neural networks. The training procedure of classification network can be modified to include uncertain samples (Lee et al. (2018a) or to discourage overconfidence through an alternative loss function (Alemi al.) 2018 Masana et al., 2018: Subramanya et al.) 2017; Sensoy et al.) 2018). Alternatively, OOD metrics can be constructed for classification based the softmax probability output (Hendrycks) jet al. 2019 Liang et al. 2018) or related uncertainty scores (Schulam & Saria 2019). In a similar
vein our proposed approach, (Lee al. 2018b) use class-conditional Mahalanobis distance
2

---

Under review as a conference paper at ICLR 2020
for classification OOD, All of these methods are designed specifically for multi-class classification problems - typically relying on the discrete output space. Consequentially, there are typically no straightforward extensions of these methods to regression problems.
There are fewer proposed methods for OOD detection on regression tasks. Most existing approaches rely on an estimate of the neural network's predictive uncertainty (Kuleshov et al. 2018). There
several proposed approaches to obtain uncertainty estimates from regression neural networks. One class of approaches is to add an additional oulput that predicts confidence interval for the network's prediction (Lakshminarayamm
et al., 2017, Kendall & Gal 2017; Malinin et al. (017). This layer can be trained in conjunction with the predictive output to minimize the negative log likelihood on the training set. Alternatively, Bayesian approaches to deep learning (Blundell et al.] [015]]
Gal & Ghahramani] 2016; Gal et al., 2017 Kingma et al., (015) can used to estimate predictive uncertainty on regression tasks. These approaches typically approximate the posterior distribution of neural networks parameters through an ensemble of models. Non- Non- Bayesian ensembling approaches achieve can produce similar uncertainty estimates (Lakshminarayanal et al., 2017 Maddox et al.) 2019). In this paper we propose an orthogonal approach to regression OOD detection using the distribution of hidden features rather than the model's predictive uncertainty.
3 CHARACTERIZING OOD PREDICTIONS ON REGRESSION TASKS
In the classification setting, the output of neural network a softmax score for each class. Because this output can be interpreted as probability distribution over classes, represents some notion of the neural network's uncer- tainty and therefore can be used to identify inputs that are likely to be OOD (Hendrycks & Gimpel [017 Liang et al. 2018). For most regression architectures however, the output is typically a point prediction á which does not convey any notion of predictive uncertainty. Moreover, in this section we show that these point predictions ÿ are insufficient for differentiating between in-distribution and out-of -iitiitribttion inputs.
Distribution of Network Predictions pi))
0.6
Ii-Dst: KY homee
0.5
OOD: CA honses
OOD: ImageNet
0.4
0.3
0.2
0.1
0.0
To illustrate this, we train a neural network predict the price of homes in Kentucky based on a front facing picture. Figure 2] (left) displays histogram of the net- work's predictions on a withheld validation set (black line). The distribution of these predictions corresponds to the distribution of actual house prices (dotted line). However, if the same network receives an OOD input, it predicts price values that lic in the same range the in-distribution predictions. For example, if houses from California are input into the neural network, the predicted housing prices (blue line) are roughly the same as that of Kentucky houses despite these houses costing up to 10 times much. The network predicts simi- lar values for Imagenet images (orange line) even though these images do not contain any houses. Based on the ý alone, it is not possible to determine that these inputs are not valid.
0
100
200
300
400
1 (thousands of dollars)
Figure 2: A neural network is trained predict Kentucky housing prices. The distribution of predictions on the valida- tion data (black line) and the actual dis- tribution of housing prices (dotted line). Out-of-diss distribution data (blue and green lines) obtain similar predictions.
The fact that the OOD predictions are similar to in-distribution predictions can be explained by a simple ciree-ff-mmnniinnaliyy argument. Let ø be regression neural network that maps an input x E X' to an output ÿ E R. The last layer of & produces a set of activations h(x) that are linearly mapped to the one-dimensional prediction: ÿ = w h(x). Let's assume that the features h(x) for some OOD inputs can be bounded within some Euclidean ball of radius r. If the dimensionality of h(() is large, then the probability of an extreme prediction becomes exponentially unlikely.
Remark 1. Let h(x) be features drawn from uniform measure on Euclidean ball of radius r. Let d be the dimensionality of h(x). The probability of large prediction (ú) > e is bounded by
2T +1
- de
p(1b > e) <
5/2pd
exp
2r w
3

---

Under review as a conference paper at ICLR 2020
Relative Size Discarded Featuzes (lu)
Singolor Valnes p(h) Comutaaoo Mot.
KY Housing Price, DenseNet
Rotated MNIST, ResNet
-4.0
Conntauoe Ratio
( Sllll// [R]p])
0.8
10"
-3.0
-200
Magaizade
log
0.6
10
-1.0
-400
10m4
-600
In-dist.
00D
10-5
0
200
400
Depth
Singulaz Value Index
Singular Value Index
The relative size of "discarded" feature information h1 (x) decreases as the depth of neural networks increases. Center: The intrinsic dimensionality of the feature vectors h(x). as measured by spectrum of its principal components, decreases with weight decay. Right: Likelihood of in-distribution and OOD features as measured by a Gaussian decomposed into the likelihood of each principal component. The Gaussian was fit on (in-distribution) training set features. OOD features have low likelihood on the small principal components, suggesting that they do not lic in the same diw-dimensional distribution.
Figure 3: Left:
(See Appendix B] for proof). In practice is not necessarily the case that OOD features will be uni- formly distributed. Nevertheless, this remark illustrates that predictive outputs discard information that would indicate whether a sample is OOD or not.
4 CHARACTERIZING THE DISTRIBUTION OF HIDDEN FEATURES
Although the network's 's predictions look very similar for in in-iistribution and OOD inputs, we expect that their hidden features h(x) are very different. Since ImageNet pictures do not contain front- facing houses, the network will extract very different types features. We similarly expect the activations of California houses to be different since the network is unable to generalize these in- puts. To that end, we hypothesize that generative model of in-distribution features p(h(x)) will distinguish between in- and duitt--disttibutionn inputs. In this section we demonstrate that despite the high dimensionality of hxx) the in-distribution features are very tightly concentrated. As result, it is possible model in-distribution features with simple generative model.
In-distribution features are intrinsically low dimensional. In the regression setting, the net- work's prediction is the inner product between the features and a weight vector: 9 = w h(x). Any feature information that is orthogonal to the weight w will not contribute to the prediction and there- fore "discarded" information. We can describe feature vector h(x) in terms of the prediction ğ and the discarded information h (x):
hxx)= jw + hi (x),
h (x) = (I-WW ww h(x),
where b_ (x) is the discarded information, computed using the orthogonal projection of w. Also, we assume, without loss of generality, that ||w||2 1. It is worth emphasizing that "useful" predictive information ýw lies within a single dimension of the d-dimensional h(x) vector (where d is often greater than 500). Though the "discarded" information h (x) occupies d-1 dimensions, it typicalll has low intrinsic dimensionality for in-distribution data as we demonstrate below. Consequentially, the in-distribution features h(x) hı(x) + ýw can be described by low-dimensional distribu- tion. We identify two factors that are correlated with this low dimensionality: network capacity and regularization.
Network capacity reduces the relative size of h1 (x). As network capacity increases, we expect that there will be less "discarded" information h (x). To understand why this is the case, assume there is a l-layer network in which there is some correlation between the discarded features and the true labels. Now assume we add an additional layer before the linear layer which computes a new set of hidden features h' hx(x). This new layer improve the network's 's predictions if it uses information in h (x) to augment the original prediction ÿ. Consequentially, this new network will likely have less discarded information than the shallower network.
4

---

Under review as a conference paper at ICLR 2020
While this is just an illustrative example, we can empirically demonstrate that network capacity does in fact decrease the amount of discarded information. First, we introduce the following metric of discarded information: let Σ be the empirical covariance matrix of hxx) for in-distribution data, and let Σn be the covariance of b (x). We measure the amount of discarded information by comparing the maximum singular values of Σ and 1. If || are comprised mostly of discarded information. Conversely, h(x) will consist primarily of predictive information if _ ||/|| 2 close to zero.
is close to 1, then the feature vectors
We apply this metric to a simple regression task using the MNIST dataset. Each input image is rotated by an amount between -90 and 90 degrees. We train residual networks with increasing depth to predict the amount of rotation in each image (details in AAppendix C). Figure 3 (left) plots the singular value ratio | shallow networks, but decreases with depth. This suggests that, for deep networks, most of h(x) lies on the one-dimensional subspace defined by the weight vector W.
as function of network depth. The ratio is quite large for
Regularization reduces the dimensionality of h(x). Applying weight decay to a neural network reduces the magnitude of the weights, which in turn reduces the magnitude of feature vectors. Re- ducing the magnitude of hxx) while maintaining predictive power will result in feature vectors with little discarded information h(x). Since all but one dimensions of the feature space are discarded, we expect that the intrinsic dimensionality of features will decrease with increasing regularization. To support this intuition we train the LeNet model on rotated MNIST with various amounts of weight decay. In Figure 3 (center) we plot the spectrum of the in-distribution feature covariance matrix Σ for these different models. We see that all covariance matrices are dominated by a few large singular values. However, the spectra of the high regularization models decay much more rapidly, suggesting that regularization simplifies the distribution of hidden features.
The fact that in-distribution inputs occupy low-dimensional portion of high-dimensional feature space is extremely advantageous. we draw a random set of random features h(x) (e.g. from a uniform measure on Euclidean ball), it will be highly unlikely that these features occupy the same low-dimensional space. Though OOD features are not truly random vectors in practice, we find empirically that they do not occupy the same low dimensional subspace. To demonstrate this, we fit a multivariate Gaussian to the training set features from the housing price network. We can decompose the fit of this Gaussian as product of its principle components and measure each component's ' likelihood for in- and out-of-distribution data. Figure 3 (right) displays the fits for the housing price validation data (in-distribution) and ImageNet samples (OOD). We find that in- distribution and OOD data are equally likely for the main principal components. However, OOD) features do not fit the smaller principal components well, and thus these small principal components are able to distinguish in- and ouf-of-distribution data. In this case the curse-of-dimensionlli makes possible to distinguish OOD and in in-distribution features.
5 OOD DETECTION WITH GENERATIVE MODELS OF HIDDEN FEATURES
As we demonstrate in the previous section, the distribution of in-distribution hidden features h(x) is intrinsically low dimensional. Since OOD data unlikely to share the same low-dimensional distribution, we propose using generative models of in-distribution h(x) to identify OOD inputs.
Possible generative model. While a simple Gaussian can model the distribution h(x), it is possi- ble that more complex generative model will fit better. There are several choices of generative mod- els, arguably the simplest choice are parametric models such as Gaussian mixture models (GMMs), which can approximate any distribution with enough mixture components Bishop (2006). Nonpa- rameteric models, such as kernel density estimation (KDE), offer complexity that scales with the number of observations. Deep generative models such as variational autoencoders (VAE) (Kingma) & Welling 2014 Rezende et al. 2014) are arguably the most powerful class of generative models.
Model fitting. After training the predictive neural network, a generative model is fit to in- distribution features. In all experiments, we use features extracted from the neural network's 's training data. However, note that features from any (unlabeled) in-distribution data could be used instead.
5

---

Under review as a conference paper at ICLR 2020
Model selection and hyperparameter tuning. Tuning hyperparameters can be challenging be- cause one typically does not have access to OOD data. For our method we must determine 1) which generative model to and 2) the hyperparameters of the generative model. We select these hy- perparameters using the generative model's log-likelihodd on the validation set: log ((avaai = Ex log 1l(((((())) From decision theoretic standpoint, the model with the largest likelihood fits the validation data with the smallest distribution (Bishop) 2006). If the distribution is strongly peaked then it has less support for samples that are not in-distribution, and therefore will be good for OOD detection. Importantly, this criterion only requires access to in-distribution data.
Comparing the Performance of Generative Models. In Section A. we examine how the choice of generative model affects OOD detection performance. We compare KDE models, VAEs, and 2-component GMMs on several benchmark OOD tasks (which are described in detail in the next section). Surprisingly, we find that GMMs detects OOD inputs well if not better- than the other models. This is especially surprising because GMMs tend not to perform well in high dimensional spaces (Bishop) 2006), which further suggests that in-distribution features are highly concentrated.
6 EXPERIMENTS
In this section, we demonstrate the performance of our OOD detection method on several regression tasks. We first illustrate the method on a 2D toy example, and then we evaluate the method on two large-scale computer vision regression tasks. For experimental details, we refer the reader to Appendix C Based on the ablation study in Section A. we model the in-distribution features with a GMM. The number of components in the GMM is chosen through model selection with the validation dataset. (The effect of the GMM size is explored in Section A.2)
Smoothed XOR.
To illustrate how feature spaces can be used for detecting OOD samples, we consider a simulated regression task. The goal is to approximate the smoothed XOR function from noisy labeled data. In-distribution inputs consist of Uniform samples in [0, 1]2, while OOD inputs come from isotropic Gaussian distribution centered at [1.5, 1.5] with variance o² 0.1. Figure 4 (left) depicts the proposed setting, i.c. the target XOR function and in-dist/OOOD samples. Here, multilayer perceptrons (MLP) the predictive model. Figure 4 (middle) plots predictions for both OOD and in-distribution (validation) samples. Note that this corresponds to the desired scenario in which one cannot identify OOD inputs based on the predictions alone.
Smnothol XOR and the data
Prodictions va. Turgels
Logelänlihood log(l))
Indist ooD
1.5
AUROC: 1.0
0.5
Tralist ooD
In-ditt oon
1.5
Figure 4: Smoothed XOR regression task. Left: Target XOR function over in-dist. and OOD data. Middle: Predictions for in-dist. and OOD inputs. Right: Gaussian on the in- and OOD features.
The proposed approach relies on the assumption that hidden features are informative, and their distribution can be approximated using simple generative models. In this regard, Figure 4 (right) shows the fit from a Gaussian the activations the last nonlinear layer of the prediction network. There is virtually no overlap between in-distribution and OOD features. As a consequence, we achieve perfect detection, represented by area under the ROC curve (AUROC) equals to 1.
Housing price prediction. We demonstrate our proposed method large-scale regression task: predicting home prices from front-facing house pictures. We train a 121-layer DenseNet (Huang al. 2017) and 50-layer ResNet (He al. 2016) dataset of nearly 50,000 middle-income

---

Under review as a conference paper at ICLR 2020
Table 1: OOD detection results for housing price prediction (top) and age estimation (bottom).
TNR at TPR 95%
AUROC
Network
OOD Dataset
MCD
Ens.
Var.
Var. MCD
GMM
MCD
Ens.
Var.
Var: MMD
GMM
ImageNet
0.223
0.587
0.757
0.200
0.999
0.825
0.909
0.940
0.764
0.999
KY Housing Price (DenseNet-121)
KITTI
0.475
0.517
0,443
0.318
0.994
0.910
0.917
0.910
0.830
0.995
StreetView
0.280
0.395
0.348
0.238
0.936
0.847
0.877
0.849
0.779
0.962
CA Houses
0,282
0.304
0.454
0.207
0.588
0.832
0.832
0.874
0.757
0.928
ImageNet
0.711
0.629
0.611
0.998
0.953
0.931
0.999
Housing Price
KITTI
0.518
0.731
0.691
0.687
0.932
0.922
0.961
0.954
0.955
0.963
(ResNer-50)
StreetView
0.317
0.880
0.800
0.913
0 0.868
0.833
0.951
0.533
0.933
0.975
CA Houses
0.598
0.261
0,263
0.571
0,803
0.930
0.839
0.812
ImageNet
0,256
0.601
0,908
0.241
0 9.933
0.718
0.503
0.981
0,760
0.994
Adult Age Estimation
Streetviow
0.120
0.625
0.963
0.361
0.954
0.873
0.916
0
0.809
0.995
(DenseNet- 121)
Pedestrian
0,590
0.794
0.998
0.313
1. 1.000
0.925
0.957
0.999
0.804
0.999
Child Age
0.201
0.338
0.125
0.844
0.866
ImngeNet
0.310
0.915
0,598
0.876
0.937
0.823
0.978
0.977
0.970
0.987
Adult Age Estimation
Streatiew
0,240
0.987
0.998
0.990
0,908
0.826
0.987
0. 0.996
0.993
0.979
(Re)
Pedestrian
0.952
0.985
Child Age
0.150
0.234
0.183
0.148
0.434
0.636
0.627
0.540
0.553
0.867
Housing
Housing
Age Predictions va. Actual Age
Age GMM OOD Scones
Predictions Actual Price
GMM OOD Scores
y (thonzuads of dolllss)
10
houges D
Adulte D)
Adults (In D)
CIA hoowwe (OOD)
Chuidlen (OPD)
Chikdren (001)
lug hh))))))))))
90% TPR
(years))
75
96% TPR
102
30
boriscs (In D)
10a
CA house (ODD)
10-
10
10
10a
10a
102
109
50
100
10P
10a
(thorsaand of dollars)
Io
8
(thorssands of dollars)
8 [years)
1y
81 [years)
Figure 5: Housing price predictions (Left) Prediction vs. actual price for in- in-distribution and OOD houses. (Middle-left) The OOD score p(h(x)) from the GMM correlates with the predictive error and can identify the most erroneous housing predictions. Age estimations (Middle-right) Predic- tion vs. actual age for adults (in-distribution) and children (OOD). (Right) Similarly to housing, the GMM's OOD score identifies high-error predictions.
house images from Fayette County in Kentucky. The CNNs are pre-trained on ImageNet (Deng ett al., 2009) and the last layer is replaced with a linear layer that produces single output. We compare the GMM-hased detector against methods that use ensemble or Bayesian model averages to estimate predictive uncertainty. Monte-Carke Dropout (MCD) (Gal & Ghahramani 2016 Kendall & Gal 2017) creates Bayesian ensemble of 10 neural networks, where each network is sampled al. 2017) is an ensemble of 4 standard neural networks. For these two baselines, the variance of the forward passes is used a metric for detecting OOD inputs. Var. (Malinin et al. 2017) adds a variance output to the neural network to predict uncertainty estimates. Var. + MCD (Kendall & Gal 2017) combines this variance estimate with MC-Dropout.
from base network with dropout (p 0.2). Deep Ensembles (Ens.) [Lakshminarayanar
Following Hendrycks & Gimpel (2017), we use several external datasets as sources for out-of- distribution inputs. Firstly, ImtgeNet (Deng et al. 2009) is a dataset of photos from 1,000 classes. This dataset contains easily detectable out-of-distribution inputs, since most of the pictures do not contain houses. KITTI (Geiger et al.) 2013) is a dataset of self-driving car images. These images may contain some houses, but any houses will be in the periphery of the image rather than being centered. StreetView (Zamir& & Shahl 2014) contains images from Google street view, which may also contain some frontal house views. The most challenging dataset is CA Houses (Ahmed & Moustafa 2016), which consists of house images from California.
METRICS. We apply each detection method to the OOD datasets as well as a withheld test set of in-distribution images. The methods assign OOD score to each input. Similar to Hendryycks & Gimpel (2017). we evaluate these scores based on two objectives: the true-negative detection rate at 95% true-positive rate (TNR) and the area under the ROC curve (AUROC).
7

---

Under review as a conference paper at ICLR 2020
RESULTS. The results for housing price OOD detection are in Table I (first two sets of rows). We observe several trends. Firstly, the GMM detector substantially outperiorms Monte-Carlo dropout and ensembles the ImageNet, KITTI, and StreetView datasets. On ImageNet, the GMM detector achieves nearly perfect OOD detection rates. These datasets contain invalid inputs (i.e. inputs that do not contain houses), and therefore the out-of-distribution and in-distribution activations will be extremely different. The CA Houses dataset is the most challenging dataset for all of the detection methods. This is because this dataset represents covariate shifted data (i.e. inputs that contain different houses), and therefore, the OOD activations will be similar to in-distribution activations. Nevertheless, the GMM detector achieves a high TNR and AUROC that are comparable with the performance of an ensemble. In Figure (far left) we plot the network's prediction and the actual housing price for in-distribution houses and CA houses (log scale). Because the CA houses are OOD, the predictions are over an order of magnitude different than the actual housing price. Figure 5] (middle-lcft) compares the error and the OOD score assigned by the GMM predictor. We see that there is a correlation between the predictive error and the OOD score. If we threshold the OOD score to obtain 95% true- suee-ositive-rate, then the most erroneous samples will be labeled
Age estimation. We evaluate GMM-based OOD detection on another large-scall regression task: predicting a person's 's age from a portrait image. We take 20,000 images of people ages 20 and older from the FTKFaaee dataset (Zhang et al. (017). 5,000 images are withheld for validation and testing. Similar to the previous setup, the CNNs are pre-trained on ImageNet and the last layer is replaced with a linear layer. We compare against the same baselines using the same metrics.
For out-of-distribution inputs, we use images from the ImageNet and Street datasets. While both datasets do contain some images of people, most of the images are not portraits and therefore are OOD. Additionally, we use 10,000 frames from the Caltech Pedestrian dataset (Dollár et al. 2012). These images, which were taken from vehicle driving through an urban environment, contain candid images of pedestrians. Finally, we generate challenging OOD dataset using images of children 10 and younger from the UTKFace dataset (Child).
RESULTS. The results for age estimation are in Table (last two sets of rows). Similar to our previous task, the GMMs achieve very high OOD detection performance on all datasets. The GMM detector for DenseNet-121 outperforms Monte-Carlo dropout and ensembles on all benchmarks by significant margin. For ResNet-50, the GMM outperforms the other methods on all but one dataset with respect to both the TNR and AUROC metrics. The child age dataset is the most challenging OOD detection dataset for all methods. This is because the images are very similar to the training data. In [Figure 5] -hiiddle-right) we plot the network's 's prediction and the actual ages for adults (in- distribution) and children (OOD). The network is unable to correctly predict the ages of children. Figure (right) shows that the GMM's score can he used to identify the most potentially erroneous predictions on this dataset.
Other considerations. Feature-based OOD detection is computationally advantagcous method. Using a GMM for OOD detection requires a constant number of matrix-vector multiplications for each input. This is fraction of the cost of the neural network's prediction. It is worth noting that ensemble- based methods have benefits besides OOD detection, such as improved predictive performance. However, ensembles require multiple forward passes, which can be quite expensive.
CONCLUSION
Regression neural networks, unlike classification networks, output point prediction rather than a distribution over possible predictions. From this prediction alone, we demonstrate that is nearly im- possible to determine whether the network's 's input is anomalous or we argue that the network's hidden features indicate whether samples are in-distribution or not. We demonstrate that the distribution of neural network features p(h(x)) are intrinsically low dimen- sional and can be well-approximmted by a simple generative model, such as mixture of Gaussians. This is simple approximation, especially considering that the feature space has more than 1.000 dimensions. Nevertheless, we find that this approach is incredibly accurate at identifying OOD inputs, whether the inputs come from nonsensical distribution slightly-shifted distribution.
In this paper,
8

---

Under review as a conference paper at ICLR 2020
REFERENCES
Eman Ahmed and Mohamed Moustafa, House price estimation from visual and textual features. In NCTA, 2016.
Alexander A Alemi, Ian Fischer, and Joshua V Dillon. Uncertainty in the variational information bottleneck. In UAI, 2018.
Dario Amodei, Chris Olah, Jacob Steinhardt, Paul Christiano, John Schulman, and Dan Mané. Con- crete problems in Al safety. arXiv preprint arXiv: 06565, 2016.
Edmon Begoli, Tanmoy Bhattacharya, and Dimitri Kusnezov. The need for uncertainty quantifica- tion in machine-assisted medical decision making. Nature Machine Intelligence, 1:20-23, 2019.
Christopher M Bishop. Puttern recognition and machine learning. springer, 2006.
Charles Blundell, Julien Cornebise, Koray Kavukcuoglu, and Daan Wierstra. Weight uncertainty in neural network. In ICML, pp. 1613-1622, 2015.
Raghavendra Chalapathy and Sanjay Chawla. Deep learning for anomaly detection: survey. arXiv preprint arXiv:1901.08:: 00300,, 2019.
Hyunsun Choi, Eric Jang, and Alexander A Alemi. Waic, but why? Generative ensembles for robust anomaly detection. arXiv preprint arXiv:1810.01322. 2018.
Chow. On optimum recognition error and reject tradeoff. IEEE Transactions on information
theory, 16(1) (6(11):1-444 1970.
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In CVPR, 2009.
Piotr Dollár, Christian Wojek, Bernt Schiele, and Pietro Perona. Pedestrian detection: An evaluation of the state of the art. IEEE Transactions Pattern Analysis and Machine Intelligence, 34(4), 2012.
Eleazar Eskin. Anomaly detection over noisy data using learned probability distributions. In ICML, 2000.
Yarin Gal and Zoubin Ghahramani. Dropout as bayesian approximation: Representing model uncertainty in deep learning. In ICML, 2016.
Yarin Gal, Jiri Hron, and Alex Kendall. Concrete dropout. In NeurIPS, 2017.
Andreas Geiger, Philip Lenz, Christoph Stiller, and Raquel Urtasun. Vision meets robotics: The kitti dataset. The International Journal of Robotics Research, 32(11:1221-1233.201 2013.
lan Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley. Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial nets. In NeurlPS, 2014.
Chuan Guo, Geoff Pleiss, Yu Sun, and Kilian Q Weinberger. On calibration of modern neural networks. ICML, 2017.
Mehadi Hassen and Philip Chan. Learning a neural-network-hased representation for open set recognition. arXiv preprint arXiv:1802.04365. 2018.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. CVPR, 2016.
Dan Hendrycks and Kevin Gimpel. A baseline for detecting misclassified and out-of-distribution examples in neural networks. In ICLR, 2017.
Dan Hendrycks, Mantas Mazeika, and Thomas G Dietterich. Deep anomaly detection with outlier exposure. In ICLR, 2019.
Victoria Hodge and Jim Austin. A survey of outlier detection methodologies. Artificial intelligence review, 22(2):85-126, 2004.

---

Under review as a conference paper at ICLR 2020
Gao Huang. Zhuang Liu, Kilian Q Weinberger, and Laurens van der Maaten. Densely connected convolutional networks. In CVPR, 2017.
Xiaoqian Jiang, Melanie Osl, Jihoon Kim, and Lucila Mano-Ma Calibrating predictive model estimates to support personalized medicine. Journal of the American Medical Informatics Asso- ciation, 19:263-277 2012.
Alex Kendall and Yarin Gal. What uncertainties do need in bayesian deep learning for computer vision? In NeurIPS, 2017.
JooSeuk Kim and Clayton D Scott. Robust kernel density estimation. Journal of Machine Learning Research, 13(Spp):2529-2555, 2012.
Diederik Kingma and Max Welling. Auto-encoding variational bayes. In ICLR, 2014.
Durk P Kingma, Tim Salimans, and Max Welling. Variational dropout and the local reparameteri- zation trick. NeurlPS, 2015.
Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. Imagenet classification with deep convo- lutional neural networks. In NeurlPS, pp. 1097-1105, 2012.
Volodymyr Kuleshov, Nathan Fenner, and Stefano Ermon. Accurate uncertainties for deep learning using calibrated regression. In /CML, 2018
Balaji Laksmminarayannn, Alexander Pritzel, and Charles Blundell. Simple and scalable predictive uncertainty estimation using deep ensembles. In NeurIPS, 2017.
Guillaume Lample, Miguel Ballesteros, Sandeep Subramanian, Kazuya Kawakami, and Chris Dyer. Neural architectures for named entity recognition. NAACL-HILT, 2016.
Yann LeCun, Bernhard Boser, John S Denker, Donnie Henderson, Richard E Howard, Wayne Hub- bard, and Lawrence Jackel. Backpropagation applied handwritten zip code recognition. Neural Computation, 1(4):541-551 1989.
Kimin Lee, Honglak Lee, Kibok Lee, and Jinwoo Shin. Training confidence-calibratee classifiers for detecting out-of-distribution samples. In ICLR, 2018a.
Kimin Lee, Kibok Lee, Honglak Lee, and Jinwoo Shin. A simple unified framework for detecting out-of-distribuution samples and adversarial attacks. In NeurlPS, 2018b.
Shiyu Liang, Yixuan Li, and R Srikant. Enhancing the reliability of out-of-distribution image detec- tion in neural networks. In ICLR, 2018.
Wesley Maddox, Timur Garipov, Pavel Izmailov, Dmitry Vetrov, and Andrew Gordon Wilson. A simple baseline for bayesian uncertainty in deep learning. arXiv preprint arXiv: 2019.
Andrey Malinin, Anton Ragni, Kate Knill, and Mark Gales. Incorporating uncertainty into deep learning for spoken language assessment. In ACL, pp. 45-50, 2017.
Mare Masana, Idoia Ruiz, Joan Serrat, Joost van de Weijer, and Antonio M Lopez. Metric learning for novelty and anomaly detection. In BMVC, 2018
Volodymyr Mnih, Adrià Puigdomènech Badia, Mehdi Mirza, Alex Graves, Tim Harley, Timothy P. Lillicrap, David Silver, and Koray Kavukcuoglu. Asynchronous methods for deep reinforcement learning. In /CML, 2016.
Eric Nalisnick, Akihiro Matsukawa, Yee Whye Teh, Dilan Gorur, and Balaji Lakshminarayanan. Do deep generative models know what they don't 't know? In ICLR, 2019.
Stanislav Pidhorskyi, Ranya Almohsen, and Gianfranco Doretto. Generative probabilistic novelty detection with adversarial autoencoders. In NeurlPS, 2018.
Marco AF Pimentel, David A Clifton, Lei Clifton, and Lionel Tarassenko. A review of novelty detection. Signal Processing, 99:215-249, 2014.
10

---

Under review as a conference paper at ICLR 2020
Danilo Jimenez Rezende, Shakir Mohamed, and Daan Wierstra. Stochastic backpropagation and approximate inference deep generative models. In JCML, 2014.
Peter Schulam and Suchi Saria. Can you trust this prediction? auditing pointwise reliability after learning. In AISTATS, pp. 1022-1031, 2019.
Murat Sensoy, Lance Kaplan, and Melih Kandemir. Evidential deep learning to quantify classifica- tion uncertainty. In NeurlPS, 2018.
Akshayvarun Subramanya, Suraj Srinivas, and R Venkatesh Babu. Confidence estimation in deep 2017.
neural networks via density modelling. arXiv preprint
Masashi Sugiyama, Matthias Krauledat, and Klaus-Robert Muller. Covariate shift adaptation by importance weighted cross validation. Journal of Machine Learning Research, 8:985-1005, 2007.
Tomasz Tkoez. An upper bound for spherical caps. The American Mathematical Monthly, 119(7): 606-607, 2012.
Aäron Van Den Oord, Nal Kalchbrenner, and Koray Kavukcuoglu. Pixel recurrent neural networks. In ICML, 2016.
Yang Yu, Wei-Yang Qu, Nan Li, and Zimin Guo. Open-category classification by adversarial sample generation. In IJCAI, 2017.
Amir Roshan Zamir and Mubarak Shah. Image geo-localization based on multiplenearest neighbor feature matching usinggeneralizzed graphs. IEEE Transactions on Pattern Analysis and Machine Intelligence, (548):1546-155) 2014.
Zhifei Zhang, Yang Song, and Hairong Qi. Age progression/regression by conditional adversarial autoencoder. In CVPR, 2017.

---

Under review as a conference paper at ICLR 2020
A
ABLATION STUDIES
A.1
GENERATIVE MODELS
Here, consider three methods for modeling hidden features: kernel density estimation (KDE), Gaussian mixture models (GMM), and variational autoencoders (VAE). The GMM consists of 2-component mixture (2-GMM) with full covariance matrices. The hyper-paramelers for KDE and VAE are selected using a withheld in-distribution validation set. The model that produces the highest average -igeliiol is selected.
For KDE, we consider gaussian kemels with widthbands in {10-5, 10 10-4,..... ..... , Regarding VAEs, we train encoder and decoder networks with 3 and 4 hidden layers. The largest architecture consists of (800, 512, 256, 128) hidden units. We have also considered simpler architectures with (64, 32, 16). The networks are trained over 800 epochs using Adam with learning rate of 10-3 We also apply weight decay of 10-4 and batch normalization (batch size of 128).
Table 2 shows the performance of the generative models on housing price data. Overall, 2-GMM the best performing model. For the easiest task (ImageNet), VAE and 2-GMM achieve very similar performances in terms of both TNR at TPR 95% and AUROC. Also, for all datasets the difference between VAE and 2- 2-GMM is not higher than 2% AUROC
On the other hand, KDE performs poorly compared to VAE and 2-GMM. A possible explanation for this erratic behavior lies in the fact that KDE is very sensitive to the choice of kernel bandwidth. Since it uses an isotropic kernel, a large bandwidth may lead to underfitting. is worth emphasizing that by limiting the bandwidth to small values, KDE performance significantly increases, approxi- mating those from VAE and GMM.
Table 2 (last two sets of rows) shows results age estimation. Similar the results housing prices, GMM outperforms both KDE and VAE. The accuracy gap is notably higher for the RestNet network.
Table 2: Performance of generative models for OOD detection.
TNR at TPR 95%
AUROC
KDE
2-GMM
VAE
KDE
2-GMM
VAE
Network
OOD Dataset
ImageNet
0.645
0.999
0.997
0.781
0.999
0.999
KY Housing Price
KITTI
0.566
0.994
0.990
0.701
0.995
0.994
(DenseNet-121)
StreetView
0.529
0.931
0.846
0.678
0.982
0.972
CA Houses
0.387
0.553
0.550
0.640
0.923
0.915
ImageNet
0.705
0.996
0.994
0.833
0.999
0.998
KY Housing Price
KITTI
0.530
0.908
0.878
0.688
0.981
0.977
(ResNet-50)
StreetView
0.485
0,820
0.722
0,653
0.968
0.957
CA Houses
0.413
0.538
0.495
0.677
0.909
0.891
ImageNet
0.723
0.983
0.955
0.840
0.994
0.990
Adult Age Estimation
Streetview
0.734
0.984
0.957
0.839
0.995
0.987
121)
Pedestrian
0.854
1.000
1.000
0.911
0.999
0.999
Child Age
0.297
0.477
0.456
0.642
0.866
0.854
ImageNet
0.393
0.845
0.664
0.725
0.974
0.927
Adult Age Estimation (ResNet-50)
Streetview
0.297
0.758
0.513
0.721
0.958
0.899
Pedestrian
0.365
0.925
0.753
0.744
0.982
0.959
Child Age
0.336
0.475
0.471
0.662
0.878
0.873
A.2 NUMBER OF GMM MIXTURE COMPONENTS.
If fit the distribution of training set features with GMM, then the only hyperparameter is the number of mixture components. We find that OOD detection performance is actually quite robust to this hyperparameter.
12

---

Under review as a conference paper at ICLR 2020
OOD detection performance function of the GMM mixture size (1, 2, or 4). We display the TNR and AUROC for each GMM, as well as the GMM's slo likelihood on in in-distribution features. Blue columns correspond the model with the highest log likelihood (the model selection criterion). Best results are in bold.
Table 3:
TNR at TPR 95%
AUROC
1-GMM
2-GMM
4-GMM
1-GMM
2-GMM
4-GMM
Network
OOD Dataset
Log Lik. (In-dist.)
2924
2970
2991
2924
2970
2991
KY Housing Price
ImageNet
0.998
0.999
0.999
0.999
0.999
0.999
(DenseNet-12l)
KITTI
0.992
0.994
0.994
0.994
0.995
0.995
StreetView
0.914
0.931
0.936
0.980
0.982
0.982
CA Houses
0,510
0.553
0.568
0.918
0.923
0.928
Log. Lik. (In-dist.)
3864
3838
3903
3864
3838
3903
KY Housing Price
ImageNet
0.997
0.996
0.998
0.999
0.999
0.999
(ResNet-50)
KITTI
0.939
0.908
0.932
0.984
0.981
0.983
StreetView
0.844
0,801
0.868
0.973
0.967
0.975
CA Houses
0.482
0,538
0.574
0.904
0.908
0.924
Log Lik. (In-dist))
2563
2589
2588
2563
2589
2588
Adult Age Estimation
ImageNet
0.986
0.983
0.964
0.996
0.994
0.993
(DenseNet-121)
Streetview
0.989
0.984
0.997
0.995
0.994
Pedestrian
1.000
1.000
1.000
1.000
0.999
0.999
Child Age
0.416
0.477
0.427
0.852
0.866
0.860
Log Lik. (In-dist.)
2630
2552
1876
2630
2552
1876
Adult Age Estimation
ImageNet
0.937
0.845
0.830
0.987
0.974
0.971
(ResNet-50)
Streetview
0.908
0.758
0.766
0.979
0.959
0.962
Pedestrian
0.986
0.924
0.906
0.993
0.982
0.981
Child Age
0.434
0.475
0.468
0.867
0.878
0.880
we display the OOD detection results for GMMs with 1, 2, and 4 mixture components. In addition, we report the log likelihood of the GMM on the in-distribution validation data, which the selection criterion. The GMMs with the highest log likelihood for a given model are displayed in blue.
In Table 3
From this figure, we can observe number of trends. Firstly, we notice that more mixture compo- nents does not always correspond to better model. 4 mixture components achieves the highest log likelihood on the housing price networks; however, fewer components is better for age estimation. Nevertheless, the log likelihood functions as a good model selection criterion. The GMMs with the highest log likelihood tend to achieve the best OOD detection performance (or comparable to the best performance) both in terms of TNR and AUROC.
Surprisingly, find the log likelihood is relatively robust to the number of mixture components. For example, the housing price ResNet's s log likelihood values vary between 3838 and 3093. Similarly, the age estimation DenseNet's log likelihood values vary between 2563 and 2589. These similar likelihoods correspond to similar OOD detection performance. On all OOD detection benchmarks, the models achieve AUROCs that are within one percentage point of each other. The only exper- iment with highly varying log likelihood is the age estimation ResNet. The 4-component GMM obtains significantly lower log likelihood than the 2- or 1-component mixtures. This low log likelihood corresponds worse OOD detection performance. The 2- and I-component mixtures perform significantly better on most OOD tasks than the 4-component mixture.
B PROOF OF REMARK
Proof. We assume that the features h(x) are drawn from uniform measure Euclidean ball of radius r. The prediction corresponding to these features is given by ÿ = w wxx), where w is the weight of the network's linear layer. In order for for some it must be the case that h(x) lies
13

---

Under review as a conference paper at ICLR 2020
in the hyperspherical cap defined by the set of points w w>h(x) |[(())]2 $ Le.
Volume cap
p > e)
Volume of ball
The hypervolume of the Euclidean ball in a d dimensional space is given by π(4/2413. where I' represents the gamma function. While there exact formula for the for the hypervolume of cap, we instead choose to use the more interpetable bound of Tkoz 2012):
4//2,/
de²
Volume of cap exp
2r |w
Finally, by symmetry we have that p(ù > c) = p(ğ < c). Putting this all together, we have that
p(lil > >) 2T 1)
de
exp
2d/2
2r |r>||
C EXPERIMENTAL DETAILS
C.I ROTATED MNIST EXPERIMENTS
The amount of rotation applied to each input is drawn from the distribution (N0, x/4). As a result, roughly 95% of the samples are rotated between - 2 and n/2 degrees.
The ResNet architecture the depth experiments based on the architecture He et al. (0066)
The first convolutional layer is reduced to accept single-channel inputs, and the ReLU activations are replaced with t tanh activations. The LeNet architecture follows that LeCun et al. (1989). All networks are trained with SGD for 50 epochs, using initial learning rate of 0.1 that is dropped by 10 after 25 and 37. We compute the covariances matrices Σ and Σ using features extracted from the training set.
C.2 TOY EXAMPLE
The training data consists of 22 input samples in [0. 1|2 and the corresponding outputs =((x + where f(xx sin sin (2xxx (22x)))sn((2r))) )sin(2nx.)) and e ~ (00 0.12). OOD inputs are represented by 200 points from the Gaussian distribution. ([1.5, 1.5), 0.1 I). An additional set of 200 in-distribution points used for validation.
The prediction network consists of overparametrized MLP with 4 hidden layers MLP(2, 20, 32, 20, 2, 1) - and ReLu activation functions. The network is trained over 400 epochs using Adam with learning rate of 10 and weight decay of 10 4, We also apply batch normaliza- tion.
C.3 HOUSING PRICE
Each network is trained to minimize the mean-squared error loss for 100 epochs using SGD with learning rate of 0.001 and weight decay of 10-1, All networks achieve a scaled mean-squared error between 0.28 and 0.29. Based on the validation log likelihood, we select 4-component GMM to model the DenseNet and ResNet features (see Section A.2
C.4 AGE ESTIMATION
All networks achieve scaled mean-squared error between 0.18 and 0.19. Based on the validation log likelihood, select 2-component GMM model the DenseNet features and single Gaussian to model the ImageNet features (see Section A.2).
14


Reviews:
[{"review_id": "ryl4xqCRKH", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This work is tackling the problem of doing out-of-distribution detection in regression. The core idea is to fit a simple generative model to hidden activations and use thresholded likelihood to decide whether an input is in or out of distribution. Using a simple generative model for this is motivated by the intrinsic low-dimensionality of hidden activations.\n\nMy issue with the narrative of this paper is two-fold. \n\nFirst, the intrinsic low-dimensionality of the hidden activation distribution seems entirely problem-dependent. One can train networks with orthonormal layers and therefor flat spectra that work fine and there are also invertible discriminative models that do not discard any information from layer to layer, albeit being very similar to standard ResNets. It might be that this heuristic works well for the problems discussed in the paper, but I don't see any reason for this to hold in general.\n\nSecond, Nalisnick et al. don't show that deep generative models are often overconfident (as stated in the paper here). What they show is that likelihood can be higher for datasets with lower entropy than the one the deep generative model has been trained on. They also show that \"graying\" inputs increase the likelihood and they can even show this analytically for flow models with data-independent determinant of their Jacobian. The GMM approach proposed here is thus a special case of what Nalisnick et al. described and should have the same problem. Reducing the variance of the features should increase the likelihood.\n\nAll in all, the presented method relies too much on heuristics that seem specific to the analyzed architectures and datasets and there is no theoretical reason for this to hold in general as far as I can see.\n\nThe originality of the paper is also somewhat limited, as generative models for OOD detection are well-studied. Thresholded likelihoods are problematic in many settings, so I am not convinced this approach will work well across the board.\n\n----\n\nEric Nalisnick, Akihiro Matsukawa, Yee Whye Teh, Dilan Gorur, and Balaji Lakshminarayanan. Do deep generative models know what they don’t know? In ICLR, 2019.", "extra_scores": {"experience_assessment": "I have published one or two papers in this area.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper thoroughly.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I carefully checked the derivations and theory."}, "extra_text": {}}, {"review_id": "SygZYijTFH", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The authors present the empirical observation that regression models trained using weight decay tend to produce high-level features that lie on low-dimensional subspaces. They then use this observation to propose an algorithm for detecting out of distribution data by fitting a simple GMM to the features produced during training.\n\nThe main contribution of the paper is a way of calculating the \"OOD scores\" that are reported in the different figures. However, after reading the paper 3 times I'm still unable to find a definition of these scores. I'm guessing it's negative log-likelihood under the fitted GMM, but I'm unsure.\n\nThe observation (features on low-dimensional manifolds) is interesting, and the proposed algorithm is potentially useful. However the experiments are quite limited, with only 2 training datasets. The analysis of why/when the algorithm is expected to work and how this depends on the model is shallow. The two main results are for models pre-trained on Imagenet: presumably this has a huge impact on the features that end up being used for OOD detection, but this is not discussed in the paper at all.\n\nQuestions to the authors:\n- Why focus only on regression? It seems to me that your analysis for the features lying on a low-dimensional subspace should also apply to classification models.\n- How do your results depend on the specific models that are used? I can come up with models with very few final features where your method would not work. Do you have any analysis or guidance?\n- What do all abbreviations in the tables mean? Please make the captions more informative so the reader does not need to search in the main text.", "extra_scores": {"experience_assessment": "I have published one or two papers in this area.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper at least twice and used my best judgement in assessing the paper.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I assessed the sensibility of the derivations and theory."}, "extra_text": {}}, {"review_id": "BJxg_FwwqB", "reviewer": "AnonReviewer4", "summary": "", "questions": "", "limitations": "", "rating": 1, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper proposes to detect inputs that are from a slightly shifted distribution (eg images of houses in CA instead of KY) or from a very shifted distribution (eg imagenet images) by fitting a density model to the last layer h(x) of an MLP trained with squared error, and using the likelhiood score p(h(x)) as a metric. This is a reasonable idea. However, it is not novel eg Aigrain'19 did essentially the same thing for classification models. (The difference between classification and regression is a trivial change to the loss function, and does not change the fundamental idea.)\n\nIn addition to lack of novelty, the experimental methodology is very weak. First, the toy 2d example is too trivial to be informative, since there is essentiallly no overlap between the two distributions of features, p(x) and q(x) - even a density model on input space could detect this. More importantly, the results on the two image datasets are suspect. First, it seems that using the predictive variance sigma(x) as the reliability metric (the \"var\" method) - which is totally standard approach known as 'heteroskedastic regression'. - works very well in several cases. I suspect when it fails it is due to  implementation problems (eg trying to predict sigma instead of log(sigma)). Also ensembles are known to be very robust to distribtution shift (see eg Ovadia'19), so  I am surprised at their poor performance. Another problem is that the datasets used are not standard, so it is impossible to compare to other papers. Finally, no error bars are reported, so it is hard to know if any of the results are statistically significant. \n\n\n\nJ. Aigrain and M. Detyniecki, “Detecting Adversarial Examples and Other Misclassifications in Neural Networks by Introspection,” in ICML Workshop on Uncertainty and Robustness in Deep Learning, 2019 [Online]. Available: http://arxiv.org/abs/1905.09186\n\n\nY. Ovadia et al., “Can You Trust Your Model’s Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shift,” arXiv [stat.ML], 06-Jun-2019 [Online]. Available: http://arxiv.org/abs/1906.02530\n\n\n\n", "extra_scores": {"experience_assessment": "I have read many papers in this area.", "review_assessment:_checking_correctness_of_derivations_and_theory": "N/A", "review_assessment:_checking_correctness_of_experiments": "I did not assess the experiments.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper at least twice and used my best judgement in assessing the paper."}, "extra_text": {}}, {"review_id": "B1xBgmDPcH", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 1, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "===Summary===\n\nThe authors propose to perform out of distribution detection for regression models by fitting a generative model in the feature space of the regression model. An input example is deemed to be out-of-distribution if it has low likelihood under this generative model.\n\n===Overall Assessment===\n\nI recommend that the paper is rejected. There are a number of aspects that need to be improved. You should fix these and resubmit to a future conference. \nThe paper focuses on the difference between regression and classification tasks and claims that the paper's method addresses an unmet need for OOD for regression. However, both the proposed method and the analysis justifying it are generic enough to be applied to both regression and classification.\nThe paper handles technical claims far too casually in sec 4 and does not provide sufficient justification that the claims are true.\nThere are natural baselines, such as using a generative model on the raw input space, that are ignored.\n\n===Comments===\n\nRemark 1 feels to me like it was added for the sake of having more math in the paper, not because it is crucial to the paper's argument. \n\nYou remark at various places that existing methods don't naturally generalize from classification to regression. However, you never fully explain why. Also, your proposed method can be applied out-of-the box to classification problems. Your analysis in sec 4 trivially applies to binary classification tasks, and could be naturally extended to multi-class classification where w is not a vector but a num_classes x num_features matrix. \n\nThe parallel should be between classification and heteroskedastic regression, since there you have a distribution per example.\n\nThe logic in \"In-distribution features are intrinsically low dimensional\" is insufficient\n\nThe connection between section 4 and your proposed method is not particularly precise. You also have lots of technical claims in 4 that are unsubstantiated. For example, you write \"this new network will likely have less discarded information than the shallower network\". What does 'likely' mean? In what sense are you making an actual technical statement? Each of the subsections in sec 4 has similar issues.\n\"The CNNs are pre-trained on ImageNet (Denget al., 2009) and the last layer is replaced with a linear layer that produces a single output.\"\nWhy did you do this? Did you fine tune or just retrain the top layer?\n\n\"For these two baselines, the variance of the forward passes is used as a metric for detecting OOD inputs\" \n   Can you explain why these are reasonable baselines for OOD? Why no baseline that fits a generative model in input space?\n\nYou should cite Ren et al. \"Likelihood Ratios for Out-of-Distribution Detection\"\n", "extra_scores": {"experience_assessment": "I have published one or two papers in this area.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I assessed the sensibility of the derivations and theory.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper at least twice and used my best judgement in assessing the paper."}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper investigates out-of-distribution detection for regression tasks.

The reviewers raised several concerns about novelty of the method relative to existing methods, motivation & theoretical justification and clarity of the presentation  (in particular, the discussion around regression vs classification). 

I encourage the authors to revise the draft based on the reviewers’ feedback and resubmit to a different venue.

Decision: Reject