Title: NEURAL NETWORK OUT-OF-DISTRIBUTION DETECTION FOR REGRESSION TASKS

Abstract: Neural network out-of-distribution (OOD) detection aims to identify when a model is unable to generalize to new inputs, either due to covariate shift or anomalous data. Most existing OOD methods only apply to classification tasks, as they assume a discrete set of possible predictions. In this paper, we propose a method for neural network OOD detection that can be applied to regression problems. We demonstrate that the hidden features for in-distribution data can be described by a highly concentrated, low dimensional distribution. Therefore, we can model these in-distribution features with an extremely simple generative model, such as a Gaussian mixture model (GMM) with 4 or fewer components. We demonstrate on several real-world benchmark data sets that GMM-based feature detection achieves state-of-the-art OOD detection results on several regression tasks. Moreover, this approach is simple to implement and computationally efficient.

1. INTRODUCTION:

The success of deep neural networks in many domains (Krizhevsky et al., 2012; Lample et al., 2016; Mnih et al., 2016) is due to their ability to learn complex functions that generalize to new samples. However, this observed generalization only extends to data that are sufficiently similar to the training data. If the neural network encounters data that deviates from the distribution of training data, its predictions are likely to be erroneous or nonsensical (Guo et al., 2017; Jiang et al., 2012; Begoli et al., 2019) . This may occur if the model is used in scenarios that experience covariate shift (Sugiyama et al., 2007) or if the model encounters previously-unseen categories of data (Yu et al., 2017; Hassen & Chan, 2018) . Such scenarios are examples of out-of-distribution (OOD) inputs.

Ideally we would like for neural networks to adapt to such shifts in the data distribution (Amodei et al., 2016) . In the absence of such adaptation, out-of-distribution detection should be used to identify when a model is unable to generalize to a previously-unseen input. While there are several proposed methods for neural network OOD detection (Hendrycks & Gimpel, 2017; Liang et al., 2018; Lee et al., 2018b) , many of these methods rely on architectural components specific to classification neural networks. Consequentially, they cannot be applied to regression problems. Regression neural networks typically output only a point prediction rather than a predictive distribution, and thus the output does not indicate its uncertainty or reliability for a given input. This is illustrated in Figure 1 , which displays predictions from a network trained to predict prices of middle-class houses in Kentucky. The network outputs a house price prediction for any possible input -even for outof-distribution images like a California mansion or chair. These predictions fall within the normal range of possible prices, and therefore do not convey that the inputs are not valid for this model.

Because the predictions cannot identify OOD inputs, we must look for alternative signals. Previous approaches perform regression OOD detection through ensembles (Gal & Ghahramani, 2016; Lakshminarayanan et al., 2017) or through an additional uncertainty prediction layer (Kendall & Gal, 2017; Malinin et al., 2017) . In this paper, we instead turn to the space of hidden features. During training a neural network learns to extract relevant features about the training data and discards irrelevant information. Whether or not a network generalizes to a given test sample depends on the extracted features from that sample. Networks do not generalize to out-of-distribution data because the distributional shift causes the network to extract the wrong information. For example, when the housing neural network is applied to the California mansion in Figure 1 , the network's features do not extract the relevant information that would indicate the true price of the house. (e.g. presence of Figure 1 : A neural network trained to predict house prices in Kentucky from curb-side images. Although the algorithm generalizes well to within-distribution test images (left two houses), it has high error for (out-of-distribution) mansion from California and is forced to make a nonsensical prediction for the armchair. Both latter cases are detected as out-of-sample with high OOD scores. palm trees). In other words, the extracted features of the California mansion will differ significantly from training data features and therefore the network does not generalize.

Based on this intuition, we investigate how to utilize the hidden features of regression neural networks for OOD detection. In particular, we make several contributions. First, we argue empirically and theoretically that we cannot identify OOD inputs simply using only the networks' predictions. However, we then demonstrate that the features of in-distribution inputs lie on an intrinsically lowdimensional portion of the feature space. It is unlikely that OOD inputs map to similar locations in feature space because of this low-dimensionality. We additionally show that, because of this lowdimensionality, it is possible to model the in-distribution features with simple generative models.

To evaluate our proposed approach, we develop new OOD detection tasks based on large-scale computer vision regression datasets. We evaluate the OOD detection performance of several generative models trained on in-distribution features. Surprisingly, we find that a simple mixture of Gaussiansoften with no more than 2 components -is better at OOD detection than more complex models such as variational autoencoders. Finally, we demonstrate that GMM models of in-distribution features are able to outperform other regression OOD methods across several benchmarks.

2. RELATED WORK:

Out-of-distribution detection and the related problems of outlier detection (Hodge & Austin, 2004; Chalapathy & Chawla, 2019) and novelty detection (Pimentel et al., 2014) are well-studied in statistics and machine learning. Arguably, the most straightforward method to these problems is generatively modeling the distribution of inputs p(x) using a parametric distribution (Chow, 1970; Eskin, 2000) or a nonparametric density estimate (Kim & Scott, 2012) . If p(x) is small for a given input, then it is likely out-of-distribution or an anomaly. Several recent works (Choi et al., 2018; Nalisnick et al., 2019; Pidhorskyi et al., 2018) have suggested identifying OOD inputs with deep generative models (Goodfellow et al., 2014; Kingma & Welling, 2014; Rezende et al., 2014; Van Den Oord et al., 2016) . One challenge with these methods is that deep generative models cannot necessarily model large-scale images or other complex distributions (Hendrycks et al., 2019) and may be overconfident when modeling input data (Nalisnick et al., 2019) .

Rather than directly modeling the input distribution, the approach proposed in this paper operates on feature spaces using a mixture model to capture potentially anomalous inputs. This falls under the category of model-dependent OOD detection methods, which are ideal when the input distribution is too complex to model with generative methods (e.g. for high resolution images) (Hendrycks et al., 2019) . Several model-dependent methods have been proposed for classification neural networks.

The training procedure of a classification network can be modified to include uncertain samples (Lee et al., 2018a) or to discourage overconfidence through an alternative loss function (Alemi et al., 2018; Masana et al., 2018; Subramanya et al., 2017; Sensoy et al., 2018) . Alternatively, OOD metrics can be constructed for classification based on the softmax probability output (Hendrycks et al., 2019; Liang et al., 2018) or related uncertainty scores (Schulam & Saria, 2019) . In a similar vein to our proposed approach, (Lee et al., 2018b) use a class-conditional Mahalanobis distance for classification OOD. All of these methods are designed specifically for multi-class classification problems -typically relying on the discrete output space. Consequentially, there are typically no straightforward extensions of these methods to regression problems.

There are fewer proposed methods for OOD detection on regression tasks. Most existing approaches rely on an estimate of the neural network's predictive uncertainty (Kuleshov et al., 2018) . There are several proposed approaches to obtain uncertainty estimates from regression neural networks. One class of approaches is to add an additional output that predicts a confidence interval for the network's prediction (Lakshminarayanan et al., 2017; Kendall & Gal, 2017; Malinin et al., 2017) . This layer can be trained in conjunction with the predictive output to minimize the negative log likelihood on the training set. Alternatively, Bayesian approaches to deep learning (Blundell et al., 2015; Gal & Ghahramani, 2016; Gal et al., 2017; Kingma et al., 2015) can be used to estimate predictive uncertainty on regression tasks. These approaches typically approximate the posterior distribution of neural networks parameters through an ensemble of models. Non-Bayesian ensembling approaches achieve can produce similar uncertainty estimates (Lakshminarayanan et al., 2017; Maddox et al., 2019) . In this paper we propose an orthogonal approach to regression OOD detection -using the distribution of hidden features rather than the model's predictive uncertainty. Out-of-distribution data (blue and green lines) obtain similar predictions.

3. CHARACTERIZING OOD PREDICTIONS ON REGRESSION TASKS:

In the classification setting, the output of a neural network is a softmax score for each class. Because this output can be interpreted as a probability distribution over classes, it represents some notion of the neural network's uncertainty and therefore can be used to identify inputs that are likely to be OOD (Hendrycks & Gimpel, 2017; Liang et al., 2018) . For most regression architectures however, the output is typically a point prediction ŷ which does not convey any notion of predictive uncertainty. Moreover, in this section we show that these point predictions ŷ are insufficient for differentiating between in-distribution and out-of-distribution inputs.

To illustrate this, we train a neural network to predict the price of homes in Kentucky based on a front facing picture. Figure 2 (left) displays a histogram of the network's predictions ŷ on a withheld validation set (black line). The distribution of these predictions corresponds to the distribution of actual house prices (dotted line). However, if the same network receives an OOD input, it predicts price values that lie in the same range as the in-distribution predictions. For example, if houses from California are input into the neural network, the predicted housing prices (blue line) are roughly the same as that of Kentucky houses despite these houses costing up to 10 times as much. The network predicts similar values for Imagenet images (orange line) even though these images do not contain any houses. Based on the ŷ alone, it is not possible to determine that these inputs are not valid.

The fact that the OOD predictions are similar to in-distribution predictions can be explained by a simple curse-of-dimensionality argument. Let φ be a regression neural network that maps an input x ∈ X to an output ŷ ∈ R. The last layer of φ produces a set of activations h(x) that are linearly mapped to the one-dimensional prediction: ŷ = w h(x). Let's assume that the features h(x) for some OOD inputs can be bounded within some Euclidean ball of radius r. If the dimensionality of h(x) is large, then the probability of an extreme prediction becomes exponentially unlikely. Remark 1. Let h(x) be features drawn from a uniform measure on a Euclidean ball of radius r. Let d be the dimensionality of h(x). The probability of a large prediction |ŷ| > is bounded by (See Appendix B for proof). In practice it is not necessarily the case that OOD features will be uniformly distributed. Nevertheless, this remark illustrates that predictive outputs discard information that would indicate whether a sample is OOD or not.

p(|ŷ| > ) ≤ 2Γ(d/2 + 1) π d/2 r d exp -d 2 2r 2 w 2 2 .

4. CHARACTERIZING THE DISTRIBUTION OF HIDDEN FEATURES:

Although the network's predictions look very similar for in-distribution and OOD inputs, we expect that their hidden features h(x) are very different. Since ImageNet pictures do not contain frontfacing houses, the network will extract very different types of features. We similarly expect the activations of California houses to be different since the network is unable to generalize these inputs. To that end, we hypothesize that a generative model of in-distribution features p(h(x)) will distinguish between in-and out-of-distribution inputs. In this section we demonstrate that -despite the high dimensionality of h(x) -the in-distribution features are very tightly concentrated. As a result, it is possible to model in-distribution features with a simple generative model.

In-distribution features are intrinsically low dimensional. In the regression setting, the network's prediction is the inner product between the features and a weight vector: ŷ = w h(x). Any feature information that is orthogonal to the weight w will not contribute to the prediction and therefore is "discarded" information. We can describe a feature vector h(x) in terms of the prediction ŷ and the discarded information h ⊥ (x):

h(x) = ŷw + h ⊥ (x), h ⊥ (x) = I -ww h(x),

where h ⊥ (x) is the discarded information, computed using the orthogonal projection of w. Also, we assume, without loss of generality, that w 2 = 1. It is worth emphasizing that "useful" predictive information ŷw lies within a single dimension of the d-dimensional h(x) vector (where d is often greater than 500). Though the "discarded" information h ⊥ (x) occupies d-1 dimensions, it typically has low intrinsic dimensionality for in-distribution data as we demonstrate below. Consequentially, the in-distribution features h(x) = h ⊥ (x) + ŷw can be described by a low-dimensional distribution. We identify two factors that are correlated with this low dimensionality: network capacity and regularization.

Network capacity reduces the relative size of h ⊥ (x). As network capacity increases, we expect that there will be less "discarded" information h ⊥ (x). To understand why this is the case, assume there is a l-layer network in which there is some correlation between the discarded features and the true labels. Now assume we add an additional layer before the linear layer which computes a new set of hidden features h (x). This new layer can improve the network's predictions if it uses information in h ⊥ (x) to augment the original prediction ŷ. Consequentially, this new network will likely have less discarded information than the shallower network.

While this is just an illustrative example, we can empirically demonstrate that network capacity does in fact decrease the amount of discarded information. First, we introduce the following metric of discarded information: let Σ be the empirical covariance matrix of h(x) for in-distribution data, and let Σ ⊥ be the covariance of h ⊥ (x). We measure the amount of discarded information by comparing the maximum singular values of Σ and Σ ⊥ . If Σ ⊥ 2 / Σ 2 is close to 1, then the feature vectors are comprised mostly of discarded information. Conversely, h(x) will consist primarily of predictive information if Σ ⊥ 2 / Σ 2 is close to zero.

We apply this metric to a simple regression task using the MNIST dataset. Each input image is rotated by an amount between -90 and 90 degrees. We train residual networks with increasing depth to predict the amount of rotation in each image (details in Appendix C). Figure 3 (left) plots the singular value ratio Σ ⊥ 2 / Σ 2 as a function of network depth. The ratio is quite large for shallow networks, but decreases with depth. This suggests that, for deep networks, most of h(x) lies on the one-dimensional subspace defined by the weight vector w.

Regularization reduces the dimensionality of h(x). Applying weight decay to a neural network reduces the magnitude of the weights, which in turn reduces the magnitude of feature vectors. Reducing the magnitude of h(x) while maintaining predictive power will result in feature vectors with little discarded information h(x). Since all but one dimensions of the feature space are discarded, we expect that the intrinsic dimensionality of features will decrease with increasing regularization. To support this intuition we train the LeNet model on rotated MNIST with various amounts of weight decay. In Figure 3 (center) we plot the spectrum of the in-distribution feature covariance matrix Σ for these different models. We see that all covariance matrices are dominated by a few large singular values. However, the spectra of the high regularization models decay much more rapidly, suggesting that regularization simplifies the distribution of hidden features.

The fact that in-distribution inputs occupy a low-dimensional portion of high-dimensional feature space is extremely advantageous. If we draw a random set of random features h(x) (e.g. from a uniform measure on a Euclidean ball), it will be highly unlikely that these features occupy the same low-dimensional space. Though OOD features are not truly random vectors in practice, we find empirically that they do not occupy the same low dimensional subspace. To demonstrate this, we fit a multivariate Gaussian to the training set features from the housing price network. We can decompose the fit of this Gaussian as a product of its principle components and measure each component's likelihood for in-and out-of-distribution data. Figure 3 (right) displays the fits for the housing price validation data (in-distribution) and ImageNet samples (OOD). We find that indistribution and OOD data are equally likely for the main principal components. However, OOD features do not fit the smaller principal components well, and thus these small principal components are able to distinguish in-and out-of-distribution data. In this case the curse-of-dimensionality makes it possible to distinguish OOD and in-distribution features.

5. OOD DETECTION WITH GENERATIVE MODELS OF HIDDEN FEATURES:

As we demonstrate in the previous section, the distribution of in-distribution hidden features h(x) is intrinsically low dimensional. Since OOD data are unlikely to share the same low-dimensional distribution, we propose using generative models of in-distribution h(x) to identify OOD inputs.

Possible generative model. While a simple Gaussian can model the distribution h(x), it is possible that a more complex generative model will fit better. There are several choices of generative models, arguably the simplest choice are parametric models such as Gaussian mixture models (GMMs), which can approximate any distribution with enough mixture components Bishop (2006) . Nonparameteric models, such as kernel density estimation (KDE), offer complexity that scales with the number of observations. Deep generative models such as variational autoencoders (VAE) (Kingma & Welling, 2014; Rezende et al., 2014) are arguably the most powerful class of generative models.

Model fitting. After training the predictive neural network, a generative model is fit to indistribution features. In all experiments, we use features extracted from the neural network's training data. However, we note that features from any (unlabeled) in-distribution data could be used instead.

Model selection and hyperparameter tuning. Tuning hyperparameters can be challenging because one typically does not have access to OOD data. For our method we must determine 1) which generative model to use and 2) the hyperparameters of the generative model. We select these hyperparameters using the generative model's log-likelihood on the validation set: log p(D valid ) =

x log p(h(x)). From a decision theoretic standpoint, the model with the largest likelihood fits the validation data with the smallest distribution (Bishop, 2006) . If the distribution is strongly peaked then it has less support for samples that are not in-distribution, and therefore will be good for OOD detection. Importantly, this criterion only requires access to in-distribution data.

Comparing the Performance of Generative Models. In Section A.1 we examine how the choice of generative model affects OOD detection performance. We compare KDE models, VAEs, and 2-component GMMs on several benchmark OOD tasks (which are described in detail in the next section). Surprisingly, we find that GMMs detects OOD inputs as well -if not better -than the other models. This is especially surprising because GMMs tend not to perform well in high dimensional spaces (Bishop, 2006) , which further suggests that in-distribution features are highly concentrated.

6. EXPERIMENTS:

In this section, we demonstrate the performance of our OOD detection method on several regression tasks. We first illustrate the method on a 2D toy example, and then we evaluate the method on two large-scale computer vision regression tasks. For experimental details, we refer the reader to Appendix C. Based on the ablation study in Section A.1, we model the in-distribution features with a GMM. The number of components in the GMM is chosen through model selection with the validation dataset. (The effect of the GMM size is explored in Section A.2.) Smoothed XOR. To illustrate how feature spaces can be used for detecting OOD samples, we consider a simulated regression task. The goal is to approximate the smoothed XOR function from noisy labeled data. In-distribution inputs consist of Uniform samples in [0, 1] 2 , while OOD inputs come from an isotropic Gaussian distribution centered at [1.5, 1.5] with variance σ 2 = 0.1. Figure 4 (left) depicts the proposed setting, i.e. the target XOR function and in-dist./OOD samples. Here, we use multilayer perceptrons (MLP) as the predictive model. Figure 4 (middle) plots predictions for both OOD and in-distribution (validation) samples. Note that this corresponds to the desired scenario in which one cannot identify OOD inputs based on the predictions alone. The proposed approach relies on the assumption that hidden features are informative, and their distribution can be approximated using simple generative models. In this regard, Figure 4 (right) shows the fit from a Gaussian on the activations at the last nonlinear layer of the prediction network. There is virtually no overlap between in-distribution and OOD features. As a consequence, we achieve perfect detection, as represented by an area under the ROC curve (AUROC) equals to 1.

Housing price prediction. We demonstrate our proposed method on a large-scale regression task: predicting home prices from front-facing house pictures. We train a 121-layer DenseNet (Huang et al., 2017) and a 50-layer ResNet (He et al., 2016) on a dataset of nearly 50,000 middle-income house images from Fayette County in Kentucky. The CNNs are pre-trained on ImageNet (Deng et al., 2009) and the last layer is replaced with a linear layer that produces a single output. We compare the GMM-based detector against methods that use ensemble or Bayesian model averages to estimate predictive uncertainty. Monte-Carlo Dropout (MCD) (Gal & Ghahramani, 2016; Kendall & Gal, 2017) creates a Bayesian ensemble of 10 neural networks, where each network is sampled from a base network with dropout (p = 0.2). Deep Ensembles (Ens.) (Lakshminarayanan et al., 2017) is an ensemble of 4 standard neural networks. For these two baselines, the variance of the forward passes is used as a metric for detecting OOD inputs. Var. (Malinin et al., 2017) adds a variance output to the neural network to predict uncertainty estimates. Var. + MCD (Kendall & Gal, 2017) combines this variance estimate with MC-Dropout.

Following Hendrycks & Gimpel (2017) , we use several external datasets as sources for out-ofdistribution inputs. Firstly, ImageNet (Deng et al., 2009) is a dataset of photos from 1,000 classes. This dataset contains easily detectable out-of-distribution inputs, since most of the pictures do not contain houses. KITTI (Geiger et al., 2013) is a dataset of self-driving car images. These images may contain some houses, but any houses will be in the periphery of the image rather than being centered. StreetView (Zamir & Shah, 2014) contains images from Google street view, which may also contain some frontal house views. The most challenging dataset is CA Houses (Ahmed & Moustafa, 2016) , which consists of house images from California. METRICS. We apply each detection method to the OOD datasets as well as a withheld test set of in-distribution images. The methods assign an OOD score to each input. Similar to Hendrycks & Gimpel (2017) , we evaluate these scores based on two objectives: the true-negative detection rate at a 95% true-positive rate (TNR) and the area under the ROC curve (AUROC).

RESULTS. The results for housing price OOD detection are in Table 1 (first two sets of rows). We observe several trends. Firstly, the GMM detector substantially outperforms Monte-Carlo dropout and ensembles on the ImageNet, KITTI, and StreetView datasets. On ImageNet, the GMM detector achieves nearly perfect OOD detection rates. These datasets contain invalid inputs (i.e. inputs that do not contain houses), and therefore the out-of-distribution and in-distribution activations will be extremely different. The CA Houses dataset is the most challenging dataset for all of the detection methods. This is because this dataset represents covariate shifted data (i.e. inputs that contain different houses), and therefore, the OOD activations will be similar to in-distribution activations. Nevertheless, the GMM detector achieves a high TNR and AUROC that are comparable with the performance of an ensemble. In Figure 5 (far left) we plot the network's prediction and the actual housing price for in-distribution houses and CA houses (log scale). Because the CA houses are OOD, the predictions are over an order of magnitude different than the actual housing price. Figure 5 (middle-left) compares the error and the OOD score assigned by the GMM predictor. We see that there is a correlation between the predictive error and the OOD score. If we threshold the OOD score to obtain 95% true-positive-rate, then the most erroneous samples will be labeled out-of-distribution.

Age estimation. We evaluate GMM-based OOD detection on another large-scale regression task: predicting a person's age from a portrait image. We take 20,000 images of people ages 20 and older from the UTKFace dataset (Zhang et al., 2017) . 5,000 images are withheld for validation and testing. Similar to the previous setup, the CNNs are pre-trained on ImageNet and the last layer is replaced with a linear layer. We compare against the same baselines using the same metrics.

For out-of-distribution inputs, we use images from the ImageNet and StreetView datasets. While both datasets do contain some images of people, most of the images are not portraits and therefore are OOD. Additionally, we use 10,000 frames from the Caltech Pedestrian dataset (Dollár et al., 2012) . These images, which were taken from vehicle driving through an urban environment, contain candid images of pedestrians. Finally, we generate a challenging OOD dataset using images of children 10 and younger from the UTKFace dataset (Child).

RESULTS. The results for age estimation are in Table 1 (last two sets of rows). Similar to our previous task, the GMMs achieve very high OOD detection performance on all datasets. The GMM detector for DenseNet-121 outperforms Monte-Carlo dropout and ensembles on all benchmarks by a significant margin. For ResNet-50, the GMM outperforms the other methods on all but one dataset with respect to both the TNR and AUROC metrics. The child age dataset is the most challenging OOD detection dataset for all methods. This is because the images are very similar to the training data. In Figure 5 (middle-right) we plot the network's prediction and the actual ages for adults (indistribution) and children (OOD). The network is unable to correctly predict the ages of children. Figure 5 (right) shows that the GMM's score can be used to identify the most potentially erroneous predictions on this dataset.

Other considerations. Feature-based OOD detection is a computationally advantageous method. Using a GMM for OOD detection requires a constant number of matrix-vector multiplications for each input. This is a fraction of the cost of the neural network's prediction. It is worth noting that ensemble-based methods have benefits besides OOD detection, such as improved predictive performance. However, ensembles require multiple forward passes, which can be quite expensive.

7. CONCLUSION:

Regression neural networks, unlike classification networks, output a point prediction rather than a distribution over possible predictions. From this prediction alone, we demonstrate that is nearly impossible to determine whether the network's input is anomalous or out-of-distribution. In this paper, we argue that the network's hidden features indicate whether samples are in-distribution or not. We demonstrate that the distribution of neural network features p(h(x)) are intrinsically low dimensional and can be well-approximated by a simple generative model, such as mixture of Gaussians. This is an simple approximation, especially considering that the feature space has more than 1,000 dimensions. Nevertheless, we find that this approach is incredibly accurate at identifying OOD inputs, whether the inputs come from a nonsensical distribution or a slightly-shifted distribution. In Table 3 we display the OOD detection results for GMMs with 1, 2, and 4 mixture components.

In addition, we report the log likelihood of the GMM on the in-distribution validation data, which we use as the selection criterion. The GMMs with the highest log likelihood for a given model are displayed in blue.

From this figure, we can observe a number of trends. Firstly, we notice that more mixture components does not always correspond to a better model. 4 mixture components achieves the highest log likelihood on the housing price networks; however, fewer components is better for age estimation. Nevertheless, the log likelihood functions as a good model selection criterion. The GMMs with the highest log likelihood tend to achieve the best OOD detection performance (or comparable to the best performance) both in terms of TNR and AUROC.

Surprisingly, we find the log likelihood is relatively robust to the number of mixture components. For example, the housing price ResNet's log likelihood values vary between 3838 and 3093. Similarly, the age estimation DenseNet's log likelihood values vary between 2563 and 2589. These similar likelihoods correspond to similar OOD detection performance. On all OOD detection benchmarks, the models achieve AUROCs that are within one percentage point of each other. The only experiment with highly varying log likelihood is the age estimation ResNet. The 4-component GMM obtains a significantly lower log likelihood than the 2-or 1-component mixtures. This low log likelihood corresponds to worse OOD detection performance. The 2-and 1-component mixtures perform significantly better on most OOD tasks than the 4-component mixture.

B PROOF OF REMARK 1:

Proof. We assume that the features h(x) are drawn from a uniform measure on a Euclidean ball of radius r. The prediction corresponding to these features is given by ŷ = w T h(x), where w is the weight of the network's linear layer. In order for ŷ > for some it must be the case that h(x) lies

:



Figures and Tables:

Figure2: A neural network is trained to predict Kentucky housing prices. The distribution of predictions on the validation data (black line) and the actual distribution of housing prices (dotted line). Out-of-distribution data (blue and green lines) obtain similar predictions.

Figure3: Left: The relative size of "discarded" feature information h ⊥ (x) decreases as the depth of neural networks increases. Center: The intrinsic dimensionality of the feature vectors h(x), as measured by spectrum of its principal components, decreases with weight decay. Right: Likelihood of in-distribution and OOD features as measured by a Gaussian decomposed into the likelihood of each principal component. The Gaussian was fit on (in-distribution) training set features. OOD features have low likelihood on the small principal components, suggesting that they do not lie in the same low-dimensional distribution.

Figure 4: Smoothed XOR regression task. Left: Target XOR function over in-dist. and OOD data. Middle: Predictions for in-dist. and OOD inputs. Right: Gaussian fit on the in-and OOD features.

Figure 5: Housing price predictions -(Left) Prediction vs. actual price for in-distribution and OOD houses. (Middle-left) The OOD score p(h(x)) from the GMM correlates with the predictive error and can identify the most erroneous housing predictions. Age estimations -(Middle-right) Prediction vs. actual age for adults (in-distribution) and children (OOD). (Right) Similarly to housing, the GMM's OOD score identifies high-error predictions.

OOD detection results for housing price prediction (top) and age estimation (bottom).

Table content: <table><tr><td/><td/><td/><td/><td colspan="3">TNR at TPR 95%</td><td/><td/><td/><td colspan="2">AUROC</td></tr><tr><td>Network</td><td/><td>OOD Dataset</td><td>MCD</td><td>Ens.</td><td>Var.</td><td>Var. + MCD</td><td>GMM</td><td>MCD</td><td>Ens.</td><td>Var.</td><td>Var. + MCD</td><td>GMM</td></tr><tr><td/><td/><td>ImageNet</td><td colspan="2">0.223 0.537</td><td>0.757</td><td>0.200</td><td>0.999</td><td colspan="2">0.825 0.909</td><td>0.940</td><td>0.764</td><td>0.999</td></tr><tr><td colspan="2">KY Housing Price</td><td>KITTI</td><td colspan="2">0.475 0.517</td><td>0.443</td><td>0.318</td><td>0.994</td><td colspan="2">0.910 0.917</td><td>0.910</td><td>0.830</td><td>0.995</td></tr><tr><td colspan="2">(DenseNet-121)</td><td>StreetView</td><td colspan="2">0.280 0.395</td><td>0.348</td><td>0.238</td><td>0.936</td><td colspan="2">0.847 0.877</td><td>0.849</td><td>0.779</td><td>0.982</td></tr><tr><td/><td/><td>CA Houses</td><td colspan="2">0.282 0.304</td><td>0.454</td><td>0.207</td><td>0.568</td><td colspan="2">0.832 0.832</td><td>0.874</td><td>0.757</td><td>0.928</td></tr><tr><td/><td/><td>ImageNet</td><td colspan="3">0.199 0.711 0.629</td><td>0.611</td><td>0.998</td><td colspan="2">0.799 0.953</td><td>0.931</td><td>0.928</td><td>0.999</td></tr><tr><td colspan="2">KY Housing Price</td><td>KITTI</td><td colspan="3">0.518 0.731 0.691</td><td>0.687</td><td>0.932</td><td colspan="3">0.922 0.961 0.954</td><td>0.955</td><td>0.983</td></tr><tr><td>(ResNet-50)</td><td/><td>StreetView</td><td colspan="3">0.317 0.680 0.609</td><td>0.603</td><td>0.868</td><td colspan="2">0.833 0.951</td><td>0.933</td><td>0.933</td><td>0.975</td></tr><tr><td/><td/><td>CA Houses</td><td colspan="3">0.224 0.598 0.261</td><td>0.263</td><td>0.574</td><td colspan="3">0.803 0.930 0.839</td><td>0.842</td><td>0.924</td></tr><tr><td/><td/><td>ImageNet</td><td colspan="2">0.256 0.601</td><td>0.908</td><td>0.241</td><td>0.983</td><td colspan="2">0.799 0.903</td><td>0.981</td><td>0.760</td><td>0.994</td></tr><tr><td colspan="2">Adult Age Estimation</td><td>Streetview</td><td colspan="2">0.420 0.625</td><td>0.963</td><td>0.361</td><td>0.984</td><td colspan="2">0.873 0.916</td><td>0.981</td><td>0.809</td><td>0.995</td></tr><tr><td colspan="2">(DenseNet-121)</td><td>Pedestrian</td><td colspan="2">0.590 0.794</td><td>0.998</td><td>0.313</td><td>1.000</td><td colspan="3">0.925 0.957 0.999</td><td>0.804</td><td>0.999</td></tr><tr><td/><td/><td>Child Age</td><td colspan="2">0.189 0.201</td><td>0.338</td><td>0.125</td><td>0.477</td><td colspan="2">0.670 0.689</td><td>0.844</td><td>0.610</td><td>0.866</td></tr><tr><td/><td/><td>ImageNet</td><td colspan="3">0.310 0.915 0.898</td><td>0.876</td><td>0.937</td><td colspan="2">0.823 0.978</td><td>0.977</td><td>0.970</td><td>0.987</td></tr><tr><td colspan="2">Adult Age Estimation</td><td>Streetview</td><td colspan="3">0.240 0.987 0.998</td><td>0.990</td><td>0.908</td><td colspan="3">0.826 0.987 0.996</td><td>0.993</td><td>0.979</td></tr><tr><td>(ResNet-50)</td><td/><td>Pedestrian</td><td colspan="3">0.277 0.982 0.998</td><td>0.998</td><td>0.986</td><td colspan="3">0.851 0.985 0.999</td><td>0.999</td><td>0.993</td></tr><tr><td/><td/><td>Child Age</td><td colspan="3">0.150 0.234 0.183</td><td>0.148</td><td>0.434</td><td colspan="2">0.656 0.627</td><td>0.540</td><td>0.553</td><td>0.867</td></tr><tr><td>10 1</td><td/><td/><td/><td/><td/><td/><td/><td/><td/><td/><td/></tr><tr><td>10 1</td><td>10 2</td><td>10 3</td><td/><td/><td/><td/><td/><td/><td/><td/><td/></tr><tr><td colspan="3">ŷ (thousands of dollars)</td><td/><td/><td/><td/><td/><td/><td/><td/><td/></tr></table>

OOD detection performance as a function of the GMM mixture size (1, 2, or 4). We display the TNR and AUROC for each GMM, as well as the GMM's log likelihood on in-distribution features. Blue columns correspond to the model with the highest log likelihood (the model selection criterion). Best results are in bold.

Table content: <table><tr><td/><td/><td colspan="3">TNR at TPR 95%</td><td/><td>AUROC</td><td/></tr><tr><td>Network</td><td>OOD Dataset</td><td colspan="3">1-GMM 2-GMM 4-GMM</td><td colspan="3">1-GMM 2-GMM 4-GMM</td></tr><tr><td/><td>Log Lik. (In-dist.)</td><td>2924</td><td>2970</td><td>2991</td><td>2924</td><td>2970</td><td>2991</td></tr><tr><td>KY Housing Price</td><td>ImageNet</td><td>0.998</td><td>0.999</td><td>0.999</td><td>0.999</td><td>0.999</td><td>0.999</td></tr><tr><td>(DenseNet-121)</td><td>KITTI</td><td>0.992</td><td>0.994</td><td>0.994</td><td>0.994</td><td>0.995</td><td>0.995</td></tr><tr><td/><td>StreetView</td><td>0.914</td><td>0.931</td><td>0.936</td><td>0.980</td><td>0.982</td><td>0.982</td></tr><tr><td/><td>CA Houses</td><td>0.510</td><td>0.553</td><td>0.568</td><td>0.918</td><td>0.923</td><td>0.928</td></tr><tr><td/><td>Log. Lik. (In-dist.)</td><td>3864</td><td>3838</td><td>3903</td><td>3864</td><td>3838</td><td>3903</td></tr><tr><td>KY Housing Price</td><td>ImageNet</td><td>0.997</td><td>0.996</td><td>0.998</td><td>0.999</td><td>0.999</td><td>0.999</td></tr><tr><td>(ResNet-50)</td><td>KITTI</td><td>0.939</td><td>0.908</td><td>0.932</td><td>0.984</td><td>0.981</td><td>0.983</td></tr><tr><td/><td>StreetView</td><td>0.844</td><td>0.801</td><td>0.868</td><td>0.973</td><td>0.967</td><td>0.975</td></tr><tr><td/><td>CA Houses</td><td>0.482</td><td>0.538</td><td>0.574</td><td>0.904</td><td>0.908</td><td>0.924</td></tr><tr><td/><td>Log Lik. (In-dist.)</td><td>2563</td><td>2589</td><td>2588</td><td>2563</td><td>2589</td><td>2588</td></tr><tr><td>Adult Age Estimation</td><td>ImageNet</td><td>0.986</td><td>0.983</td><td>0.964</td><td>0.996</td><td>0.994</td><td>0.993</td></tr><tr><td>(DenseNet-121)</td><td>Streetview</td><td>0.989</td><td>0.984</td><td>0.982</td><td>0.997</td><td>0.995</td><td>0.994</td></tr><tr><td/><td>Pedestrian</td><td>1.000</td><td>1.000</td><td>1.000</td><td>1.000</td><td>0.999</td><td>0.999</td></tr><tr><td/><td>Child Age</td><td>0.416</td><td>0.477</td><td>0.427</td><td>0.852</td><td>0.866</td><td>0.860</td></tr><tr><td/><td>Log Lik. (In-dist.)</td><td>2630</td><td>2552</td><td>1876</td><td>2630</td><td>2552</td><td>1876</td></tr><tr><td>Adult Age Estimation</td><td>ImageNet</td><td>0.937</td><td>0.845</td><td>0.830</td><td>0.987</td><td>0.974</td><td>0.971</td></tr><tr><td>(ResNet-50)</td><td>Streetview</td><td>0.908</td><td>0.758</td><td>0.766</td><td>0.979</td><td>0.959</td><td>0.962</td></tr><tr><td/><td>Pedestrian</td><td>0.986</td><td>0.924</td><td>0.906</td><td>0.993</td><td>0.982</td><td>0.981</td></tr><tr><td/><td>Child Age</td><td>0.434</td><td>0.475</td><td>0.468</td><td>0.867</td><td>0.878</td><td>0.880</td></tr></table>

Peer review (iclr2020):
The authors present the empirical observation that regression models trained using weight decay tend to produce high-level features that lie on low-dimensional subspaces. They then use this observation to propose an algorithm for detecting out of distribution data by fitting a simple GMM to the features produced during training. The main contribution of the paper is a way of calculating the *OOD scores* that are reported in the different figures. However, after reading the paper 3 times I*m still unable to find a definition of these scores. I*m guessing it*s negative log-likelihood under the fitted GMM, but I*m unsure. The observation (features on low-dimensional manifolds) is interesting, and the proposed algorithm is potentially useful. However the experiments are quite limited, with only 2 training datasets. The analysis of why/when the algorithm is expected to work and how this depends on the model is shallow. The two main results are for models pre-trained on Imagenet: presumably this has a huge impact on the features that end up being used for OOD detection, but this is not discussed in the paper at all. Questions to the authors: - Why focus only on regression? It seems to me that your analysis for the features lying on a low-dimensional subspace should also apply to classification models. - How do your results depend on the specific models that are used? I can come up with models with very few final features where your method would not work. Do you have any analysis or guidance? - What do all abbreviations in the tables mean? Please make the captions more informative so the reader does not need to search in the main text.