Neural Network Out-of-Distribution Detection for Regression Tasks

1 INTRODUCTION . The success of deep neural networks in many domains ( Krizhevsky et al. , 2012 ; Lample et al. , 2016 ; Mnih et al. , 2016 ) is due to their ability to learn complex functions that generalize to new samples . However , this observed generalization only extends to data that are sufficiently similar to the training data . If the neural network encounters data that deviates from the distribution of training data , its predictions are likely to be erroneous or nonsensical ( Guo et al. , 2017 ; Jiang et al. , 2012 ; Begoli et al. , 2019 ) . This may occur if the model is used in scenarios that experience covariate shift ( Sugiyama et al. , 2007 ) or if the model encounters previously-unseen categories of data ( Yu et al. , 2017 ; Hassen & Chan , 2018 ) . Such scenarios are examples of out-of-distribution ( OOD ) inputs . Ideally we would like for neural networks to adapt to such shifts in the data distribution ( Amodei et al. , 2016 ) . In the absence of such adaptation , out-of-distribution detection should be used to identify when a model is unable to generalize to a previously-unseen input . While there are several proposed methods for neural network OOD detection ( Hendrycks & Gimpel , 2017 ; Liang et al. , 2018 ; Lee et al. , 2018b ) , many of these methods rely on architectural components specific to classification neural networks . Consequentially , they can not be applied to regression problems . Regression neural networks typically output only a point prediction rather than a predictive distribution , and thus the output does not indicate its uncertainty or reliability for a given input . This is illustrated in Figure 1 , which displays predictions from a network trained to predict prices of middle-class houses in Kentucky . The network outputs a house price prediction for any possible input – even for outof-distribution images like a California mansion or chair . These predictions fall within the normal range of possible prices , and therefore do not convey that the inputs are not valid for this model . Because the predictions can not identify OOD inputs , we must look for alternative signals . Previous approaches perform regression OOD detection through ensembles ( Gal & Ghahramani , 2016 ; Lakshminarayanan et al. , 2017 ) or through an additional uncertainty prediction layer ( Kendall & Gal , 2017 ; Malinin et al. , 2017 ) . In this paper , we instead turn to the space of hidden features . During training a neural network learns to extract relevant features about the training data and discards irrelevant information . Whether or not a network generalizes to a given test sample depends on the extracted features from that sample . Networks do not generalize to out-of-distribution data because the distributional shift causes the network to extract the wrong information . For example , when the housing neural network is applied to the California mansion in Figure 1 , the network ’ s features do not extract the relevant information that would indicate the true price of the house . ( e.g . presence of palm trees ) . In other words , the extracted features of the California mansion will differ significantly from training data features and therefore the network does not generalize . Based on this intuition , we investigate how to utilize the hidden features of regression neural networks for OOD detection . In particular , we make several contributions . First , we argue empirically and theoretically that we can not identify OOD inputs simply using only the networks ’ predictions . However , we then demonstrate that the features of in-distribution inputs lie on an intrinsically lowdimensional portion of the feature space . It is unlikely that OOD inputs map to similar locations in feature space because of this low-dimensionality . We additionally show that , because of this lowdimensionality , it is possible to model the in-distribution features with simple generative models . To evaluate our proposed approach , we develop new OOD detection tasks based on large-scale computer vision regression datasets . We evaluate the OOD detection performance of several generative models trained on in-distribution features . Surprisingly , we find that a simple mixture of Gaussians – often with no more than 2 components – is better at OOD detection than more complex models such as variational autoencoders . Finally , we demonstrate that GMM models of in-distribution features are able to outperform other regression OOD methods across several benchmarks . 2 RELATED WORK . Out-of-distribution detection and the related problems of outlier detection ( Hodge & Austin , 2004 ; Chalapathy & Chawla , 2019 ) and novelty detection ( Pimentel et al. , 2014 ) are well-studied in statistics and machine learning . Arguably , the most straightforward method to these problems is generatively modeling the distribution of inputs p ( x ) using a parametric distribution ( Chow , 1970 ; Eskin , 2000 ) or a nonparametric density estimate ( Kim & Scott , 2012 ) . If p ( x ) is small for a given input , then it is likely out-of-distribution or an anomaly . Several recent works ( Choi et al. , 2018 ; Nalisnick et al. , 2019 ; Pidhorskyi et al. , 2018 ) have suggested identifying OOD inputs with deep generative models ( Goodfellow et al. , 2014 ; Kingma & Welling , 2014 ; Rezende et al. , 2014 ; Van Den Oord et al. , 2016 ) . One challenge with these methods is that deep generative models can not necessarily model large-scale images or other complex distributions ( Hendrycks et al. , 2019 ) and may be overconfident when modeling input data ( Nalisnick et al. , 2019 ) . Rather than directly modeling the input distribution , the approach proposed in this paper operates on feature spaces using a mixture model to capture potentially anomalous inputs . This falls under the category of model-dependent OOD detection methods , which are ideal when the input distribution is too complex to model with generative methods ( e.g . for high resolution images ) ( Hendrycks et al. , 2019 ) . Several model-dependent methods have been proposed for classification neural networks . The training procedure of a classification network can be modified to include uncertain samples ( Lee et al. , 2018a ) or to discourage overconfidence through an alternative loss function ( Alemi et al. , 2018 ; Masana et al. , 2018 ; Subramanya et al. , 2017 ; Sensoy et al. , 2018 ) . Alternatively , OOD metrics can be constructed for classification based on the softmax probability output ( Hendrycks et al. , 2019 ; Liang et al. , 2018 ) or related uncertainty scores ( Schulam & Saria , 2019 ) . In a similar vein to our proposed approach , ( Lee et al. , 2018b ) use a class-conditional Mahalanobis distance for classification OOD . All of these methods are designed specifically for multi-class classification problems – typically relying on the discrete output space . Consequentially , there are typically no straightforward extensions of these methods to regression problems . There are fewer proposed methods for OOD detection on regression tasks . Most existing approaches rely on an estimate of the neural network ’ s predictive uncertainty ( Kuleshov et al. , 2018 ) . There are several proposed approaches to obtain uncertainty estimates from regression neural networks . One class of approaches is to add an additional output that predicts a confidence interval for the network ’ s prediction ( Lakshminarayanan et al. , 2017 ; Kendall & Gal , 2017 ; Malinin et al. , 2017 ) . This layer can be trained in conjunction with the predictive output to minimize the negative log likelihood on the training set . Alternatively , Bayesian approaches to deep learning ( Blundell et al. , 2015 ; Gal & Ghahramani , 2016 ; Gal et al. , 2017 ; Kingma et al. , 2015 ) can be used to estimate predictive uncertainty on regression tasks . These approaches typically approximate the posterior distribution of neural networks parameters through an ensemble of models . Non-Bayesian ensembling approaches achieve can produce similar uncertainty estimates ( Lakshminarayanan et al. , 2017 ; Maddox et al. , 2019 ) . In this paper we propose an orthogonal approach to regression OOD detection – using the distribution of hidden features rather than the model ’ s predictive uncertainty . 3 CHARACTERIZING OOD PREDICTIONS ON REGRESSION TASKS . In the classification setting , the output of a neural network is a softmax score for each class . Because this output can be interpreted as a probability distribution over classes , it represents some notion of the neural network ’ s uncertainty and therefore can be used to identify inputs that are likely to be OOD ( Hendrycks & Gimpel , 2017 ; Liang et al. , 2018 ) . For most regression architectures however , the output is typically a point prediction ŷ which does not convey any notion of predictive uncertainty . Moreover , in this section we show that these point predictions ŷ are insufficient for differentiating between in-distribution and out-of-distribution inputs . To illustrate this , we train a neural network to predict the price of homes in Kentucky based on a front facing picture . Figure 2 ( left ) displays a histogram of the network ’ s predictions ŷ on a withheld validation set ( black line ) . The distribution of these predictions corresponds to the distribution of actual house prices ( dotted line ) . However , if the same network receives an OOD input , it predicts price values that lie in the same range as the in-distribution predictions . For example , if houses from California are input into the neural network , the predicted housing prices ( blue line ) are roughly the same as that of Kentucky houses despite these houses costing up to 10 times as much . The network predicts similar values for Imagenet images ( orange line ) even though these images do not contain any houses . Based on the ŷ alone , it is not possible to determine that these inputs are not valid . The fact that the OOD predictions are similar to in-distribution predictions can be explained by a simple curse-of-dimensionality argument . Let φ be a regression neural network that maps an input x ∈ X to an output ŷ ∈ R. The last layer of φ produces a set of activations h ( x ) that are linearly mapped to the one-dimensional prediction : ŷ = w > h ( x ) . Let ’ s assume that the features h ( x ) for some OOD inputs can be bounded within some Euclidean ball of radius r. If the dimensionality of h ( x ) is large , then the probability of an extreme prediction becomes exponentially unlikely . Remark 1 . Let h ( x ) be features drawn from a uniform measure on a Euclidean ball of radius r. Let d be the dimensionality of h ( x ) . The probability of a large prediction |ŷ| > is bounded by p ( |ŷ| > ) ≤ 2Γ ( d/2 + 1 ) πd/2rd exp ( −d 2 2r2‖w‖22 ) . ( See Appendix B for proof ) . In practice it is not necessarily the case that OOD features will be uniformly distributed . Nevertheless , this remark illustrates that predictive outputs discard information that would indicate whether a sample is OOD or not . 4 CHARACTERIZING THE DISTRIBUTION OF HIDDEN FEATURES . Although the network ’ s predictions look very similar for in-distribution and OOD inputs , we expect that their hidden features h ( x ) are very different . Since ImageNet pictures do not contain frontfacing houses , the network will extract very different types of features . We similarly expect the activations of California houses to be different since the network is unable to generalize these inputs . To that end , we hypothesize that a generative model of in-distribution features p ( h ( x ) ) will distinguish between in- and out-of-distribution inputs . In this section we demonstrate that – despite the high dimensionality of h ( x ) – the in-distribution features are very tightly concentrated . As a result , it is possible to model in-distribution features with a simple generative model . In-distribution features are intrinsically low dimensional . In the regression setting , the network ’ s prediction is the inner product between the features and a weight vector : ŷ = w > h ( x ) . Any feature information that is orthogonal to the weight w will not contribute to the prediction and therefore is “ discarded ” information . We can describe a feature vector h ( x ) in terms of the prediction ŷ and the discarded information h⊥ ( x ) : h ( x ) = ŷw + h⊥ ( x ) , h⊥ ( x ) = ( I−ww > ) h ( x ) , where h⊥ ( x ) is the discarded information , computed using the orthogonal projection of w. Also , we assume , without loss of generality , that ‖w‖2 = 1 . It is worth emphasizing that “ useful ” predictive information ŷw lies within a single dimension of the d-dimensional h ( x ) vector ( where d is often greater than 500 ) . Though the “ discarded ” information h⊥ ( x ) occupies d−1 dimensions , it typically has low intrinsic dimensionality for in-distribution data as we demonstrate below . Consequentially , the in-distribution features h ( x ) = h⊥ ( x ) + ŷw can be described by a low-dimensional distribution . We identify two factors that are correlated with this low dimensionality : network capacity and regularization . Network capacity reduces the relative size of h⊥ ( x ) . As network capacity increases , we expect that there will be less “ discarded ” information h⊥ ( x ) . To understand why this is the case , assume there is a l-layer network in which there is some correlation between the discarded features and the true labels . Now assume we add an additional layer before the linear layer which computes a new set of hidden features h′ ( x ) . This new layer can improve the network ’ s predictions if it uses information in h⊥ ( x ) to augment the original prediction ŷ . Consequentially , this new network will likely have less discarded information than the shallower network . While this is just an illustrative example , we can empirically demonstrate that network capacity does in fact decrease the amount of discarded information . First , we introduce the following metric of discarded information : let Σ be the empirical covariance matrix of h ( x ) for in-distribution data , and let Σ⊥ be the covariance of h⊥ ( x ) . We measure the amount of discarded information by comparing the maximum singular values of Σ and Σ⊥ . If ‖Σ⊥‖2/‖Σ‖2 is close to 1 , then the feature vectors are comprised mostly of discarded information . Conversely , h ( x ) will consist primarily of predictive information if ‖Σ⊥‖2/‖Σ‖2 is close to zero . We apply this metric to a simple regression task using the MNIST dataset . Each input image is rotated by an amount between −90 and 90 degrees . We train residual networks with increasing depth to predict the amount of rotation in each image ( details in Appendix C ) . Figure 3 ( left ) plots the singular value ratio ‖Σ⊥‖2/‖Σ‖2 as a function of network depth . The ratio is quite large for shallow networks , but decreases with depth . This suggests that , for deep networks , most of h ( x ) lies on the one-dimensional subspace defined by the weight vector w. Regularization reduces the dimensionality of h ( x ) . Applying weight decay to a neural network reduces the magnitude of the weights , which in turn reduces the magnitude of feature vectors . Reducing the magnitude of h ( x ) while maintaining predictive power will result in feature vectors with little discarded information h ( x ) . Since all but one dimensions of the feature space are discarded , we expect that the intrinsic dimensionality of features will decrease with increasing regularization . To support this intuition we train the LeNet model on rotated MNIST with various amounts of weight decay . In Figure 3 ( center ) we plot the spectrum of the in-distribution feature covariance matrix Σ for these different models . We see that all covariance matrices are dominated by a few large singular values . However , the spectra of the high regularization models decay much more rapidly , suggesting that regularization simplifies the distribution of hidden features . The fact that in-distribution inputs occupy a low-dimensional portion of high-dimensional feature space is extremely advantageous . If we draw a random set of random features h ( x ) ( e.g . from a uniform measure on a Euclidean ball ) , it will be highly unlikely that these features occupy the same low-dimensional space . Though OOD features are not truly random vectors in practice , we find empirically that they do not occupy the same low dimensional subspace . To demonstrate this , we fit a multivariate Gaussian to the training set features from the housing price network . We can decompose the fit of this Gaussian as a product of its principle components and measure each component ’ s likelihood for in- and out-of-distribution data . Figure 3 ( right ) displays the fits for the housing price validation data ( in-distribution ) and ImageNet samples ( OOD ) . We find that indistribution and OOD data are equally likely for the main principal components . However , OOD features do not fit the smaller principal components well , and thus these small principal components are able to distinguish in- and out-of-distribution data . In this case the curse-of-dimensionality makes it possible to distinguish OOD and in-distribution features .

Summary: This paper proposes to detect inputs that are from a slightly shifted distribution (eg images of houses in CA instead of KY) or from a very shifted distribution (eg imagenet images) by fitting a density model to the last layer h(x) of an MLP trained with squared error, and using the likelhiood score p(h(x)) as a metric. This is a reasonable idea. However, it is not novel eg Aigrain'19 did essentially the same thing for classification models. (The difference between classification and regression is a trivial change to the loss function, and does not change the fundamental idea.)