Igeood: An Information Geometry Approach to Out-of-Distribution Detection

1 INTRODUCTION . Deep neural networks ( DNNs ) achieve state-of-the-art in several classification tasks as they are known to generalize well on data with similar distribution to the training set . Whereas , in many practical applications , the training set does not reflect well enough the real-life environment ( QuioneroCandela et al. , 2009 ) which is often non-stationary and sometimes with unpredictable events . As a consequence , matching the training scenario to reality can be either not possible or too complex . The failure of machine learning ( ML ) models to adapt to non-stationary distributions could limit their adoption in critical systems ( e.g. , autonomous devices , healthcare applications ) . Out-of-Distribution ( OOD ) or novelty detection is one of the main objectives in conceiving reliable ML systems ( Amodei et al. , 2016 ) . A typical application is in monitoring ML-based online services for periodically shifting distributions . However , tracking changes in the underlying data distribution is challenging as they contain unusual ( irregular or unexpected ) events and have large dimensions . For instance , relying on the intrinsic properties of the ML models and their statistical behavior in the presence of in-distribution data is key to identifying OOD samples . Classic approaches to OOD detection consist of deriving metrics for detecting those abnormalities from the lens of ML models ( e.g. , softmax output , latent representations across layers ) , provided that often only a single test example is available . Besides , those metrics are subject to potential limitations inherent to practical scenarios depending on the level of information access in the ML model , e.g. , only having access to the last layer or all intermediate layers . The baseline approach for OOD detection relies on the predictive uncertainty of DNNs . Hendrycks & Gimpel ( 2017 ) demonstrated that OOD samples , in general , induce DNN classifiers to output less confident softmax scores . Whereas existing state-of-the-art methods on classification problems still output high accuracy even under dataset shift . For instance , Ovadia et al . ( 2019 ) show that as the accuracy of the underlying DNN increases , the supervisors ’ outlier detection accuracy also improves . Unfortunately , also the variance increases . Henriksson et al . ( 2021 ) observed that small changes in model parameters that marginally impact the accuracy could have a degrading impact on the performance of the OOD discriminator . This challenge is not exclusive to discriminative models . Deep generative models also fall short in discerning OOD from in-distribution samples . Nalisnick et al . ( 2019 ) raise awareness of the fact that deep generative models also may output a higher likelihood to OOD samples . They show that , even though the samples from the in-distribution CIFAR-10 ( Krizhevsky et al. , 2009 ) dataset ( e.g. , cats , dogs , airplanes , ships ) are conceptually and visually different from house numbers from SVHN ( Netzer et al. , 2011 ) dataset , DNN-based classifiers may still assign a high likelihood to SVHN samples . In this paper , we propose IGEOOD , a new unified and effective method to perform OOD detection by rigorously exploring the information-geometric properties of the feature space on various depths of a DNN . IGEOOD provides a flexible framework that applies to any pre-trained softmax neural classifier . A key ingredient of IGEOOD is the Fisher-Rao distance . This distance is used as an effective differential geometry tool for clustering and as a distance in the context of multivariate Gaussian pdfs ( Pinele et al. , 2020 ; Strapasson et al. , 2016 ) . In our context , we measure the dissimilarity between probability ( in and out ) distributions , as the length of the shortest path within the manifold induced by the underlying class of distributions ( i.e. , the softmax probabilities of the neural classifier or the densities modeling the learned representations across the layers ) . By doing so , we can explore statistical invariances of the geometric properties of the learned features ( Bronstein et al. , 2021 ) . Our method adapts to the various scenarios depending on the level of information access of the DNN and uses only in-distribution samples but can also benefit ( if available ) from OOD samples . 1.1 CONTRIBUTIONS . Our work studies the problem of OOD detection and advances state-of-the-art in very different ways . • To the best of our knowledge , this is the first work studying information geometry tools to devise a unified metric for OOD detection . We derive an explicit characterization of the Fisher-Rao distance based on the information-geometric properties of the softmax probabilities of the neural classifier and the class of multivariate Gaussian pdfs . In general terms , our Fisher-Rao-based metric measures the mismatch–in the geometry space–between probability density functions of the pre-trained DNN classifier conditioning on test and indistribution samples . Section 3 shows in detail how IGEOOD is used for OOD detection . • Experiments on BLACK-BOX and GREY-BOX setups using various datasets , architectures , and classification tasks show that IGEOOD is competitive with state-of-the-art methods . In the BLACK-BOX setup , we assume that only the outputs , i.e. , the logits of the DNN , are available . In the GREY-BOX setup , we allow access to all parameters of the network ; however , the detection must be performed using only the output softmax probabilities . The latter permits input pre-processing which introduces a small ( additive ) noise according to the direction of the gradients w.r.t the test sample . This pre-processing allows for further discrimination between in- and out-of-distribution samples . Our benchmark contains two DNN architectures , three in-distribution datasets , and nine OOD datasets . • In a WHITE-BOX setting , we combine the logits with the low-level features of the DNN to leverage further useful statistical information of the encoded in-distribution data . We model the pre-trained latent representations as a mixture of Gaussian pdfs with a diagonal covariance matrix . Under this assumption , we derive a confidence score based on the Fisher-Rao distance between conditional pdfs corresponding to the test and the closest indistribution samples . Experiments based on various datasets , architectures , and classification tasks clearly show consistent improvement of IGEOOD , achieving new state-of-the-art performance on a couple of benchmarks . In particular , we increased the average TNR at 95 % TPR by 11.2 % with tuning on OOD data and by 2.5 % with tuning on adversarial data compared to Lee et al . ( 2018 ) . 1.2 RELATED WORKS . OOD discriminators consist of a binary classifier to distinguish between in- and out-of-distribution samples . A few works ( Shalev et al. , 2018 ; Hendrycks et al. , 2019 ; Bitterwolf et al. , 2020 ; Mohseni et al. , 2020 ; Winkens et al. , 2020 ; Vyas et al. , 2018 ; Hein et al. , 2019 ) propose retraining the base ( or an auxiliary ) model with synthetic or ground truth OOD samples to serve as a classifier and as an OOD discriminator . Disposing of both OOD and in-distribution samples during training enables the latent representations to learn the decision boundaries to facilitate OOD detection . These methods will not be compared to ours in this work , as they entail retraining or modifying the base neural network by using OOD data to further train parameters . Nagarajan et al . ( 2021 ) studies failure modes of OOD detection methods to better understand how to improve them , especially how spurious features like the background can vastly degrade detection performance . Lee et al . ( 2021 ) leverage OOD data as a regularization technique to improve the generalization and robustness of current neural networks . References ( Schlegl et al. , 2017 ; Kirichenko et al. , 2020 ; Choi & Jang , 2019 ; Vernekar et al. , 2019 ; Xiao et al. , 2020 ; Ren et al. , 2019 ; Zhang et al. , 2021 ; Mahmood et al. , 2021 ; Zhang et al. , 2020 ; Zisselman & Tamar , 2020 ) study OOD detection in the context of generative models for density estimation . Open set recognition ( Bendale & Boult , 2016 ) , outlier or anomaly detection ( Pimentel et al. , 2014 ) , concept drift detection ( Quionero-Candela et al. , 2009 ) , and adversarial attacks detection ( Goodfellow et al. , 2015 ; Madry et al. , 2018 ) are related topics . BLACK-BOX and GREY-BOX scenarios . It is often the case on ML as a service ( Ribeiro et al. , 2015 ) that the model ’ s parameters knowledge and access are restricted to the end-user , granting access only to the computation of the forward and the logits or softmax outputs . The baseline work ( Hendrycks & Gimpel , 2017 ) for BLACK-BOX techniques considers simply the unscaled maximum value of the softmax ( MSP ) as OOD score . In some cases , this confidence score is enough to distinguish between in-distribution and out-of-distribution examples , but it also may assign overconfident values to OOD examples ( Hein et al. , 2019 ) . ODIN ’ s ( Liang et al. , 2018 ) method has two variations . The BLACK-BOX variation consists of temperature scaling the softmax outputs . While the GREYBOX variation also uses an input pre-processing technique that calculates the gradient of the model parameters and adds to the input in an adversarial manner for a more effective OOD detection . Hsu et al . ( 2020 ) proposes a variation of ODIN that does not need access to OOD data for validation . Liu et al . ( 2020 ) proposes an energy-based OOD score . Without retraining , they substitute the softmax confidence score by the free energy function with a temperature parameter . They also propose a GREY-BOX variation with posterior processing for improved results . Fine-tuning is done differently across the literature and should be considered when comparing methodologies . WHITE-BOX scenario . This class of OOD detectors has access to all intermediate layer outputs . Naturally , discriminators have access to more information than the BLACK-BOX or GREY-BOX setups , warranting greater detection capacity . Batch-normalization statistics between layers are used ( Quintanilha et al. , 2019 ) to fit a logistic regression that serves as an OOD detection score . Sastry & Oore ( 2020 ) proposes high order Gram matrices to perform OOD detection by computing classconditional pairwise feature correlations between the test sample and the training set across the hidden layers of the network . Lee et al . ( 2018 ) assume that latent features of DNN models trained under the softmax score follow a class-conditional Gaussian mixture distribution with tied covariance matrix and different class-conditional mean vectors . They calculate the Mahalanobis distance between a test sample as a single estimator of the mean of a class-conditional Gaussian distribution with a tied covariance matrix estimated on the training set . The importance of each low-level component and hyperparameters are tuned using validation data . Ren et al . ( 2021 ) modifies this method to improve detection of near-OOD data . They fit the layer-wise background distribution with a Gaussian distribution with mean and standard calculated empirically from the training set . They subtract the Mahalanobis distance between the test example and this distribution from the score proposed in Lee et al . ( 2018 ) , reducing the importance of features shared by in- and out-of-distribution data . 2 BACKGROUND . Let X ⊆ Rd be the feature space ( continuous ) and Y a label space . Moreover , let pXY be the underlying unknown probability density function ( pdf ) over X × Y . We define the in-distribution training dataset as DN , { ( xi , yi ) } N i=1 ∼ pXY , where xi ∈ X is the input feature data , yi ∈ Y , { 1 , . . . , C } is the output class among C possible classes and N denotes the number of training samples . The training dataset is characterized by the joint pdf pXY with in-distribution marginals X ∼ pX and Y ∼ PY . The predictor denoted by fDN : X → Y is based on the inferred model PŶ |X , i.e. , fDN ( x ) ≡ fn ( x ; DN ) , arg maxy∈Y PŶ |X ( y|x ; DN ) . In order to model the underlying problem , we introduce an artificial binary random variable Z ∈ { 0 , 1 } indicating with z = 1 that the test sample x is OOD and otherwise , it is in-distribution . The open-world data can then be modeled as a mixture distribution pX|Z defined by pX|Z ( x|z = 0 ) , pX ( x ) , pX|Z ( x|z = 1 ) , qX ( x ) . ( 1 ) The intrinsic difficulty arises from the fact that very little can be assumed about the unknown distributions pX and qX , in particular for out-of-distribution .

Summary: UPDATE:  I acknowledge that I've read the author responses as well as the other reviews.   While the authors added further analysis on the proposed method, I am skeptical of the performance of the method since the proposed method requires validation OOD data to achieve SOTA performance and the runtime is much larger than MSP and energy baseline. However, I think the rebuttal clarified much of my concerns and therefore raise the score to 5 weak reject.   =================================================================  The paper proposes an out-of-distribution detection (OOD) metric based on Fisher-Rao distance which can be applied for pre-trained classifiers. First, the authors derive a Fisher-Rao distance applied to the distribution of softmax. Also, they motivate a toy example where the FIsher-Rao distance outperforms the conventional OOD metric, Mahalanobis distance. Furthermore, they formulate the Fisher-Rao distance-based framework, IGEOOD on Black(Grey)-Box, where we can only get access on the logit of the network output, and White-Box, where we can get access to intermediate feature layers. Finally, the authors compare IGEOOD against conventional OOD metrics on various out-of-distribution data and in-distribution data.