Exploring Covariate and Concept Shift for Detection and Confidence Calibration of Out-of-Distribution Data

1 INTRODUCTION . Out-of-distribution ( OOD ) detection is a fundamental research area important for downstream tasks with open-world assumptions such as continual learning ( Smith et al. , 2021 ) , open-set learning ( Yu et al. , 2020 ) and safety-critical applications such as self-driving ( Bojarski et al. , 2016 ) . However , common OOD detection benchmarks1 are not well categorized . This prevents us from more systematically studying of OOD detection . A recent attempt ( Winkens et al. , 2020 ; Fort et al. , 2021 ) to provide a more granular characterization of OOD data introduces the concept of near OOD ( Winkens et al. , 2020 ; Fort et al. , 2021 ) . Near OOD detection is posed as a more challenging problem than far OOD detection because near OOD datasets usually share similar semantics , and style , e.g. , all natural images of similar environment , as the training dataset . Existing works characterize OOD in terms of difficulty of OOD detection such as Confusion Log Probability ( CLP ) ( Winkens et al. , 2020 ) . However , this characterization only conveys difficulty of OOD detection across a single dimension and does not expose intrinsic characteristics of OOD data . In this paper we provide a more systematic categorization of OOD data , which motivates a more robust OOD detection method and helps us reflect on some existing approaches . Specifically , there are at least two dimensions along which we can characterize the spectrum of OOD data . Intuitively , out-of-distribution data refers to data that are sampled from distributions different from the training distribution . This is well known in machine learning as distribution shift ( Moreno-Torres et al. , 2012 ) . There are two dominant shift types : covariate shift2 and concept shift . The former usually 1Texturess ( Cimpoi et al. , 2014 ) , SVHN ( Netzer et al. , 2011 ) , Place365 ( Zhou et al. , 2017 ) , LSUNCrop/Resize ( Yu et al. , 2015 ) , and iSUN ( Xu et al. , 2015 ) 2Covariate shift is often associated with model calibration ( Ovadia et al. , 2019 ; Chan et al. , 2020 ) refers to change in style , e.g. , clean to noised and natural images to cartoon , and the latter refers to change in semantics , e.g. , dog to cat . Even though existing works Hsu et al . ( 2020 ) have acknowledged the different types of OOD datasets and the difficulty of detecting them , their methods do not distinguish them and do not explicitly disentangle them . To study these two distribution shifts , we propose to create a benchmark with multiple magnitude of distribution shift in each dimension . Specifically , we use corrupted CIFAR10C/CIFAR100C ( Hendrycks & Dietterich , 2019 ) , originally developed for testing robustness , with varying degrees of severity to represent increasing covariate shift ; we also introduce a new benchmark , CIFAR100 Splits , which divides the CIFAR100 dataset into 10 splits with increasing conceptual shift from CIFAR10 classes , measured by semantic similarity in a word embedding space ( Pennington et al. , 2014 ) . We show that the increasing conceptual difference measured by semantic similarity in language translates to a spectrum of OOD datasets under gradual concept shift measured by the difficulty of OOD detection in the image space . To address these two distribution shifts , we need to approach it from two aspects : representation and modeling , analogous to the role of entropy and Bayesian inference in uncertainty quantification ( Depeweg et al. , 2018 ) where entropy is a score function that represents uncertainty and Bayesian inference is a method that models uncertainty ( Hüllermeier & Waegeman , 2021 ) . To represent these two shifts , we can derive score functions which output scalars indicating the severity of distribution shift . Specifically we derive two score functions by expanding and decomposing the KL divergence between a predictive distribution from a classifier and a discrete uniform distribution . Arising from the decomposition is a covariate shift score which is a function of feature norms and a concept shift score which is a function of feature angles . To model these two shifts , and motivated by the covariate score ( a function of norms ) and the concept score ( a function of angles ) , we propose to directly improve the sensitivity of norms and angles to distribution shift through a geometric perspective ( Tian et al. , 2021 ) . Specifically , we adopt the Geometric Sensitivity Decomposition proposed in Tian et al . ( 2021 ) , originally developed for model calibration , which decomposes norms and angles into a variance component and a scalar offset , and improve it by parametrizng the scalar offsets as standalone networks . Our approach improves detection of OOD data under both covariate and concept shifts . While OOD detection and calibration3 has been studied separately in the literature , they share the same underlying motivation : confidence/uncertainty should be low/high when distribution shift occurs . Our method to improve OOD detection naturally yields a powerful calibration function in the family of intra class-preserving functions ( Rahimi et al. , 2020 ) . This class of functions provides enough expressive power to calibrate complex decision boundaries in neural networks and proves to be effective in calibration on in-distribution data . Therefore , we apply a post-calibration ( Guo et al. , 2017 ) method on a validation set as described in those works . Additionally , thanks to the improved sensitivity to distribution shift , our model also is on par with state-of-the-art calibration performance on distribution shifted data as well . To summarize our contributions : • We propose to characterize the spectrum of OOD data in terms of covariate and concept shift and unify the notion of near OOD in different literature ( Sec . 3.1 ) . Additionally , we construct a new benchmark ( e.g. , CIFAR100 Splits ) to represent gradual distribution shift in each direction ( Sec . 4.1.2 and 4.1.3 ) . • To represent distribution shifts , we analytically derive two OOD score functions based on KL-divergence to capture both shifts more effectively ( Sec . 3.2 ) . • To model distribution shifts , we propose Geometric ODIN to improve the sensitivity of neural networks during training ( Sec . 3.3 ) and calibration during inference ( Sec . 3.4 ) . This method achieves state-of-the-art performance on both detection ( Sec . 4.1 ) and calibration ( Sec . 4.2 ) of OOD data . 2 RELATED WORK . Out-of-Distribution ( OOD ) detection methods can be largely divided into two camps depending on whether they require OOD data during training . Hendrycks et al . ( 2018 ) ; Thulasidasan et al . ( 2020 ) ; Roy et al . ( 2021 ) leverages anomalous data in training . Our method belongs to the class of methods that do not assume the availability of OOD data during training . Hendrycks & Gimpel ( 2016 ) uses the maximum softmax probability ( MSP ) to detect incorrect predictions and OOD data . Lee et al . 3OOD detection focuses on concept-shifted data while calibration focuses on covariate-shifted data . ( 2018 ) proposes to use Mahalanobis distance by fitting a Gaussian mixture model ( GMM ) in the feature space . Mukhoti et al . ( 2021 ) uses log density of the GMM model instead . Liu et al . ( 2020b ) uses an energy score as the uncertainty metric . ODIN ( Liang et al. , 2017 ) uses a combination of input processing and post-training tuning to improve OOD detection performance . Generalized ODIN Hsu et al . ( 2020 ) ( also Techapanurak et al . ( 2019 ) ) includes an additional network in the last layer to improve OOD detection during training . There are other interesting OOD detection approaches without OOD data such as using contrastive learning with various transformations ( Winkens et al. , 2020 ; Tack et al. , 2020 ) , training a deep ensemble of multiple models ( Lakshminarayanan et al. , 2016 ) and leveraging large pretrained models ( Fort et al. , 2021 ) . They require extended training time , hyperparameter tuning and careful selections of transformations , whereas our method does not introduce any hyperparameters and has negligible influence on standard cross-entropy training time . Model Calibration methods can also be largely divided in two categories : 1 ) training time calibration using augmentations ( Thulasidasan et al. , 2019 ; Jang et al. , 2021 ) , using modified losses ( Kumar et al. , 2018 ) ; 2 ) post-hoc calibration ( Guo et al. , 2017 ; Kull et al. , 2019 ; Kumar et al. , 2019 ; Rahimi et al. , 2020 ) . Recently , Rahimi et al . ( 2020 ) formally generalizes a family of expressive functions for calibration , the intra order-preserving functions . This class of functions has more representation power to calibrate more complex decision boundaries in neural networks . Our proposed method belongs to this class of functions . However , all these works focus on calibration of in-distribution data . To obtain better calibration on OOD data , on which the confidence of a model needs to decrease accordingly , sensitivity and deterministic uncertainty modeling is explored by SNGP ( Van Amersfoort et al. , 2020 ) and DUQ ( Liu et al. , 2020a ) . Our proposed model not only belongs to the family of intra order-preserving functions , which ensures good in-distribution calibration , but also improves sensitivity to distribution shift , which improves out-of-distribution calibration simultaneously . Please refer to Appendix 6.8 for a more detailed discussion on related works . 3 METHOD . 3.1 MOTIVATION : COVARIATE AND CONCEPT SHIFT . We follow Moreno-Torres et al . ( 2012 ) for the formal definition of distribution shift , covariate shift and concept shift . LetX ∈ RD denote the covariate which is the input and Y ∈ R denote the output label . Distribution shift happens when the training joint distribution is not equal to the testing joint distribution Ptr ( X , Y ) 6= Ptst ( X , Y ) . Covariate shift appears when Ptr ( Y |X ) = Ptst ( Y |X ) and Ptr ( X ) 6= Ptst ( X ) . Concept shift appears when Ptr ( Y |X ) 6= Ptst ( Y |X ) and Ptr ( X ) = Ptst ( X ) . However , in the image domain , it is rare that such concept shift occurs without changes in P ( X ) . We therefore modify the equality in concept shfit to Ptr ( X ) ≈ Ptst ( X ) where superficial , low-level statistics may be retained . Specifically , covariate shift happens when the testing data is non-semantically different from the training data and concept shift happens when the testing data is semantically different from the training data . We illustrate a spectrum of OOD dataset in Fig . 1 . In this example , CIFAR10 is the training dataset . CIFAR100 represents concept shift because CIFAR100 has a non-overlapping label space , i.e. , Ptr ( Y |X ) 6= Ptst ( Y |X ) , but similar style with CIFAR10 . The corrupted CIFAR10C dataset ( Hendrycks & Dietterich , 2019 ) represents covariate shift because it has the same labels but different style , i.e. , Ptr ( X ) 6= Ptst ( X ) , compared to CIFAR10 . SVHN represents both covariate and concept shift due to its non-overlapping label space and very different style . As shown in Fig . 2 , the first goal of this paper is to derive score functions to represent the shift in either P ( X ) or P ( Y |X ) . We denote the score function that reflects change from Ptr ( X ) to Ptst ( X ) as the covariate shift score function : g ( x ) : RD → R and the score function that reflects change from Ptr ( Y |X ) to Ptst ( Y |X ) as the concept shift score function : h ( y , x ) : Y × RD → R. The second goal is to improve the sensitivity of these scores to their corresponding distribution shift . Specifically , we propose a geometrically inspired out-of-distribution detection method with only indistribution data ( Geometric ODIN ) . The third goal is to show that , unlike prior works which treat OOD detection ( Mukhoti et al. , 2021 ) and calibration ( Van Amersfoort et al. , 2020 ; Liu et al. , 2020a ) separately , our proposed Geomeric ODIN naturally leads to an expressive calibration function in the family of intra order-preserving functions ( Rahimi et al. , 2020 ) , which ensures good calibration .

Summary: The paper starts from the KL divergence between a uniform distribution and the predicted distribution, based on which two score functions are derived for covariate shift and concept shift, respectively. The covariate shift score measures the feature norm while the concept shift score is essentially the difference between the cosine distance of the predicted class and the average cosine distance of all classes.  Furthermore, the paper integrates a variant of the geometric sensitivity decomposition method introduced in a recent work into the proposed score functions, aiming to make them more sensitive to distribution shift and to help calibrate the model as well.  A new dataset called CIFAR 100 Splits is also built, which contains 10 sub-datasets with an increasing level of concept shift compared to CIFAR 10.  The results show that the proposed approach achieves decent performance in OOD detection (in both the scenarios of covariate shift and concept shift) as well as calibration.