E XPLORING C OVARIATE AND C ONCEPT S HIFT FOR D ETECTION AND C ONFIDENCE C ALIBRATION OF O UTOF -D ISTRIBUTION D ATA
Anonymous authors
Paper under double-blind review
A BSTRACT
Moving beyond testing on in-distribution data, works on Out-of-Distribution (OOD) detection have recently increased in popularity.
A recent attempt to categorize OOD data introduces the concept of near and far OOD detection.
Speciﬁcally, prior works deﬁne characteristics of OOD data in terms of detection difﬁculty.
We propose to characterize the spectrum of OOD data using two types of distribution shifts: covariate shift and concept shift, where covariate shift corresponds to change in style, e.g., noise, and concept shift indicates change in semantics.
This characterization reveals that sensitivity to each type of shift is important to the detection and conﬁdence calibration of OOD data.
Consequently, we investigate score functions that capture sensitivity to each type of dataset shift and methods that improve them.
To this end, we theoretically derive two score functions for OOD detection, the covariate shift score and concept shift score , based on the decomposition of KL-divergence for both scores, and propose a geometricallyinspired method (Geometric ODIN) to improve OOD detection under both shifts with only in-distribution data.
Additionally, the proposed method naturally leads to an expressive post-hoc calibration function which yields state-of-the-art calibration performance on both in-distribution and out-of-distribution data.
We are the ﬁrst to propose a method that works well across both OOD detection and calibration, and under different types of shifts.
I NTRODUCTION
Out-of-distribution (OOD) detection is a fundamental research area important for downstream tasks with open-world assumptions such as continual learning (Smith et al., 2021), open-set learning (Yu et al., 2020) and safety-critical applications such as self-driving (Bojarski et al., 2016).
However, common OOD detection benchmarks 1 are not well categorized.
This prevents us from more systematically studying of OOD detection.
A recent attempt (Winkens et al., 2020; Fort et al., 2021) to provide a more granular characterization of OOD data introduces the concept of near OOD (Winkens et al., 2020; Fort et al., 2021).
Near OOD detection is posed as a more challenging problem than far OOD detection because near OOD datasets usually share similar semantics, and style, e.g., all natural images of similar environment, as the training dataset.
Existing works characterize OOD in terms of difﬁculty of OOD detection such as Confusion Log Probability (CLP) (Winkens et al., 2020).
However, this characterization only conveys difﬁculty of OOD detection across a single dimension and does not expose intrinsic characteristics of OOD data.
In this paper we provide a more systematic categorization of OOD data, which motivates a more robust OOD detection method and helps us reﬂect on some existing approaches.
Speciﬁcally, there are at least two dimensions along which we can characterize the spectrum of OOD data.
Intuitively, out-of-distribution data refers to data that are sampled from distributions different from the training distribution.
This is well known in machine learning as distribution shift (Moreno-Torres et al., 2012).
There are two dominant shift types: covariate shift 2 and concept shift .
The former usually refers to change in style, e.g., clean to noised and natural images to cartoon, and the latter refers to change in semantics, e.g., dog to cat .
Even though existing works Hsu et al. (2020) have acknowledged the different types of OOD datasets and the difﬁculty of detecting them, their methods do not distinguish them and do not explicitly disentangle them.
To study these two distribution shifts, we propose to create a benchmark with multiple magnitude of distribution shift in each dimension.
Speciﬁcally, we use corrupted CIFAR10C/CIFAR100C (Hendrycks & Dietterich, 2019), originally developed for testing robustness, with varying degrees of severity to represent increasing covariate shift; we also introduce a new benchmark, CIFAR100 Splits, which divides the CIFAR100 dataset into 10 splits with increasing conceptual shift from CIFAR10 classes, measured by semantic similarity in a word embedding space (Pennington et al., 2014).
In our experiments, we show that the increasing conceptual difference measured by semantic similarity in language translates to a spectrum of OOD datasets under gradual concept shift measured by the difﬁculty of OOD detection in the image space.
To address these two distribution shifts, we need to approach it from two aspects: representation and modeling , analogous to the role of entropy and Bayesian inference in uncertainty quantiﬁcation (Depeweg et al., 2018) where entropy is a score function that represents uncertainty and Bayesian inference is a method that models uncertainty (H¨ullermeier & Waegeman, 2021).
To represent these two shifts, we can derive score functions which output scalars indicating the severity of distribution shift.
Speciﬁcally we derive two score functions by expanding and decomposing the KL divergence between a predictive distribution from a classiﬁer and a discrete uniform distribution.
Arising from the decomposition is a covariate shift score which is a function of feature norms and a concept shift score which is a function of feature angles.
To model these two shifts, and motivated by the covariate score (a function of norms) and the concept score (a function of angles), we propose to directly improve the sensitivity of norms and angles to distribution shift through a geometric perspective (Tian et al., 2021).
Speciﬁcally, we adopt the Geometric Sensitivity Decomposition proposed in Tian et al.
(2021), originally developed for model calibration, which decomposes norms and angles into an instance-dependent component and an instance-independent scalar, and improve it by parametrizng the instance-independent scalars as standalone networks.
This approach improves detection of OOD data under both covariate and concept shifts.
While OOD detection and calibration has been studied separately in the literature, they share the same underlying motivation: conﬁdence/uncertainty should be low/high when distribution shift occurs.
Our method to improve OOD detection naturally yields a powerful calibration function in the family of intra class-preserving functions (Rahimi et al., 2020).
This class of functions provides enough expressive power to calibrate complex decision boundaries in neural networks and proves to be effective in calibration on in-distribution data.
Therefore, we apply a post-calibration (Guo et al., 2017) method on a validation set as described in those works.
Additionally, thanks to the improved sensitivity to distribution shift, our model also is on par with state-of-the-art calibration performance on distribution shifted data as well.
To summarize our contributions:
• We propose to characterize the spectrum of OOD data in terms of covariate and concept shift and unify the notion of near OOD in different literature (Sec. 3.1).
Additionally, we construct a new benchmark (e.g., CIFAR100 Splits) to represent gradual distribution shift in each direction (Sec. 4.1.2 and 4.1.3).
• To represent distribution shifts, we analytically derive two OOD score functions based on KL-divergence to capture both shifts more effectively (Sec. 3.2).
• To model distribution shifts, we propose Geometric ODIN to improve the sensitivity of neural networks during training (Sec. 3.3) and calibration during inference (Sec. 3.4).
This method achieves state-of-the-art performance on both detection (Sec. 4.1) and calibration (Sec. 4.2) of OOD data.
R ELATED W ORK
Out-of-Distribution (OOD) detection methods can be largely divided into two camps depending on whether they require OOD data during training.
Hendrycks et al.
; Thulasidasan et al.
Roy et al.
(2021) leverages anomalous data in training.
Our method belongs to the class of methods that do not assume the availability of OOD data during training.
Hendrycks & Gimpel (2016) uses the maximum softmax probability (MSP) to detect incorrect predictions and OOD data.
Lee et al.
(2018) proposes to use Mahalanobis distance by ﬁtting a Gaussian mixture model (GMM) in the feature space.
Mukhoti et al.
(2021) uses log density of the GMM model instead.
Liu et al.
(2020b) uses an energy score as the uncertainty metric.
ODIN
(Liang et al., 2017) uses a combination of input processing and post-training tuning to improve OOD detection performance.
Generalized ODIN
Hsu et al.
(also Techapanurak et al.
(2019)) includes an additional network in the last layer to improve OOD detection during training.
There are other interesting OOD detection approaches without OOD data such as using contrastive learning with various transformations (Winkens et al., 2020; Tack et al., 2020), training a deep ensemble of multiple models (Lakshminarayanan et al., 2016) and leveraging large pretrained models (Fort et al., 2021).
They require extended training time, hyperparameter tuning and careful selections of transformations, whereas our method does not introduce any hyperparameters and has negligible inﬂuence on standard cross-entropy training time.
Model Calibration methods can also be largely divided in two categories: 1) training time calibration using augmentations (Thulasidasan et al., 2019; Jang et al., 2021), using modiﬁed losses (Kumar et al., 2018); 2) post-hoc calibration (Guo et al., 2017; Kull et al., 2019; Kumar et al., 2019; Rahimi et al., 2020).
Recently, Rahimi et al.
(2020) formally generalizes a family of expressive functions for calibration, the intra order-preserving functions .
This class of functions has more representation power to calibrate more complex decision boundaries in neural networks.
Our proposed method belongs to this class of functions.
However, all these works focus on calibration of in-distribution data.
To obtain better calibration on OOD data, on which the conﬁdence of a model needs to decrease accordingly, sensitivity and deterministic uncertainty modeling is explored by SNGP (Van Amersfoort et al., 2020) and DUQ (Liu et al., 2020a).
Our proposed model not only belongs to the family of intra order-preserving functions, which ensures good in-distribution calibration, but also improves sensitivity to distribution shift, which improves out-of-distribution calibration simultaneously.
Please refer to Appendix 6.8 for a more detailed discussion on related works.
M ETHOD 3. M OTIVATION : C OVARIATE AND C ONCEPT S HIFT
We follow Moreno-Torres et al.
(2012) for the formal deﬁnition of distribution shift, covariate shift and concept shift.
Let X ∈ R D denote the covariate which is the input and Y ∈ R denote the output label.
Distribution shift happens when the training joint distribution is not equal to the testing joint distribution P tr ( X, Y ) (cid:54)
=
P tst ( X, Y ) .
Using product rule, the joint distribution can be decomposed as P ( X, Y ) =
P ( Y | X ) P ( X ) .
Covariate shift appears when P tr ( Y | X )
=
P tst ( Y | X ) and
P tr ( X ) (cid:54) =
P tst ( X ) .
Concept shift appears when P tr ( Y | X ) (cid:54) =
P tst ( Y | X ) and P tr ( X ) =
P tst ( X ) .
However, in the image domain, it is rare that such concept shift occurs without changes in P ( X ) . We therefore modify the equality in concept shﬁt to P tr ( X ) ≈ P tst ( X ) where superﬁcial, low-level statistics may be retained.
Speciﬁcally, covariate shift happens when the testing data is non-semantically different from the training data and concept shift happens when the testing data is semantically different from the training data.
We illustrate a spectrum of OOD dataset in Fig.
In this example, CIFAR10 is the training dataset.
CIFAR100 represents concept shift because CIFAR100 has a non-overlapping label space, i.e., P tr ( Y | X ) (cid:54)
=
P tst ( Y | X ) , but similar style with CIFAR10.
The corrupted CIFAR10C dataset (Hendrycks & Dietterich, 2019) represents covariate shift because it has the same labels but different style, i.e., P tr ( X ) (cid:54)
=
P tst ( X ) , compared to CIFAR10.
SVHN represents both covariate and concept shift due to its non-overlapping label space and very different style.
As shown in Fig.
2, the ﬁrst goal of this paper is to derive score functions to represent the shift in either P ( X ) or P ( Y | X ) .
We denote the score function that reﬂects change from P tr ( X ) to P tst ( X ) as the covariate shift score function : g ( x ) : R D
→ R and the score function that reﬂects change from P tr ( Y | X ) to P tst ( Y | X ) as the concept shift score function : h ( y, x ) : Y × R D
→ R .
The second goal is to improve the sensitivity of these scores to their corresponding distribution shift.
Speciﬁcally, we propose a geometrically inspired o ut-of-distribution d etection method with only indistribution data ( Geometric ODIN ).
The third goal is to show that, unlike prior works which treat
OOD detection (Mukhoti et al., 2021) and calibration (Van Amersfoort et al., 2020; Liu et al., 2020a) separately, our proposed Geomeric ODIN naturally leads to an expressive calibration function in the family of intra order-preserving functions (Rahimi et al., 2020), which ensures good calibration.
3. C OVARIATE AND C ONCEPT S CORE F UNCTIONS
In this section, we theoretically derive two score functions, g ( x ) and h ( y, x ) , based on the KLdivergence between a uniform distribution U and a predicted distribution P ∈ R M , where M is the number of classes.
By starting from KL-divergence, we hinge the subsequent derivation of score functions on a physical meaningful uncertainty measure, i.e., how far the predicted distribution is from a uniform distribution.
This relationship ensures a natural interpretation of score functions because predictions on distribution shifted data should have larger uncertainty, i.e. smaller distance from uniform.
We are speciﬁcally interested in softmax-linear models for classiﬁcation.
They typically consist of a feature extractor and a linear layer followed by a softmax activation.
Let f ∈ R D denote a feature vector from the feature extractor 3 .
The output of the linear layer, i.e., logits, l ∈ R M = < f , W > is deﬁned as the inner product between the feature vector and a weight matrix in the linear layer.
Let l i = (cid:107) f
(cid:107) 2 (cid:107) w i (cid:107) 2 cos φ i denote the i th logit, P i =
[-exp l i (cid:80) Mj =
-][Equation]
denote the predicted probability of the i th class.
The KL-divergence KL ( U || P ) can be written as following:
[Equation]
Now we can use the inequality property of Log-Sum-Exp (LSE) 4 functions in Eq. 2 to bound Eq.
[Equation]
Therefore the KL-divergence (Eq. 1) can be bounded as follows:
[Equation]
where U = max j l j − 1 M (cid:80) Mi
=1 l i .
U can be further decomposed into two multiplicative components by plugging in the deﬁnition of logits l i :
[Equation]
We deﬁne the covariate shift score function as g ( x ) (cid:44) (cid:107) f (cid:107) 2 because the norm of a feature vector is the sum of squared activation values and only depends on the input.
Intuitively, activation of a neural network on covariate-shifted data should be weaker than in-distribution data.
Therefore, g ( x ) assigns a higher value to in-distribution data than to OOD data.
We deﬁne the concept shift score function as h ( y, x ) (cid:44) [Equation] M [Equation]
[Equation]
w i
[Equation]
2 cos φ i because it is the difference between the cosine distance of the predicted class and the average cosine distance of all classes and depends on both the input and ﬁnal class membership, assigned by the max operator.
Intuitively, class assignment should be less obvious under concept shift and the difference should be small.
Consequently, h ( y, x ) assigns a higher value to in-distribution data than to OOD data.
In retrospect, our deﬁnition of covariate shift and concept shift scores supports existing ﬁndings that the feature norms correspond to intra-class variance and angles reﬂect inter-class variation (Liu et al., 2018).
Intuitively, covariate shift represents non-semantic change within a speciﬁc class, i.e., intra-class variance; concept shift represents semantic changes, i.e, inter-class variation.
Here we formalize the intuition and observations in Liu et al.
as score functions derived analytically from a
KL-divergence viewpoint.
We provide an extended discussion of these scores in Appendix 7.
More importantly, the combined score function U (Eq. 4) carries a physical meaning: it bounds the KL-divergence between a uniform distribution and the predictive distribution.
Intuitively, a small U indicates large uncertainty because U upper bounds KL ( U || P ) , and a large U indicates small uncertainty because it also appears in the lower bound.
A similar scoring function,
S ( x ) = (cid:107) f (cid:107) 2 max j (cid:107) w j (cid:107) 2 cos φ j , is used in Tack et al.
but is only empirically motivated based on observations with limited analytical insights such as its relationship to uncertainty and the functionality of each of components.
In contrast, our derivation clearly shows the relationship between these score functions and the KL-divergence, which is as an uncertainty measure, and disentangles their roles.
We will compare the g ( x ) , h ( y, x ) and U as score functions in subsequent experiments and analyze their respective sensitivity to different dataset shifts (Sec. 4.1).
3. G EOMETRIC O UT - OF -D ISTRIBUTION D ETECTION WITH I N -D ISTRIBUTION D ATA
As derived in Eq. 4, the covariate score is a function of feature norms and the concept score is a function of feature angles.
Consequently, improving the sensitivity of feature norms and feature angles to data shifts seems to be the natural next step to improve OOD detection.
Therefore, we adopt Geometric Sensitivity Decomposition (GSD) (Tian et al., 2021) (reviewed in appendix 6.3) to improve sensitivity to covariate and concept shifts.
Speciﬁcally, GSD improves sensitivity by extracting sensitive components from norms (cid:107) f ∗ [Equation] 2 5 and angles | φ ∗ i | through a decomposition of them into: an instance-independent scalar and an instance-dependent variance factor as shown in
Eq. 5.
Instance-independent scalars C f and C φ minimize the loss on the training set and instancedependent components f
and φ i account sensitively for variances in samples.
[Equation]
With the decomposed components, the original logit l ∗ i =
[Equation] [Equation]
w i
[Equation]
2 cos φ ∗ i , can be written as:
[Equation]
where l i denote the new i th logit.
In Eq.
6, the new 6 feature f is a direct output of a feature extractor, and is modiﬁed by α and β as also illustrated Fig. 2.
Note that the calculation of score functions in
Sec. 3.2 only uses the feature and is independent of α and β .
Because cos C φ and C f are instance-independent, we can parametrize them separately from the main network.
Unlike GSD which parametrizes them as instance-independent scalars, inspired by Hsu et al.
; Techapanurak et al.
(2019), we make α ( f ) and β ( f ) instance-dependent scalars and use a single linear layer to learn them.
To enforce numerical constraints, i.e., 0 < α < 1 and β > 0 , α ( f ) uses a sigmoid activation and β ( f ) uses a softplus activation .
Finally, the relaxed output is:
[Equation]
Now the new predicted norm (cid:107) f (cid:107) 2 and angle φ
i are more sensitive to input changes because they encode variances in samples as shown in Eq.
Therefore, including β (related to norms) improves sensitivity to covariate shift and including α (related to angles) improves sensitivity to concept shift.
Note that, under this construction, Generalized ODIN (Hsu et al., 2020) is a special case of our proposed method.
Generalized ODIN only includes the α ( f ) which only improves angle sensitivity but not norm sensitivity.
Unlike Hsu et al.
(2020)’s probabilistic perspective 7 , our model builds on a geometric perspective and captures both covariate and concept shifts by improving norm and angle sensitivity.
The new model can be trained identically as the vanilla network without additional hyperparameter tuning and extended training time.
Combined with score functions derived in Sec. 3.2,
Geometric ODIN achieves state-of-the-art OOD detection performance (Sec. 4.1)
C ALIBRATION OF G EOMETRIC ODIN
Like most softmax-linear classiﬁcation models, Geometric ODIN also suffers from miscalibration right after training.
Speciﬁcally, predictions tend to be overconﬁdent (Guo et al., 2017).
In the case of Geometric ODIN, the cause of overconﬁdence is obvious by construction.
The offset scalars β ( f ) and α ( f ) are designed to minimize the training loss.
Practically, they are optimized to push the predicted probability of the ground truth class to be close to 1 during training, and this intended behavior unsolicitedly continues to inference time when accuracy is not as high as during training.
Nevertheless, our construction of α ( f ) and β ( f ) belongs to a family of intra order-preserving functions, which has potentially strong representation to calibrate complex functions in deep networks according to Theorem 1 in Rahimi et al.
To prove that, it’s sufﬁce to show that that α ( l ) and β ( l ) 8 forms a strictly positive function, m ( l ) as shown in Eq.
[Equation]
Because 0 < α ( l ) < 1 and β ( l ) > 0 , the function m ( l ) is strictly positive and thus satisﬁes
Theorem 1 in Rahimi et al.
(2020) (See appendix 6.4 for more details).
We follow a simple procedure proposed in Guo et al.
Rahimi et al.
(2020) to calibrate Geometric ODIN by minimizing the negative log likelihood (NLL) on a evaluation dataset for a few epochs.
During calibration the gradients to other parts of the model are stopped and only these two linear layers, α ( l ) and β ( l ) , are optimized (Fig. 2).
Naturally with the improved sensitivity and calibrated functions, the calibration of Geometric ODIN is on par with state-of-the-art models on both in-distribution and OOD data.
Note that, as mentioned in Sec. 3.3, because the calculation of score functions for OOD detection only depends on the feature f and is independent of α and β , calibration does not affect the OOD detection performance and does not change prediction results due to the order-preserving property.
[Table]
E XPERIMENTS
In this section [Table] we present results for Out-of-Distribution detection in Sec.
4.1 and calibration in
Sec. 4.2.
We are the ﬁrst to present a method that works well on both OOD detection and calibration, disentangling shift types.
Implementation:
Following prior works (Liu et al., 2020a;
Van Amersfoort et al., 2020; Mukhoti et al., 2021), we use Wide-ResNet-28-10 (Mukhoti et al., 2021) for all experiments.
We train the model using SGD with an initial learning rate of 0 .
1 forepochs.
The learning rate is annealed with a cosine scheduler (Loshchilov & Hutter, 2016) (more details in appendix 6.5).
Metrics:
For OOD detection in Sec.
4.1, we use the common AUROC and
TNR @ TPR95 as benchmark metrics.
For calibration in Sec.
4.2, we use the Expected Calibration
Error (ECE) (Guo et al., 2017) with 15 bins and Negative Log Likelihood (NLL) which is a strictly proper scoring rule (Gneiting & Raftery, 2007).
Datasets: CIFAR10 (Krizhevsky et al., a) and CIFAR100 (Krizhevsky et al., b) are considered near OOD datasets (Winkens et al., 2020; Fort et al., 2021) to each other.
SVHN (Netzer et al., 2011) is considered a far OOD dataset to both CIFAR and CIFAR100 due to its shift in both concept and style.
The CIFAR10C/CIFAR100C (Hendrycks & Dietterich, 2019) dataset is a variant of CIFAR10/CIFAR100 corrupted by 15 types of noises with 5 degrees of severity.
We also introduce CIRAF100 Splits benchmark which consists of 10 datasets with increasing concept shift from CIFAR10 classes (Sec. 4.1.3).
4. O UT - OF -D ISTRIBUTION D ETECTION R ESULTS 4.1. N EAR AND F AR OOD D ETECTION
The α - β model is the best.
In Tab. 1, we present OOD detection results against state-of-the-art methods on existing near and far OOD categorization.
For near OOD detection under strong concept shift, CIFAR10 (ID) vs. CIFAR100 (OOD), both our α -only and α - β variants achieve the the best performance.
This demonstrates that α ( x ) improves the sensitivity of angles and hence the sensitivity to concept-shifted data.
For near OOD detection under strong covariate shift, CIFAR (ID) vs. CIFAR10C (OOD), the α - β variant achieves the the best performance.
This suggests that β ( x ) improves the sensitivity of norms and hence the sensitivity to covariate-shifted data.
In CIFAR100 (ID) vs. CIFAR10 (OOD) experiments, the performance of the α -only and α - β variants are within variance and is close to some other compared methods, we can not make clear observations from those experiments 9 .
For far OOD detection, CIFAR10/CIFAR100 (ID) vs. SVHN (OOD), the α - β model achieves state-of-the-art performance.
This reconﬁrms that β ( x ) improves sensitivity to covariate-shifted data, because SVHN has both covariate and concept shifts compared to the CIFAR datasets, and the α - β model outperforms the α -only variant, which only improves on concept shift, by a noticeable margin.
In terms of score functions, the best performing one for the α - β model is U , which is a product of g ( x ) and h ( y, x ) (Sec. 3.2), while that of the α -only model is h ( y, x ) .
This shows that depending on which component is more sensitive, different scoring functions are
[Figure]
α - β , α -only, β -only and vanilla models using functions g ( x ) and ( y, x )
[Figure]
All results are averaged over covairate shift ( α - β model)
yields the best performance as shown in Fig.
3b.
g ( x ) is more responsive to covariate shift than h ( y, x ) as shown in Fig.
3a with the α - β model
[Figure]
[Figure]
h ( y, x ) is more responsive to concept shift than g ( x ) as shown in Fig.
4a with the preferred.
When the sensitivity of both norms and angles are improved, as in the α - β variant, the combined score function U performs well under different distribution shifts.
4.1. OOD D ETECTION UNDER C OVARIATE S HIFT WITH I MAGE C ORRUPTION
The g ( x ) score captures covariate shift.
We compare the α - β , α -only, β -only variants and the vanilla model on CIFAR10C (Hendrycks & Dietterich, 2019) corrupted by motion blur in Fig.
3 with increasing degrees of noise (see appendix 6.7 for additional experiments).
From 3a, we observe that 1) as covariate shift severity increases, OOD detection becomes easier because AUROC increases with increasing severity.
2) in practice, covariate shift does not happen in isolation from concept shift and vice versa (Sec. 4.1.3) because AUROC using either g ( x ) or h ( y, x ) increases.
3) the vanilla model is more sensitive to the concept shift component because h ( y, x ) > g ( x ) in the vanilla model plot even though covariate shift is the dominant distribution shift in this example.
4) when sensitivity to both covariate and concept shift is improved , the α - β model becomes more sensitive to the covariate shift component.
This suggests that the dominant shift in this example is indeed covariate shift.
From Fig.
3b, we observe that the α - β model outperforms the β -only model using the combined score function U .
This suggests that improving sensitivity to both shifts and using
U yield the best OOD detection performance.
In retrospect, the performance of OOD detection has always relied on two components: the model and the score function.
Some works propose more responsive score functions, e.g., energy in Liu et al.
(2020b), and some works propose more sensitive models, e.g., DDU in Mukhoti et al.
(2021) to improve the performance of existing score functions.
4.1. OOD D ETECTION UNDER C ONCEPT S HIFT WITH CIFAR100 S PECIAL S PLITS
Finding a dataset to benchmark gradual concept shift is not straightforward because concept shift is traditionally thought as binary: overlapping or non-overlapping.
However, not all non-overlapping labels are the same.
For example, pickup truck (CIFAR100) is semantically much closer to truck (CIFAR10) than sunﬂowers (CIFAR100) is.
To create this gradual concept/semantic shift, we propose to divide the CIFAR100 dataset into 10 sub-datasets with increasing conceptual difference from
CIFAR10 classes.
Specially, we use 300 dimensional Glove word embeddings 10 (Pennington et al., 2014) trained on the entire wikipedia2014 and Gigaword
(Napoles et al., 2012) to measure semantic closeness (inner product) between CIFAR100 and CIFAR10 classes.
The result is 10 subdatasets split from CIFAR100.
Please refer to appendix 6.6 for the full splits.
The h ( y, x ) score captures concept shift.
Following the previous section, we benchmark the α - β , α -only, β -only variants and the vanilla model on the newly created CIFAR100 Splits.
From Fig.
4a, we observe that 1) as concept shift severity increases, OOD detection becomes easier because AUROC increases with increasing severity.
2) both concept shift and covariate shift are present because
AUROC using either h ( y, x ) or g ( x ) increases.
3) the vanilla model is dominantly more sensitive to the concept shift component because concept shift is the dominant distribution shift in CIFAR
Splits by construction and vanilla ResNet is more sensitive to concept shift (the same behavior is also observed on covariate-shift-heavy data in Sec. 4.1.2).
4) when sensitivity to both shifts is improved, the α - β model is still more sensitive to concept shift ( h ( y, x ) > g
( x ) ).
This reconﬁrms that the dominant shift type is indeed concept shift.
From Fig.
4b, interestingly, we observe that all three variants perform similarly and all outperform the vanilla model.
Combined with previous observations that the dataset has strong concept shift and the vanilla model is already very sensitive to concept shift, improving sensitivity to the covariate shift component yields equally good performance as improving sensitivity to both shifts.
4. O UT - OF -D ISTRIBUTION C ALIBRATION R ESULTS
[Table] [Table]
As proved in Sec. 3. [Table] our model naturally leads to a calibration function in the family of intra order-preserving functions (Rahimi et al., 2020), which ensures good in-distribution calibration performance.
Moreover, thanks to the improved sensitivity, the model can potentially achieve state-of-the-art out-of-distribution calibration as well.
We benchmark our models on both clean CIFAR10/CIFAR100 as well as corrupted CIFAR10C/CIFAR100C (Hendrycks & Dietterich, 2019).
Following prior works (Rahimi et al., 2020), we use 5-fold cross validation for in-distribution calibration experiments.
As shown in Tab. 2 and Tab. 3, our model is comparable to state-of-the-art models on both in-distribution and out-of-distribution calibration.
In the literature, OOD detection and calibration have been studied separately, but our method and experimentation tackles both tasks.
C ONCLUSION
In this work, we propose to characterize the spectrum of out-of-distribution (OOD) data using covariate shift and concept shift.
Unlike difﬁculty of OOD detection, distribution shift exposes intrinsic characteristics of OOD data.
Consequently, to achieve good OOD detection performance, a model needs to consider both distribution shifts.
At representation level, we derive two score functions that represent and capture each shift separately.
At modeling level, inspired by these score functions, we propose a geometrically-inspired method, Geometric ODIN, to improve a model’s sensitivity to both shift.
Furthermore, Geometric ODIN is the ﬁrst method that considers both OOD detection and calibration, targeting both concept and covariate shifts, and yields state-of-the-art performance in both tasks.
Finally, we hope that the distribution shift perspective can lead to a new direction to study OOD data and unify the the ﬁelds of detection and calibration of OOD data.
A PPENDIX 6. I NTRODUCTION TO O UT - OF -D ISTRIBUTION D ETECTION
Out-of-Distribution detection is the task of separating in-distribution data from out-of-distribution (OOD) data.
For example, a classiﬁer, trained on a training distribution P tr
( X, Y ) , is tested on a different test joint distribution P tst ( X, Y ) .
When the test joint distribution P tst ( X, Y ) (cid:54) =
P tr ( X, Y ) , data sampled from P tst ( X, Y ) are considered OOD data.
P tst ( X, Y ) can be different in two ways, i.e., covariate shift and concept shift (Sec. 3.2).
Data sampled from the same distribution as training, P tr ( X, Y ) , such as a conventional validation/test split, are in-distribution (ID) data.
To detect
OOD data, a score function is used such as maximum softmax probability (Hendrycks & Gimpel, 2016) and energy (Liu et al., 2020b).
The score function assigns higher value to OOD data than ID data.
Consequently, OOD data can be detected by setting a suitable threshold.
The commonly used benchmark, Area Under the Receiver Operating Characteristic curve (AUROC), plots true positive rate of ID data against false positive rate of OOD data by sweeping through a range of thresholds, and is a threshold-independent metric for measuring OOD detection performance.
6. I NTRODUCTION TO C ONFIDENCE C ALIBRATION
Colloquially, conﬁdence calibration refers to aligning the maximum predicted probability (conﬁdence) from a multi-class classiﬁer to the empirical accuracy of the predicted class to be the ground truth.
For example, if a classiﬁer classiﬁes 100 images to dog with a conﬁdence of 0 . 8 for each prediction, than 80 out of 100 those images should contain a dog.
Formally, we adopt the deﬁnition from Kull et al.
(2019) to deﬁne conﬁdence calibration.
Let ˆP : X → ∆ x be a probabilistic multiclass classiﬁer for M classes.
For any input x ∈ X , ˆP ( x ) =
( ˆ P 1 ( x ) , ..., ˆ P M ( x )) is a probability vector.
Deﬁnition
A probabilistic classiﬁer ˆP : X → ∆ x is conﬁdence-calibrated, if for any p ∈ [0 , 1]
[Equation]
One notion of conﬁdence calibration is the expected difference between the conﬁdence and accuracy.
[Equation]
Expected Calibration Error (ECE) (Naeini et al., 2015)
(Guo et al., 2017) is a metric that approximates this expectation.
ECE partitions predictions into several M equally-sized bins according to the predicted conﬁdence, and then calculates average accuracy and conﬁdence for each bin.
Let ˆ y and ˆ p be the predicted class and conﬁdence of an input x with corresponding label y .
B n denotes the n -th bin.
The accuracy and conﬁdence are calculated for each bin as follows:
[Equation]
Than ECE is deﬁned as a weighted sum of the difference between conﬁdence and accuracy in each bin.
Therefore, lower ECE indicates better calibration.
[Equation]
6. G EOMETRIC S ENSITIVITY D ECOMPOSITION R EVIEW
Geometric sensitivity decomposition (GSD) is proposed in Tian et al.
(2021) to improve the sensitivity of neural networks to input changes.
Speciﬁcally, GSD views the output of the last linear layer, i.e., logits, in a softmax-linear model as inner products, between the feature and weights in the linear layer, , l ∈ R M = < f , W > .
The inner products can be written as products of norms and cosine similarity, e.g., the i th logit is l ∗
i = (cid:107) f ∗ (cid:107) 2 (cid:107) w i (cid:107) 2 cos φ ∗ i .
GSD proposes to decompose norms, (cid:107) f ∗ (cid:107) 2 , and angles, φ ∗ i , into two components: an instance-dependent residual component and an instance-independent scalar as shown in Eq.
The instance-independent scalar acts as freely optimizable parameters during training to minimize training loss.
[Equation]
Decomposed components in Eq.
11 can be plugged into equation for logits.
[Equation]
  
Eq.
12 can be simpliﬁed by assuming that the angle | φ ∗ i | is small.
This equivalent assumes that training data are closely clustered to linear classiﬁers speciﬁed by the linear weight W .
[Equation]
Therefore, logits can be written as the following.
[Equation]
Because cos C φ and C f are instance-independent, we can parametrize them separately from the main network as α and β respectively.
6. I NTRA O RDER P RESERVING F UNCTIONS R EVIEW
Rahimi et al.
(2020) formalizes a family of powerful calibration function: intra order preserving functions .
Many familiar functions are in this family such as the softmax function and temperature scaling (Guo et al., 2017).
Formally, a function f : R n → R n is intra order-preserving if for any x ∈ R n , both x and f ( x ) share share the same ranking.
The following theorem from Rahimi et al.
(2020) provides sufﬁcient and necessary conditions for constructing an intra order preserving function.
Theorem 1 A continuous function f : R n → R n is intra order-preserving, if and only if f ( x ) =
S ( x ) − 1 U w ( x ) with U being an upper-triangular matrix of ones and w : R n → R n being a continuous function such that
• w i ( x ) =
if y i =
y i −
and i < n , • w i ( x ) >
if y i > y i
and i < n , • w n ( x ) is arbitrary
where y = S ( x ) x is the sorted version (descending, i.e., y 1 ≥ y 2 ) of x and S ( x ) : R n → R n is a sorting matrix.
The task of constructing an intra order preserving function comes down to designing the function w ( x ) .
A speciﬁc form of w ( x ) is proposed in Rahimi et al.
(2020): w i ( x ) = σ ( y i − y
i − 1 )
m i ( x ) .
As long as σ is a positive function, i.e., σ ( x ) = 0 when x = 0 and σ ( x ) > 0 otherwise, and m ( x ) > 0 is a strictly positive function, the contructed function will satisfy Theorem 1.
Our construction of α and β satisﬁes the necessary conditions.
For calibration, the function f acts on the logits l
so we use l instead of x in subsequent proof.
Let l i = (cid:107) f
(cid:107) 2 (cid:107) w i (cid:107) 2 cos φ i denote the sorted 11 i th logit and ¯ l i = (cid:16) 1 α ( l ) (cid:107) f
(cid:107) 2 + β ( l ) α
( l ) (cid:17) (cid:107) w i (cid:107) 2 cos φ i detnote the actual output i th logit ( sorted ) with α and β .
The goal is to show that ¯l and l share the same ranking by satisfying
Theorem 1.
We start by extracting the m ( x ) function in Eq. 15.
[Equation]
Therefore, we can construct the w ( x ) function by taking the difference between two adjacent logits for i < n .
[Equation]
l ) (cid:107) f
(cid:107)(cid:19) (cid:124) (cid:123)(cid:122) (cid:125) m
( l ) >σ ≥ 0 (cid:122) (cid:125)(cid:124) (cid:123)
( l i − l
i − 1 ) , for i < n w i ( l ) =
l i for i = n (16)
where
l i ≥ l i − 1 .
Because 0 < α ( l ) < 1 and β ( l ) > 0 , the function m ( l ) is strictly positive and thus satisﬁes Theorem 1 in Rahimi et al.
Speciﬁcally, m ( l ) corresponds to the strictly positive function m ( x ) and l i − l
i − 1 corresponds to the positive function σ ( y i − y
i − 1 ) , due to the sorting in descending order.
6. I MPLEMENTATION DETAILS
Out-of-Distribution Detection Following prior works (Liu et al., 2020a; Van Amersfoort et al., 2020;
Mukhoti et al., 2021), we use Wide-ResNet-28-10 (Mukhoti et al., 2021) for all OOD detection experiments.
We train the model using SGD and cross entropy loss with an initial learning rate of 0 .
1 for 200 epochs.
The SGD optimizer is conﬁged with 5 .
e − 4 weight decay and 0 .
9 momentum.
Batch size is 128.
The learning rate is annealed with a cosine scheduler (Loshchilov & Hutter, 2016).
We adopt the ofﬁcial code 12 for the implementation of DDU (Mukhoti et al., 2021).
DDU (Mukhoti et al., 2021) and Mahanobis (Lee et al., 2018) distance use density estimation by ﬁtting a Gaussian mixture model (GMM) to the learned feature space.
We use the ofﬁcial implementation of DDU to ﬁt a GMM to the feature space (feature from the CNN).
For Manhanobis distance, we ﬁt a GMM to the low dimensional logit space instead for computational stability.
Calibration The backbone training follows the same procedure as the previous section.
During the calibration/tuning stage, we train all models for 20 epochs with cosine learning rate decay using
SGD and cross entropy loss.
In this stage, we freeze the backbone models and only tune the calibration parameters/functions following (Guo et al., 2017).
In our case, only α and β are tuned.
For calibration on out-of-distribution data, the models are tuned on the entire validation set.
For calibration on in-distribution data, we use 5-fold cross validation using the validation set following prior work (Rahimi et al., 2020).
For matrix scaling Guo et al.
and Dirichlet scaling (Kull et al., 2019), we use the Off-Diagonal and Intercept Regularisation (ODIR) (Kull et al., 2019) with a default scaling of 1 × 10 − 7 .
Note that we use the same initial learning rate, 0 . 1 , for all methods except for Dirichlet scaling.
Dirichlet scaling requires smaller learning rate and we tuned the learning rate on different datasets.
For CIFAR10, we use 0 .
01 and for CIFAR100 we use 0 .
6. CIFAR100 C ONCEPT S HIFT S PLITS
While it is natural to associate covariate shift with increasing degrees of image corruption, ﬁnding a dataset to benchmark gradual concept shift is not straightforward because concept shift is traditionally thought as binary: overlapping or non-overlapping.
However, not all non-overlapping labels
AVE.
STD.
[Table] airplane automobile bird cat deer dog frog horse ship truck [Table] cattle shrew motorcycle squirrel snake trout sea tractor bus pickup[-24.96 2.39bear elephant leopard camel lizard rabbit beaver spider raccoon orchid 21.99 0.44lion mountain crab bicycle turtle beetle train mouse snail otter 20.18 1.14possum shark forest pine dinosaur boy porcupine wolf road butterﬂy 17.79 0.32girl rocket man tiger bee tank whale baby kangaroo dolphin 16.26 0.44willow worm chimpanzee skunk cup mushroom oak cockroach crocodile hamster 14.64 0.55castle can bridge lobster house bed fox maple pear woman 12.
-]
[Table]
[Table]
bear elephant leopard camel lizard rabbit beaver spider raccoon orchid [Table] lion mountain crab bicycle turtle beetle train mouse snail otter [Table]
possum shark forest pine dinosaur boy porcupine wolf road butterﬂy
[Table]
rocket man tiger bee tank whale baby kangaroo dolphin
[Table]
[Table]
willow worm chimpanzee skunk cup mushroom oak cockroach crocodile hamster
[Table]
castle can bridge lobster house bed fox maple pear woman
[Table]
palm streetcar pepper keyboard bottle seal rose couch caterpillar [Table] ﬂatﬁsh apple orange plate table tulip bowl television skyscraper ray
[Table]
lamp plain lawnmower chair poppy clock cloud sunﬂower telephone
[Table] [Table]
are the same.
For example, pickup truck (CIFAR100) is much closer to truck (CIFAR10) than sunﬂowers (CIFAR100) is semantically.
To create this gradual concept/semantic shift, we propose to divide the CIFAR100 dataset into 10 sub-datasets with increasing conceptual difference from CIFAR10 classes.
Specially, we use Glove word embeddings (Pennington et al., 2014) wtih 6 billion tokens trained on the entire wikipedia2014 and Gigaword
(Napoles et al., 2012) to measure semantic closeness (inner product) between CIFAR100 and CIFAR10 classes.
The result is 10 subdatasets split from CIFAR100 as shown in Tab. 4.
Our experiments demonstrate that the conceptual difference measured by semantic similarity in language translates to a spectrum of near OOD datasets measured by the difﬁculty of OOD detection in the image space.
6. A DDITIONAL R ESULTS
We provide additional results and tabulated results of ﬁgures in the main paper.
Speciﬁcally, Tab.and Tab. 8 are tabulated versions of Fig .3 and Fig. 4 in the main paper respectively.
In addition to motion blur, we also provide out-of-distribution detection results on shot noise in Tab. 7 and zoom blur in Tab. 6.
6. E XTENDED R ELATED W ORK
Out-of-Distribution (OOD) detection methods can be largely divided into two camps depending on whether they require OOD data during training.
Hendrycks et al.
(2018) leverages anomalous
[Table] [Table]
[Table]
data in training which enables classiﬁers to generalize and detect unseen OOD data.
Thulasidasan et al.
(2020) extends existing classiﬁer to include an OOD class.
Roy et al.
(2021) assigns multiple classes for outliers instead of a single class.
Our method belongs to the class of methods that do not assume the availability of OOD data during training.
Hendrycks & Gimpel (2016) discovers that correctly classiﬁed examples have larger maximum softmax probability (MSP) and propose to use it to detect incorrect predictions and OOD data.
Lee et al.
(2018) proposes to use Mahalanobis distance by ﬁtting a Gaussian mixture model (GMM) in the feature space.
Mukhoti et al.
(2021) uses log density of the GMM model instead.
Liu et al.
(2020b) uses an energy score as the uncertainty metric to distinguish between in-distribution and OOD data.
ODIN (Liang et al., 2017) uses a combination of input processing and post-training tuning to improve OOD detection performance.
Generalized ODIN Hsu et al.
(also Techapanurak et al.
(2019)) includes an additional network in the last layer to improve OOD detection during training.
There are many other interesting
OOD detection approaches that have achieved state-of-the-art performance without OOD data such as using contrastive learning with various transformations (Winkens et al., 2020; Tack et al., 2020), training a deep ensemble of multiple models (Lakshminarayanan et al., 2016) and leveraging large pretrained models (Fort et al., 2021).
They require extended training time, hyperparameter tuning and careful selections of transformations, whereas our method does not introduce any hyperparameters and has negligible inﬂuence on standard cross-entropy training time.
For example, Winkens et al.
(2020) trains a constrastive model for 1200 epochs on CIFAR100 whereas our model requires only 200 epochs (same as standard classiﬁer training) on the same dataset.
Model Calibration methods can also be largely divided in two categories: 1) training time calibration using augmentations (Thulasidasan et al., 2019; Jang et al., 2021), using modiﬁed losses (Kumar et al., 2018); 2) post-hoc calibration (Guo et al., 2017; Kull et al., 2019; Kumar et al., 2019; Rahimi et al., 2020).
Guo et al.
(2017) proposes to calibrate the conﬁdence of a trained classiﬁer using temperature scaling, a single scalar that softens overconﬁdent softmax predictions.
Kull et al.
(2019) extends single-class conﬁdence calibration to multiclass calibration using the Dirichlet distribution.
Kumar et al.
(2019) proposes a scaling-binning calibrator that is more sample efﬁcient.
Recently,
Rahimi et al.
(2020) formally generalizes a family of expressive functions for calibration, the intra order-preserving functions .
This class of functions has more representation power to calibrate more complex decision boundaries in neural networks.
Our proposed method belongs to this class of functions.
However, all these works focus on calibration of in-distribution data.
To obtain better calibration on OOD data, on which the conﬁdence of a model needs to decrease accordingly, sensitivity and deterministic uncertainty modeling is explored by SNGP (Van Amersfoort et al., 2020) and DUQ (Liu et al., 2020a).
SNGP uses a bounded spectral normalization regularization (Miyato et al., 2018) during training and DUQ adopts a two-sided gradient penalty (Gulrajani et al., 2017) to improve model sensitivity to distribution shift.
Our proposed model not only belongs to the family of intra order-preserving functions, which ensures good in-distribution calibration, but also improves sensitivity to distribution shift, which improves out-of-distribution calibration simultaneously. [Figure]
D ISCUSSION ON C OVARIATE AND C ONCEPT S CORE D ECOMPOSITION
In Sec.
3.2 in the main paper, two score functions are proposed to capture changes due to covariate and concept shift.
As shown in Eq.
17, g ( x ) denotes the covariate shift score function and h
( y, x ) denotes the concept shift score function .
[Equation]
Conceptually, these two scores disentangle covariate shift and concept shift.
However, in practice, both shifts almost always happen at the same time in the image domain, i.e., covariate shifted data, e.g., noised data, often result in concept shift, i.e., increasing ambiguity in class assignment.
Therefore, both scores can increase and decrease simultaneously as observed in Sec.
4.1.2 and Sec.
The importance of separating them conceptually is to provide a clean perspective to study robust out-of-distribution detection methods that will work well on different situations under either a single distribution shift or a mixed of shifts.
It very likely that better score functions can be derived to better disentangle the effects of distribution shifts.
In turn, methods that improve sensitivity of those new score functions can be motivated.
A DDITIONAL F IGURES
While it seems impossible that concept shift could happen without change in P ( x ) , this is possible if we deﬁne the support, P ( x ) , as the distribution of pixels, ie., the lowest level characteristics, completely stripped of any semantic information.
A simple example would be reshufﬂing the pixels of a CIFAR10 image where the semantics, P ( y | x ) , is completely changed, while P ( x ) remains the same.
Using CIFAR100 to represent concept shift is based on this reasoning.
We provide a visualization of pixel distributions of CIFAR10, CIFAR100 and SVHN in Fig.
5c.
CIFAR and CIFAR100 share very similar pixel distributions while the pixel distributions of SVHN are very different from those of CIFAR10.
The similarity in pixel distributions between CIFAR10 and
CIFAR100 could be attributed to the data generation/collection process.
To further support the statement that norms are sensitive to covariate shift and justify the choice of deﬁning the covariate shift score g ( x )
=
|| x
|| 2 , we show a visualization of CIFAR10 classes in the feature space of a model trained on CIFAR10 in Fig.
5a and 5b.
Similar to Liu et al.
(2018), we change the last feature dimension of a ResNet to 3.
In this way, we can directly visualize the learned feature space without resorting to dimension reduction techniques such as T-SNE (Wattenberg et al., 2016).
In addition to the clean validation data from CIFAR10, we also visualize CIFAR10 data corrupted by Gaussian noise.
The noised data represents severe covariate shift.
We can observe that the noised data are clustered around the origin indicating very small norms.