Exploring Covariate and Concept Shift for Detection and Confidence Calibration of Out-of-Distribution Data
Authors: Junjiao Tian
Yen-Chang Hsu
Yilin Shen
Hongxia Jin
Zsolt Kira
Venue: ICLR 2022 Submitted (2022)
Keywords: out-of-distribution detection
calibration
distribution shift

Abstract: Moving beyond testing on in-distribution data, works on Out-of-Distribution (OOD) detection have recently increased in popularity. A recent attempt to categorize OOD data introduces the concept of near and far OOD detection. Specifically, prior works define characteristics of OOD data in terms of detection difficulty. We propose to characterize the spectrum of OOD data using two types of distribution shifts: covariate shift and concept shift, where covariate shift corresponds to change in style, e.g., noise, and concept shift indicates change in semantics. This characterization reveals that sensitivity to each type of shift is important to the detection and model calibration of OOD data. Consequently, we investigate score functions that capture sensitivity to each type of dataset shift and methods that improve them. To this end, we theoretically derive two score functions for OOD detection, the covariate shift score and concept shift score, based on the decomposition of KL-divergence for both scores, and propose a geometrically-inspired method (Geometric ODIN) to improve OOD detection under both shifts with only in-distribution data. Additionally, the proposed method naturally leads to an expressive post-hoc calibration function which yields state-of-the-art calibration performance on both in-distribution and out-of-distribution data. We are the first to propose a method that works well across both OOD detection and calibration, and under different types of shifts. Specifically, we improve the previous state-of-the-art OOD detection by relatively 7% AUROC on CIFAR100 vs. SVHN and achieve the best calibration performance of 0.084 Expected Calibration Error on the corrupted CIFAR100C dataset.  

EXPLORING COVARIATE AND CONCEPT SHIFT FOR DETECTION AND CONFIDENCE CALIBRATION OF OUT- OF-DISTRIBUTION DATA
Anonymous authors Paper under double-bling blind review
ABSTRACT
Moving beyond testing in-distribution data, works Out-of-Distribution (OOD) detection have recently increased in popularity. A recent attempt to catego- rize OOD data introduces the concept of near and far OOD detection. Specifically, prior works define characteristics of OOD data in terms of detection difficulty. We propose characterize the spectrum of OOD data using two types of distribu- tion shifts: covariate shift and concept shift, where covariate shift corresponds to change in style, e-g., noise, and concept shift indicates change in semantics. This characterization reveals that sensitivity to each type of shift is important to the detection and confidence calibration of OOD data. Consequently, we investigate score functions that capture sensitivity to each type of dataset shift and methods that improve them. To this end, we theoretically derive two score functions for OOD detection, the covariate shift score and concept shift score, based the decomposition of KI-divergence for both scores, and propose a geometrically- inspired method (Geometric ODIN) improve OOD detection under both shifts with only in-distribution data. Additionally, the proposed method naturally leads an expressive post-hoc calibration function which yields state-of-the-art cali- bration performance on both in-distribution and nut-of-diitribution data. We are the first to propose method that works well across both OOD detection and cali- bration, and under different types of shifts.
INTRODUCTION
Out-of-distribution (OOD) detection is a fundamental research area important for downstream tasks with open-world assumptions such continual learning (Smith et al., 2021), open-set learning (Yu et al., 2020) and safety-critical applications such as self-driving (Bojarski et al., 2016). However, from more sys-
common OOD detection benchmarks not well categorized. This prevents tematically studying of OOD detection, recent attempt (Winkens et al., 2020; Fort et al., 2021) to provide more granular characterization of OOD data introduces the concept of near OOD (Winkens et al., 2020; Fort et al., 2021). Near OOD detection is posed as a more challenging problem than far OOD detection because near OOD datasets usually share similar semantics, and style, e.g., all natural images of similar environment, as the training dataset. Existing works characterize OOD terms of difficulty of OOD detection such as Confusion Log Probability (CLP) (Winkens al., 2020). However, this characterization only conveys difficulty of OOD detection across single dimension and does not expose intrinsic characteristics of OOD data.
In this paper we provide more systematic categorization of OOD data, which motivates a more robust OOD detection method and helps us reflect on some existing approaches. Specifically, there are at least two dimensions along which we can characterize the spectrum of OOD data. Intuitively, out-of- di--istribttion data refers to data that are sampled from distributions different from the train- ing distribution. This is well known in machine learning distribution shift (Moreno-Towe TorressTTToesss al., 2012). There are two dominant shift types: covariate shift? and concept shift. The former usually
Texturess (Cimpoi et al., 2014), SVHN (Netzer et al., 2011), Place365 (Zhou et al., 2017), LSUN- Crop/Resize (Yu et al., 2015), and iSUN (Xu ct al., 2015) Covariate shift is often associated with model calibration (Ovadia et al., 2019; Chan al., 2020)

---

refers to change in style, e.g., clean to noised and natural images to cartoon, and the latter refers to change in semantics, e.g. dog to cat. Even though existing works Hsu et al. (2020) have acknowl- edged the different types of OOD datasets and the difficulty of detecting them, their methods do not distinguish them and do not explicitly disentangle them. To study these two distribution shifts, propose create benchmark with multiple magnitude of distribution shift in each dimension. Specifically, we use corrupted CIFARIOC MARIIO//IIARR 100C (Hendrycks & Dietterich, 2019), originally developed for testing robustness, with varying degrees of severity to represent increasing covariate shift; we also introduce a new benchmark, CIFAR 100 Splits, which divides the CIFAR 100 dataset into 10 splits with increasing conceptual shift from CIFARIO classes, measured by semantic simi- larity in a word embedding space (Pennington et al., 2014). We show that the increasing conceptual difference measured by semantic similarity in language translates to a spectrum of OOD datasets under gradual concept shift measured by the difficulty of OOD detection in the image space.
To address these two distribution shifts, we need to approach from two aspects: representation and modeling, analogous to the role of entropy and Bayesian inference in uncertainty quantification (De- peweg et al., 2018) where entropy is a score function that represents uncertainty and Bayesian in- ference is method that models uncertainty (Hüllermeier & Waegeman, 2021). To represent these two shifts, we can derive score functions which output scalars indicating the severity of distribution shift. Specifically we derive two score functions by expanding and decomposing the KL divergence between a predictive distribution from a classifier and a discrete uniform distribution. Arising from the decomposition is covariate shift score which is function of feature norms and a concept shift score which is a function of feature angles. To model these two shifts, and motivated by the covariate score (a function of norms) and the concept score (a function of angles), we propose to directly im- prove the sensitivity of norms and angles to distribution shift through a geometric perspective (Tian et al., 2021). Specifically, we adopt the Geometric Sensitivity Decomposition proposed in Tian al. (2021), originally developed for model calibration, which decomposes norms and angles into a vari- ance component and scalar offset, and improve it by parametrizng the scalar offsets as standalone networks. Our approach improves detection of OOD data under both covariate and concept shifts.
While OOD detection and calibration has been studied separately in the literature, they share the same underlying motivation: confidence//unertaing should be low/high when distribution shift oc- curs. Our method to improve OOD detection naturally yields a powerful calibration function in the family of intra class-pressd preserving functions (Rahimi et al., 2020). This class of functions provides enough expressive power to calibrate complex decision boundaries in neural networks and proves to be effective in calibration 2017) method on a validation set as described in those works. Additionally, thanks to the improved sensitivity distribution shift, our model also is on par with state-of- of-te-of-the-rr calibration performance on distribution shifted data as well. To summarize our contributions:
in-distribution data. Therefore, we apply post-calibration (Guo al.,
We propose to characterize the spectrum of OOD data in terms of covariate and concept shift and unify the notion of near OOD in different literature (Sec. 3.1.. Additionally, we construct a new benchmark (e.g., CIFAR100 Splits) to represent gradual distribution shift in each direction (Sec. 4.2 and 4.1.3).
To represent distribution shifts, we analytically derive two OOD score functions based on KL-divergence to capture both shifts more effectively (Sec. 3.2).
To model distribution shifts, we propose Geometric ODIN to improve the sensitivity of neural networks during training (Sec. 3.3) and calibration during inference (Sec. 3.4). This method achieves ofte-of-thhe-rt performance on both detection (Sec. 4.1) and calibration (Sec. 4.2) of OOD data.
2 RELATED WORK
(OOD) detection methods can be largely divided into two camps depending on whether they require OOD data during training. Hendrycks et al. (2018); Thulasidasan et al. (2020); Roy et al. (2021) leverages anomalous data in training. Our method belongs to the class of methods that do not assume the availability of OOD data during training. Hendrycks & Gimpel (2016) uses the maximum softmax probability (MSP) detect incorrect predictions and OOD data. Lee et al.
OOD detection focuses concept-shifted data while calibration focuses on covariate-shifted data.
2

---

Concept Shift P..0KK) PozOP13
Covtrate Shin Pu.00 Fox
CRAS2S
9 189
18
C
Figure 1: Illustration of near and far
data. Top left: CIFARIO dataset (training).
Top right: CIFARIOR 100 dataset. Lower left: Corrupted CIFARIO. 10. Lower right: SVHN dataset.
(2018) proposes to use Mahalanobis distance by fitting a Gaussian mixture model (GMM) in the feature space. Mukhoti et al. (2021) uses log density of the GMM model instead. Liu et al. (2020b) uses an energy score the uncertainty metrie. ODIN (Liang et al., 2017) uses combination of input processing and post-traine post-training tuning to improve OOD detection performance. Generalized ODIN Hsu et al. (2020) (also Techapanurak al. (2019)) includes an additional network in the last layer to improve OOD detection during training. There are other interesting OOD detection approaches without OOD data such as using contrastive learning with various transformations (Winkens al., 2020; Tack et al., 2020), training a deep ensemble of multiple models (Lakshminarayanan et al., 2016) and leveraging large pretrained models (Fort et al., 2021). They require extended training time, hyperparameter tuning and careful selections of transformations, whereas method does not introduce any hyperparameters and has negligible influence on standard cross-entropy training time.
Model Calibration methods also be largely divided in two categories: 1) training time calibra- tion using augmentations (Thulasidasan et al., 2019; Jang et al., 2021), using modified losses (Kumar al., 2018); 2) post-hoc calibration (Guo et al., 2017; Kull al., 2019; Kumar et al., 2019; Rahimi et al., 2020). Recently, Rahimi et al. (2020) formally generalizes a family of expressive functions for calibration, the intra order-presee order-preserving functions. This class of functions has more representation power to calibrate more complex decision boundaries in neural networks. Our proposed method be- longs to this class of functions. However, these works focus on calibration of in-distribution data. To obtain better calibration on OOD data, on which the confidence of a model needs to decrease ac- cordingly, sensitivity and deterministic uncertainty modeling is explored by SNGP (Van Amersfoort et al., 2020) and DUQ (Liu et al., 2020a). Our proposed model not only belongs to the family of intra order-preserving functions, which ensures good in-distribution calibration, but also improves sen- sitivity distribution shift, which improves out-of-dit distiiiiiiibttonn calibration simultancously. Please refer to Appendix 6.8 for a more detailed discussion on related works.
3 METHOD
3.1 MOTIVATION: COVARIATE AND CONCEPT SHIFT
We follow Moreno- Torres et al. (2012) for the formal definition of distribution shift, covariate shift and concept shift. Let X E RD denote the covariate which is the input and Y E R denote the output label. Distribution shift happens when the training joint distribution is not equal to the testing joint distribution PX, # Px,)(X,,)) Y). Covariate shift appears when Per P()(( = Pise X) and (X). Concept shift appears when (tr X) * (Yse(Y|X) and (X) (X) * (X) However, in the image domain, it is rare that such concept shift occurs without changes in P(X).
We therefore modify the equality in concept shfit to P., (X) Ri (X) where superficial, low-level statistics may be retained.
Specifically, covariate shift happens when the testing data is non-semantically different from the training data and concept shift happens when the testing data is semantically different from the training data. We illustrate a spectrum of OOD dataset in Fig. 1. this example, CIFAR10 the training dataset. CIFAR100 represents concept shift because CIFAR 100 has non-overlapping label space, i.e., (X # Ptse (Y)x). but similar style with CIFAR1O. The corrupted CIFARIOC 10C dataset (Hendrycks & Dietterich, 2019) represents covariate shift because it has the same labels but
3

---

Score Functions
Geometric ODIN
Covariate Shift Madl Po) 6r)
Feature
|1
Training
#
|wwlll.com cos
Logits
Predictive Distribution
Data
Calibration
lIwill, s
SoftMax
Diagram of Score functions, Geometric ODIN training and calibration The paper proposes two score functions and an OOD detection and calibration method: Geometric ODIN. Loss backprobagated all components during training and only to and , during calibration.
Figure 2:
different style, i.e., (X) * (X), compared to CIFARIO. SVHN represents both covariate and concept shift due to non-overlapping label space and very different style.
As shown in Fig. 2, the first goal of this paper is to derive score functions to represent the shift in either P(X) or PX) We denote the score function that reflects change from (:r(X) to Pist (X) as the covariate shift score function: g(x) RDD R. and the score function that reflects change from (Y to (Y as the concept shift score function: hly, x) y x RD - R. The second goal is to improve the sensitivity of these scores to their corresponding distribution shift. Specifically, we propose a geometrically inspired out-of-distribution detection method with only in- distribution data (Geometric ODIN). The third goal is to show that, unlike prior works which treat OOD detection (Mukhoti et al., 2021) and calibration (Van Amersfoort et al., 2020; Liu et al., 2020a) separately, our proposed Geomeric ODIN naturally leads an expressive calibration function in the family of intra order-preserving functions (Rahimi et al., 2020), which ensures good calibration.
3.2 COVARIATE AND CONCEPT SCORE FUNCTIONS
In this section, we theoretically derive two score functions, g(x) and h(y, x). based on the KL- divergence between uniform distribution U and predicted distribution R.M, where M is the number of classes. By starting from KL-divergence, we hinge subsequent derivation of score functions physical meaningful uncertainty measure, i.c., how far the predicted distribution is from uniform distribution. This relationship ensures a natural interpretation of score functions because predictions on distribution shifted data should have larger uncertainty, i.e. smaller dis- tance from uniform. We are specifically interested in softmax-linear models for classification. They typically consist of feature extractor and linear layer followed by a softmax activation. Let f RD denote feature vector from the feature extractor". The output of the linear layer, i.e., logits, 1 E R =< f, W defined as the inner product between the feature vector and a weight matrix in the linear layer. Let 4 || ///ll ll2 cos o, denote the ith logit, P. Σト exply
exp denote
the predicted probability of the ith class. The Kl.-divergence (U PP) can be written as following:
M
KL(U = -- Σ M
In MP, In Σ exply
Σ
In
j=1
Log Sogg-Sum Exy
Now we can use the inequality property of Log-Sum-Exp (LSE) functions in Eq. 2 to bound Eq. 1.
M
max ly < In
exply S maxi, + M
(2)
j=1
Therefore the KL-divergence (Eq. 1) can be bounded as follows:
M KL <u
(3)
*Bold letter indicates vectors
"Note that the negative L.S. function also defined as free energy Liu (2020h).
4

---

where 21 max, M Σi= 4. 2 can be further decomposed into two multiplicative components by plugging in the definition of logits 4;:
s(x)
M
M
= maxla
Σ M ==1
1
max || cos bb
Σ
will2 cos
M
A(y.m)
We define the covariate shift score function as g(x) - the sum of squared activation values and only depends on the input. Intuitively, activation of neural network on covariate-sho -hifttdd data should be weaker than in-distribution data. Therefore, g(x) assigns higher value to ni-distribution data than OOD data. We define the concept shift score function as h x) A max, [|w, ll2 cos 0j - 1 Σm= l/w,ll2 cos@, because it is the difference between the cosine distance of the predicted class and the average cosine distance of all classes and depends on both the input and final class membership. assigned by the max operator. Intuitively, class assignment should be less obvious under concept shift and the difference should be small. Consequently, h(y, x) assigns a higher value to in-distribution data than to OOD data. retrospect, our definition of covariate shift and concept shift scores supports existing findings that the feature norms correspond to intra-class variance and angles reflect inter-class variation (Liu et al., 2018). Intuitively, covariate shift represents non-semantic change within a specific class, i.e., intra-class variance; concept shift represents semantic changes, i.e, inter-class variation. Here we formalize the intuition and observations in Liu et al. (2018) as score functions derived analytically from a KL-divergence viewpoint. We provide an extended discussion of these scores in Appendix 7.
[|r|2 because the norm of a feature vector
M
More importantly, the combined score function (Eq. 4) carries a physical meaning: it bounds the KL-divergence between uniform distribution and the predictive distribution. Intuitively, small U indicates large uncertainty because 2/ upper bounds KIL [ and a large U indicates small un- certainty because it also appears in the lower bound. We can derive U also by softmax equation as in Liang et al. (2017), which claims that 2/ is responsible for good OOD detec- tion. Our findings confirm this and push it further by decomposing it into different components. A similar scoring function, S(x)= = [l max; [w, |l2 cos Φj. is used in Tack et al. (2020), but is only empirically motivated based on observations with limited analytical insights such as its relationship to uncertainty and the functionality of each of components. In contrast, our derivation clearly shows the relationship between these score functions and the KL-divergence, which is as an uncertainty measure, and disentangles their roles. We will compare the g(r), h x) and Ui as score functions subsequent experiments and analyze their respective sensitivity to different shifts (Sec. 4.1).
the
3.3 GEOMETRIC OUT-OF-DISTRIBUUS
DETECTION WITH IN- DISTRIBUTION DATA
As derived in Eq. 4, the covariate score is function of feature norms and the concept score is function of feature angles. Consequently, improving the sensitivity of feature norms and feature angles to data shifts seems to be the natural next step improve OOD detection. Therefore, adopt Geometric Sensitivity Decomposition (GSD) (Tian et al., 2021) (reviewed in appendix 6.3) improve sensitivity covariate and concept shifts. Specifically, GSD improves sensitivity by extracting sensitive components from norms |f' and angles | through a decomposition of them into: scalar offset and variance component shown in Eq. 5. Scalar offsets Cy and C. minimize the loss on the training set and the variance components f and Φ; account sensitively for variances samples.
| 2|2
|f[la Cj
|Φ]1 = l0v - |Col
With the decomposed components, the original logit 17 = ||[*||/||w/] cos oi. can be written as:
1 = ||f" ||/l/ cos of 2
||f|| 12 +
C₁
|| l| cos Φ;
(6)
cos
cos Co
TThe superscript denotes the original component before decomposition.
5

---

where l denote the new ith logit. In Eq. 6, the new" feature is direct output of feature extractor, and is modified by a and 3 as also illustrated Fig. 2. Note that the calculation of score functions in Sec. 3.2 only uses the feature and is independent of and g.
Because cos C. and y are scalar offsets, we can parametrize them separately from the main network. Unlike GSD which parametrizes them as instance-indee intrance-iddeeeneent scalars, inspired by Hsu et (2020); Techapanurak et al. (2019), we make (f) and (f( destance-dependent scalars and use a single linear layer to learn them. To enforce numerical constraints, i.e., 0 < a 1 and /3 0, ((f) uses sigmoid activation and (()) uses softplus activation. Finally, the relaxed output is:
exp ((t))) || + x(F)) || cos
((f)
expl ΣE= exply
P(Y = il,a) =
Σ3 exp (( a(T) |f +
3( a() ||wj l2 cos 65)
Now the new predicted norm ||f]|2 and angle o; are more sensitive input changes because they encode variances in samples as shown in Eq. 5. Therefore, including / (related to norms) improves sensitivity covariate shift and including a (related angles) improves sensitivity to concept shift. Note that, under this construction, Generalized ODIN (Hsu et al., 2020) is special case of our proposed method. Generalized ODIN only includes the (f) which only improves angle sensitivity but not norm sensitivity. Unlike Hsu et al. (2020)'s probabilistic perspective our model builds geometric perspective and captures both covariate and concept shifts by improving norm and angle sensitivity. The new model can be trained identically the vanilla network without additional hy- perparameter tuning and extended training time. Combined with score functions derived in Sec. 3.2, Geometric ODIN achieves state-of-the-art OOD detection performance (Sec. 4.1)
3.4
CALIBRATION OF GEOMETRIC ODIN
Like most softmax-linew classification models, Geometric ODIN also suffers from miscalibration right after training. Specifically, predictions tend be overconfident (Guo al., 2017). In the case of Geometric ODIN, the cause of overconfidence is obvious by construction. The offset scalars ((f) and (ff) are designed to minimize the training loss. Practically, they are optimized to push the predicted probability of the ground truth class to be close to 1 during training, and this intended behavior unsolicitedly continues to inference time when accuracy not as high as during training- func- tions, which has potentially strong representation to calibrate complex functions in deep networks according Theorem 1 in Rahimi et al. (2020). To prove that, it's suffice to show that that (:l) and ((1) forms a strictly positive function, mIl) as shown in Eq. 8
Nevertheless, construction of (f) and ((f) belongs to family of intra
3(1) 4; 4- (a1) |ff| + a(l)
||wil2 c cos = ((c1) + ((1) a())
3(1)
|| 22
W:
1//22 cos Φ;
(8)
a(1)
m())
Because 0 < a11) < 1 and 3() > 0, the function m(l) is strictly positive and thus satisfies Theorem in Rahimi et al. (2020) (See appendix 6.4 for more details). We follow a simple procedure proposed in Guo et al. (2017); Rahimi et al. (2020) to calibrate Geometric ODIN by minimizing the negative log likelihood (NLL) on a evaluation dataset for a few epochs. During calibration the gradients to other parts of the model are stopped and only these two linear layers, a()) and 5(1), are optimized (Fig. 2). Naturally with the improved sensitivity and calibrated functions, the calibration models
both in-distribution and OOD data. Note that, as mentioned in Sec. 3.3, because the calculation of score functions for OOD detection only depends on the feature and is independent of a and B, calibration does not affect the OOD detection performance and does not change prediction results due to the prder-preserving property.
of Geometric ODIN is on par with state-of-the-ee
"Even though both f* and f are outputs directly from the feature extractor, we use original and new to indicate whether GSD is applied.
* *f()) interpreted as P(d,, (x), the probability of x heing in-distribution.
FFllooing Rahimi et al. (2020), which uses logits as input instead of the feature f. Our derivation in previous sections is still valid because salely depends f given model.

---

AUROC
Sonee Parations
CHPARII
IICERPRRO
Nea:
Flr SVAN
Far
Near
Nour
Eai
Far
CIFARIO
TINER)
CIFARIOG
CIFARIOC
SVHN
TINIRO
MSP (Ilenryycic Cieppe, 20100
A1.5510.14
70-56-00 9M
0% 01:66-1.29
Vinill Wide-RtaNaa----111 HI
Energy (1 au al, a0h000
M6.74 9,45
79
12.01
83. 12000-11
RX 0914-0444
70.00-0 50
96 19 01330
38 18.16-11
(Zapeayloa & Krmastaii 2006)
Matumorii iI
2018))
01-99 18
AK 18:16.111 50
80 1:1
GMM Dessity Ofalluui 2021)
08.0189711
76.58600
75.71:00.00
19:0222
97.32=1.30
BUQ (ian Amnzudoutt a., 2020)
Kemnll Distance
95.224.55 35
SNOP tLa st 2000aa
SoliMax Eatrey
85.712981
91.333 15
013 20
(Nhithor al 2001)
CMMM Deruity
87
M0 -0-00-4
26.00-11110
97 12-1120 21
8088-10
Hbpoofirrttaaaaale S.loornie (1
2016)
SottMaa Hhrrapy
19.5619991
92.13-0100
07-21-0031
Dur assjamll
Myy x)
10.24
83:2543..0
$2.720022
02:29:00155
71.0020.00
V7.5630.00
91:31000
Oure oxii
M4. 0.16
01.15 12.05
10.34.10.4
89 6434
76.00 10
09 40
00110.1144
98.74 10.79
7142:0027
95.77:0.33
50.5659.33
02.31:0.33
78:00:171
99 99.54:9.155
92.85-0119
Table I: AUROC 1 for Near and Far OOD detection. Results are averaged over 5 runs. Our (z)-onny model is the same as Generalized ODIN (Hsu et al., 2020) (h' (h'(r) variant) without its input processing. Hyper-free (Techapanurak et al., 2019) and Generalized ODIN (1*(x) variant) are the same. * denotes results from Mukhoti et al. (2021).
4 EXPERIMENTS
In this section, we present results for Sec. 4.2. We are the first to present a method that works well on both OOD detection and cal- ibration, disentangling shift types. Implementation: Following prior works (Liu et al., 2020a; Van Amersfoort et al., 2020; Mukhoti et al., 2021), we use Wide-ResNet-28-11 (Mukhoti et al., 2021) for all experiments. We train the model using SGD with an initial learning rate of 0.1 for 200 epochs. The learning rate is annealed with a cosine scheduler (Loshchilov & Hutter, 2016) (more details in appendix 6.5). Metries: For OOD detection in Sec. 4.1, we use the common AUROC and TNR@TPR95 as benchmark metrics. For calibration in Sec. 4.2, we use the Expected Calibration Error (ECE) (Guo et al., 2017) with 15 bins and Negative Log Likelihood (NLL) which is a strictly proper scoring rule (Gneiting & Raftery, 2007). Datasets: CIFARIO (Krizhevsky et al., a) and CI- 100 (Krizhevsky et al., b) are considered near OOD datasets (Winkens et al., 2020; Fort et al., 2021) to each other. SVHN (Netzer et al., 2011 1) is considered far OOD dataset to both CIFARIO and CIFAR100 due to its shift in both concept and style. The CIFARIOC/CIFARII & Dietterich, 2019) dataset is variant of CIFARIO/CIFAL TO/CIOAAARRTTT 100 corrupted by 15 types of noises with 5 degrees of severity. We also introduce CIRAF100 Splits benchmark which consists of 10 datasets with increasing concept shift from CIFARIO classes (Sec, 4.1.3).
detection in Sec. 4.1 and calibration in
(Hendrycks
4.1
DETECTION RESULTS
4.1.1 NEAR AND FAR OOD DETECTION
The a-9 model is the best. In Tab. 1, we present OOD detection results against state-of-the-an methods on existing near and far OOD categorization. For near OOD detection under strong con- cept shift, CIFARIO (ID) vs. CIFAR 100 (OOD), both our a-only and a-3 variants achieve the the best performance. This demonstrates that a(xx) improves the sensitivity of angles and hence the sensitivity to concept-shifted data. For near OOD detection under strong covariate shift, CIFARIO (ID) vs. CIFARIOC (OOD), the a-9 variant achieves the the best performance. This suggests that 3(x) improves the sensitivity of norms and hence the sensitivity to novariate-shifter data. In CI- FAR100 (ID) vs. CIFARIO (OOD) experiments, the performance of the an-oly and a-3 variants are within variance and is close to some other compared methods, we can not make clear observations from those experiments 10_ For far OOD detection, CIFAR 10/CIFAR 100 (ID) vs. SVHN (OOD), the a-3 model achieves state-of-the-ar performance. This reconfirms that 3(x) improves sensitivity to covariate-shiffe data, because SVHN has both covariate and concept shifts compared to the CIFAR datasets, and the a-3 model outperforms the a-only variant, which only improves on concept shift, by noticeable margin. In terms of score functions, the best performing one for the a-9 model is U, which is a product of g(x) and h x) (Sec. 3.2), while that of the a-only model is ly x). This shows that depending on which component is more sensitive, different scoring functions are
Otthe confounding factors could contribute to the close performance. Prior works either omit comparisons under these settings (Mukhoti al., 2021) report only marginal improvement (Winkens et al., 2020)
7

---

O- 3i AUHOC
D-CRYy AUROC
p-snty AUROC
varils AUROC
AUHDOC Laing Score Function
elki higal
NUROC
AUROC
AUROC
AUROC
AUROC
0143 My.00
N15000
serurit
Servan by
Savee by
waweriy
Suveriy
Ct-3, c--olyy B-only and vanilla models using functions g(z) and h(y,a) a)
b All models using U
Figure 3: Capturing Covariate Shift (Motion Blur) All results are averaged over 5 runs. Modeling covairate shift (a-3 model) yields the best performance as shown in Fig. 3b. g(x) is more responsive to covariate shift than hyy, x) as shown in Fig. 3a with the a-B model.
2-0 AUBOC
da AUBOC
Panty AlRor
vania
AUROC
AUBOC Using Score function (
AUROC
AUROC
AUNIE
tyysi
AUROC
hlya0
Severity
severity
severity
severity
severity
Ct-B, c--nlyy B-only and vanilla models using score functions g(z) and A(y, X))
b All models using L
Figure 4: Capturing Concept Shift (CIFARI00 Splits) All results are averaged over runs. Mod- cling concept shift (both a-only and a-3 models) yields the best performance shown in Fig. 4b. h(y, x) is more responsive to concept shift than g(x) as shown in Fig. 4a with the a-3 model.
preferred. When the sensitivity of both norms and angles are improved, as in the a-3 variant, the combined score function 4/ performs well under different distribution shifts.
4.1.2 OOD DETECTION UNDER COVARIATE SHIFT WITH IMAGE CORRUPTION
The g(x) score captures covariate shift. We compare the a-3, a--oly (3 = 0), B-only (a = 1) variants and the vanilla model on CIFAR1OC (Hendrycks & Dietterich, 2019) corrupted by mo- tion blur in Fig. 3 with increasing degrees of noise (see appendix 6.7 for additional experiments). From 3a, we observe that 1) covariate shift severity increases, OOD detection becomes easier because AUROC increases with increasing severity. 2) the vanilla model is more sensitive to the concept shift component because h(y, x) > 9(x) the vanilla model plot even though covariate shift is the dominant distribution shift in this example. 3) when sensitivity to both covariate and concept shift is improved, the a-3 model becomes more sensitive to the covariate shift component. This suggests that the dominant shift in this example is indeed covariate shift. From Fig. 3b, we ob- serve that the a-3 model outperforms the 8-only model using the combined score function U. This suggests that improving sensitivity to both shifts and using U yield the best OOD detection perfor- mance. In retrospect, the performance of OOD detection has always relied on two components: the model and the score function. Some works propose more responsive score functions, e.g., energy in Liu et al. (2020b), and some works propose more sensitive models, e.g., DDU in Mukhoti et al. (2021) to improve the performance of existing score functions.
4.1.3 OOD DETECTION UNDER CONCEPT SHIFT WITH CIFAR100 SPECIAL SPLITS
Finding a dataset benchmark gradual concept shift is not straightforward because concept shift is traditionally thought as binary: overlapping or non-overlapping. However, not all non-overlapping labels are the same. For example, pickup truck (CIFAR 100) is semantically much closer truck 10) than sunflowers (CIFAR is. To create this gradual concept/semantic shift, we pro- pose divide the CIFAR 100 dataset into 10 sub-datasets with increasing conceptual difference from CIFARIC 10 classes. Specially, we use 300 dimensional Glove word embeddings" (Pennington et al., 2014) trained on the entire wikipedia2014 and Gigaword5 (Napoles al., 2012) measure seman- tic closeness (inner product) between CIFAR 100 and CIFAR10 classes. The result is 10 subdatasets split from CIFAR100. Please refer to appendix 6.6 for the full splits.
The h(y. æ) score captures concept shift. Following the previous section, we benchmark the a-3, a-only, 9-only variants and the vanilla model on the newly created CIFAR 100 Splits. From Fig. 4a, we observe that 1) as concept shift severity increases, OOD detection becomes easier because AU-
Inttps://n stanppotttffddda.o.o wtt.taafrrddddppoooooeeeeeeee wwwppp/pepppeeeeeeee/ee / ove/
8

---

ROC increases with increasing severity. 2) both concept shift and covariate shift present because AUROC using either h(y, x) or g(z) increases. 3) the vanilla model is dominantly more sensitive to the concept shift component because concept shift is the dominant distribution shift in CIFARIO0 Splits by construction and vanilla ResNet is more sensitive to concept shift (the same behavior is also observed on covariate-shiff-haavy data in Sec. 4.1.2). 4) when sensitivity to both shifts is im- proved, the a-8 model is still more sensitive to concept shift (h(y, æ) > 9(x)). This reconfirms that the dominant shift type is indeed concept shift. From Fig. 4b, interestingly, we observe that all three variants perform similarly and all outperform the vanilla model. Combined with previous observations that the dataset has strong concept shift and the vanilla model is already very sen- sitive to concept shift, improving sensitivity to the covariate shift component yields equally good performance as improving sensitivity to both shifts.
4.2 OUT-OI-DISTRIBUTION CALIBRATION RESULTS
Accuracy Comupted
BCR
NLL.
Clean
Clean
Comupted
Clean
Corrupted
Vanilla
06.23-0.13
69.78 1.22
0.01510.0011
0.148
0.048=00.05
1.07:00.022
Temp Scaling (Gin al., 2017)
96.23-0.13
69.78+1.22
D103+00001
0.107+00009
0.131-0.005
0.906+00029
Matrix scaling (Guo al., 2017)
95,98=0.10
69:81:1:111
0:005:0.000
0.107:0.009
0.145=0.008
0.966100041
Dirichlet (Kall et al., 2019)
96.10=0010
69.75.1.09
0.00410.002
0.11410.007
0.130
0.977
DUQ (Van Amersfoort SNGP (Liu et al., 2020g)
20209
94.7+0.02
71.6+0.02
0.034+00002
0,183+0.011
0.239-0.02
1.348-0.01
95.9=00.11
74.6::0.01
0.018:0.001
0,090=0.012
0.138:20.0]
0.935=0.01
GSD (Tian et al., 2021)
95.9110.01
74.9.10.05
0.008100.002
0.085
0.140
0.853.0.039
Ours: a-9
05.99-0.12
70.41+0..55
0.001=0.000
0.071-+0.0112
0.130 0.003
0,854+0029
Table 2: Calibration on CIFAR1O averaged over 5 seed. i denotes results from Liu et al. (2020a).
Accuracy 1
ECE
NLL |
Clean
Comppted
Clean
Comppted
Clean
Corupied
Vanilla
80.66 0,20
50.25:10.48
0.03510.002
0.17110.017
2.38410.039 0.774=0.007
Tomp Scaling (Guo al., 2017)
80.99=0.20
50.25:1.44
0.033:0.002
0.163:10:17
0.776=0.006
2.368110039
Matrix scaling (Guo et al., 2017)
79.31 1:025
48.74:10.55
0.03:10.003
0.160100.16
0.791
2.532110051
Dirichler (Kall et al., 2019)
80.70-0.36
50.06+00.46
0.015-0..02
0,144+0.017
0.743 0,025
2.346+00177
DUQ (Van Amersfoort al. 2020)
78.5=0.02
50:420.02
0.119:00001
0.281:0:012
0.980=0.02
284122001
SNGP (Liu et al., 2020at
79.9-0003
49.010.02
0.025110.012
0.11710.014
0.847=0.01
2.62610.01
GSD (Tian al., 2021)
79.8=0.03
49.8++0.03
0.027=00003
0.088
0.784-0.011
2.236+0.021
Ours: a-9
79.21=0.19
49.2110061
0.010:0.001
0.084110.009
0.751:0.005
2.32340.557
Table 3: Calibration on CIFAR100 averaged over 5 seeds. 1 denotes results from Liu et al. (2020a).
As proved in Sec. 3.4, our model naturally leads to a calibration function in the family of in- tra order-preserving functions (Rahimi et al., 2020), which ensures good in-distribution calibra- tion performance. Moreover, thanks to the improved sensitivity, the model can potentially achieve state-of-the-art out-of-distribution calibration as well. We benchmark our models on both clean CI- FARIO/CIFARI00 100 as well as corrupted CIFARIOC/CIFAL 1O0C (Hendrycks & Dietterich, 2019). Following prior works (Rahimi et al., 2020), use 5-fold cross validation for in-distribution cal- ibration experiments. As shown in Tab. and Tab. 3, our model is comparable to state-of-the-an models both in-distribution and out-of-distributtin calibration. In the literature, OOD detection and calibration have been studied separately, but our method and experimentation tackles both tasks.
5 CONCLUSION
In this work, we propose to characterize the spectrum of out-of-distribution (OOD) data using co- variate shift and concept shift. Unlike difficulty of OOD detection, distribution shift exposes intrinsic characteristics of OOD data. Consequently, to achieve good OOD detection performance, a model needs to consider both distribution shifts. At representation level, we derive two score functions that represent and capture each shift separately. At modeling level, inspired by these score functions, we propose a geometrically-insiree method, Geometric ODIN, to improve a model's sensitivity to both shift. Furthermore, Geometric ODIN is the first method that considers both OOD detection and calibration, targeting both concept and covariate shifts, and yields state-of-the-ar performance in both tasks. Finally, we hope that the distribution shift perspective can lead to new direction to study OOD data and unify the the fields of detection and calibration of OOD data.

---

REFERENCES
Mariusz Bojarski, Davide Del Testa, Daniel Dworakowski, Bernhard Firner, Beat Flepp. Prasoon Goyal, Lawrence Jackel, Mathew Monfort, Urs Muller, Jiakai Zhang, al. End end learning for self-driving cars. arXiv preprint /604. 1e0..60044...... 2016.
Alex Chan, Ahmed Alaa, Zhaozhi Qian, and Mihaela Van Der Schaar. Unlabelled data improves bayesian uncertainty calibration under covariate shift. In International Conference on Machine Learning, pp. 1392-1402. PMLR, 2020.
Mircea Cimpoi, Subhransu Maji, lasonas Kokkinos, Sammy Mohamed, and Andrea Vedaldi. De- scribing textures in the wild. In Proceedings of the IEEE Conference Computer Vision and Pattern Recognition, pp. 3606-3613, 2014.
Stefan Depeweg, Jose-Miguel Hernandez-Lobato. Finale Doshi-Velez, and Steffen Udluft. Decom- position of uncertainty in bayesian deep learning for efficient and risk-sensitive learning. In International Conference Machine Learning, pp. 1184-1101 1193. PMLR, 2018.
Stanislav Fort, Jie Ren, and Balaji Lakshminarayanan. Exploring the limits of diitrr--dsttibuttonn
detection. arXiv preprint arXiv:2106.0300 2021.
Tilmann Gneiting and Adrian E Raftery. Strictly proper scoring rules, prediction, and estimation. Journal of the American statistical Association, 102(477):359-377 2007.
Ishaan Guirajani, Faruk Ahmed, Martin Arjovsky, Vincent Dumoulin, and Aaron Courville. Im- proved training of wasserstein gans. In I. Guyon, U. V. Luxburg. S. Bengio, H. Wallach, R. Fer- gus, S. Vishwanathan, and Garnett (eds.), Advances in Neural Information Processing Systems, volume 30. Curran Associates, Inc., 2017. URL pttppe///proeeeiinggs nwwrips.cr.mmm cc/ paper/2017 /atee//e////////////eeeeeeeee wee.ee2c3cccceceooco (tt.ce/222666660000833366---
pdf.
Chuan Guo, Geoff Pleiss, Yu Sun, and Kilian Q Weinberger. On calibration of modern neural networks. In International Conference on Machine Learning, pp. 1321-1330. PMLR, 2017.
Dan Hendrycks and Thomas Dictterich. Benchmarking neural network robustness to common cor- ruptions and perturbations. Proceedings of the International Conference Learning Represen- tations, 2019.
Dan Hendrycks and Kevin Gimpel. A baseline for detecting misclassified and out-of out-o--distribution examples in neural networks. arXiv preprint arXiv:1610.02136, 2016.
Dan Hendrycks, Mantas Mazeika, and Thomas Dietterich. Deep anomaly detection with outlier exposure. arXiv preprint arXiv: 18/:. 04606, 2018.
Yen-Chang Hsu, Yilin Shen, Hongxia Jin, and Zsolt Kira. Generalized odin: Detecting out- of-distribution image without learning from out-of-distribution data. In Proceedings of the JEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 10951-10960, 2020.
Eyke Hüllermeier and Willem Wacgeman. Alcatoric and epistemic uncertainty in machine learning: An introduction to concepts and methods. Machine Learning, 110(3)457-506, 2021.
Sooyong Jang, Insup Lee, and James Weimer. Improving classifier confidence using lossy label- invariant transformations. International Conference on Artificial Intelligence and Statistics, pp. 4051-4055 PMLR, 2021.
Alex Krizhevsky, Vinod Nair, and Geoffrey Hinton. Cifar-10 (canadian institute for advanced re- search). a. URL www. cs toronto. adu/-kriz/ /
Alex Krizhevsky, Vinod Nair, and Geoffrey Hinton. Cifar-100 (canadian institute for advanced research). b. URL http://i :ttt:://wwww wwww cs toronto. www.otooooooooooooooooooo kriz/ /cciaararr..//////// html.
Meelis Kull, Miquel Perello-Nieto Markus Kängsepp. Hao Song, Peter Flach, et al. Beyond temper- ature scaling: Obtaining well call-cali multiclass probabilities with dirichlet calibration. arXiv preprint arXiv: 09...9911.2... 2019.
10

---

Ananya Kumar, Percy Liang, and Tengyu Ma. Verified uncertainty calibration. arXiv preprint arXiv:1909.10155, 2019.
Aviral Kumar, Sunita Sarawagi. and Ujjwal Jain. Trainable calibration measures for neural networks from kernel mean embeddings. In International Conference on Machine Learning, pp. 2805- 2814. PMLR, 2018.
Balaji Lakshminarayanan, Alexander Pritzel, and Charles Blundell. Simple and scalable predictive uncertainty estimation using deep ensembles. arXiv preprint arXiv:16/2.01144, 2016.
Kimin Lee, Kibok Lee, Honglak Lee, and Jinwoo Shin. simple unified framework for detecting out-of-distribution samples and adversarial attacks. Advances in neural information processing systems, 31, 2018.
Shiyu Liang, Yixuan Li, and Rayadurgam Srikant. Enhancing the reliability of out-of-distribution image detection in neural networks. arXiv preprint arXiv: arXiv:/706.02690 2017.
Jeremiah Zhe Liu, Zi Lin, Shreyas Padhy, Dustin Tran, Tania narayanan. Simple and principled uncertainty estimation with deterministic deep learning via distance awareness. arXiv preprint 10X1:22006.10108, 2020a.
and Balaji Lakshmi-
Weitang Liu, Xiaoyun Wang, John Owens, and Yixuan Li. Energy-based diitt---dittibutionn detection. arXiv preprini arXiv:2010.03759, 2020b.
Weiyang Liu, Zhen Liu, Zhiding Yu, Bo Dai, Rongmei Lin, Yisen Wang, James M Rehg, and Le Song. Decoupled networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2771-2779 2018
Iyy Loshchilov and Frank Hutter. Sgdr: Stochastic gradient descent with warm restarts. arXiv preprint arXiv:1608.0388 2016.
Takeru Miyato, Toshiki Kataoka, Masanori Koyama, and Yuichi Yoshida. Spectral normalization for generative adversarial networks. arXiv preprint arXi::1802.05957. 2018
Jose G Moreno-Torres, Troy Raeder, Rocío Aldiz-Rorr Nitesh Chawla, and Francisco Her- rera. A unifying view on dataset shift in classification. Pattern recognition, 45(1))521-553 2012.
Jishnu Mukhoti, Andreas Kirsch, Joost van Amersfoort, Philip HS Torr, and Yarin Gal. Deterministic neural networks with appropriate inductive biases capture epistemic and aleatoric uncertainty. arXiv preprini arXiv:2102. 1582, 2021.
Mahdi Pakdaman Naeini, Gregory Cooper, and Milos Hauskrecht. Obtaining well calibrated proba- bilities using bayesian binning. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 29, 2015.
Courtney Napoles, Matthew Gormley, and Benjamin Van Durme. Annotated gigaword. In Proceedings of the Joint Workshop on Automatic Knowledge Base Construction and Web-scale Knowledge Extraction (AKBC-WEKEX), pp. 95-100, 2012.
Yuval Netzer, Tao Wang, Adam Coates, Alessandro Bissacco, Bo Wu, and Andrew Y. Ng. Reading digits in natural images with unsupervised feature learning. 2011. URL http::/ufld/ stanford. edu wwd.hhusseummeee.. http/mmbrrrr/ninnnnnnnmmm htt.200111_tuuuummbrrrr.... pdd..
Yaniv Ovadia, Emily Fertig, Jie Ren, Zachary Nado, David Sculley, Sebastian Nowozin, Joshua V Dillon, Balaji Lakshminarayanan, and Jasper Snock. Can you trust your model's uncertainty? evaluating predictive uncertainty under dataset shift, arXiv preprint
2019.
Jeffrey Pennington, Richard Socher, and Christopher D Manning. Glove: Global vectors for word representation. In Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP), pp. 1532-1543, 2014.
Amir Rahimi, Amirreza Shaban, Ching-An An Cheng, Richard Hartley, and Byron Boots. Intra order- preserving functions for calibration of multi-class neural networks. Advances in Neural Informa- fion Processing Systems, 33:13456-1346) 2020.

---

Abhijit Guha Roy, Jie Ren, Shekoofeh Azizi, Aaron Loh, Vivek Natarajan, Basil Mustafa, Nick Pawlowski, Jan Freyberg, Yuan Liu, Zach Beaver, et al. Does your dermatology classifier know what it doesn't know? detecting the long-tail of unseen conditions. arXiv preprint arXiv:2104.0360 2021.
James Smith, Jonathan Balloch, Yen-Chang Hsu, and Zsolt Kira. Memorr--fficceettsssisssseeeeeee continual learning: The world is its own replay buffer. arXiv preprint arXiv:2101.09536, 2021.
Jihoon Tack, Sangwoo Mo, Jongheon Jeong, and Jinwoo Shin. Csi: Novelty detection via contrastive learning distributionally shifted instances. arXiv preprint arXiv:2007.08776, 2020.
Engkarat Techapanurak, Masanori Suganuma, and Takayuki Okatani. Hyperparameete-ffee out-of- distribution detection using softmax of scaled cosine similarity. arXiv preprint arXiv: 10628, 2019.
Sunil Thulasidasan. Gopinath Chennupati, Jeff Bilmes, Tanmoy Bhattacharya, and Sarah Michalak. On mixup training: Improved calibration and predictive uncertainty for deep neural networks. arXiv preprint arXiv: /000) 2019.
Sunil Thulasidasan, Sushil Thapa, Sayera Dhaubhadel, Gopinath Chennupati, Tanmoy Bhat- tacharya, and Jeff Bilmes. A simple and effective baseline for out-of-distribution detection using abstention. 2020.
Junjiao Tian, Dylan Yung, Yen-Chang Hsu, and Zsolt Kira. A geometric perspective towards neural calibration via sensitivity decomposition. In NeurIPS, 2021.
Joost Van Amersfoort, Lewis Smith, Yee Whye Teh, and Yarin Gal. Uncertainty estimation using single deep deterministic neural network. In International Conference on Machine Learning, pp- 9690-9700. PMIR, 2020.
Martin Wattenberg, Fernanda Viégas, and Ian Johnson. How to use t-sne effectively. Distill, 1(10): e2, 2016.
Jim Winkens, Rudy Bunel, Abhijit Guha Roy, Robert Stanforth, Vivek Natarajan, Joseph R Ledsam, Patricia MacWilliams, Pushmeet Kohli, Alan Karthikesalingam, Simon Kohl, et al. Contrastive training for improved out-of-distribution detection. arXiv preprint arXiv:2007.05566, 2020.
Pingmei Xu, Krista Ehinger, Yinda Zhang, Adam Finkelstein, Sanjeev Kulkarni, and Jianxiong Xiao. Turkergaze: Crowdsourcing saliency with webcam based eye tracking. arXiv preprint 150.0055 2015.
Fisher Yu, Ari Seff, Yinda Zhang, Shuran Song, Thomas Funkhouser, and Jianxiong Xiao. Lsun: Construction of large-scale image dataset using deep learning with humans in the loop. arXiv preprint IrXiv:1506. 5336,0366,, 2015.
Qing Yu, Daiki Ikami, Go Irie, and Kiyoharu Aizawa. Multi-task curriculum framework for open-set semi-supervised learning. In European Conference on Computer Vision, pp. 438-454. Springer, 2020.
Sergey Zagoruyko and Nikos Komodakis. arXiv:1605.07146. 2016.
Wide residual networks.
arXiv preprint
Bolei Zhou, Agata Lapedriza, Aditya Khosla, Aude Oliva, and Antonio Torralba. Places: A 10 million image database for scene recognition. IEEE transactions on patlern analysis and machine intelligence, 40(6):1452-15 2017.
12

---

APPENDIX
6.1 INTRODUCTION TO
DETECTION
Out-of-Distributie detection the task of separating in-distribution data from out-of-distribution (OOD) data. For example, classifier, trained on a training distribution (x, Y), is tested on dif- ferent test joint distribution Ptst (t,r(X,Y), Y). When the test joint distribution Pise (X, # (rr(X,Y), data sampled from Pr (X, Y) are considered OOD data. (X,, Y) can be different in two ways, ie., covariate shift and concept shift (Sec. 3.2). Data sampled from the same distribution as train- ing, (x,(X, , Y), such as a conventional validation/test split, are in-distribution (ID) data. To detect OOD data, a score function used such as maximum softmax probability (Hendrycks & Gimpel, 2016) and energy (Liu et al., 2020b). The score function assigns higher value OOD data than ID data. Consequently, OOD data can be detected by setting a suitable threshold. The commonly used benchmark, Area Under the Receiver Operating Characteristic curve (AUROC), plots true positive rate of ID data against false positive rate of OOD data by sweeping through a range of thresholds, and is weissolld-idependenn metric for measuring OOD detection performance.
6.2 INTRODUCTION CONFIDENCE CALIBRATION
Colloquially, confidence calibration refers to aligning the maximum predicted probability (confi- dence) from multi-class classifier to the empirical accuracy of the predicted class to be the ground truth. For example, if classifier classifies 100 images to dog with confidence of 0.8 for each prediction, than 80 out of 100 those images should contain a dog. Formally, we adopt the definition from Kull et al. (2019) to define confidence calibration.
Let P X - A be a probabilistic multi- class classifier for M classes. For any input x E X, P(x) = (P, (z), (M(()) is probability vector.
Definition 1 A probabilistic classifier P: : X + A2 is confidence-caliirraeedd for any E [0, I]
P(Y arg max (xx) max (x) = =p p
One notion of confidence calibration is the expected difference between the confidence and accuracy.
Ep P(Y = arg max Pxx) max P(x)) p)
Expected Calibration Error (ECE) (Naeini et al., 2015) (Guo et al., 2017) mates this expectation. ECE partitions predictions into several M equally-sized bins according to the predicted confidence, and then calculates average accuracy and confidence for each bin. Let and ý be the predicted class and confidence of input with corresponding label y. B,, denotes the n-th bin. The accuracy and confidence are calculated for each bin as follows:
7) is a metric that approxi-
1
aco(Bn)
Σ 1( = m)
(9)
[Bal
iEB.
conf(BBn) =
Σ P.
(10)
Bal
iEB,
Than ECE is defined weighted sum of the difference between confidence and accuracy in each bin. Therefore, lower ECE indicates better calibration.
Bu (acc(B,)) c
ECE
6.3 GEOMETRIC SENSITIVITY DECOMPOSITION REVIEW
Geometric sensitivity decomposition (GSD) is proposed in Tian et al. (2021) to improve the sen- sitivity of neural networks to input changes. Specifically, GSD views the output of the last linear layer, i.e., logits, in softmax-linear model inner products, between the feature and weights in
13

---

the linear layer, E RM =< f, W >. The inner products be written products of norms and cosine similarity, c.g., the ith logit is 5 = ||f 2 wallz cos cos GSD proposes to decompose instanceddepnddent residual component and norms, || and angles, 67 into two components: an instance-indepenndet scalar as shown in Eq. 11. The instance-indepennennt scalar acts as freely optimizable parameters during training minimize training loss.
f 1l2 = || + Cj | |o - Ccol
(11)
Decomposed components in Eq. can be plugged into equation for logits.
|| 112 cos of ||f* ||2 cos | (llf 22 + Cy) cos (10il (C/l)
(12)
cos |Cg sin losa 1
= (I1fl12 Cy) cos (col
cos loil
1 1 sin 1csl
sin |Cyll cos (
E.q. 13
Eq. 12 can simplified by assuming that the angle | is small. This equivalent assumes that training data closely clustered to linear classifiers specified by the linear weight W.
cos sin [osl sin (lo.) (C&l) + sin |d
(13)
sin [Col cos kisl sin (lod + [Cal) sin b
Therefore, logits can be written as the following.
|| |2 wolllillle cos of ~
||f||2
Cj
|/wll2 cos
(14)
cos Co.
cos Co
Because cos Co and Cj instance-independerr network as a and 3 respectively.
parametrize them separately from the main
6.4 INTRA ORDER PRESERVING FUNCTIONS REVIEW
Rahimi al. (2020) formalizes family of powerful calibration function: intra order preserving functions. Many familiar functions are in this family such as the softmax function and temperature scaling (Guo et al., 2017). Formally, a function f RD - R" intra order-preserving for any E RR, both and f(x) share share the same ranking. The following theorem from Rahimi et al. (2020) provides sufficient and necessary conditions for constructing an intra order preserving function.
Theorem A continuous function f: RD + RD is intra order-preserving, if and only f(x) = (wx(() with U being an upper-triangular matrix of ones and R R being continuous function such that
wi(x) = vf = Vi-1 and < n.
w;(x) > 0 y: > Vi, and < n,
wx(x is arbitrary
where y y==St S(x)x the sorted version (descending, i.e., y1 2 82) of and S(x) : R" R" is
sorting matrix.
The task of constructing an intra order preserving function comes down to designing the function
w(x). A specific form of w(x) is proposed in Rahimi et al. (2020): wi(x) a(yi yi m
14

---

As long is positive function, i.e., a(z) 0 when z and (xx) 0 otherwise, and m(x) > 0 is a strictly positive function, the contructed function will satisfy Theorem 1.
Our construction of and 3 satisfies the necessary conditions. For calibration, the function f acts on the logits so we use instead of x in subsequent proof. Let 4; = |fflz||w:llz cos Φ; denote the so ith logit and I, = (2d)) [fll2 + atl) |/wi]l2 cos @ detnote the actual output ith logit (sorted) with a and B. The goal is to show that and share the same ranking by satisfying Theorem 1. We start by extracting the m(x) function in Eq. 15.
a(1)
8() 4 (2)) 1f a(1)) l/ 22 cos oi
1 (1))
B(1)
a(1)
|| wi cos di
(15)
m(
Therefore, we can construct the w(x) function by taking the difference between two adjacent logits for <
30)
wi(l) = 1144-11- a(1) +
(4 4-1).
for i< n
at)]lfll2
(16)
m()>00
wI( 4
for i=
where 4, 2 4-11
Because 0 (1) and ((1) > O, the function m(1 is strictly positive and thus satisfies Theorem I in Rahimi et al. (2020). Specifically, m(l) corresponds to the strictly positive function m(x) and 4- 4-1 corresponds to the positive function o1ys 31-1), due the sorting descending order.
6.5 IMPLEMENTATION DETAILS
Duttof-Distribution Detection Following prior works (Liu et al., 2020a; Van Amersfoort et al., 2020; Mukhoti et al., 2021), we use Nede-ResNett22-110 (Mukhoti et al., 2021) for all OOD detection experiments. We train the model using SGD and cross entropy loss with an initial learning rate of 0.1 for 200 epochs. The SGD optimizer is configed with 5.0e weight decay and 0.9 momentum. Batch size is 128. The learning rate is annealed with a cosine scheduler (Loshchilov & Hutter, 201 6). We adopt the official "3de for the implementation of DDU (Mukhoti et al., 2021). DDU (Mukhoti et al., 2021) and Mahanobis (Lee al., 2018) distance use density estimation by fitting Gaussian mixture model (GMM) to the learned feature space. We use the official implementation of DDU to fit GMM to the feature space (feature from the CNN). For Manhanobis distance, fit GMM to the low dimensional logit space instead for computational stability.
Calibration The backbone training follows the same procedure the previous section. During the calibration/tuning stage, we train all models for 20 epochs with cosine learning rate decay using SGD and cross entropy loss. In this stage, we freeze the backbone models and only tune the cal- ibration parameters/functions following (Guo et al., 2017). In our case, only a and 33 are tuned. For calibration on out-of-distribution data, the models are tuned on the entire validation set. For calibration on in-distribution data, we use 5-fold cross validation using the validation set following prior work (Rahimi et al., 2020). For matrix scaling Guo al. (2017) and Dirichlet scaling (Kull et al., 2019), we use the Off- Off-Diagonal and Intercept Regularisation (ODIR) (Kull et al., 2019) with default scaling
x 10-7 Note that we use the same initial learning rate, 0.1, for all methods ex- cept for Dirichlet scaling. Dirichlet scaling requires smaller learning rate and we tuned the learning rate on different datasets. For CIFAR1O, we use 0.01 and for CIFAR we use 0.005.
6.6 CIFAR 100 CONCEPT SHIFT SPLITS
While it is natural to associate covariate shift with increasing degrees of image corruption, finding dataset to benchmark gradual concept shift is not straightforward because concept shift tradition- ally thought binary: overlapping non-overlapping. However, not all non-overlapping labels
The original order can he restored by applying 5(1) to htstps: /gi thub. com/ omega tragger/i00
15

---

AVE
CIARRI
aipllane
dog
frog
house
stop
Group
cattle
shrow
motonyyle
squirrel
snake
trout
sea
tracior
pickup
24.96
2.39
Group
bear
tiephant
Jeopard
cameil
lizurd
rabbit
beaver
spider
raceoon
orchid
21.99
0.44
Group?
lion
mounian
crub
bicycle
tunle
boeile
unan
mouse
smuil
outer
20.18
L.14
Group
possun
sharks
forest
pane
dinosauu
boy
porcupune
welf
road
bunerrly
17.79
0.32
Group
rocket
liger
hee
tank
while
baby
kanganoo
dolphin
16.26
0.44
Group
willow
chimpanzee
ckank
cup
ouk
cockreach
14.64
Gruup
can
house
fox.
maple
pear
woman
12.65
Group paim
streescar
pepper
keyboard
bottle
rose
couch
caterpillar
goldiish
10.18
Group Balfish
upple
ocange
plate
table
wlip
bowl
television
skyseraper
ray
8.95
0.25
Group 0
wardrohe
lamp
plain
lownmower
chair
poppy
clock
cloud
sunflower
telephone
7.5
0.97
Table CIFARIO0 Concept Shift Splits Small group numbers indicate less conceptual similarity to CIFAR 10 classes. The similarity is calculated using inner product between the Glove embeddings of CIFAR 100 class and CIFARIO class. For each CIFARION class, the largest similarity each CIFAR10 10 class is taken as the overall similarity to CIFARIO. The average shows average similarity CIFAR1O 10 and the standard deviation shows in- grouroup variance.
AUROCC
TNRETPROOT
vanilla
58.66-0.95
64.59+1.4 142
69. 60.17+1.00
69 18-2.01 2.01
72. 2.44
10.75+0.50 50
17.31+0.50
2406+1
24 14-3--1224
20.80+2.01
62.01 as3
69 6.:0 55
75.29++1900 90
75 15.46=1.91
79. 70.50+2.32
9.21+1186
1404=3.07
19.3 00.394-5.64
19.56-5.66
24.38-6.88
67.00=0.55
77.87:0.51
$5.77=0.53
05.6:-0.3
91.25:0.56
14,07=1500
26.32:2.55
41,7743013
42.33:3.11
57,99:283
sunila
61.2210.44
69 09.8010.744
76,4310.79
76.4720.811
$1.3910900
9,9813.70
16.111115
24.701155
24.111.78
32.1322008
[n[])-mlly
63
74.39:01.97
$1,58:0.91
01.66:0.3
M6.07=1255
10.52=1.07
15.18-2.0
27.69:3.68
27.65:3.6
36.79:543
59.11
68.45:0077
76,195008
76.1620.85
82. 22.20:90.33
10.22:0025
17.72:1155
27.2841.32
27.19:1.26
36.91:1555
sumila
60.66.10:51
68 68.42.0.66
74.32.10.00
14.34.1:00
78, 1.21
10.36.1..77
17.60.067
26 26.08.10.22
25 25.97.11.10
33 12.891124
14
efs)-only
66.20.10.65
2.0551002
81. 16.10.99
81 19.29....99
86 18.100997
10.2810.500
10.23
17.52.108
26.651193
26 22.62:2.115
35 57.5713352
(12)-3(2)
66.36:9.477
77:42:0.45
85.82:0.52
85.83-00.47
91.19:0047
13,32:1.11
25.21::.112
40.67:3.13
40.74:2.65
56.48:2.62
Table 5: Capturing Covariate Shift (Motion Blur) All results are averaged over 5 runs.
are the same. For example, pickup truck (CIFAR100) much closer to truck (CIFAR10) than sun- flowers (CIFAR100) is semantically. To create this gradual concept/semantic shift, we propose to divide the CIFAR 100 dataset into sub-datasets with increasing conceptual difference from CI- FARI0 classes. Specially, we use Glove word embeddings (Pennington et al., 2014) wtih 6 billion tokens trained on the entire wikipedia201- 14 and Gigaword5 (Napoles et al., 2012) to measure seman- tic closeness (inner product) between CIFARI00 and CIFARIO classes. The result is 10 subdatasets split from CIFAR100 as shown in Tab. 4. Our experiments demonstrate that the conceptual differ- ence measured by semantic similarity in language translates to spectrum of near OOD datasets measured by the difficulty of OOD detection in the image space.
6.7 ADDITIONAL RESULTS
We provide additional results and tabulated results of figures in the main paper. Specifically, Tab. 5 and Tab. 8 are tabulated versions of Fig .3 and Fig. 4 in the main paper respectively. In addition to motion blur, we also provide out-of-distribution detection results on shot noise in Tab. 7 and zoom blur in Tab. 6.
6.8 EXTENDED RELATED WORK
Out-of- of-Distribution (OOD) detection methods can be largely divided into two camps depending on whether they require OOD data during training. Hendrycks al. (2018) leverages anomalous
AUROCT
TNR INRUPPR95T
vmilla
61.8811.33
67.2711711
72.8262.27
77.22:2.00
$1.58 11.48:3.0 3.16
1345:0.40
19.58:1.00
27.452232
549114211
44.66:6.3 32
[(x)
miz)-nnl mls)-8(s)
66.6+1.5 59
68.22-2.50
72 13.86
7334-5.01
75.34-620
10.88-1.0 1.70
10:30+3.0
13.88-492
14
17.00-8.3
71.21-5.89
77.52-2.37
$1.78-668
88.53-7744
39.74-7.36
20.6-5.50
26.17-806
38.67111388
4335-14553
5488-18899
vanille
66.1910.78
72.12=1.44
78.6721.67
83.14:1.51
87.5321.65
13.4320.58
18.11:1.57
26.31:2.54
34.3724000
45.89:4.44
ais) only
6934:1.63
75.49-2.40
$1.93-2611
96.37-2.55
90.651228
1,,4
32.86:2.80
3..11:4.52
42.9116.14
55.99-7.39
ela] B(x)
63.7321.26 1.26
69.2-2229
74,8012.31
79.4223.61
15.1124.0
13.3321.00
18.4011.67
27.09.2.53
35.6011881
48.24-2.39
Sanilla
64.3-0.50
70.96-0.88
77 77.26+1.11
41.87-145
36.661
13.550..0 50
19.35+1.10 18
28-32-201
3.0043000
48.52-1.00
niz]ennly
60.55-111
74.51-167
50.42+1.4 38
84 86.06-2.00
48
14.5500.0 90
19 19-19-100 50
67.6-2.55
35 33.07+8881
45.46-5.12
71.25=5.14
7800011.4
82.23-5.11
M6.4-6.44
9150-6641
19.93:3.05
25:54=4.08
3671-55772
46.54+7.28
60.70-8.87
Table 6: Capturing Covariate Shift (Zoom Blur) All results averaged over 5 runs.
16

---

AUROCY
TNRAFPRO911
1984-181
65,064244
1308 2
28 25 1850-503
32.46
08,75+700004
[6x)
mizl-nnly
61,86+275
67.054365
75.054464
77.35+4.66
80.49-449
8.8-1.75
10.27-285
12,08+577
1582+7.16
1542+9.8
64.78-1999
73.2+11844
37.01=3055
90.3-2.99
9455-2242
1133=1
10
18.58-4555
46.1112911
55A1-1472
71:7-14996
vanilla
681421172
75.9062.32
15.9223.13
59.2013.34
90971334
1693 50
24.752:005
40.5566.11
45.58.7.59
12.05110066
My.
ofxt only
698888879
78.301003
89. 19.
91.79
13
1864:1112
139
52.1243.99
59.6614800
046227.44
08662-0.66
76.771000
87.2 67.26-11001
90:00:140
93:42-1566
1741-0.52
26 1.77
46.0014..33
54 54.2677299
65.65=11144
vanilla
65.0722117
29 29
23.05
556212234
88952224
1.12:1111
23966195
29.0633777
41.2815513
51,647,23
44
01.1 only
1.811:123
33.91
17.1911611
89.8111.711
92.6.1.74
15.070082
12.29-184
80.7715877
47.0717781
67.2550555
76.131998
89.12:2002
92.19:2.112
1549
049
24.27
$1,7719888
61.50.11..77
15.45
Table 7: Capturing Covariate Shift (Shot Noise) All results are averaged over 5 runs.
AUBOOC
TNRMTTRUS
15.85
70.64
10.35
90.21
$1.45
19.85
10.111
78.00
$3.31
33,12
55.52
41.94
41:18
41.42
04.33
16.3 14:54
42.18
38:72
4:.66
0(x)
88.25
$7:35
16:77
89-11 10
$7-31
$9.59
$6.00
86. 23
85.65
67.83
29:55
58.23
23:14
44:38
16.90
13:40
27.3 01:33
25.30
13:31
45:00
78:05
$9.30
$8.51
90.49
$3.97
96.95
$9.29
90.98
91.78
91.15
31.30
42.18
41.82
49:14
48.16
50.76
42.94
51,00
49.36
51.32
66.75
01.74
91. (1
30
80.81
$1:10
91 85
01.90
94.09
16
53.90
97.5
51.66
48.55
53.61
51.38
58.10
61.60
68:38
kl..2)
57.33
91.36
91.68
91.36
91.43
92.43
92.75
93.58
94,87
95.53
47.46
53.78
55.46
55.90
53.44
61,44
58.35
63.68
65.93
12.54
88.16
$5:53
50.15
50 08
91.53
91.31
91.81
92.59
94.41
95.22
48.14
51.66
53.14
53.88
$2.56
59.32
$7.70
64.56
65.10
72.54
41
$6.54
91.84
91.99
02.56
92.86
93.36
92.00
03.10
94.12
95.00
46.68
54.83
56.30
57.82
53.92
02.20
$7.85
62.36
64.55
10.6
$1.55
91.96
91.85
92.68
91 94
95.31
90.55
V3.54
05.06
95.07
66.72
54.62
55.10
58.94
53.76
03.54
57.73
66.43
69.30
10.00
Table 8: Capturing Concept Shift (CIFAR100 Splits) All results averaged over 5 runs.
data in training which enables classifiers generalize and detect unseen OOD data. Thulasidasan et al. (2020) extends existing classifier to include an OOD class. Roy et al. (2021) assigns multiple classes for outliers instead of single class. Our method belongs to the class of methods that do not assume the availability of OOD data during training. Hendrycks & Gimpel (2016) discovers that correctly classified examples have larger maximum softmax probability (MSP) and propose to use it to detect incorrect predictions and OOD data. Lee et al. (2018) proposes to use Mahalanobis distance by fitting a Gaussian mixture model (GMM) in the feature space. Mukhoti et al. (2021) uses log density of the GMM model instead. Liu et al. (2020b) uses an energy score as the uncer- tainty metric to distinguish between in-distribution and OOD data. ODIN (Liang et al., 2017) uses combination of input processing and post-training tuning to improve OOD detection performance. Generalized ODIN Hsu et al. (2020) (also Techapanurak et al. (2019)) includes an additional net- work in the last layer to improve OOD detection during training. There are many other interesting OOD detection approaches that have achieved state-of- performance without OOD data such as using contrastive learning with various transformations (Winkens et al., 2020; Tack et al., 2020), training a deep ensemble of multiple models (Lakshminarayanan et al., 2016) and leveraging large pretrained models (Fort et al., 2021). They require extended training time, hyperparameter tuning and careful selections of transformations, whereas our method does not introduce any hyperparam- eters and has negligible influence on standard cross-entropy training time. For example, Winkens et al. (2020) trains constrastive model for 1200 epochs on CIFAR 100 whereas our model requires only 200 epochs (same as standard classifier training) on the same dataset.
Model Calibration methods can also be largely divided in two categories: 1) training time calibra- tion using augmentations (Thulasidasan et al., 2019; Jang et al., 2021), using modified losses (Kumar et al., 2018): 2) post-hoc calibration (Guo et al., 2017; Kull et al., 2019; Kumar et al., 2019; Rahimi et al., 2020). Guo et al. (2017) proposes to calibrate the confidence of a trained classifier using tem- perature scaling, single scalar that softens overconfident softmax predictions. Kull al. (2019) extends single-class confidence calibration to multiclass calibration using the Dirichlet distribution. Kumar al. (2019) proposes scaling-binning calibrator that is more sample efficient. Recently, Rahimi et al. (2020) formally generalizes a family of expressive functions for calibration, the in- tra peder-preserving functions. This class of functions has more representation power to calibrate more complex decision boundaries in neural networks. Our proposed method belongs to this class of functions. However, all these works focus on calibration of in-distribution data. To obtain better calibration on OOD data, on which the confidence of model needs to decrease accordingly, sensi- tivity and deterministic uncertainty modeling is explored by SNGP (Van Amersfoort et al., 2020) and DUQ (Liu et al., 2020a). SNGP uses a bounded spectral normalization regularization (Miyato et al., 2018) during training and DUQ adopts a two-sided gradient penalty (Gulrajani et al., 2017) to improve model sensitivity distribution shift. Our proposed model not only belongs to the family of intra order- snder-preservig functions, which ensures good in-distribution calibration, but also improves sensitivity to distribution shift, which improves duitt--distri calibration simultaneously.
17

---

SWIN CIFARLOD CHAROO
a Feature Space! I
Feature Space
Histogram of Pixel Values
Figure 5: Fig. 5a and Fig. 5b show visualization of 3-dimensional feature space of model trained on CIFARIO. The black cluster represents Gaussian- noise corrupted CIFARIO data. Fig. 5c shows histogram of pixels CIFARIO. CIFARI00 and SVHN.
7 DISCUSSION ON COVARIATE AND CONCEPT SCORE DECOMPOSITION
In Sec. 3.2 the main paper, two score functions are proposed to capture changes due to covariate and concept shift. As shown in Eq. 17, (xx) denotes the covariate shift score function and hyy x)) denotes the concept shift score function.
2(x)
M
M
u = m-mal - M
Σ 4
|f
max |wll2 cos 03 - M
Σ
/ws/la cos
(17)
i=1
h(g.a) az)
Conceptually, these two scores disentangle covariate shift and concept shift. However, in practice, both shifts almost always happen at same time the image domain, i.e., covariate shifted data, e.g., noised data, often result in concept shift, i.e., increasing ambiguity in class assignment. There- fore, both scores can increase and decrease simultaneously as observed in Sec. 4.1.2 and Sec. 4.1.3. The importance of separating them conceptually is to provide clean perspective to study robust out-of-distribution detection methods that will work well on different situations under either a single distribution shift mixed of shifts. It very likely that better score functions be derived to better disentangle the effects of distribution shifts. In turn, methods that improve sensitivity those new score functions can be motivated.
ADDITIONAL FIGURES
While it seems impossible that concept shift could happen without change in P(x), this is possible we define the support,P(z), as the distribution of pixels, ie., the lowest level characteristics, completely stripped of any semantic information. A simple example would be reshufiling the pixels of a CIFARIO image where the semantics, P(y2z). is completely changed, while P(x) remains the same. Using CIFAR100 to represent concept shift is based on this reasoning. We provide visualization of pixel distributions of CIFAR10, CIFAR 100 and SVHN in Fig. 5c. CIFARIO and CIFAR100 share very similar pixel distributions while the pixel distributions of SVHN are
very different from those of CIFAR 10. The similarity in pixel distributions between CIFAR10 10 and CIFAR 100 could be attributed the data generation/collection process.
To further support the statement that norms sensitive to covariate shift and justify the choice of defining the covariate shift score g(z) = ||x||2, we show visualization of CIFARIO classes in the feature space of model trained CIFARIO in Fig. 5a and 5b. Similar to Liu et al. (2018), change the last feature dimension a ResNet to 3. In this way, we can directly visualize the learned feature space without resorting to dimension reduction techniques such T-SNE (Wattenberg al., 2016). In addition to the clean validation data from CIFAR10, 10, we also visualize CIFARIO 10 data corrupted by Gaussian noise. The noised data represents severe covariate shift. We can observe that the noised data are clustered around the origin indicating very small norms.
18


Reviews:
[{"review_id": "nHN5l_QmUok", "reviewer": "Reviewer_5ox7", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strength: \n1. This paper is well-written, and I enjoyed reading it. It is self-contained and also provides sufficient introduction to guide people less familiar with this topic. \n2. The derivation of covariate shift score function and concept shift score function is solid and the justification/reasoning make a lot of sense. \n3. Leveraging geometric sensitivity decomposition and model the learnable instance-dependent scalar alpha and beta is a neat idea. The design of neural network that approximates these two variables make sense, though more choices of activation can be studied in addition to the sigmoid and softplus.\n\nWeakness/Question to Authors (order does not matter):\n1. Overall, I like the explanation of Figure 1, but CIFAR 10 vs CIFAR 100 is a quite extreme example, because they are using the same source of data but categorized differently (I believe that CIFAR100 is a more fine-grained label set). Does concept shift always happens when P(X) follow the identical distribution? It seems not to be the case, given that SVHN simultaneously has concept and covariant shift. Also, I believe here you did not explain near OOD and far OOD? Is near OOD the case where either covariant or concept shift happens and far OOD the case where they both happen?\n2. In section 3.4, is the reason why intra order-preserving function can better calibrate complex function in deep neural networks a conclusion of the theorem 1 in Rahimi et. Al. 2020? If so, probably it would be better to make this more explicit.\n ", "summary_of_the_paper": "This paper studies the problem of Out-Of-Distribution detection, which a focus on analyzing the covariate and concept shift. Leveraging these analysis, it further investigate score functions that capture sensitivity to those shifts and then theoretically derive new score functions, to improve OOD detection. This proposed method leads to a calibration function that obtain the state-of-the-art calibration performance on both in-distribution and out-of-distribution data. Experiments are conducted on small image recognition benchmarks, such as CIFAR100 and SVHN.\n", "summary_of_the_review": "I am not an expert in OOD so my comments needs to be largely discounted. With that being said, I like this paper overall.  I believe the paper is very clearly written, well justified, and provides strong theoretical result and sufficient empirical result. Given this, I am leaning towards accepting this paper. \n", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "SBC9hH_-npc", "reviewer": "Reviewer_rWjW", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Positives:\n\n- The paper is very well written, with a clear presentation and writing flow. I enjoyed reading through it. \n\n- To simulate the gradual concept shift, the paper proposes a new benchmark CIFAR100 Splits using word-embeddings. \n\n- The papers compared with a broad spectrum of baseline methods on OOD detection and calibration. \n\nConcerns and questions for the authors:\n\n- First and foremost, while the need for detecting semantic shifts is clear, the motivation of detecting covariate shifts confusing and questionable w.r.t the existing literature. In the former case, the goal is to **abstain from making a prediction** on something that the model's class space does not support, which is natural in real-world applications. However, in the case of covariate shift (say CIFAR10C), wouldn't it be desirable to generalize to the shifted data? This seems to be commonly formulated and tackled as a robustness and OOD generalization problem rather than an OOD detection problem. If the model is indeed robust and generalizable to the covariate shift, one would expect the OOD score statistics to be almost indistinguishable between the clean and corrupted data. However, the paper seems to challenge otherwise. The contribution and evaluation would have a much clearer and well-defined focus with near OOD and far OOD (on the semantic side). \n\n- It'd be great if the authors can provide further justification on why the feature norm (the g(x) after decomposition) captures the covariate shift but not semantic shift? As the core contribution, this score decomposition (for representing covariate shift vs semantic shift) isn't rigorously justified. It'd be interesting to also measure the feature norm and see the separability between ID and semantic OOD. Table 1 can be expanded to include g(x) and U for the vanilla model (trained with cross-entropy), alpha model as well. Based on Table 1, it seems like g(x) is also effective in distinguishing semantic OOD (SVHN), which weakens the argument and covariate-semantic score decomposition. \n\n- A missing baseline [1] also utilizes the cosine similarity for OOD detection, which shows strong performance together with generalized ODIN [2]. The paper can strengthen its SOTA claim by performing a comparison with these two methods. \n\n- The authors could have provided an additional evaluation on other datasets like Texture, LSUN, and Places 365 similar to previous works in OOD detection literature (as listed in footnote 1). Together with CIFAR10-CIFAR100 evaluation, this would make the experiments more comprehensive. \n\n- The definition of U in Equation (4) seems to coincide with the ODIN score, which also showed that the separability between ID and OOD can be approximated as $\\frac{1}{M}\\sum_i^M [\\max l_j - l_i]$ (under a large temperature). This connection might be interesting to draw explicitly in the revision.\n\n- The authors should clearly define the concept of \"out-of-distribution calibration\". The notion of calibration commonly refers to making the posterior matching the empirical accuracy on the in-distribution data. Why is it necessary to calibrate for OOD data, if the model shouldn't make an erroneous prediction anyway? For example, given the CIFAR-10 trained model, what does calibration error mean for an SVHN input? \n\n- Code is missing so the reproducibility is unclear. \n\nMinor: \n- Page 3, \"Concept shift appears when P_tr(Y |X)= P_tst(Y |X) and P_tr(X) = P_tst(X)\"---> this is not correct as concept shift would imply the input shifted accordingly.\n\n[1] E. Techapanurak, M. Suganuma, T. Okatani. Hyperparameter-Free Out-of-Distribution Detection Using Cosine Similarity. \n\n[2] Y. Hsu, Y. Shen, H. Jin, Z. Kira. Generalized ODIN: Detecting Out-of-Distribution Image Without Learning From Out-of-Distribution Data. CVPR 2020. \n\n\n\n\n", "summary_of_the_paper": "This paper proposes a method for detecting two types of distributional shifts: covariate shifts in the input space $\\mathcal{X}$ (due to input corruption) and semantic shifts (due to test data falling outside the support set of ID classes, $ y_\\text{test} \\notin \\mathcal{Y}$). The idea is based on the decomposition of KL-divergence between softmax prediction and a uniform vector. Furthermore, the authors propose Geometric ODIN to improve OOD detection and calibration, outperforming strong baseline on CIFAR10, CIFAR 100, and SVHN datasets. ", "summary_of_the_review": "Overall the paper is well written, with comprehensive evaluations. However, the reviewer is concerned about several conceptual confusion that the paper may introduce (as raised in my detailed comments above). The reviewer hopes to see the satisfactory changes made, and increase the score accordingly. ", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "NLgOKiRVbeO", "reviewer": "Reviewer_kVK3", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "**Strengths**\n\n- The paper provides a novel perspective about the KL divergence and the softmax linear classifier, and how they can be turned into a sensible OOD detection score function that works well. I like the proposed formulation and the interpretation presented in Sec.3.2, which gives the community some new insights.\n\n- The fields of OOD detection and confidence calibration are often studied separately though they are closely related to each other. The paper offers a unified approach for them, which is a good contribution to the literature.\n\n- The new CIFAR 100-based dataset is also a good contribution, which would be of interest to the community.\n\n**Weaknesses**\n\nThe major weakness is in the experiments on the covariate shift datasets (i.e., CIFAR 10C/100C) and how the findings are presented. Covariate shift is something we want the model to overcome in practice, which has been extensively studied in the OOD generalization (or domain generalization) literature. The paper merely focuses on distinguishing between ID and OOD data resulting from covariate shift, which might send a misleading message that contradicts with the field of OOD generalization. CIFAR 10C and 100C have been widely used to test model robustness. The paper should also report the accuracy numbers to justify whether the proposed approach (and the compared ones) still maintain the discriminative power. Such a study shouldn't be isolated from OOD detection.\n\n**Questions**\n\nSome places are unclear and need further clarifications.\n\n1. Is it necessary to use a linear layer to learn the alpha/beta in GSD? What is the motivation for such a modification? And what if the original GSD formulation is used?\n\n2. In Sec.3.4, I don’t fully understand why having a positive function helps calibration. Appx.6.4 proves that the proposed formulation preserves the order of logits when updating the parameters but does not explain why doing so helps calibrate the confidence values. Also, would it be possible to provide the confidence-vs-accuracy figure and the metrics as used in [a]?\n\n3. What are alpha-beta, alpha-only and beta-only? There is no explanation for these variants. I’d like to confirm if alpha-only means only the alpha in Eq.7 is used while the beta is set to a constant?\n\n4. In Sec.4.1.2, the statement of “covariate shift does not happen in isolation from concept shift” seems inaccurate. Covaraite shift can certainly occur without shift in the label space, like changes in lighting or viewpoint that create a divergence in the input marginal distribution but do not change the labels anyway. Maybe I misunderstood the sentence?\n\n[a] Guo, C., Pleiss, G., Sun, Y., & Weinberger, K. Q. (2017, July). On calibration of modern neural networks. In International Conference on Machine Learning (pp. 1321-1330). PMLR.", "summary_of_the_paper": "The paper starts from the KL divergence between a uniform distribution and the predicted distribution, based on which two score functions are derived for covariate shift and concept shift, respectively. The covariate shift score measures the feature norm while the concept shift score is essentially the difference between the cosine distance of the predicted class and the average cosine distance of all classes.\n\nFurthermore, the paper integrates a variant of the geometric sensitivity decomposition method introduced in a recent work into the proposed score functions, aiming to make them more sensitive to distribution shift and to help calibrate the model as well.\n\nA new dataset called CIFAR 100 Splits is also built, which contains 10 sub-datasets with an increasing level of concept shift compared to CIFAR 10.\n\nThe results show that the proposed approach achieves decent performance in OOD detection (in both the scenarios of covariate shift and concept shift) as well as calibration.", "summary_of_the_review": "Overall, I like the technical contributions, which have been proved effective and provide new insights to the community. But I am concerned with the weakness part mentioned above, which should be addressed. I give borderline accept for now and will reassess the paper after the discussion period.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "Ag4ZTBR0Nwe", "reviewer": "Reviewer_kPST", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The overall aim of the submission to explore OOD detection for both covariate shift as well as concept shift (along a spectrum of increasing differences in concepts) is interesting and thought-provoking. However, I’m slightly confused by how concept shift is defined: $P_{tr}(Y | X) \\neq P_{tst}(Y | X)$ but $P_{tr}(X) = P_{tst}(X)$? How can the distributions over X be the same if different concepts are being represented in tr and tst? This would only make sense if the exact same objects are being redefined significantly over the same support in different domains, which is not quite what is being explored in this paper, as far as I can tell (different objects are showing up in the test OOD set). In the context of this paper, I’d recommend simply describing concept shift as cases when the concepts have shifted but the more superficial characteristics are unchanged, such as low-level image  statistics (as discussed earlier in [1,2]). (This issue does not apply for covariate shift, since the question there is would you apply the same label to images with the same object but different styles, so we can have the same $P(Y|X)$ with shift in $P(X)$.)\n\n\nWhile the score functions are described as being derived “theoretically”, it seems that the use of the norm-score does not really have any theoretical backing (that I could spot), instead it is claimed that “Intuitively, activation of a neural network on covariate-shifted data should be weaker”. I’m not sure I fully understand this intuition; it seems to me that one cannot say what would happen to the activations of a neural network on covariate-shifted data; in fact, I imagine one could easily synthesize some sort of adversarial example to maximally activate neurons with data that is highly covariate-shifted from the in-distribution. Also, if the derivation of U is based so strongly upon sandwiching the KL, why not simply use the KL as a score? \n\n\nI don’t fully understand the reasoning behind making alpha and beta functions of f. I followed the initial motivation that a decoupling of norms and angles might help, and that an existing method has shown how to perform that decoupling. But there isn’t a clear explanation for why alpha, beta needed to be functions of f, and I couldn’t think of one either; it seemed more reasonable to expect, based on the motivation presented in the submission, that these should be instance-independent. Could you unpack the reason in the rebuttal?\n\n\nI’d recommend rewording some of the framings of novelty such as “existing works do not distinguish between different types of OOD datasets”: Hsu et al. [1] discussed the distinction between “non-semantic” and “semantic” shift, which as far as I can tell, is pretty much synonymous with the paper’s “covariate” and “concept” shift terminology, but without the notational difficulties discussed above. Ahmed et al. [2] discussed the need for detecting “semantic” anomalies, since we typically want to be robust to non-semantic shift rather than flag it down (which is also part of the reason benchmarks such as corrupted-CIFAR are typically used to test for robustness rather than as OOD detection). The novel evaluation perspective in the submission, which is quite interesting, is the breakdown of CIFAR-100 into sets of classes with increasing semantic shift. \n\n\nIn the experiments, it is not clear why quite a few of the numbers deviate significantly from those reported elsewhere. For example, [3] reported the Mahalanobis method to score 98.4 AUROC for CIFAR100/SVHN, while the submission reports 84.4. The CIFAR10/SVHN performance is reported in [3] as 99.1 for Mahalanobis, while the submission reports 93.1. The proposed method is characterized as achieving “state-of-the-art results”, but it appears from works such as [4] that the Mahalanobis baseline and the Gram method often outperform the proposed method (the original papers developing the method also report similar numbers). If the difference is due to differences in architectural/training choices, these choices need to be motivated since most of the reported baseline numbers in the submission seem significantly worse than in existing literature.\n\n[1] Generalized ODIN, Hsu et al.\n\n[2] Detecting semantic anomalies, Ahmed et al.\n\n[3] Detecting OOD examples with gram matrices, Sastry et al.\n", "summary_of_the_paper": "The submission proposes to evaluate covariate shift and \"concept shift\" through two scoring functions, one based on feature norms (for covariate shift) and the other based on angles (for concepts). To help the network decouple norms and angles, (a modified version of) a very recent method is used to decompose the feature norm and the cosine-distances and also calibrate the network. Experimental results appear to demonstrate good performance on both near and far OOD examples, as well as on calibration metrics.\n", "summary_of_the_review": "While the submission is certainly intriguing, promoting further the decoupling of semantic and non-semantic distributional shift detection, and also exploring semantic shift along a spectrum of increasing distance, questions remain in my mind about some of the terminology, motivations, claims of novelty, and inconsistencies of reported results with existing numbers that need to be resolved.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper proposes a method for detecting two types of distributional shifts: covariate shifts in the input space  (due to input corruption) and semantic shifts (due to test data falling outside the support set of ID classes). The idea is based on the decomposition of KL-divergence between softmax prediction and a uniform vector. Furthermore, the authors propose Geometric ODIN to improve OOD detection and calibration, outperforming strong baseline on CIFAR10, CIFAR 100, and SVHN datasets. The paper aims to solve a very important problem in ML and the approach is thought-provoking.

However, there were several questions and confusions raised by the reviewers, such as the applicability of the model, justification of use of feature norm, discussion on sensitivity vs robustness, framing of the novelty, clear definition of OOD detection, definition of parameters, etc. (please see reviews for a comprehensive list). I invite authors to incorporate these points in the next version of the paper which will significantly improve the paper.
Decision: Reject