SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Jinggang Chen†*, Junjie Li†*, Xiaoyang Qu‡§, Jianzong Wang‡§, Jiguang Wan†, Jing Xiao‡
† Huazhong University of Science and Technology, China
$\ddagger$  Ping An Technology (Shenzhen) Co., Ltd.
{chen.jinggang98, 22162176691jj, quxiaoy}@gmail.com, jzwang@188.com,
jgwan@hust.edu.cn, xiaojing661@pingan.com.cn
Detecting out-of-distribution (OOD) examples is crucial to guarantee the reliability and safety of deep neural networks in real-world settings. In this paper, we offer an innovative perspective on quantifying the disparities between in-distribution (ID) and OOD data—analyzing the uncertainty that arises when models attempt to explain their predictive decisions. This perspective is motivated by our observation that gradient-based attribution methods encounter challenges in assigning feature importance to OOD data, thereby yielding divergent explanation patterns. Consequently, we investigate how attribution gradients lead to uncertain explanation outcomes and introduce two forms of abnormalities for OOD detection: the zero-deflation abnormality and the channel-wise average abnormality. We then propose GAIA, a simple and effective approach that incorporates Gradient Abnormality Inspection and Aggregation. The effectiveness of GAIA is validated on both commonly utilized (CIFAR) and large-scale (ImageNet-1K) benchmarks. Specifically, GAIA reduces the average FPR95 by  $23.10\%$  on CIFAR10 and by  $45.41\%$  on CIFAR100 compared to advanced post-hoc methods.
Deep neural networks have been extensively applied across various domains, demonstrating remarkable performance. However, when they are deployed in real-world scenarios, particularly in contexts that require high levels of security [1-3], an urgent challenge arises. Namely, these models must be able to ensure the reliability of their outcomes, even in the face of out-of-distribution (OOD) inputs from the open world that differ from in-distribution (ID) training data and thus surpass their cognitive capabilities. That underscores the importance of OOD detection, which involves estimating uncertainty from the model to identify the "unknown" samples, serving as an alert mechanism before making predictive decisions.
Recently, a rich line of literature has emerged to address the challenge of OOD detection [4-13]. Indeed, the majority of previous approaches focus on defining more suitable measures of OOD uncertainty by using model outputs [4, 5, 7-9] or feature representations [11, 13-15]. Despite the above mainstream approaches, estimating uncertainty from gradients is readily implemented with a fixed model and has received increasing research attention lately. Prior gradient-based OOD detection methods [10, 16, 17] have primarily emphasized utilizing parameter gradients as the measurement, while giving limited attention to the in-depth exploration of gradients related to the inputs (i.e., attribution gradients [18]).
$\S$  Corresponding Author.
Figure 1: Motivation of our work. Gradient-based attribution algorithms use attribution gradients to explain where models look for predicting final outputs. An intriguing question is: when encountering OOD sample  $X_{\mathrm{out}}$  whose label falls outside the in-distribution label space  $Y_{\mathrm{in}}$ , how does the model interpret its overconfident prediction? In order to unearth uncertainty from the explanatory result, we conduct our research by inspecting the abnormalities in attribution gradients and then aggregate them for OOD detection.
In this paper, we put our eye on a novel and insightful perspective — let models explain the uncertainty themselves with attribution approaches. Gradient-based attribution algorithms [19-21] are ubiquitous for the visual explanation of why the model makes such a decision to the predicted class. An intuition comes up that well-trained networks can clearly attribute the region of target ID objects, but what if they face OOD samples that are totally unknown to them? As shown in Fig. 1, we observe through the utilization of attribution gradients that the pre-trained model is capable of generating reasonable visual interpretation for the ID input  $X_{\mathrm{in}}$  from ImageNet [22]. However, when attempting to interpret an OOD image  $X_{\mathrm{out}}$  from iNaturalist [23] with a label that does not belong to  $Y_{\mathrm{in}}$ , it confuses the model, leading to a meaningless attribution result.
Following the observation, we delve into investigating the gradient-based attribution abnormality when inferring OOD examples. Our further study finds that this phenomenon can be caused by the attribution gradient, which is constructed by taking the value of the partial derivative of the target output  $S_{c}(\cdot)$  w.r.t. one unit  $z_{i}$  of the input variables  $z$  (i.e.,  $\frac{\partial S_c(z)}{\partial z_i}$ ). To enlarge the discrepancy between ID and OOD without prior knowledge from training data, we introduce the channel-wise average abnormality and the zero-deflation abnormality as two measurements for detecting distributional shifts. Then, we propose our detection framework GAIA with Gradient Abnormality Inspection and Aggregation and conduct comprehensive experiments on both CIFAR benchmarks and large-scale ImageNet-1K benchmark to validate the effectiveness of our proposed method. Code is available at https://github.com/JGethanChen/GAIA-OOD.
Our key results and contributions are summarized as follows:
- We provide insights into the attribution abnormality for OOD detection. Our intuition is that unreliability from visual explanations can be a direct alarm to distinguish OOD examples. Hence, we delve further into the underlying causality of the abnormality. Then, we provide a theoretical explanation for the causes of attribution abnormality.
- We propose a simple yet effective post-hoc detection framework via Gradient Abnormality Inspection and Aggregation (GAIA), which consists of two independent measurements: the Channel-wise Average abnormality (GAIA-A) and the Zero-deflation abnormality (GAIA-Z). Both of them are lightweight and plug-and-play—hyperparameter-free, training-free, with no ID data and outliers required for estimation.
- Thorough experiments demonstrate that GAIA surpasses most advanced post-hoc methods on both commonly utilized (CIFAR) and large-scale (ImageNet-1K) benchmarks. GAIA-Z exhibits superior performance on CIFAR benchmarks, reducing the average FPR95 by  $23.10\%$  on CIFAR10 and by  $45.41\%$  on CIFAR100. GAIA-A performs well on the ImageNet-1K benchmark and reduces by  $17.28\%$  compared to the advanced gradient-based detection method GradNorm.
We consider the general setting of a supervised machine learning problem, where  $\mathcal{X}$  denotes the input space and  $\mathcal{Y}_{\mathrm{in}} = \{1,2,\dots,C\}$  denotes the ID label space. Especially, we denote the output score w.r.t. class  $c$  before softmax layer as  $S_{c}(\cdot)$ .
Out-of-distribution detection. The goal of out-of-distribution (OOD) detection is to distinguish the sample  $x_{\mathrm{out}}$  that exhibits substantial deviation from the distribution  $\mathcal{X}$ . In literature, OOD data originates from an unknown distribution  $\mathcal{X}_{\mathrm{out}}$ . And the label space of the OOD samples has no intersection with  $\mathcal{Y}_{\mathrm{in}}$ . This problem can be formulated as a binary classification task using a score function  $\Delta(x)$ . More specifically, when provided with an input sample  $x$ , the level-set estimation can be expressed as follows:
$$
\mathcal {G} (x) = \left\{ \begin{array}{l l} \text {o u t}, & \text {i f} \quad \Delta (x) > \gamma \\ \text {i n}, & \text {i f} \quad \Delta (x) \leq \gamma \end{array} \right. \tag {1}
$$
In our work, lower scores correspond to a higher likelihood of classifying the sample  $x$  as indistribution (ID), and  $\gamma$  denotes a threshold for separating the ID and OOD data.
Gradients from attribution algorithms. The attribution gradient is first introduced by sensitivity analysis (SA) [18] and widely utilized in visual explainability techniques [19-21, 24, 25]. It refers to the sensitivity of a particular input variable (input or feature unit) w.r.t.  $c$ -class predictive output  $S_{c}(\cdot)$ . Denotes  $k$ -th channel feature map at layer  $l$  as  $\mathbf{A}^{kl} \in \mathbb{R}^{W \times H}$ . The attribution gradient of one feature unit  $A_{ij}^{kl}$  is computed by:
$$
\operatorname {G r a d} _ {i j} = \frac {\partial S _ {c} \left(\boldsymbol {A} ^ {k l}\right)}{\partial A _ {i j} ^ {k l}} \tag {2}
$$
It is unrelated to the gradients commonly associated with the typical understanding of network optimization (i.e., gradients of the parameters). In most attribution algorithms, the attribution gradient is used for quantifying the contribution of each feature unit to the model's prediction.
3 Investigating Attribution Abnormality for Out-of-distribution Detection
In this section, we aim to investigate how attribution gradients can lead to abnormality when explaining OOD examples. We also attempt to provide a unified theoretical analysis.
Channel-wise average abnormality. We first focus on the abnormality in the Gradient-based Class Activation Mapping (GradCAM) algorithm [19], which is one of the most widely applied attribution strategies. Its paradigm is to channel-wise sum up feature maps for a saliency map  $M \in \mathbb{R}^{W \times H}$ . Here we denote feature maps  $A \in \mathbb{R}^{K \times W \times H}$  with  $K$ -channels in the convolutional layer as the input variables. The attribution  $\alpha_{ij}$  of each unit  $M_{ij}$  can be formulated as follows:
$$
\alpha_ {i j} = \operatorname {R e L U} \left(\sum_ {k = 1} ^ {K} w _ {k} A _ {i j} ^ {k}\right), \quad \text {w h e r e} \quad w _ {k} = \frac {\partial g (\boldsymbol {A})}{\partial A _ {i j} ^ {k}} = \frac {1}{W \times H} \sum_ {i = 1} ^ {W} \sum_ {j = 1} ^ {H} \frac {\partial S _ {c} (\boldsymbol {A})}{\partial A _ {i j} ^ {k}} \tag {3}
$$
Figure 2: Demonstration of the attribution abnormality from gradient-based weights. The toy experiment is conducted on ResNet34 with four blocks trained on CIFAR10. We select four attribution layers from different blocks and calculate the average attribution gradients for each channel.
where  $w_{k}$  is the channel-wise weight that re-weights feature maps in different channels, and  $g(A)$  denotes the explanatory function of the DNN output from  $A$ . Detailed elaboration is provided in Appendix C. Taking different layers as the attribution targets, we visualize the distribution of channel-wise average attribution gradients in Fig. 2. It can be observed that the discrepancy of the weights  $w_{k}$  is distinguishable in that OOD samples tend to produce more noisy and abnormal outliers compared to ID samples. Additionally, as the layers increase in depth, the magnitude of the average gradients also increases.
Zero-deflation abnormality. Then, we closely examine the abnormality that may arise in attribution gradients themselves due to distributional shifts. Fig. 3(a) shows attribution gradients on feature maps across all channels at a specific layer. We observe that the quantity of zero partial derivation  $\frac{\partial S_c(A)}{\partial A_{ij}^k}$  in OOD is extremely less than ID, leading to a high occurrence of dense gradient matrices. As shown in Fig. 3(b), this phenomenon is more pronounced in deeper layers, indicating an abnormal behavior.
3.1 Theoretical Explanation for Attribution Abnormality
We consider a unified explanation for attribution algorithms with Taylor expansion. As proved in [26], attribution algorithms are mathematically equivalent to the perspective that the network  $c$ -class output  $S_{c}(z)$  is explained as a Taylor expansion model. For variables  $z = [z_{1},\dots,z_{n}]$  (e.g., feature units to be attributed or inputs), here we perform  $P$ -order expansion of zero baseline output  $S_{c}(\mathbf{0})$  at  $z$ :
$$
S _ {c} (\mathbf {0}) = S _ {c} (\mathbf {z}) + \sum_ {p = 1} ^ {P} \sum_ {i = 1} ^ {n} \frac {1}{p !} \frac {\partial^ {p} S _ {c} (\mathbf {z})}{\partial (z _ {i}) ^ {p}} (z _ {i}) ^ {p} + \frac {1}{2 !} \frac {\partial^ {2} S _ {c} (\mathbf {z})}{\partial z _ {1} \partial z _ {2}} z _ {1} z _ {2} + \dots + R _ {P} (\mathbf {z}) \tag {4}
$$
where  $R_P(z)$  denotes the remainder term for the  $P$ -order expansion. In our paper, we consider the feature values to be all zeros as the zero baseline, which is commonly adopted for analyzing gradient-based attribution algorithms. Then all terms can be represented by vector  $\kappa = [\kappa_1, \dots, \kappa_n] \in \mathbb{N}^n$ , where  $\kappa_i \in \mathbb{N}$  reflects the integral degree of the input variable  $z_i$  (e.g.,  $\kappa_i = 1$  indicates the corresponding item only contains first-order partial derivative w.r.t.  $z_i$ ). Thus we can represent the  $c$ -label output change caused by variables  $z$  as:
$$
\left| S _ {c} (\boldsymbol {z}) - S _ {c} (\mathbf {0}) \right| = \left| \sum_ {p = 1} ^ {P} \sum_ {\boldsymbol {\kappa} \in \Omega , | \boldsymbol {\kappa} | = p} \mathcal {C} (\boldsymbol {\kappa}) \cdot \frac {\partial^ {\kappa_ {1} + \cdots + \kappa_ {n}} S _ {c} (\boldsymbol {z})}{\partial^ {\kappa_ {1}} z _ {1} \cdots \partial^ {\kappa_ {n}} z _ {n}} (z _ {1}) ^ {\kappa_ {1}} \dots (z _ {n}) ^ {\kappa_ {n}} + R _ {P} (\boldsymbol {z}) \right| \tag {5}
$$
where  $\mathcal{C}(\kappa)$  is a non-negative constant related to vector  $\kappa$ . The expansion formula reflects the contribution of each variable  $z_{i}$  to the  $c$ -label output change. Thus, we can attribute importance  $\alpha_{i}$  to  $z_{i}$  based on how much it contributes to such a change. Furthermore, the effect of  $z_{i}$  to  $S_{c}(z)$  can be decomposed into Taylor independent effect term  $\phi_{i}(\kappa)$  and Taylor interaction effect term  $\psi_{i}(\kappa)$ . For independent term  $\phi_{i}(\kappa)$ , only  $z_{i}$  is contained, where  $\kappa = [0,\dots ,\kappa_i,\dots ,0]\in \Omega_\phi$  and  $\kappa_{i} > 0$ . And
(b)
Figure 3: Left (a): Visualization of attribution gradients on feature maps. Right (b): Proportion of non-zero values across different channels. Each data point represents one single channel.
the overall effect of term  $\psi_i(\kappa)$  is caused by the interactions between  $z_{i}$  and other variables, where  $\kappa \in \Omega_{\psi}$  has at least two non-negative values and  $\kappa_{i} > 0$  . Attribution methods are formulated as:
$$
\alpha_ {i} = \sum_ {p = 1} ^ {P} \sum_ {\kappa \in \Omega_ {\phi}, | \kappa | = p} \omega_ {i, \kappa} \phi_ {i} (\kappa) + \sum_ {p = 1} ^ {P} \sum_ {\kappa \in \Omega_ {\psi}, | \kappa | = p} \omega_ {i, \kappa} \psi_ {i} (\kappa) \tag {6}
$$
where  $\omega_{i,\kappa}$  denotes the ratio of a specific term (either the independent term or the interaction term) allocated to  $\alpha_{i}$ .
Attribution abnormality in zero importance. A reliable attribution result requires accurate identification of the features that are useful for the output. Here, we consider the Null-player axiom [27] (see Appendix  $E$ ), which states that in the reliable attribution, a feature should be considered as having zero importance when it makes no contribution to the model's output. In other words, if a feature does not contribute to the model's prediction, it should be considered as having zero importance.
Proposition 1. Given input variables  $z$ , for one variable  $z_i \in \mathbf{z}$  to be attributed, if  $\frac{\partial S_c(\mathbf{z})}{\partial z_i}$  is zero throughout the analysis, then  $\alpha_i = 0$  always holds.
Given variables  $z$  in one analysis, it is assumed that the partial derivative function w.r.t.  $z_{i}$  is a constant zero. As shown in Eq. 7, all independent and interaction terms related to  $z_{i}$  are zero. Thus,  $z_{i}$  is of zero importance to the prediction. This is, zero attribution gradient values will directly impact the final attribution result.
$$
\frac {\partial S _ {c} (\boldsymbol {z})}{\partial z _ {i}} = 0 \Rightarrow \frac {\partial^ {\kappa_ {1} + \cdots + \kappa_ {n}} S _ {c} (\boldsymbol {z})}{\partial^ {\kappa_ {1}} z _ {1} \cdots \partial^ {\kappa_ {n}} z _ {n}} = 0, \kappa_ {i} > 0 \Rightarrow \phi_ {i} (\kappa) = \psi_ {i} (\kappa) = 0 \Rightarrow \alpha_ {i} = 0 \tag {7}
$$
This provides us with an explanatory perspective for our observation — visual explanation for OOD data tends to be messy and unreliable due to the model's uncertainty about the unknown distribution, resulting in an abundance of intricate non-zero importance attributions.
Attribution abnormality in gradient-based weights. Following Eq. 6, GradCAM in Eq. 3 can be reformulated in form that includes only the first-order Taylor independent terms (see Appendix  $D$  for the proof), where  $\kappa = [0,\dots,\kappa_{i} = 1,\dots,0]$  is a one-hot vector, and  $\kappa_{j} = 0$  if  $j\neq i$ . This simplifies our analysis of the abnormality in weights, focusing solely on the correlation between first-order partial derivatives and the attribution result to reflect the uncertainty on each independent feature.
4 GAIA: A Simple and Effective Framework for Abnormality Aggregation
We propose our GAIA framework, which aggregates the channel-wise average abnormality (GAIA-A) or the zero-deflation abnormality (GAIA-Z) for out-of-distribution detection.
Abnormality aggregation from label space. General attribution algorithms focus on the final predictive output  $S_{c}(\mathbf{A})$ , where  $c = \operatorname{argmax}_{c_i \in C} S_{c_i}(\mathbf{A})$ . This is adequate for the zero-deflation abnormality as we aim to ascertain the model's confidence in interpreting its own classification result. While for the channel-wise average abnormality, our aspiration is to gather abnormalities from a broader label space. Hence, all outputs in the ID label space are informative for collecting the model's tendency towards identifying samples as ID categories. For GAIA-A, we fuse all the outputs with  $\log (\operatorname{softmax}(\cdot))$ :
$$
\frac {\partial \mathcal {S} (\boldsymbol {A})}{\partial A _ {i j} ^ {k}} = \frac {\partial \sum_ {c \in C} \log \operatorname {s o f t m a x} \left(S _ {c} (\boldsymbol {A})\right)}{\partial A _ {i j} ^ {k}} \tag {8}
$$
This strategy first accumulates the model's outputs and simultaneously performs backpropagation w.r.t. the features  $A_{ij}^{k}$ . It is more efficient compared to individually backpropagating through each category and then accumulating them, which is impractical in scenarios with large label space (e.g., 1000 categories in ImageNet). Furthermore, we find that GAIA-A can be enhanced with a two-stage fusion strategy. Let us denote the neural network prediction function based on input feature variables  $\mathbf{A} \in \mathbb{R}^{K \times W \times H}$  by  $S_{c}(\mathbf{A}) = \Psi(\mathbf{A}_{\mathrm{last}}, \Theta_{\Psi})$ , where  $\Psi(\cdot)$  represents the classification function and  $\mathbf{A}_{\mathrm{last}} \in \mathbb{R}^{1 \times W_{\mathrm{last}} \times H_{\mathrm{last}}}$  is the feature map at the last layer. Then the network feature extraction function is defined as  $\Phi(\cdot)$  and  $\mathbf{A}_{\mathrm{last}} = \Phi(\mathbf{A}, \Theta_{\Phi})$ . In our methods, we consider the gradient matrix on the  $\mathbf{A}_{\mathrm{last}}$  and the inner feature map  $\mathbf{A}^{k} \in \mathbb{R}^{1 \times W \times H}$  ( $k$ -th channel from  $\mathbf{A}$ ) separately, with the former
regarded as the output component  $\nabla A_{\mathrm{last}}$  and the latter as the inner component  $\nabla A^k$ :
$$
\begin{array}{l} \nabla \boldsymbol {A} ^ {k} = \frac {\partial \Phi (\boldsymbol {A} , \Theta_ {\Phi})}{\partial \boldsymbol {A} ^ {k}} \quad \partial S (\boldsymbol {A} _ {- - }) \quad \partial \Sigma \quad \log \operatorname {s o f t m a x} (S (\boldsymbol {A} _ {- - })) \tag {9} \\ \nabla A _ {\text {l a s t}} = \frac {\partial S (A _ {\text {l a s t}})}{\partial A _ {\text {l a s t}}} = \frac {\partial \sum_ {c \in C} \log \operatorname {s o f t m a x} \left(S _ {c} \left(A _ {\text {l a s t}}\right)\right)}{\partial A _ {\text {l a s t}}} \\ \end{array}
$$
Abnormality aggregation from input space. We start by defining the anomalies expectation on  $k$ -th channel feature map at  $l$  layer as  $A^{kl} \in \mathbb{R}^{1 \times W \times H}$ . The zero-deflation abnormality can be described as the non-zero density of  $A^{kl}$ :
$$
\mathbb {E} [ \epsilon | \boldsymbol {A} ^ {k l} ] = \frac {1}{W \times H} \mid \left\{A _ {i j} ^ {k l} \right. \mid \frac {\partial S _ {c} (\boldsymbol {A} ^ {k l})}{\partial A _ {i j} ^ {k l}} \neq 0 \} \mid \tag {10}
$$
For the channel-wise average abnormality, we observed that average gradients on  $A_{\mathrm{last}}$  from the output component and the average attribution gradients obtained from the inner component exhibit opposite behaviors in terms of ID and OOD data (We discuss its effectiveness in Section 5.3 and provide theoretical analysis in Appendix F). Consequently, we use division to get the expectation of the final fusion channel-wise average abnormality abnormality:
$$
\mathbb {E} [ \epsilon | \boldsymbol {A} ^ {k l} ] = \frac {\mathbb {E} _ {\text {i n n e r}} [ \epsilon | \boldsymbol {A} ^ {k l} ]}{\sqrt {\mathbb {E} _ {\text {o u t p u t}} [ \epsilon | \boldsymbol {A} _ {\text {l a s t}} ]}} = \frac {| \frac {1}{W \times H} \sum_ {g ^ {k l} \in \nabla \boldsymbol {A} ^ {k l}} g ^ {k l} |}{| \frac {1}{W _ {\text {l a s t}} \times H _ {\text {l a s t}}} \sum_ {g _ {\text {l a s t}} \in \nabla \boldsymbol {A} _ {\text {l a s t}}} g _ {\text {l a s t}} | ^ {\frac {1}{2}}} \tag {11}
$$
Consider networks have  $L$  layers to be utilized, and each layer has  $K_{l}$  channels. Our framework accumulates them into an abnormality matrix  $\Lambda \in \mathbb{R}^{L\times K_m}$ , where  $K_{m} = \max \{K_{i}|1\leq i\leq L\}$  and  $\Lambda_{ij} = 0$  if  $j > K_{i}$ . Then, we use the Frobenius norm as a non-parameter measuring score to represent the global abnormality. For instance, assuming  $K_{m} = K_{L}$ ,  $\| \Lambda \|_{F}$  is calculated as:
$$
\left\| \boldsymbol {\Lambda} \right\| _ {F} = \left\| \begin{array}{c c c c c c} \mathbb {E} [ \epsilon | \boldsymbol {A} ^ {1, 1} ] & \dots & \mathbb {E} [ \epsilon | \boldsymbol {A} ^ {1, K _ {1}} ] & 0 & \dots & 0 \\ \vdots & & \vdots & & \vdots \\ \mathbb {E} [ \epsilon | \boldsymbol {A} ^ {L, 1} ] & \dots & \mathbb {E} [ \epsilon | \boldsymbol {A} ^ {L, K _ {1}} ] & \dots \dots \dots \dots \dots \mathbb {E} [ \epsilon | \boldsymbol {A} ^ {L, K _ {m}} ] \end{array} \right\| _ {F} = \sqrt {\sum_ {i} ^ {L} \sum_ {j} ^ {K _ {m}} (\mathbb {E} [ \epsilon | \boldsymbol {A} ^ {i , j} ]) ^ {2}} \tag {12}
$$
The overall process are formulated in Algorithm 1.
Algorithm 1: GAIA
Input: Test sample  $x$ ; Fixed model  $f_{\theta}$ .
Output: OOD score  $\Delta (x)$
Compute label output set  $\{S_c(x)|c\in C\}$  by  $f_{\theta}(x)$
Backpropagate attribution gradients by  $\frac{\partial S_c(A^{kl})}{\partial A_{ij}^{kl}}$  (GAIA-Z) or Eq. 9 (GAIA-A);
Calculate  $\mathbb{E}[\epsilon |\pmb{A}^{kl}]$  by Eq. 10 (GAIA-Z) or Eq. 11 (GAIA-A);
Calculate global abnormality  $\| \Lambda \| _F$  by Eq. 12;
return  $\| \Lambda \|_{F}$  as OOD score  $\Delta (x)$
In this section, we describe our experimental setup in Section 5.1. Then, we demonstrate the effectiveness of our method on the large-scale ImageNet-1K benchmark [28] and the CIFAR benchmarks [7] in Section 5.2. We also conduct ablation studies in Section 5.3.
Benchmarks. In accordance with [10, 11, 15, 28], we employ the large-scale ImageNet-1K benchmark [28], which offers a more realistic and challenging environment due to its use of high-resolution images and an large label space that encompasses 1,000 distinct categories. Four OOD datasets in this
Table 1: Main Results on CIFAR Benchmarks [7]. We evaluate on ResNet34 [31] and WideRes-net40 [32], which are both pre-trained with cross-entropy loss. For Rankfeat and ASH, we choose their best performance.  $\uparrow$  indicates larger values are better, while  $\downarrow$  indicates smaller values are better. All values are percentages. The best results are in Bold and the second best results are underlined.