SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Our experimental studies present the promising performance of OrthoGrad in OOD detection on two benchmarks, ImageNet and CIFAR benchmarks.
4.2.1 Experimental Results on Out-of-Distribution Detection
ImageNet Benchmark: Our method demonstrates competitive performance, reaching the state-of-the-art level, as indicated in table 1. On the Resnet pre-trained network, GradOrth surpasses ASH-S, ASH-B, and ASH-S by  $0.45\%$ ,  $2.47\%$ , and  $0.93\%$  in terms of FPR95 on the iNaturalist, SUN, and Textures OOD datasets, respectively. When evaluated on the Places OOD dataset, our method achieves an FPR95 of  $33.67\%$  and secures the second rank after ASH-B. Furthermore, GradNorm demonstrates an average FPR95 performance of  $18.57\%$ , outperforming ASH-B by  $3.98\%$ .
It is important to acknowledge the fact that GradOrth boasts a low computational complexity. It only requires computing the subspace of the pre-trained network once and can be conveniently utilized through a simple gradient calculation, without the need for hyper-parameter tuning or additional training during OOD detection. In contrast, certain methods like Mahalanobis [29] require collecting feature representations from intermediate layers for the entire training set, which can be computationally expensive for large-scale datasets like ImageNet. Additionally, GradOrth presents a stable performance across most datasets whereas the performance of ASH versions varies across the four OOD datasets. ASH-B outperforms other baselines on the Places dataset but ranks third, second, and third on the other three datasets. A similar pattern is observed for ASH-S in terms of FPR95, where it ranks second, sixth, sixth, and second across the iNaturalist, SUN, Places, and Textures datasets, respectively.
GradOrth also exhibits superior performance in terms of AUROC, outperforming ASH-S by an average of  $2.80\%$  across the four datasets. Particularly, GradOrth surpasses ASH-S, ASH-B, and ASHS by  $0.13\%$ ,  $0.66\%$ , and  $0.46\%$  on the iNaturalist, SUN, and Textures OOD datasets, respectively.
For the pre-trained MobileNet model, our GradOrth approach also demonstrates outstanding performance. We present experimental results on leveraging MobileNet as the pre-trained ID network and evaluate the OOD detection performance on the iNaturalist, SUN, Places, and Textures datasets (the bottom section of table 1). In these experiments, GradOrth demonstrates outstanding performance. In terms of FPR95, GradOrth outperforms ASH-B, DICE+ReAct, DICE+ReAct, and ASH-S by  $4.65\%$ ,  $0.40\%$ ,  $6.50\%$ , and  $0.43\%$ , respectively, across the four datasets. Regarding AUROC, GradOrth
outperforms other baselines on average by at least  $4.01\%$  and  $0.57\%$  in terms of FPR95 and AUROC, respectively.
Model	Methods	OOD Datasets	Average
iNaturalist	SUN	Places	Textures	FPR95	AUROC
FPR95↓	AUROC↑	FPR95↓	AUROC↑	FPR95↓	AUROC↑	FPR95↓	AUROC↑	↓	↑
ResNet	Softmax score	54.99	87.74	70.83	80.86	73.99	79.76	68.00	79.61	66.95	81.99
ODIN	47.66	89.66	60.15	84.59	67.89	81.78	50.23	85.62	56.48	85.41
Mahalanobis	97.00	52.65	98.50	42.41	98.40	41.79	55.80	85.01	87.43	55.47
Energy score	55.72	89.95	59.26	85.89	64.92	82.86	53.72	85.99	58.41	86.17
GradNorm	42.46	90.33	40.73	89.96	43.48	80.64	34.48	88.43	40.29	87.34
ExGrad	54.11	76.91	46.73	69.74	50.62	74.27	38.12	79.37	47.40	75.07
ReAct	20.38	96.22	24.20	94.20	33.85	91.58	47.30	89.80	31.43	92.95
DICE	25.63	94.49	35.15	90.83	46.49	87.48	31.72	90.30	34.75	90.77
DICE + ReAct	18.64	96.24	25.45	93.94	36.86	90.67	28.07	92.74	27.25	93.40
VRA-DN	16.82	96.92	30.65	93.65	39.94	90.75	26.72	95.04	28.53	94.09
VRA-P	15.70	97.12	26.94	94.25	37.85	91.27	21.47	95.62	25.49	94.57
ASH-P	44.57	92.51	52.88	88.35	61.79	85.58	42.06	89.70	50.32	89.04
ASH-B	14.21	97.32	22.08	95.10	33.45	92.31	21.17	95.50	22.73	95.06
ASH-S	11.49	97.87	27.98	94.02	39.78	90.98	11.93	97.60	22.80	95.12
GradOrth (Ours)	11.04±0.23	98.00±0.09	19.61±1.26	95.76±0.49	33.67±0.18	91.78±0.22	11.19±0.20	98.06±0.28	18.57±0.47	96.31±0.27
MobileNet	Softmax score	64.29	85.32	77.02	77.10	79.23	76.27	73.51	77.30	73.51	79.00
ODIN	55.39	87.62	54.07	85.88	57.36	84.71	49.96	85.03	54.20	85.81
Mahalanobis	62.11	81.00	47.82	86.33	52.09	83.63	92.38	33.06	63.60	71.01
Energy score	59.50	88.91	62.65	84.50	69.37	81.19	58.05	85.03	62.39	84.91
ReAct	42.40	91.53	47.69	88.16	51.56	86.64	38.42	91.53	45.02	89.47
DICE	43.09	90.83	38.69	90.46	53.11	85.81	32.80	91.30	41.92	89.60
DICE + ReAct	32.30	93.57	31.22	92.86	46.78	88.02	16.28	96.25	31.64	92.68
ASH-P	54.92	90.46	58.61	86.72	66.59	83.47	48.48	88.72	57.15	87.34
ASH-B	31.46	94.28	38.45	91.61	51.80	87.56	20.92	95.07	35.66	92.13
ASH-S	39.10	91.94	43.62	90.02	58.84	84.73	13.12	97.10	38.67	90.95
GradOrth (Ours)	26.81±1.19	93.17±0.21	30.82±0.94	93.18±0.46	40.27±1.33	89.12±0.84	12.69±0.21	97.52±0.12	27.65±0.92	93.25±0.41
Table 1: OOD detection results with ImageNet-1k as ID. GradOrth present outstanding performance on average and across most datasets. We adopted the identical table format and evaluation metrics as introduced in [48, 12]. The ResNet and MobileNet models are pre-trained solely with ID data from the ImageNet-1k dataset. We use  $\uparrow$  to denote that larger values are preferable, and  $\downarrow$  to denote that smaller values are preferable. All values are presented as percentages. All values in the table are directly taken from table 1 of [12] except for the gradient-based methods (GradNorm, ExGrad, GradOrth (ours)). For GradNorm and ExGrad, we run this experiment leveraging the code provided by the authors.
CIFAR Benchmark: In this research study, we further investigate the performance of GradOrth by conducting additional experimental studies on the CIFAR10 and CIFAR100 datasets. The key observation is that no single method consistently outperforms all other methods across diverse datasets. However, it is noticeable that GradOrth rank is always among the top three across six OOD datasets. This feature presents its promising performance for OOD detection. On the CIFAR10 dataset, GradOrth demonstrates superior performance compared to other baseline methods across six OOD datasets, namely SVHN, LSUN-c, LSUN-r, iSUN, Textures, and Places365. On average, GradOrth outperforms these baselines by  $2.71\%$  and  $0.32\%$  in terms of FPR95 and AUROC, respectively. Detailed experimental results can be found in table 2. In the LSUN-c OOD dataset, DICE demonstrates superior performance with an impressive  $0.26\%$  FPR95, placing it at the top. Our method, on the other hand, ranks second with a respectable  $0.81\%$  FPR95. However, the ranking differs when examining the Textures and Places365 datasets. Notably, Gradorth outperforms other baseline methods in both cases, achieving noteworthy FPR95 values of  $20.63\%$  and  $38.22\%$ , respectively. In contrast, DICE attains the sixth and ninth positions in these datasets, displaying comparatively higher FPR95 rates of  $41.90\%$  and  $48.59\%$ .
On the CIFAR100 dataset, GradOrth surpasses its competitors in both FPR95 and AUROC by an average margin of  $8.0\%$  and  $2.80\%$ , respectively, across six well-known OOD datasets. Detailed experimental results are provided in table 3. For a comprehensive discussion and analysis of the CIFAR benchmark, please refer to the appendix, specifically section C.
Method	OOD Datasets	Average
SVHN	LSUN-c	LSUN-r	iSUN	Textures	Places365	FPR95	AUROC
FPR95	AUROC	FPR95	AUROC	FPR95	AUROC	FPR95	AUROC	FPR95	AUROC	FPR95	AUROC	↓	↑
Softmax score	47.24	93.48	33.57	95.54	42.10	94.51	42.31	94.52	64.15	88.15	63.02	88.57	48.73	92.46
ODIN	25.29	94.57	4.70	98.86	3.09	99.02	3.98	98.90	57.50	82.38	52.85	88.55	24.57	93.71
Mahalanobis	6.42	98.31	56.55	86.96	9.14	97.09	9.78	97.25	21.51	92.15	85.14	63.15	31.42	89.15
Energy score	40.61	93.99	3.81	99.15	9.28	98.12	10.07	98.07	56.12	86.43	39.40	91.64	26.55	94.57
GradNorm	18.63	94.11	1.03	99.61	3.38	98.87	36.89	91.67	50.26	89.72	50.43	84.29	26.77	93.04
ReAct	41.64	93.87	5.96	98.84	11.46	97.87	12.72	97.72	43.58	92.47	43.31	91.03	26.45	94.67
VRA-P	18.75	96.68	1.32	99.63	5.80	98.69	5.70	98.69	34.89	93.42	39.98	91.69	17.74	96.47
DICE	25.99	95.90	0.26	99.92	3.91	98.30	4.36	97.55	41.90	93.36	48.59	89.13	20.83	95.24
ASH-P	30.14	95.29	2.82	99.34	7.97	98.33	8.46	98.29	50.85	88.29	40.46	91.76	23.45	95.22
ASH-B	17.92	96.86	2.52	99.48	8.13	98.54	8.59	98.45	35.73	92.88	48.47	89.93	20.23	96.02
ASH-S	6.51	98.65	0.90	99.73	4.96	98.92	5.17	98.90	24.34	95 (.9)	48.45	88.34	15.05	96.61
GradOrth	5.84±0.29	98.72±0.09	0.81±0.04	99.78±0.05	2.33±0.07	98.71±0.11	4.25±0.09	98.32±0.57	20.63±1.14	94.77±0.19	38.22±0.38	91.64±0.12	12.34±0.34	96.99±0.19
Table 2: Detailed results on six common OOD benchmark datasets with CIFAR-10 as ID: Textures [8], SVHN [40], Places365 [63], LSUN-Crop [61], LSUN-Resize [61], and iSUN [58]. GradOrth outperforms other baselines on FPR95 and AUROC on average. For each ID dataset, we use the same DenseNet pre-trained on CIFAR-10. We present the first, second, and third ranks in blue, green, and orange colors, respectively.  $\uparrow$  indicates larger values are better and  $\downarrow$  indicates smaller values are better.
Method	OOD Datasets	Average
SVHN	LSUN-c	LSUN-r	ISUN	Textures	Places365	FPR95	AUROC
FPR95	AUROC	FPR95	AUROC	FPR95	AUROC	FPR95	AUROC	FPR95	AUROC	FPR95	AUROC	FPR95	AUROC
↓	↑	↓	↑	↓	↑	↓	↑	↓	↑	↓	↑	↓	↑	↓	↑
Softmax score	81.70	75.40	60.49	85.60	85.24	69.18	85.99	70.17	84.79	71.48	82.55	74.31	80.13	74.36
ODIN	41.35	92.65	10.54	97.93	65.22	84.22	67.05	83.84	82.34	71.48	82.32	76.84	58.14	84.49
Mahalanobis	22.44	95.67	68.90	86.30	23.07	94.20	31.38	93.21	62.39	79.39	92.66	61.39	55.37	82.73
Energy score	87.46	81.85	14.72	97.43	70.65	80.14	74.54	78.95	84.15	71.03	79.20	77.72	68.45	81.19
GradNorm	31.57	93.66	9.89	96.75	58.22	87.76	59.60	84.21	59.42	88.09	57.14	82.10	45.98	88.76
ExGrad	29.17	92.47	8.91	96.80	60.12	88.21	56.43	84.73	57.29	88.79	53.47	84.38	44.23	89.23
ReAct	83.81	81.41	25.55	94.92	60.08	87.88	65.27	86.55	77.78	78.95	82.65	74.04	62.27	84.47
VRA-P	66.38	89.02	10.34	98.12	54.39	89.49	55.16	89.48	48.12	88.48	78.31	77.84	53.24	88.74
DICE	54.65	88.84	0.93	99.74	49.40	91.04	48.72	90.08	65.04	76.42	79.58	77.26	49.72	87.23
ASH-P	81.86	83.86	11.60	97.89	67.56	81.67	70.90	80.81	78.24	74.09	77.03	77.94	64.53	82.71
ASH-B	53.52	90.27	4.46	99.17	48.38	91.03	47.82	91.09	53.71	84.25	84.52	72.46	48.73	88.04
ASH-S	25.02	95.76	5.52	98.94	51.33	90.13	46.67	91.30	34.02	92.35	85.86	71.62	41.40	90.02
GradOrth	24.2720.33	93.47±1.02	3.71±0.14	99.07±1.04	48.09±0.12	91.26±0.12	42.73±0.59	91.48±0.41	32.71±0.41	92.62±0.11	48.61±0.35	89.03±0.43	33.35±0.32	92.82±0.50
Table 3: Detailed results on six common OOD benchmark datasets with CIFAR-100 as ID: Textures [8], SVHN [40], Places365 [63], LSUN-Crop [61], LSUN-Resize [61], and iSUN [58]. GradOrth outperforms other baselines on FPR95 and AUROC on average. For each ID dataset, we use the same DenseNet pre-trained on CIFAR-100. We present the first, second, and third ranks in blue, green, and orange colors, respectively.  $\uparrow$  indicates larger values are better and  $\downarrow$  indicates smaller values are better.
4.2.2 Experimental Results on Near-OD Detection
In this section, we demonstrate the effectiveness of our proposed method on near-OOD detection using competitive deep convolutional neural network architectures such as DenseNet and ResNet on computer vision benchmark datasets: CIFAR-10, CIFAR-100. We follow the experimental setting presented in [29, 46].
In-dist (model)	OOD	TNR at TPR 95%	AUROC	Detection Acc.
Baseline [18] / ODIN / Mahalanobis / Gram [45] / GradOrth (Ours)
CIFAR-10 (ResNet)	CIFAR-100	33.3 / 42.0 / 41.6 / 32.9 / 45.3±1.32	86.4 / 85.8 / 88.2 / 79.0 / 90.3±0.81	80.4 / 78.6 / 81.2 / 71.7 / 84.6±1.09
CIFAR-100 (ResNet)	CIFAR-10	19.1 / 18.7 / 20.2 / 12.2 / 19.6±0.31	77.1 / 77.2 / 77.5 / 67.9 / 76.9±0.93	71.0 / 71.2 / 72.1 / 63.4 / 75.4±0.53
CIFAR-10 (DenseNet)	CIFAR-100	40.3 / 53.1 / 14.5 / 26.7 / 58.3±0.26	89.3 / 90.2 / 58.5 / 72.0 / 94.1±0.31	82.9 / 82.7 / 57.2 / 67.3 / 84.7±0.62
CIFAR-100 (DenseNet)	CIFAR-10	18.9 / 16.8 / 7.7 / 10.6 / 21.42±0.52	75.9 / 74.2 / 60.1 / 64.2 / 77.5±0.37	69.7 / 68.6 / 57.8 / 60.4 / 72.9±0.68
Table 4: GradOrth presents stable and reliable performance on Near-OOD data setting (CIFAR datasets). This table presents comparison of OOD detection performance for all combinations of model architecture and training datasets following [45].
4.2.3 Experimental Results on Semantic Shift versus Non-Semantic Shift
Exploring out-of-distribution data uncovers a compelling challenge: the differentiation between semantic and non-semantic shifts. In pursuit of this investigation, we adopt the experimental methodology outlined in [19] to evaluate the efficacy of the GradOrth approach in this context.
DomainNet [43] presents a collection of high-resolution images, spanning dimensions from 180x180 to 640x880 pixels, distributed across 345 classes within six distinct domains. Notably, four of these domains—real, sketch, infographic, and quickdraw—incorporate class labels, thus facilitating the exploration of various distribution shift scenarios.
Our analysis focused on discerning semantic shifts by segregating classes into two subsets: Split A (comprising classes 0 to 172) and Split B (with classes 173 to 344). In our experiment, real-A served as the in-distribution dataset, while the remaining subsets assumed roles as OOD datasets. Real-B represented a notable example of a semantic shift from real-A, whereas sketch-A exemplified a non-semantic shift. Sketch-B exhibited a convergence of both shift types. The classifier, implemented with a Resnet-34 architecture, underwent a 100-epoch training regimen initiated with a learning rate of 0.01. Images were uniformly center-cropped and resized to dimensions of  $224 \times 224$  pixels for this experiment.
The results, as depicted in table 5, highlight two prominent trends. Firstly, datasets characterized by both distribution shift types are notably more discernible, with non-semantic shifts following as the next most distinguishable category. Detecting semantic shifts presents the most formidable challenge. Importantly, GradOrth exhibits promising performance, as evidenced by favorable AUROC and TNR@TPR95 metrics, underscoring its robustness and scalability in addressing practical problems. Nevertheless, there remains ample potential for further refinement and enhancement.
Choice of  $L_{p}$ -norm in GradOrth: In order to investigate the impact of the choice of  $L_{p}$ -norm in Equation 7 on OOD detection performance, we conducted an ablation study. Figure 3 illustrates the comparison using  $L_{1\sim 4}$ -norm,  $L_{\infty}$ -norm, and the fraction norm (with  $p = 0.3$ ).
Among the different norms considered, we observed that the  $L_{2}$ -norm consistently achieved the best OOD detection performance across all four datasets. This finding suggests that the  $L_{2}$ -norm, which equally captures information from all dimensions in the gradient space, is better suited for this task. In contrast, higher-order norms tend to unfairly emphasize larger elements over smaller ones due
OOD	Shift	AUROC	TNR@TPR95
S	NS	Baseline [18]/ODIN* / Maha* / DeConf-C [19]*/ GradOrth (ours)
real-B	✓	75.1 / 69.9 / 53.6 / 69.8 / 79.3	15.3 / 15.4 / 5.09 / 14.0 / 17.8
sketch-A	✓	75.5 / 80.7 / 59.5 / 84.5 / 86.2	20.1 / 31.2 / 7.30 / 37.5 / 40.3
sketch-B	✓	✓	81.8 / 85.7 / 60.4 / 89.1 / 88.4	25.2 / 36.8 / 7.55 / 44.1 / 42.2
infograph-A	79.6 / 82.7 / 81.5 / 89.0 / 92.1	23.5 / 27.8 / 21.6 / 45.4 / 46.8
infograph-B	✓	✓	82.1 / 85.3 / 80.9 / 90.9 / 92.6	24.8 / 31.7 / 21.9 / 49.6 / 51.4
quickdraw-A	✓	78.8 / 96.4 / 67.4 / 96.9 / 96.5	21.1 / 79.9 / 3.38 / 83.1 / 80.6
quickdraw-B	✓	✓	80.5 / 96.9 / 66.1 / 97.4 / 97.0	22.1 / 83.6 / 2.38 / 86.6 / 85.7
Table 5: GradOrth reliable and stable performance on semantic shift (S) and non-semantic shift settings (NS). Performance of five OOD detection methods using DomainNet. The in-distribution is the real-A subset. Each value is averaged over three runs. All values in the table are directly taken from table 4 of [19] except for GradOrth. The type of distribution shift presents a trend of difficulty to the OOD detection problem: Semantic shift (S) > Non-semantic shift (NS) > Semantic + Non-semantic shift.
to the effect of the exponent  $p$ . Notably, the  $L_{\infty}$ -norm, which only considers the largest element (in absolute value), resulted in the worst OOD detection performance among all the norms tested. Additionally, we evaluated the fraction norm but found that it did not outperform the  $L_{2}$ -norm in terms of overall OOD detection performance.
It is worth to mention that while both GradNorm and GradOrth leverage gradients for OOD detection, they adopt differing methodologies. GradNorm places a strong emphasis on the L1 norm, utilizing it to assess gradient magnitudes and discern disparities between ID and OOD data. This focus on gradient magnitude forms the crux of GradNorm's approach. In contrast, GradOrth takes a unique path by employing orthogonal gradient projection within a pre-trained network subspace. In this context, the L2 norm is more suitable for approximating projection length, offering a distinct perspective from the L1 norm employed by GradNorm.
Figure 3: Comparison of OOD detection performance using various  $L_{p}$ -norms.  $L_{2}$ -norm is the best  $L$ -norm for GradOrth as it provides the lowest FPR95 and largest AUROC for GradOrth. Results are presented for False Positive Rate at 95% True Positive Rate (FPR95) on the left, and Area Under the Receiver Operating Characteristic curve (AUROC) on the right.  $\uparrow$  indicates larger values are better and  $\downarrow$  indicates smaller values are better.
To the best of our knowledge, there exists only limited prior research on the use of gradients for detecting OOD inputs. This section aims to explore the connections and differences between our proposed GradOrth method and prior OOD detection approaches that also utilize gradient information. In particular, we will discuss the connection between our approach and the ODIN method, the approach proposed by [28], GradNorm method [23], and ExGrad method [24]. Moreover, [28] utilized gradients from all layers to train a distinct binary classifier, which can lead to a computationally burdensome process for deeper and larger models. Nevertheless, our findings with GradOrth demonstrate that the gradient from the last layer consistently achieves the highest performance compared to other gradient selections. Hence, the computational cost incurred by GradOrth is negligible.
ODD Gradient-Based Methods ODIN [32] introduced the concept of utilizing gradient information for OOD detection. Their approach involved a pre-processing technique that added small perturbations obtained from the input gradients. The objective was to enhance the model's confidence in its predictions by increasing the softmax score for each input. This resulted in a larger gap between the softmax scores of ID and OOD inputs, making them more distinguishable and improving OOD detection performance. It is important to note that ODIN indirectly employed gradients through
input perturbation, and the OOD scores were still calculated based on the output space of the perturbed inputs. GradNorm [23] also utilizes gradient information from a neural network to detect distributional shifts between ID and OOD samples. By measuring the norm of gradients with respect to the network's input, it quantifies uncertainty and identifies OOD samples causing significant output changes. ExGrad, proposed by [24], introduces a method akin to GradNorm with two notable distinctions. Firstly, the label distribution of  $y$  is derived from the model's predicted distribution  $(P)$  as opposed to the uniform distribution. Secondly, ExGrad computes the expected norm of the gradient, in contrast to GradNorm which calculates the norm of the expected gradient.
Discriminative Models for OOD Uncertainty Estimation The problem of classification with rejection has a long history, dating back to early works on abstention such as [6] and [15], which considered simple model families like SVMs [9]. However, the phenomenon of neural networks' overconfidence in OOD data was not revealed until the work of [41].
Early efforts aimed to improve OOD uncertainty estimation by proposing the ODIN score [32] and Mahalanobis distance-based confidence score [29]. More recently, [34] proposed using an energy score derived from a discriminative classifier for OOD uncertainty estimation, showing advantages over the softmax confidence score both empirically and theoretically. [54] demonstrated that an energy-based approach can improve OOD uncertainty estimation for multi-label classification networks. Additionally, [22] revealed that approaches developed for common CIFAR benchmarks might not effectively translate into a large-scale ImageNet benchmark, highlighting the need to evaluate OOD uncertainty estimation in a large-scale real-world setting. These developments have brought renewed attention to the problem of classification with rejection and the need for effective OOD uncertainty estimation.
Generative Models for OOD Uncertainty Estimation Detection of OOD inputs is a crucial problem in machine learning. One popular approach is to use generative models that estimate the density directly. Such models can identify OOD inputs as those lying in low-likelihood regions. To this end, a plethora of literature has emerged to leverage generative models for OOD detection. However, recent studies have shown that deep generative models can assign high likelihoods to OOD data, rendering such models less effective in OOD detection. Additionally, these models can be challenging to train and optimize, and their performance may lag behind their discriminative counterparts. In contrast, our approach relies on a discriminative classifier, which is easier to optimize and achieves stronger performance. While some recent works have attempted to improve OOD detection with generative models using improved metrics, likelihood ratios, and likelihood regret, our approach leverages the energy score from a discriminative classifier and has demonstrated significant advantages over generative models in OOD detection.
Distributional Shifts The problem of distributional shift has garnered significant attention in the research community. It is essential to recognize and distinguish between different types of distributional shift problems. In the literature on OOD detection, the focus is typically on ensuring model reliability and detecting label-space shifts [18, 32, 34], where OOD inputs have labels that are disjoint from the ID data, and as such, should not be predicted by the model. On the other hand, some studies have examined covariate shifts in the input space [17, 37, 42], where inputs may be subject to corruption or domain shifts. However, covariate shifts are commonly used to evaluate model robustness and domain generalization performance, where the label space  $\mathcal{V}$  remains the same during test time. It is worth noting that our work focuses on the detection of shifts where the model should not make any predictions, as opposed to covariate shifts where the model is expected to generalize.
In this paper, we propose GradOrth, a novel OOD uncertainty estimation approach utilizing information extracted from the important parameter space for ID data and gradient space. Extensive experimental results show that our gradient-based method can improve the performance of OOD detection by up to  $8.05\%$  in FPR95 on average, establishing superior performance. We hope that our research brings to light the informativeness of gradient subspace, and inspires future work to utilize it for OOD uncertainty estimation. In our future research, our objective is to investigate GradOrth's capabilities considering different directions like influence functions [25], novelty detection in open-world context [13], data pre-selection [30], and underspecification [36].