Title: Revisiting Logit Distributions for Reliable Out-of-Distribution Detection
Venue: NeurIPS 2025
Accepted: yes
Keywords: Out-of-Distribution Detection

Logits Analysis

Open-set Recognition

Model Reliability
Abstract: Out-of-distribution (OOD) detection is critical for ensuring the reliability of deep learning models in open-world applications. While post-hoc methods are favored for their efficiency and ease of deployment, existing approaches often underexploit the rich information embedded in the model’s logits space. In this paper, we propose LogitGap, a novel post-hoc OOD detection method that explicitly exploits the relationship between the maximum logit and the remaining logits to enhance the separability between in-distribution (ID) and OOD samples. To further improve its effectiveness, we refine LogitGap by focusing on a more compact and informative subset of the logit space. Specifically, we introduce a training-free strategy that automatically identifies the most informative logits for scoring. We provide both theoretical analysis and empirical evidence to validate the effectiveness of our approach. Extensive experiments on both vision-language and vision-only models demonstrate that LogitGap consistently achieves state-of-the-art performance across diverse OOD detection scenarios and benchmarks.

Revisiting Logit Distributions for Reliable Out-of-Distribution Detection

Abstract

Out-of-distribution (OOD) detection is critical for ensuring the reliability of deep learning models in open-world applications. While post-hoc methods are favored for their efficiency and ease of deployment, existing approaches often underexploit the rich information embedded in the model's logits space. In this paper, we propose LogitGap, a novel post-hoc OOD detection method that explicitly exploits the relationship between the maximum logit and the remaining logits to enhance the separability between in-distribution (ID) and OOD samples. To further improve its effectiveness, we refine LogitGap by focusing on a more compact and informative subset of the logit space. Specifically, we introduce a training-free strategy that automatically identifies the most informative logits for scoring. We provide both theoretical analysis and empirical evidence to validate the effectiveness of our approach. Extensive experiments on both vision-language and vision-only models demonstrate that LogitGap consistently achieves state-of-the-art performance across diverse OOD detection scenarios and benchmarks. Code is available at https://github.com/GIT-LJc/LogitGap .

1 Introduction

Deep learning models have demonstrated remarkable success across various computer vision tasks, typically under the closed-set assumption. However, this assumption often fails to hold in real-world applications such as autonomous driving, medical imaging, and access control systems. In open-world scenarios, deployed models are inevitably exposed to out-of-distribution (OOD) samples, which can result in unreliable predictions, thereby introducing significant risks to the safety and reliability of the system. To mitigate these risks, OOD detection has been proposed to detect and reject such OOD inputs before making decisions.

In recent years, the post-hoc OOD detection methods have attracted significant attention. These methods directly operate on pre-trained models without modifying their parameters [30, 14, 59, 25, 29, 45, 17], offering high deployment flexibility and computational efficiency. A core problem for the post-hoc methods is how to design a scoring function that effectively maximize the separability between ID and OOD samples. Two representative scoring functions are Maximum Logit (MaxLogit) [14] and Maximum Concept Matching (MCM 2 [33]). MaxLogit simply uses the largest logit value as the OOD score, which completely disregards information from the remaining logits. In contrast,

1 , 2

ID

OOD

overlap

(a) MaxLogit

ID

OOD

overlap

(b) MCM

ID

OOD

overlap

(c) LogitGap

MCMapplies a softmax function into the logits and then takes the maximum softmax probability as the OOD score. By applying a softmax function, MCM significantly reducing the overlap between the score distributions of ID and OOD samples (Figure 1). However, the softmax function used in MCMignores the absolute magnitudes of original logits, resulting in information loss. Consequently, different logit patterns may collapse into similar probability distributions, limiting the effectiveness of OOD detection. A natural question arises: How can we more effectively leverage the discriminative information embedded in non-maximum logits to enhance ID-OOD separability?

To answer the above question, we first systematically examine the logit distributions of ID and OOD samples. We observe a consistent phenomenon: the relationship between the maximum logit and the remaining logits displays fundamentally different characteristics for ID and OOD data. As shown in Figure 2, ID samples generally have higher maximum logits accompanied by lower non-maximum logits compared to OOD samples. This results in a pronounced 'logit gap", defined as the numerical difference between the maximum and the remaining logits, which is consistently and significantly larger for ID samples than for OOD samples. This observed pattern suggests that the relative logit gap can naturally serve as a discriminative criteria for effective OOD detection.

Motivated by this observation, we propose LogitGap , a novel post-hoc OOD detection approach that effectively leverages the logit-gap to distinguish between ID and OOD samples. By explicitly leveraging the entire logit space, our proposed method significantly enhances the separability between ID and OOD samples (as shown in Figure 1). To further improve its effectiveness, we refine LogitGap by focusing on a more compact and informative subset of the logit space. This improvement is inspired by our empirical observation: the tail region (logits from least likely classes) exhibits substantial overlap between ID and OOD samples, which brings irrelevant and noisy signal. To mitigate this issue, we constrain LogitGap's computation to the topN logits, naturally excluding the noisy tail logits from the OOD score. To balance information retention (larger N ) and noise suppression (smaller N ), we propose a simple yet effective strategy for selecting N : choose the value that maximizes the difference between the mean topN logits of ID and OOD samples, using only a small number of ID samples.

We conduct experiments across diverse OOD scenarios to evaluate the effectiveness of our LogitGap method. Results show that LogitGap outperforms existing OOD scoring functions in both zero-shot and few-shot OOD detection tasks based on the CLIP model. Furthermore, when combined with training-based OOD detection methods, LogitGap brings notable performance gains, highlighting its complementary nature. To assess the generality of our approach, we also evaluate it under traditional OOD settings, where models are trained from scratch using cross-entropy loss on ID data. Across all scenarios, LogitGap consistently achieves significant performance gains, validating its versatility and robustness in various OOD detection scenarios.

Our main contributions can be summarized as follows:

Wepropose a novel post-hoc OOD detection method for a pre-trained model, which leverages the information from entire logit space to enhance ID-OOD separability.

We analyze the limitations of existing OOD scoring methods and theoretically derive their relationship to our LogitGap method.

We conduct experiments to demonstrate the versatility and effectiveness of proposed LogitGap method in various OOD scenarios.

2 Related Work

OODDetection for Models Trained with Cross-Entropy Loss. OOD detection for classification models trained with cross-entropy loss typically falls into two categories. The first involves trainingtime strategies that endow models with OOD awareness, often requiring access to training data [20, 11, 18]. For example, VOS [11] synthesizes virtual outliers, LogitNorm [52] applies logit normalization, and MOS [22] partitions the class space hierarchically. Other approaches leverage auxiliary OOD samples, including OE [18], which encourages flat predictions, MCD [56], which amplifies entropy discrepancies, UDG [53], which clusters out mixed ID data, and MixOE [57], which mixes ID and OOD data to expand coverage. A key challenge for these methods lies in selecting appropriate auxiliary data while mitigating the risk of overfitting. Another major line of work focuses on post-hoc methods, which are training-free and computationally efficient. These approaches can be broadly categorized into logits-based and feature-based methods. Logits-based approaches derive OOD scores from model outputs, such as MaxLogit [14], ODIN [27], Energy [30], GradNorm [21], and MaxCosine [59]. In contrast, feature-based methods analyze internal representations, including Mahalanobis [25], KNN [45], FDBD [28], ViM [50].

OODDetection for Contrastive Learning Pretrained Models. For contrastive learning models, MCM[33] extends MSP [17] to vision-language models, while GL-MCM [35] incorporates local feature cues to enhance detection performance. Recent works have increasingly focused on teaching models to recognize 'what an input is not'. For example, CLIPN [51] fine-tunes CLIP to generate negative prompts corresponding to absent concepts. With the rise of prompt engineering, many approaches improve OOD detection by fine-tuning prompts with a few ID samples. For example, LoCoOp [34] builds on CoOp's [62] by applying OOD regularization through local OOD features. Approaches such as LSN [37] and NegPrompt [26] jointly learn positive and negative prompts to better capture OOD characteristics. ID-like [1] introduces prompts aligned with ID-like regions near the OOD boundary to refine detection through targeted fine-tuning. Additionally, some CLIP-based approaches leverage external concept vocabularies [23, 4] or language models [3] to map OOD samples to auxiliary semantic categories, but such methods are generally constrained to semanticshift scenarios. While CLIP-based OOD detection methods have demonstrated strong performance, most depend on additional data or external models, limiting their scalability and general applicability. In this work, we propose a lightweight, post-hoc OOD method for pre-trained models, which does not require additional supervision.

3 Preliminary

3.1 Problem Setting

We study the zero-shot and few-shot OOD detection problem with a pre-trained model f . Let Y in denote the set of known classes (in-distribution, ID) for a given classification task. Given a test sample x drawn from the input space X , if its true label y / ∈ Y in , then x is considered as an OOD sample. The goal of zero-shot OOD detection is to identify samples that do not belong to any of known ID classes, without requiring additional training or exposure to OOD data during model development. Similarly, few-shot OOD detection aims to achieve the same goal, but with access to a small number of training samples from the ID class set Y in .

Formally, in the zero-shot and few-shot OOD problem, the decision function D : X → { ID, OOD } is typically constructed as:

where S : X → R is a scoring function based on pre-trained model f , which assigns a scalar confidence score to each input sample. The decision threshold λ establishes the decision boundary: samples satisfying S ( x ) < λ are identified as OOD, while those with S ( x ) ≥ λ are considered ID.

3.2 OODDetection Scoring Function

The design of the scoring function is crucial, as it directly determines the effectiveness of OOD detection. In this subsection, we revisit the design principles of existing logit-based scoring functions,

with a focus on two representative approaches: Maximum Logit (MaxLogit) [14] and Maximum Concept Matching (MCM) [33]. The most straightforward approach, MaxLogit [14], uses the maximum logit value as the confidence score. Formally, given K known classes ( i . e ., |Y in | = K ), for an input sample x , the pre-trained model f produces a logit vector as z = f ( x ; Y in ) ∈ R K . The OOD score by MaxLogit is then calculated as:

where z k denotes the predicted logit for class k . However, MaxLogit focus solely on the most confident prediction, disregarding potentially useful information from other classes. A more widely used scoring function is MCM [33], which applies softmax normalization to the logits and takes the maximum probability as the OOD score:

where τ is a temperature scaling parameter. Unlike MaxLogit, MCM leverages the full logit distribution through the denominator of the softmax operation. As a result, MCM generally achieves superior separability between ID and OOD samples compared to MaxLogit.

4 Method

4.1 OODscore: LogitGap

We observe an interesting phenomenon in real-world data: While the mean logit values of ID and OOD samples are often similar, their logit distributions exhibit systematically distinct patterns, as shown in Figure 2. Specifically, ID samples (red line in Figure 2) tend to produce sharper, more peaked logit distributions, typically with one dominant logit value corresponding to the predicted class. In contrast, OOD samples often yield flatter, more uniform logit distributions, with less pronounced maxima and elevated non-maximum logits. This is quantitatively reflected as: (i) Higher maximum logit values for ID samples compared to OOD samples; (ii) Higher non-predicted class logits in OOD samples relative to ID samples.

0

20

40

60

80

100

Index of topN Logits

7.5

10.0

12.5

15.0

17.5

20.0

22.5

25.0

27.5

Logits

ID

OOD

Building upon these observations, we propose a simple yet effective OOD scoring function, call LogitGap, which exploits the inherent distributional differences between ID and OOD logits. The core idea is to quantify the average gap between the maximum logit and the remaining logits -a measure that naturally amplifies the contrast between ID and OOD samples. This formulation effectively captures the distinctive peakedness of ID logit predictions, while highlighting the relative flatness of OOD logit distributions, thereby enhancing ID-OOD separability.

Formally, given a logit vector z predicted for a test sample x , we first sort its elements in descending order to obtain z ′ , where z ′ n denotes the n -th largest logit. The LogitGap score is then defined as:

The main difference between MCM [33] and our proposed LogitGap lies in how they utilize nonmaximum class information. MCM implicitly incorporates this information through the normalization process in the softmax denominator, while LogitGap explicitly quantifies the average gap between the maximum logit and all other logits. This explicit formulation allows for more direct utilization of the discriminative patterns observed in logit distributions. To establish a theoretical connection between MCM and LogitGap, we present the following analysis.

Theorem 4.1. Given a K -way classification task, let the predicted logit vector for a sample x be z = [ z 1 , z 2 , . . . , z K ] . Let z ′ = [ z ′ 1 , z ′ 2 , . . . , z ′ K ] denote the sorted logits in descending order, such

that z ′ 1 = max k z k . Then, for the temperature scaling parameter τ used in the softmax function, if τ > 2( K -1) , we have

where FPR LogitGap ( λ L ) is the false positive rate based on LogitGap with threshold λ L . Similarly, FPR MCM ( τ, λ MCM ) is the false positive rate based on MCM with temperature τ and threshold λ MCM.

For clarity, we provide a brief proof sketch below (complete proof is provided in Appendix A).

proof sketch. False Positive Rate (FPR) measures the likelihood of an ID sample being misclassified as OOD. A lower FPR indicates better OOD detection performance. Let Q x denotes the out-ofdistribution P x | OOD . By definition, we express the FPR of MCM, denoted as FPR MCM ( τ, λ MCM ) , in the following:

Similarly, the FPR of LogitGap can be written as

By introducing an intermediate OOD score function, LogitGap with softmax (LogitGap_softmax), we establish a connection between MCM and LogitGap. We define it as S LogitGap _ softmax ( x ; f ) =

where λ MCM = λ LM + 1 K .

Then, we aim to demonstrate the performance of LogitGap is guaranteed to surpass that of MCM, which is equal to finding the conditions under which the inequality holds: FPR LogitGap ( τ, λ L ) < FPR MCM ( τ, λ MCM ) . By combining the results from Eq.(5) Eq.(6), we can derive that the inequality holds with the condition of τ > 2( K -1) .

Theorem 4.1 demonstrates that when the temperature scaling parameter τ exceeds a certain threshold, the OOD detection performance of LogitGap is guaranteed to surpass that of MCM [33]. This result stems from fundamental differences in how the two methods process logit information: (i) Information Preservation. MCMoperates on softmax-normalized probabilities, which inherently compress logit magnitudes. Differently, LogitGap directly exploits the raw logit margins, preserving richer discriminative cues. (ii) Temperature Sensitivity. The performance gap increases with higher τ : for MCM: higher temperatures exacerbate information loss by overly dispersing the probability mass; For LogitGap, the margin-based formulation remains robust regardless of τ .

4.2 Logits Selection for Focused Scoring

While LogitGap utilize the complete set of K predicted logits, we identify a critical limitation in K -way classification: certain classes consistently yield negligible activations, particularly those that are semantically or visually unrelated to the input. These inactive logits provide limited discriminative information and thus contribute little to OOD detection. The issue becomes more pronounced as K increases, since a larger proportion of irrelevant logits is introduced. These irrelevant logits add noise, reducing the clarity of the distributional differences between ID and OOD samples.

By eliminating redundant tail information, the structural differences between ID and OOD prediction distributions become more pronounced and easier to exploit. To achieve this, we propose narrowing the focus to a more compact and informative subset of the logit space. Formally, we sort the predicted

logits in descending order and select the top N (out of K ) logits, resulting in a reduced prediction vector. We then apply the LogitGap scoring function to this truncated logit vector

How to Determine the Hyperparameter N . The effectiveness of LogitGap largely depends on the choice of the hyperparameter N , which controls the number of top logits retained for OOD scoring. An appropriate selection of N strikes a balance between capturing informative signals and avoiding noisy, uninformative logits. Formally, given the sorted logits z ′ , the LogitGap -topN score can be equivalently reformulated as:

where ¯ z ′ N denotes the mean of the logits ranked from second to the N -th largest. Then, for OOD samples drawn from P OOD ( x ) and ID samples drawn from P ID ( x ) , our objective is to identify the optimal value of N that maximizes the score discrepancy between them. This can be expressed as:

Equation (10) provides a criterion for selecting the hyperparameter N . However, in practical scenarios, it is often challenging to obtain authentic OOD samples. To address this, we assume access to a small ID validation set (containing no more than 100 samples) and simulate potential OOD data by applying interpolation-based transformations to the ID features and injecting random noise. Based on these synthesized samples, we can estimate an appropriate value for N through Equation (10).

5 Experiments

5.1 Experimental Settings

Datasets. We evaluate the effectiveness of LogitGap across multiple OOD detection benchmarks. Specifically, we use ImageNet [7] or ImageNet-100 as ID datasets, while NINCO [2], ImageNet-OOD [55], and ImageNet-O [19] are used as OOD datasets. Each OOD dataset contains categories that are disjoint from those in the corresponding ID dataset. Following MCM [33], we also construct a more challenging OOD detection task by alternately using ImageNet-10 and ImageNet-20 as the ID and OOD datasets. This setting leverages semantic similarity within the label space to increase task difficulty, providing a more rigorous evaluation of OOD separability.

Comparison Methods. We adopt CLIP [39] with ViT-B/16 [10] backbone as the pre-trained model for zero-shot OOD detection. Our evaluation encompasses both zero-shot and few-shot settings. For the few-shot OOD detection setting, we randomly select one (one-shot) or four (fourshot) samples from each ID class and use these limited samples to fine-tune CLIP. To provide a comprehensive comparison, we benchmark against several representative post-hoc OOD detection methods, including MCM [33], MaxLogit [14], Energy [30] and GL-MCM [35], as well as an enhancing method TAG [31]. Additionally, we evaluate two few-shot OOD approaches: CoOp [62] and ID-Like [1]. Specifically, CoOp [62] fine-tunes prompts using limited labeled samples, serving as a simple few-shot baseline. ID-Like [1] jointly optimizes ID-like prompts and synthesizes outliers from semantically related ID data, thereby improving detection of semantically similar OOD samples. We compare these methods against two variants of our approach: LogitGap , which uses a fixed value of N set to 20% of the total number of classes, and LogitGap* , where N is adaptively determined using an ID validation set along with synthetic OOD data.

Evaluation Metrics. We use three standard metrics commonly used in OOD detection literature: (1) False Positive Rate (FPR95): Measures the probability that an OOD sample is misclassified as ID

when the true positive rate of ID samples is fixed at 95% . (2) Area Under the Receiver Operating Characteristic Curve (AUROC), and (3) Area Under the Precision-Recall Curve (AUPR).

5.2 Results

Zero-Shot OOD Detection. We first conduct a comprehensive evaluation of various methods in the zero-shot OOD detection setting. Our experimental setup utilizes ImageNet and ImageNet100 as ID datasets, comparing multiple OOD scoring functions against our proposed LogitGap approach. As shown in Table 1 and Table 2, both LogitGap and its adaptive variant LogitGap* achieve superior performance compared to existing methods. Specifically, on ImageNet as ID dataset, LogitGap reduces FPR95 by 3 . 65% compared to MCM [33]. On ImageNet-100, the improvement reaches 5 . 78% . These significant performance gains highlight the effectiveness of our logit-gap based scoring approach. Furthermore, LogitGap* typically provides additional improvements over the fixed-parameter LogitGap, validating the advantage of our adaptive parameter selection strategy 3 .

Few-Shot OOD Detection. We further investigate the integration of LogitGap with several recent few-shot OOD detection approaches, including CoOp [62] and ID-Like [1]. Specifically, CoOp [62] enhances in-distribution classification by prompt tuning, and ID-Like [1] extends this framework by incorporating negative prompts to enhance OOD detection. Following [1], we conduct rigorous experiments under both one-shot and four-shot finetuning configurations. The results are summarized in Table 1 and Table 2. As shown, LogitGap* consistently enhances the performance of both baseline methods across all evaluation metrics. For instance, using ImageNet as ID dataset, integrating LogitGap with ID-Like [1] improves FPR95 and AUROC by 7 . 39% and 7 . 08% under the 1-shot setting, respectively (Table 1). These significant performance gains demonstrate that LogitGap can serve as an effective complementary module to existing OOD detection frameworks, effectively amplifying their OOD detection capabilities.

Hard OOD Detection. To thoroughly evaluate the robustness of LogitGap, we further conduct specialized experiments focusing on two particularly challenging categories of hard OOD samples: Semantically Hard OOD Detection and Covariate Shifts OOD Detection.

1. Semantically Hard OOD Detection. Prior work [33] has demonstrated that OOD detection becomes particularly challenging when OOD samples exhibit semantic similarity to ID data. To evaluate this difficult scenario, we adopt the experimental setup from MCM [33], alternately

using ImageNet-10 and ImageNet-20 as ID and OOD datasets. As shown in Table 3, LogitGap demonstrates superior performance across all evaluation metrics compared to existing methods. Specifically, when detecting ImageNet-20 as OOD against ImageNet-10 ID, LogitGap reduces FPR95 by 3 . 40% compared to MCM [33]. In the reverse configuration (ImageNet-20 as ID vs. ImageNet-10 as OOD), the improvement reaches 8 . 40% . These significant margins highlight LogitGap's enhanced capability to distinguish semantically related distributions, addressing a key challenge in real-world OOD detection scenarios.

2. Covariate Shifts OOD Detection. While CLIP exhibits strong generalization, its performance degrades significantly under covariate shifts. For instance, fine-tuning CLIP on ImageNet leads to notable drops on covariate-shifted variants like ImageNet-Sketch [49] and ImageNet-A [19], where the visual appearance of test samples significantly differs from that of ImageNet [62]. To assess LogitGap under hard OOD conditions, we consider a covariate shift setting. Specifically, we construct ID data using limited samples (one-shot or four-shot) from ImageNet, while treating ImageNet-R [16], ImageNet-A, and ImageNet-Sketch as OOD datasets. LogitGap is integrated into several recent few-shot OOD detection methods. The results are summarized in Table 4.

From Table 4, we observe that: (1) Increasing ID training samples does not consistently enhance OOD detection performance in some few-shot OOD methods. For instance, ID-like [1] shows higher FPR95 in 4-shot versus 1-shot training. We hypothesize that this phenomenon arises from the strong generalization capability of fine-tuned CLIP models: while aiding ID classification, it may also lead to overconfident prediction, thereby misclassifying OOD samples as incorrect ID classes and ultimately impairing OOD detection. (2) The proposed LogitGap method consistently enhances OOD detection performance across various covariate shift scenarios and different fewshot OOD methods. For instance, LogitGap reduces FPR95 by 2 . 20% (1-shot) and 1 . 17% (4-shot) for ID-Like [1], underscoring its effectiveness under varying few-shot conditions.

5.3 Further Discussion

Influence of the Hyperparameter N on LogitGap Performance. We examine the impact of the hyperparameter N on the performance of the LogitGap method. We conduct experiments by utilizing ImageNet-100 as the ID dataset in the semantic shift senario. As illustrated in Figure 3, the FPR95 performance on all three OOD datasets degrades when N is either too small or too large. This trend can be attributed to the trade-off in the representation capacity of the logits space. When N is too small, the logits space is overly compact and lacks sufficient discriminative information, thereby limiting the effectiveness of LogitGap. In contrast, an excessively large N introduces a substantial amount of tail information, where the distinction between ID and OOD samples becomes less pronounced. This weakens the discriminative signal and reduces overall performance. Importantly, we find that LogitGap exhibits relatively stable and consistently strong performance when N is set between 20% and 50% of the total number of classes. This observation suggests that a moderately compact logits space provides a favorable N in real-world OOD detection scenarios.

0

20

40

60

80

100

45

50

55

60

65

70

75

FPR

NINCO

ImageNet-O

ImageNetOOD

balance between retaining information and suppressing noise, offering a robust choice for selecting

Logit Distribution Differences Between ID and OOD

Data. The design of LogitGap is motivated by the observation that the logit distributions of ID and OOD samples exhibit distinct patterns: ID samples tend to have higher maximum logit, whereas OOD samples display higher logits among the non-predicted classes. To verify the generality of this phenomenon, we analyze the logit distribution statistics using ImageNet as the ID dataset on CLIP ViTL/14. Specifically, z ′ 1 denotes the average maximum logit, while ¯ z ′ N represents the average logit over all non-predicted classes. As shown in Figure 4, ID samples consistently show larger z ′ 1 and smaller ¯ z ′ N than OOD samples, which confirms the broad consistency of this observation.

z

′

1

z

′

N

0

5

10

15

20

25

Logits

ImageNet

iNaturalist

Textures

ImageNet-O

Different Variants of LogitGap. Motivated by the distinct logit distribution patterns of ID and OOD samples, we design LogitGap, which subtracts the average of non-maximum logits from the maximum logit. Since ID samples usually have higher maximum and lower remaining logits, this formulation effectively enhances ID-OOD separation. In addition, we introduce three variants. LogitGap_exp, LogitGap_square, and LogitGap_sqrt apply exponential, square, square root transformations, respectively, to further transform score differences. As shown in Table 5, these variants perform comparably to the original LogitGap, suggesting that different nonlinear transformations can achieve similarly effective discrimination.

Effectiveness on Traditional OOD Detection. To rigorously assess the generalizability of LogitGap, we extend our evaluation to traditional OOD detection setting. Following the OpenOOD protocol [54], we use CIFAR-10 [24] as ID dataset and adopt the standard OpenOOD benchmark splits for OOD evaluation. The OOD benchmarks include both near-OOD datasets : CIFAR-100, Tiny ImageNet, and far-OOD datasets : MNIST [8], SVHN [36], Texture [5], and Places365 [61]. In this setup, we employ a ResNet-50 [13] backbone trained from scratch on ID data using cross-entropy loss. We compare LogitGap against a comprehensive suite of baselines: (i) Logit-based post-hoc methods: MSP [17], MaxLogit [14], ODIN [27], Energy [30], GEN [32]; (ii) Internal network statistics-based methods: Mahalanobis [25], ReAct [44], NCI [29], SHE [58]; (iii) Training-based methods with auxiliary data: Outlier Exposure (OE) [18], MixOE [57].

As shown in Table 6, LogitGap consistently outperforms all baselines across benchmarks. We observe that: (1) Compared to the logit-based method MSP [17], LogitGap reduces FPR95 by 2 . 26% . (2) Unlike NCI [29], which relies on the relationship between features and classifier weights, LogitGap is fully end-to-end without extra feature extraction. (3) When combined with auxiliary-data methods such as OE [18] and MixOE [57], LogitGap further improves their baselines - achieving a notable 8 . 29% FPR95 reduction on MixOE. These results demonstrate the strong generalization capability of LogitGap across diverse OOD detection paradigms in the traditional setting.

6 Conclusion

In this work, we revisit the OOD detection problem from the perspective of logits space and reveal key limitations of existing logits-based scoring methods. To overcome these challenges, we propose LogitGap, a simple yet effective approach that enhances ID-OOD separability by explicitly leveraging the gap between the maximum logit and the remaining logits. Furthermore, we develop a lightweight, data-efficient strategy to automatically determine the optimal subset size for scoring. Extensive experiments show that LogitGap consistently outperforms existing methods across both traditional vision architectures and pre-trained vision-language models, while maintaining ease of deployment.

Limitations and Broader Impact. While LogitGap demonstrates strong performance in visual classification tasks, its current design is primarily tailored to visual models. In future work, we plan to extend LogitGap to other modalities, such as text or speech, to enable robust OOD detection across a broader range of input types.

Acknowledgments

This work is partially supported by National Science and Technology Major Project 2021ZD0111901 and National Natural Science Foundation of China 62376259, 62576334, 62306301.

References

Yichen Bai, Zongbo Han, Changqing Zhang, Bing Cao, Xiaoheng Jiang, and Qinghua Hu. Id-like prompt learning for few-shot out-of-distribution detection. In CVPR , 2024.

Julian Bitterwolf, Maximilian Müller, and Matthias Hein. In or out? fixing imagenet out-ofdistribution detection evaluation. In ICML , 2023.

Chentao Cao, Zhun Zhong, Zhanke Zhou, Yang Liu, Tongliang Liu, and Bo Han. Envisioning outlier exposure by large language models for out-of-distribution detection. In ICML , 2024.

Mengyuan Chen, Junyu Gao, and Changsheng Xu. Conjugated semantic pool improves ood detection with pre-trained vision-language models. In NeurIPS , 2024.

Mircea Cimpoi, Subhransu Maji, Iasonas Kokkinos, Sammy Mohamed, and Andrea Vedaldi. Describing textures in the wild. In CVPR , 2014.

Aron Culotta and Andrew McCallum. Reducing labeling effort for structured prediction tasks. In AAAI , 2005.

Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In CVPR , 2009.

Li Deng. The mnist database of handwritten digit images for machine learning research [best of the web]. IEEE signal processing magazine , 29(6):141-142, 2012.

Andrija Djurisic, Nebojsa Bozanic, Arjun Ashok, and Rosanne Liu. Extremely simple activation shaping for out-of-distribution detection. In ICLR , 2023.

Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. An image is worth 16x16 words: Transformers for image recognition at scale. In ICLR , 2021.

Xuefeng Du, Zhaoning Wang, Mu Cai, and Yixuan Li. Vos: Learning what you don't know by virtual outlier synthesis. In ICLR , 2022.

Benjamin Elizalde, Soham Deshmukh, Mahmoud Al Ismail, and Huaming Wang. Clap learning audio concepts from natural language supervision. In ICASSP , 2023.

Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In CVPR , 2016.

Dan Hendrycks, Steven Basart, Mantas Mazeika, Mohammadreza Mostajabi, Jacob Steinhardt, and Dawn Song. Scaling out-of-distribution detection for real-world settings. In ICML , 2022.

Dan Hendrycks, Steven Basart, Mantas Mazeika, Andy Zou, Joe Kwon, Mohammadreza Mostajabi, Jacob Steinhardt, and Dawn Song. Scaling out-of-distribution detection for realworld settings. In ICML , 2022.

Dan Hendrycks, Steven Basart, Norman Mu, Saurav Kadavath, Frank Wang, Evan Dorundo, Rahul Desai, Tyler Zhu, Samyak Parajuli, Mike Guo, et al. The many faces of robustness: A critical analysis of out-of-distribution generalization. In CVPR , 2021.

Dan Hendrycks and Kevin Gimpel. A baseline for detecting misclassified and out-of-distribution examples in neural networks. In ICLR , 2017.

Dan Hendrycks, Mantas Mazeika, and Thomas Dietterich. Deep anomaly detection with outlier exposure. In ICLR , 2019.

Dan Hendrycks, Kevin Zhao, Steven Basart, Jacob Steinhardt, and Dawn Song. Natural adversarial examples. In CVPR , 2021.

Yen-Chang Hsu, Yilin Shen, Hongxia Jin, and Zsolt Kira. Generalized odin: Detecting out-ofdistribution image without learning from out-of-distribution data. In CVPR , 2020.

Rui Huang, Andrew Geng, and Yixuan Li. On the importance of gradients for detecting distributional shifts in the wild. In NeurIPS , 2021.

Rui Huang and Yixuan Li. Mos: Towards scaling out-of-distribution detection for large semantic space. In CVPR , 2021.

Xue Jiang, Feng Liu, Zhen Fang, Hong Chen, Tongliang Liu, Feng Zheng, and Bo Han. Negative label guided ood detection with pretrained vision-language models. In ICLR , 2024.

A. Krizhevsky and G. Hinton. Learning multiple layers of features from tiny images. Handbook of Systemic Autoimmune Diseases , 1(4), 2009.

Kimin Lee, Kibok Lee, Honglak Lee, and Jinwoo Shin. A simple unified framework for detecting out-of-distribution samples and adversarial attacks. In NeurIPS , 2018.

Tianqi Li, Guansong Pang, Xiao Bai, Wenjun Miao, and Jin Zheng. Learning transferable negative prompts for out-of-distribution detection. In CVPR , 2024.

Shiyu Liang, Yixuan Li, and Rayadurgam Srikant. Enhancing the reliability of out-ofdistribution image detection in neural networks. In ICLR , 2018.

Litian Liu and Yao Qin. Fast decision boundary based out-of-distribution detector. In ICML , 2024.

Litian Liu and Yao Qin. Detecting out-of-distribution through the lens of neural collapse. In CVPR , 2025.

Weitang Liu, Xiaoyun Wang, John D Owens, and Yixuan Li. Energy-based out-of-distribution detection. In NeurIPS , 2020.

Xixi Liu and Zach Christopher. TAG: Text prompt augmentation for zero-shot out-of-distribution detection. In ECCV , 2024.

Xixi Liu, Yaroslava Lochman, and Zach Christopher. Gen: Pushing the limits of softmax-based out-of-distribution detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2023.

Yifei Ming, Ziyang Cai, Jiuxiang Gu, Yiyou Sun, Wei Li, and Yixuan Li. Delving into out-of-distribution detection with vision-language representations. In NeurIPS , 2022.

Atsuyuki Miyai, Qing Yu, Go Irie, and Kiyoharu Aizawa. Locoop: Few-shot out-of-distribution detection via prompt learning. In NeurIPS , 2023.

Atsuyuki Miyai, Qing Yu, Go Irie, and Kiyoharu Aizawa. Gl-mcm: Global and local maximum concept matching for zero-shot out-of-distribution detection. International Journal of Computer Vision , pages 1-11, 2025.

Yuval Netzer, Tao Wang, Adam Coates, Alessandro Bissacco, Baolin Wu, Andrew Y Ng, et al. Reading digits in natural images with unsupervised feature learning. In NeurIPS Workshop , 2011.

Jun Nie, Yonggang Zhang, Zhen Fang, Tongliang Liu, Bo Han, and Xinmei Tian. Out-ofdistribution detection with negative prompts. In ICLR , 2023.

Karol J. Piczak. ESC: Dataset for Environmental Sound Classification. In ACM Multimedia , 2015.

Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In ICML , 2021.

Sudarshan Regmi. Going beyond conventional ood detection. arXiv preprint arXiv:2411.10794 , 2024.

Tal Ridnik, Emanuel Ben-Baruch, Asaf Noy, and Lihi Zelnik-Manor. Imagenet-21k pretraining for the masses. In NeurIPS , 2021.

Dan Roth and Kevin Small. Margin-based active learning for structured output spaces. In Proceedings of the 17th European Conference on Machine Learning , 2006.

Burr Settles and Mark Craven. An analysis of active learning strategies for sequence labeling tasks. In EMNLP , 2008.

Yiyou Sun, Chuan Guo, and Yixuan Li. React: Out-of-distribution detection with rectified activations. In NeurIPS , 2021.

Yiyou Sun, Yifei Ming, Xiaojin Zhu, and Yixuan Li. Out-of-distribution detection with deep nearest neighbors. In ICML , 2022.

Antonio Torralba, Rob Fergus, and William T Freeman. 80 million tiny images: A large data set for nonparametric object and scene recognition. TPAMI , 2008.

Grant Van Horn, Oisin Mac Aodha, Yang Song, Yin Cui, Chen Sun, Alex Shepard, Hartwig Adam, Pietro Perona, and Serge Belongie. The inaturalist species classification and detection dataset. In CVPR , 2018.

Sagar Vaze, Kai Han, Andrea Vedaldi, and Andrew Zisserman. Open-set recognition: a good closed-set classifier is all you need? In ICLR , 2022.

Haohan Wang, Songwei Ge, Zachary Lipton, and Eric P Xing. Learning robust global representations by penalizing local predictive power. In NeurIPS , 2019.

Haoqi Wang, Zhizhong Li, Litong Feng, and Wayne Zhang. Vim: Out-of-distribution with virtual-logit matching. In CVPR , 2022.

Hualiang Wang, Yi Li, Huifeng Yao, and X. Li. Clipn for zero-shot ood detection: Teaching CLIP to say no. In ICCV , 2023.

Hongxin Wei, Renchunzi Xie, Hao Cheng, Lei Feng, Bo An, and Yixuan Li. Mitigating neural network overconfidence with logit normalization. In ICML , 2022.

Jingkang Yang, Haoqi Wang, Litong Feng, Xiaopeng Yan, Huabin Zheng, Wayne Zhang, and Ziwei Liu. Semantically coherent out-of-distribution detection. In ICCV , 2021.

Jingkang Yang, Pengyun Wang, Dejian Zou, Zitang Zhou, Kunyuan Ding, Wenxuan Peng, Haoqi Wang, Guangyao Chen, Bo Li, Yiyou Sun, et al. Openood: Benchmarking generalized out-of-distribution detection. In NeurIPS , 2022.

William Yang, Byron Zhang, and Olga Russakovsky. Imagenet-ood: Deciphering modern out-of-distribution detection algorithms. In ICLR , 2024.

Qing Yu and Kiyoharu Aizawa. Unsupervised out-of-distribution detection by maximum classifier discrepancy. In ICCV , 2019.

Jingyang Zhang, Nathan Inkawhich, Randolph Linderman, Yiran Chen, and Hai Li. Mixture outlier exposure: Towards out-of-distribution detection in fine-grained environments. In WACV , 2023.

Jinsong Zhang, Qiang Fu, Xu Chen, Lun Du, Zelin Li, Gang Wang, Shi Han, Dongmei Zhang, et al. Out-of-distribution detection based on in-distribution data patterns memorization with modern hopfield energy. In ICLR , 2023.

Zihan Zhang and Xiang Xiang. Decoupling maxlogit for out-of-distribution detection. In CVPR , pages 3388-3397, 2023.

Zihan Zhang and Xiang Xiang. Decoupling maxlogit for out-of-distribution detection. In CVPR , 2023.

Bolei Zhou, Agata Lapedriza, Aditya Khosla, Aude Oliva, and Antonio Torralba. Places: A 10 million image database for scene recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence , 2017.

Kaiyang Zhou, Jingkang Yang, Chen Change Loy, and Ziwei Liu. Learning to prompt for vision-language models. International Journal of Computer Vision , 130:2337-2348, 2022.

NeurIPS Paper Checklist

1. Claims

Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?

Answer: [Yes]

Justification: A summary of the paper's contributions is provided at conclusion.

Guidelines:

The answer NA means that the abstract and introduction do not include the claims made in the paper.

The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.

The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.

It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.

2. Limitations

Question: Does the paper discuss the limitations of the work performed by the authors?

Answer: [Yes]

Justification: See Limitations in Section 6.

Guidelines:

The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.

The authors are encouraged to create a separate "Limitations" section in their paper.

The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.

The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.

The authors should reflect on the factors that influence the performance of the approach. For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.

The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.

If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.

While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.

3. Theory assumptions and proofs

Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?

Answer: [Yes]

Justification: The proof is provided in Appendix A.

Guidelines:

The answer NA means that the paper does not include theoretical results.

All the theorems, formulas, and proofs in the paper should be numbered and crossreferenced.

All assumptions should be clearly stated or referenced in the statement of any theorems.

The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.

Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.

Theorems and Lemmas that the proof relies upon should be properly referenced.

4. Experimental result reproducibility

Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?

Answer: [Yes]

Justification: See Section 5.1, 5.2, and supplemental material for implementation details.

Guidelines:

The answer NA means that the paper does not include experiments.

If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.

If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.

Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.

While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example

(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.

(b) If the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.

(c) If the contribution is a new model (e.g., a large language model), then there should either be a way to access this model for reproducing the results or a way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).

(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.

5. Open access to data and code

Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?

Answer: [Yes]

Justification: See https://github.com/GIT-LJc/LogitGap .

Guidelines:

The answer NA means that paper does not include experiments requiring code.

Please see the NeurIPS code and data submission guidelines ( https://nips.cc/ public/guides/CodeSubmissionPolicy ) for more details.

While we encourage the release of code and data, we understand that this might not be possible, so 'No' is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).

The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines ( https: //nips.cc/public/guides/CodeSubmissionPolicy ) for more details.

The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.

The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only a subset of experiments are reproducible, they should state which ones are omitted from the script and why.

At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).

Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.

6. Experimental setting/details

Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?

Answer: [Yes]

Justification: See Section 5.1, 5.2, 5.3 and supplemental material for implementation details.

Guidelines:

The answer NA means that the paper does not include experiments.

The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.

The full details can be provided either with the code, in appendix, or as supplemental material.

7. Experiment statistical significance

Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?

Answer: [NA]

Justification: Not Applicable.

Guidelines:

The answer NA means that the paper does not include experiments.

The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).

The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)

The assumptions made should be given (e.g., Normally distributed errors).

It should be clear whether the error bar is the standard deviation or the standard error of the mean.

It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified.

For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).

If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.

8. Experiments compute resources

Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?

Answer: [Yes]

Guidelines:

The answer NA means that the paper does not include experiments.

The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.

The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.

The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).

9. Code of ethics

Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines ?

Answer: [Yes]

Justification: The research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics.

Guidelines:

The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.

If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.

The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction).

10. Broader impacts

Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?

Answer: [Yes]

Justification: See Section 6.

Guidelines:

The answer NA means that there is no societal impact of the work performed.

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.

Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.

The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.

The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.

If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).

11. Safeguards

Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?

Answer: [NA]

Justification: Not Applicable.

Guidelines:

The answer NA means that the paper poses no such risks.

Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.

Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.

We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.

12. Licenses for existing assets

Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?

Answer: [Yes]

Justification: See References.

Guidelines:

The answer NA means that the paper does not use existing assets.

The authors should cite the original paper that produced the code package or dataset.

The authors should state which version of the asset is used and, if possible, include a URL.

The name of the license (e.g., CC-BY 4.0) should be included for each asset.

For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset.

For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.

If this information is not available online, the authors are encouraged to reach out to the asset's creators.

13. New assets

Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?

Answer: [NA]

Justification: Not Applicable.

Guidelines:

The answer NA means that the paper does not release new assets.

Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.

The paper should discuss whether and how consent was obtained from people whose asset is used.

At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.

14. Crowdsourcing and research with human subjects

Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?

Answer: [NA]

Justification: Not Applicable.

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper.

According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.

15. Institutional review board (IRB) approvals or equivalent for research with human subjects

Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?

Answer: [NA]

Justification: Not Applicable.

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.

For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.

16. Declaration of LLM usage

Question: Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.

Answer: [No]

Justification: We only use LLM for writing.

Guidelines:

The answer NA means that the core method development in this research does not involve LLMs as any important, original, or non-standard components.

Please refer to our LLM policy ( https://neurips.cc/Conferences/2025/LLM ) for what should or should not be described.

A Theoretical Analysis: LogitsGap vs Softmax Scaling

In this section, we provide the proof of Theorem 4.1 from Section 4, which offers a detailed analysis of the relationship between LogitGap and MCM [33]. We first introduce the necessary notation and definitions.

Notations. Formally, in the out-of-distribution problem, the decision function D : X → { ID, OOD } is typically constructed as:

where S : X → R is a scoring function based on pre-trained model f , which assigns a scalar confidence score to each input sample. The decision threshold λ establishes the decision boundary: samples satisfying S ( x ) < λ are identified as OOD, while those with S ( x ) ≥ λ are considered ID.

We present the OOD scoring functions for LogitGap and MCM. Given a logit vector z predicted for a test sample x , we sort its elements in descending order to obtain z ′ . LogitGap exploits the inherent distributional differences between ID and OOD logits 4 :

where z ′ j denotes the j -th largest logit.

MCM [33] applies softmax normalization to the logits and takes the maximum probability as the OOD score:

where τ is a temperature scaling parameter.

Furthermore, we give the OOD decision function based on the corresponding score:

Remarks. By convention, λ LogitGap and λ MCM are typically chosen such that the true positive rate is at 95%. For brevity, we will use λ L to denote λ LogitGap throughout the rest of this paper.

Theorem A.1. Given a K -way classification task, let the predicted logit vector for a sample x be z = [ z 1 , z 2 , . . . , z K ] . Let z ′ = [ z ′ 1 , z ′ 2 , . . . , z ′ K ] denote the sorted logits in descending order, such that z ′ 1 = max k z k . Then, for the temperature scaling parameter τ used in the softmax function, if τ > 2( K -1) , we have

where FPR LogitGap ( λ L ) is the false positive rate based on LogitGap with threshold λ L . Similarly, FPR MCM ( τ, λ MCM ) is the false positive rate based on MCM with temperature τ and threshold λ MCM .

Proof. Let Q x denotes the out-of-distribution P x | OOD . By definition, we express the false positive rate of MCM, denoted as FPR MCM ( τ, λ MCM ) , in the following:

Next, we introduce a intermediate OOD score function, LogitGap with softmax (LogitGap_softmax) to bridge the MCM and our LogitGap. We define the score function of LogitGap_softmax as S LogitGap _ softmax ( x ; f ) = ∑ K i =1 [ e z ′ 1 /τ -e z ′ i /τ ] K ∑ K j =1 e z ′ j /τ . Therefore, the false positive rate of LogitGap_softmax can be expressed as

Combining Eq.(16) and Rq.(17), we have λ MCM = λ LM + 1 K . Similarly, we can express the false positive rate of LogitGap as

Since the outputs of CLIP are bounded within [ -1 , 1] 5 , it follows that

Furthermore, we have

λ λ

Next, for λ LM and λ L , we consider two cases: (1) λ LM ≤ L τ ; (2) λ LM > L τ . For the case (1), with the assumption λ LM ≤ λ L , we have

τ

By comparing Eq.(16) and Eq.(19), we can derive that FPR LogitGap ( λ L ) ≤ FPR MCM ( τ, λ MCM ) , if the following condition holds:

Note that, λ LM = λ MCM -1 K . Combining this equation with Eq.(20), we have

For the case (2), by directly comparing Eq.(16) and Eq.(18), we can derive that FPR LogitGap ( λ L ) ≤ FPR MCM ( τ, λ MCM ) , if the following condition holds:

In the case (2), we have λ LM > λ L τ . With this inequation, Eq.(22) can be rewitten as

Note that Eq.(23) is the same as Eq.(21), thus we can derive the same conclusion.

B Analysis: Logits Distribution Differences Between ID and OOD Samples

B.1 Motivation Example

To illustrate a key limitation of the softmax-based scoring function MCM [33], we present two examples where logit distributions differ significantly yet yield nearly identical MCM scores . This reveals how the softmax operation can suppress informative structural differences in the logit space, which are often critical for distinguishing ID and OOD samples. Consider a 3-way classification problem with two test samples, x 1 and x 2 , whose corresponding logit vectors are z 1 = [0 . 5596 , -0 . 9808 , -0 . 9808] and z 2 = [0 . 9783 , -0 . 6311 , -0 . 4976] , respectively. Despite the significant difference in magnitude and sharpness ( z 2 exhibits a much more confident prediction), the maximum softmax probability for both is identical: softmax ( z 1 ) = [ 0 . 70 , 0 . 15 , 0 . 15] and softmax ( z 2 ) = [ 0 . 70 , 0 . 14 , 0 . 16] .

This arises because softmax normalizes logits into a probability distribution, preserving their relative ordering while discarding absolute magnitudes. As a result, distinct logit patterns can be mapped to probability vectors with similar maximum values, thereby limiting the discriminative power of softmax-based OOD detection. This issue is further exacerbated by temperature scaling τ > 1 , which flattens the softmax distribution and amplifies information loss. This observation naturally raise a key question: How can we more effectively leverage the discriminative information embedded in non-maximum logits to enhance ID-OOD separability?

B.2 Motivation Evidence

We observe an interesting phenomenon: the relationship between the maximum logit and the remaining logits differs notably between in-distribution (ID) and out-of-distribution (OOD) samples. As shown in Figure 5, ID samples tend to have a larger maximum logit value than OOD samples. Conversely, for the non-maximum logits, OOD samples typically exhibit higher values than ID samples.

To verify the generality of this phenomenon, we conduct experiments across various ID/OOD datasets and model architectures in Table 7. Specifically, we report the average maximum logit, and the average logit of non-predicted classes, using ResNet-50, ViT-B/16, and ViT-L/14 as backbone models.

B.3 Theoretical Analysis

Our theoretical motivation stems from the observed distinction in logit patterns between ID and OOD samples: (i) Higher maximum logit values for ID samples; (ii) Higher non-predicted class logits in OOD samples. This indicates that the gap between the maximum logit and the remaining logits is generally wider for ID samples than for OOD samples.

To understand the rationale behind this, we provide a simple analysis from the theoretical perspective.

Theorem B.1. In a binary classification problem, given a well-trained feature extractor ϕ and a classifier W , we assume that the feature distribution of ID samples for class i follows a Gaussian distribution, ϕ ( x ID | y = 0) ∼ N ( µ 0 , Σ 0 ) and ϕ ( x ID | y = 1) ∼ N ( µ 1 , Σ 1 ) . We further assume that the features of OOD samples can be modeled as an interpolation of two ID feature distribution with a Gaussian noise, i.e. ϕ ( x OOD ) = α · N ( µ 1 , Σ 1 ) + (1 -α ) · N ( µ 0 , Σ 0 ) + β · N (0 , Σ) . Let z ID = [ z ID 0 , z ID 1 ] denotes the predicted logit vector for ID sample x ID , z OOD = [ z OOD 0 , z OOD 1 ] denotes the predicted logit vector for OOD sample x OOD . If the ID sample x ID belongs to class i , we have

Proof. If the ID sample x ID belongs to class i , we have

0

20

40

60

80

100

Index of topN Logits

7.5

10.0

12.5

15.0

17.5

20.0

22.5

25.0

27.5

Logits

ID

OOD

(a) Logit distribution on ResNet-50 using CIFAR-10 as ID dataset.

(c) Logit distribution on CLIP ViT-L/14 using ImageNet as ID dataset.

Similarly, we have

Therefore, we have

Assuming the features of x OOD can be denoted as the interpolation of two ID feature distribution with a random Gaussian noise, i . e . ϕ ( x OOD ) = α · N ( µ 1 , Σ 1 ) + (1 -α ) · N ( µ 0 , Σ 0 ) + β · N (0 , Σ) , we have

We assume that the classifier W is well trained, thus w T 1 -i µ i < w T 1 -i µ 1 -i . Then, we have

C Experimental Details

C.1 Implementation Details

We run all OOD detection experiments on NVIDIA GeForce RTX-4090Ti GPUs with Pytorch 2.3.1. For CLIP-based OOD detection, we adopt the pre-trained ViT-B/16 [10] model from CLIP [39]. For conventional OOD detection task using CIFAR-10 [24] as ID dataset, we use a ResNet-50 [13] model pre-trained on the CIFAR-10 training set as the backbone.

C.2 Datasets

ImageNet-10, ImageNet-20, ImageNet100 [33] creates ImageNet-10 that mimics the class distribution of CIFAR-10 [24] but with high-resolution images. For semantically hard OOD evaluation with realistic datasets, [33] curates ImageNet-20, which consists of 20 classes semantically similar to ImageNet-10 (e.g., dog (ID) vs. wolf (OOD)). ImageNet-100 is created by randomly sample 100 classes from ImageNet-1k [7].

NINCO The NINCO [2] main dataset comprises 64 OOD classes with a total of 5,879 samples. These classes were carefully selected to ensure no categorical overlap with any of the ImageNet-1K [7] classes. Additionally, each sample was manually inspected by the authors to confirm the absence of ID objects, making NINCO a reliable benchmark for evaluating out-of-distribution detection on ImageNet-1K.

ImageNetOOD ImageNet-OOD [55] is a clean, manually curated, and diverse dataset containing 31,807 images from 637 classes, designed for evaluating semantic shift detection using ImageNet-1K [7] as the in-distribution (ID) dataset. To minimize covariate shifts, images are sourced directly from ImageNet-21K [41], with human verification ensuring the removal of any ID contamination from ImageNet-1K. The dataset also addresses multiple sources of semantic ambiguity caused by inaccurate hierarchical relationships in ImageNet labels and eliminates visually ambiguous images stemming from inconsistencies in ImageNet's data curation process.

ImageNet-O ImageNet-O is a dataset containing image concepts absent from ImageNet-1K [7], specifically designed to evaluate the robustness of ImageNet models. These out-of-distribution (OOD) images consistently cause models to misclassify them as high-confidence in-distribution examples. As the first anomaly and OOD dataset tailored for testing ImageNet-1K models, ImageNet-O provides a valuable benchmark for assessing OOD detection performance under label distribution shifts.

ImageNet-A ImageNet-A [19] contains images sampled from a distribution distinct from the standard ImageNet-1k [7] training distribution. Although its examples belong to existing ImageNet1k classes, they are deliberately more challenging, frequently leading to misclassifications across a range of models. ImageNet-A enables we to test image classification performance when the input data has covariate distribution shifts.

ImageNet-R ImageNet-R (Rendition) [16] consists of artistic and non-photographic renditions of ImageNet-1k [7] classes, including art, cartoons, graffiti, embroidery, graphics, origami, paintings, patterns, plastic figures, plush toys, sculptures, sketches, tattoos, video game assets, and more. The dataset covers 200 ImageNet-1k classes with a total of 30,000 images, offering a diverse benchmark for evaluating model robustness to distribution shifts in visual style and appearance.

ImageNet-Sketch The ImageNet-Sketch [49] dataset contains 50,889 images, with approximately 50 images per class for all 1,000 ImageNet-1k [7] categories. It was constructed by querying Google Images using the phrase 'sketch of [class name],' restricted to a black-and-white color scheme. An initial set of 100 images was collected for each class, followed by manual filtering to remove irrelevant images and those belonging to visually similar but incorrect classes. For categories with fewer than 50 valid images after cleaning, data augmentation techniques such as flipping and rotation were applied to balance the dataset.

C.3 Logits Selection Methods

Logits Selection with a Fixed Hyperparameter N The effectiveness of our proposed LogitGap method relies on the selection of an informative region within the logits space, which is governed by a hyperparameter N . To determine an appropriate value for N , we consider the number of classes in the dataset. For datasets with a large number of classes (e.g., ImageNet and ImageNet-100), we set N to 20% of the total number of classes K . In contrast, for datasets with fewer classes (e.g., ImageNet-10 and ImageNet-20), we set N to 50% of K . This strategy is motivated by the following intuition: when K is large, using a lower proportion (e.g., 20%) helps reduce the influence of noise; whereas when K is small, a higher proportion (e.g., 50%) preserves more useful information. Moreover, our empirical results demonstrate a consistent performance improvement when N is set within the 20%-50% range of the total number of classes across different datasets. The specific N values chosen for different datasets are summarized in Table 8.

Logits Selection with an Adaptive Hyperparameter N To enhance LogitGap, we introduce an improved strategy to adaptively choice the value of N . To this end, we firstly construct a small validation set randomly sampled from the in-distribution (ID) data, with a fixed size of 100 samples. Using the model's feature extractor, we obtain the image features of these ID samples. We then perform random inter-class interpolation on these features to generate synthetic OOD samples. To further increase the diversity of these synthetic samples, Gaussian noise is added to the interpolated features. The interpolation process is formally defined as follows:

where x i and x j represent two samples from the ID validation set, α denotes the mixing coefficient between samples, while β controls the weight of the added noise.

In our experiments, we generate a set of synthetic OOD samples equal in size to the ID validation set to serve as the OOD validation set. For datasets with a large number of categories, such as ImageNet [7] and ImageNet-100 [33], we set the interpolation parameters to α = 0 . 3 and β = 0 . 8 . For smaller-scale datasets like ImageNet-10 [33] and ImageNet-20 [33], we set α = 0 . 3 and β = 0 . 0 . We then compute the logits scores for both the ID and synthetic OOD validation samples and determine the optimal value of N adaptively based on the following criterion:

where ¯ z ′ N represents the mean of the logits ranked from second to the N -th largest.

C.4 Implementation Details on LogitGap Combined with Negative-Prompt-Based OOD Method

Negative-Prompt-Based OOD detection methods jointly train both positive and negative prompts to separately capture the characteristics of ID and OOD samples. In these methods, the model's predicted logits take the form z = [ z ID 1 , z ID 2 , ..., z ID K , z OOD 1 , z OOD 2 , ..., z OOD M ] , where z ID i denotes the logit for the i -th ID class, and z OOD j denotes the logit for the j -th OOD class. Since negative prompts tend to increase the logits of OOD samples, directly applying LogitGap over the entire logits space is not appropriate. In our experiments, when combining ID-Like [1] with LogitGap, we rely solely on the ID logits, z ID = [ z ID 1 , z ID 2 , ..., z ID K ] , to compute the LogitGap score.

D Experimental Results

D.1 Results on Traditional OOD Detection with CIFAR100 as ID Dataset

To further evaluate the effectiveness and generality of our LogitGap, we conduct experiments in traditional OOD detection settings with CIFAR100 [24] as ID dataset. Following the OpenOOD [54] protocol, we adopt the standard OpenOOD benchmark splits for OOD evaluation. The OOD benchmarks include both near-OOD datasets : CIFAR-10 [24], Tiny ImageNet [46], and far-OOD datasets : MNIST [8], SVHN [36], Textures [5], and Places365 [61]. In this setup, we employ a ResNet-50 [13] backbone trained from scratch on ID data using cross-entropy loss. We compare LogitGap against a comprehensive suite of baselines: (i) Logit-based post-hoc methods: MSP [17], MaxLogit [14], KL-Matching [15], ODIN [27], IODIN [40]; (ii) Internal network statistics-based methods: Mahananobis [25], ASH [9], SHE (Hopfield energy) [58]; (iii) Training-based methods with auxiliary data: Outlier Exposure (OE) [18], MixOE [57]. The results are summarized in Table 9.

As shown in Table 9, our proposed LogitGap demonstrates superior performance across all evaluated benchmarks. We observe that: (1) Among logit-based methods, IODIN [40] achieves the best performance. ODIN [27] enhances OOD detection by introducing input perturbations to amplify the difference between ID and OOD samples in the softmax output. Building upon this, IODIN proposes to mask low-magnitude regions and perturb only invariant features, effectively encouraging the model to ignore environmental factors and focus on more informative regions. This requires calculating an additional invariant feature mask on top of the gradient computation in ODIN. In contrast, LogitGap improves detection performance by applying a simple transformation in the output space, reducing the FPR95 by 1 . 98% and 0 . 78% compared to MSP [17] and IODIN, respectively. (2) Internal network statistics-based methods generally perform worse because they heavily rely on the representation quality and discriminative ability of the internal network. When the backbone lacks sufficient expressive power, the extracted feature distributions of ID and OOD samples tend to overlap, making it difficult to effectively distinguish between them. (3) When combined with methods that leverage extra out-of-distribution data, such as OE [18] and MixOE [57], LogitGap further enhances performance over their respective baselines. For example, a straightforward substitution of the OOD score function with LogitGap leads to a 3.57% and 1.23% performance gain in FPR95 and AUROC respectively over the original OE [18] method. These experiments highlight the advantages of LogitGap in terms of ease of integration and wide applicability across different settings.

D.2 Results on Traditional OOD Detection with ImageNet as ID Dataset

In the traditional OOD detection setting, we further evaluate the performance of LogitGap using a ResNet-50 [13] backbone on the large-scale ImageNet [7] dataset. Following the OpenOOD [54] protocol, we adopt SSB-hard [48] and NINCO [2] as near-OOD datasets, iNaturalist[47], Textures[5], and OpenImage-O [50] as far-OOD datasets. In this setup, the ResNet-50 model [13] is trained from scratch on the ID data using cross-entropy loss. We compare LogitGap against several representative

post-hoc OOD detection methods, including MSP [17], MaxLogit [14], KL-Matching [15], ODIN [27], IODIN [40], Energy [30], and GradNorm [21].

As summarized in Table 10, LogitGap achieves comparable or superior performance across all benchmarks and obtains the best overall average performance, demonstrating its robustness and effectiveness in the traditional OOD detection scenario.

D.3 Results on Far-OOD Evaluation with ImageNet as ID Dataset in Zero-Shot OOD Detection

In the main paper, to isolate the effect of semantic shift, we construct a challenging OOD detection benchmark specifically designed for semantic shift evaluation. This benchmark is built using ImageNet [7] as ID dataset, while NINCO [2], ImageNet-O [19], and ImageNet-OOD [55] as OOD datasets, enabling a more accurate assessment of the model's ability to detect semantic-level distribution changes.

Moreover, following the OpenOOD [54] evaluation protocol, we assess the performance of OOD detection methods in the far-OOD scenario under a zero-shot setting. In the far-OOD scenario, ImageNet is used as the ID dataset, while iNaturalist [47], OpenImage-O [50], and Textures [5] serve as the OOD datasets. As shown in Table 11, LogitGap maintains robust performance under the broader distributional shift.

D.4 Results on More Architectures with ImageNet as ID Dataset in Zero-Shot OOD Detection

We extend our experiments to other CLIP variants with ImageNet as ID dataset, including ResNet-50, ResNet-101, ViT-B/32 and ViT-L/14 in zero-shot OOD detection. As shown in Table 12, LogitGap consistently outperforms baselines across all backbones. For instance, on ViT-L/14, LogitGap reduces FPR95 by 6 . 03% and improves AUROC by 2 . 65% compared to MCM. This demonstrate that LogitGap generalizes well across architectures.

(a) Results on ResNet-50.

(d) Results on ViT-L/14.

E More Analysis about LogitGap

E.1 Generalization Beyond Visual Tasks

Although LogitGap is primarily designed for visual classification, one of the core settings for OOD detection, we further verify its generalization to non-visual tasks. Specifically, we conduct an evaluation on the ESC-50 [38] audio classification dataset, where 50 categories are randomly divided into 25 ID and 25 OOD classes. The method is implemented with the CLAP [12] model, following the same hyperparameter selection strategy used in the visual domain, with N = 20% of the class count ( N = 5 ).

As shown in Table 13, LogitGap maintains strong performance on this audio task, indicating that it does not rely on modality-specific architectures or features. This confirms its broad applicability across domains. In addition, we introduce MCM_topN, a comparative baseline that applies the top-N strategy to MCM [33]. For fair comparison, we use the same N for MCM_topN and LogitGap. Results show that applying the top-N strategy to MCM does not consistently improve performance, which indicates LogitGap's advantage goes beyond top-N filtering.

E.2 Relationship Between LogitGap and Other Logit-Pattern-Based Methods

We review related works in both active learning and OOD detection, and summarize several representative methods in Table 14. Here, z c denotes the logit and p c represents the predicted probability for c -th class. Specifically, we compare LogitGap with three classical active learning approaches: LC [6], Margin [42], and Entropy [43]; and four representative OOD detection methods: MSP [17], MaxLogit[14], DML [60], and GEN [32].

Among these methods, Margin Sampling is the most closely related to ours, as it also measures a margin between model outputs. However, two key distinctions set LogitGap apart: (1) Representation Level: Margin Sampling operates on predicted probabilities, whereas LogitGap computes directly on raw logits, avoiding softmax-induced normalization effects. (2) Scope of Comparison: Margin Sampling considers only the top-2 predictions, while LogitGap generalizes this to the top-N logits, enabling a more holistic uncertainty estimation.

To emphasize the importance of these distinctions, we adapt Margin Sampling for OOD detection under zero-shot setting using CLIP ViT-B/16, with ImageNet as the ID dataset in a semantic shift scenario. As shown in Table 15, LogitGap consistently outperforms Margin Sampling across all benchmarks, demonstrating the effectiveness and robustness of our formulation.

E.3 The Effectiveness of Synthetic OOD Data

As described in Section C.3, we propose an OOD data synthesis strategy to adaptively select the hyperparameter N . Since the synthesized OOD data are derived from in-distribution (ID) information, they may not fully capture the characteristics of real-world OOD data. Nevertheless, our empirical findings indicate that such synthetic samples are sufficiently informative for selecting a robust hyperparameter N . As shown in Table 16, the optimal N determined using synthetic OOD samples ( i . e ., N syn) is highly consistent with the one obtained from multiple real OOD datasets ( i . e ., N real ), achieving comparable detection performance. These results validate the practicality of the N -selection strategy, even in the absence of real OOD data.

E.4 The Impact of Hyperparameter N on LogitGap Performance

In Table 17, we provide a more ablation study on hyparameter N . Specifically, we conduct experiments under the zero-shot OOD detection setting, using either ImageNet-100 or ImageNet-1K as ID dataset. For simplicity, we report the FPR95 of our LogitGap method based on CLIP ViT-B/16 model. We can observe that: (1) Optimal N varies by dataset (e.g., 19 for ImageNet-100, 195 for ImageNet-1K); (2) Setting N to 20% of total classes consistently provides strong performance. Therefore, we adopt this value as default in LogitGap.

(a) ImageNet as ID dataset.

(b) ImageNet-100 as ID dataset.

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper addresses the under-utilization of the logit space in post-hoc OOD detection by proposing the LogitGap scoring function and its focused subset variant (LogitGap-topN). The main idea is to explicitly measure the average gap between the maximum logit and the other logits, and to improve separability between ID and OOD samples by retaining only the top-N logits to filter out noisy tail values.\n\nSTRENGTHS AND WEAKNESSES:\n**strengths**\n\n- The writing is very clear, articulating the motivation and method clearly\n- The method is simple and easy to use\n\n**weaknesses**\n\n- The biggest issue may be that the method itself is very trivial and lacks novelty. Research around logit magnitudes can be traced back to the last century, and the concept of logit gap proposed in this paper has similar concepts that were introduced as early as 2001 (Margin Sampling) [1], and has been discussed in many subsequent works. Therefore, the method and insights proposed in this paper are not novel.\n- The comparisons in the method section are not comprehensive enough. The zero-shot comparison targets in Tables 1 and 2 are all works from three years ago; while the effects when combined with other methods in Table 4 are often negative. In Table 5, the paper did not include several top-ranking works from the OpenOOD leaderboard for comparison, only comparing against methods with poorer performance.\n- The ablation study on hyperparameter N with only one figure (Fig 3) is insufficient. It would be better to provide more comprehensive data. As stated in the paper, N has a significant impact on logitgap performance, and it clearly needs different tuning for different datasets and models. Although the authors provide a synthetic OOD data scheme, whether synthetic OOD data aligns with real OOD distributions, and the impact of the alignment degree on final performance, need further discussion.\n\n**minor weaknesses**\n\n- The title of Table 2 in the supplementary material is incorrect\n\n[1] Tong, Simon, and Daphne Koller. \"Support vector machine active learning with applications to text classification.\" *Journal of machine learning research* 2.Nov (2001): 45-66.\n\nQUESTIONS:\nHow well do synthetic OOD samples align with real OOD distributions, and how does this affect performance?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 4, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-06-23"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis work proposes a post-hoc OOD score based on the gap between the maximum logit and the remaining logits. Specifically, they focus on the Top-$N$ logits,  the threshold $N$ is computed using only a small number of ID samples.  If ID samples are not available, $N$ is set to be 20% of the total number of classes $K$.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n- The idea is simple and effective compared to Maxi-Logit.\n- It demonstrates the performance of LogitGap on both semantic-shifted OOD detection and covariate-shifted OOD detection.\n- Experiments on semantic-shift OOD detection are conducted under both zero-shot and few-shot settings. \n\nWeaknesses:\n- The introduction section is poorly written. \n    - Line 27-18, I would say energy score is the most representative OOD score because of its simplicity and effectiveness compared to MSP score. \n    -  Line 32-33, it is true that MCM/MSP lacks information from other logits.  However, it's abrupt that the authors simply ignore the energy score (using full logit information) and directly raise the question of utilizing information from non-maximum logits. Could the authors clarify their reasoning for overlooking the energy score and elaborate on their motivation at this point?\n    - Line 38-40, although the authors make an effort to demonstrate the effectiveness of LogiGap in Figure 1, the caption does not provide any information about the datasets or models used. Additionally, I believe the authors should include energy score for comparison in Figure 1.\n\n\n- The experiments are not sufficient. \n    -  For zero-shot OOD detection setting, two baselines are missing including GL-MCM [1] and TAG [2]. Meanwhile, the performed   OOD benchmark is not standard. Specifically, the evaluation of far-OOD detection is missing.  See https://zjysteven.github.io/OpenOOD/ for complete evaluation. \n    -  For traditional OOD detection, two baselines are missing in the experiments section including ViM (using information from logit and feature) and GEN [3] (using information from probability space). Meanwhile, is there any reason that the authors switch from the large OOD benchmark (ImageNet-1k as ID dataset) to small OOD benchmark (CIFAR-10 as ID dataset)? Additionally, the results across different models such as ViT and BiT are also expected to present.\n    - All results are only demonstrated in CLIP ViT-B-/16, which fails to show the generalization of the proposed method across different models. If the authors focus on the vision-language models,  it is expected to evaluate on the other ViT models including ViT-B/32 and ViT-L/14 and ResNet-based models including ResNet-50 and ResNet-101. \n    - To support the claim in Line 159 -161, a more comprehensive empirical evaluation across different ID and OOD datasets with different models should be done.\n \n\nMinor weaknesses:\n- What datasets and models are utilized to produce Figure 2? \n- Line 50-51, the concept of focusing on the top-N logits is also mentioned in GEN [1], which demonstrates that using the energy score with top 10% of logit values improves OOD performance.  Again, the author probably could try to compare the performance of LogitGap with the energy score using top 10% logit information. \n- Based on the results in Table 3, specifically the FPR95 and AUROC metrics, it does not appear that OOD detection on ImageNet-10 and ImageNet-20 is particularly challenging, as the FPR95 values are close to 0 and the AUROC values are near 100.\n\n \n[1] GL-MCM: Global and Local Maximum Concept Matching for Zero-Shot Out-of-Distribution Detection, 2023.\n\n[2] TAG: Text Prompt Augmentation for Zero-Shot Out-of-Distribution Detection, 2024.\n\n[3] GEN: Pushing the Limits of Softmax-Based Out-of-Distribution Detection, 2023.\n\nQUESTIONS:\nSee weaknesses.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 5, "CLARITY": 2, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-06-30"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes LogitGap, a novel scoring function for out-of-distribution (OOD) detection. LogitGap is designed to address the limitations of existing methods such as MaxLogit and Maximum Concept Matching (MCM). Specifically, while MaxLogit considers only the highest class logit and ignores the rest of the distribution, MCM relies on the softmax function, which normalizes the logit values and may obscure informative differences. LogitGap is formulated as the gap between the top logit and the rest of the distribution, thereby combining the advantages of both MaxLogit (absolute logit norm) and MCM (distributional awareness). The paper provides both theoretical insights and extensive empirical results to demonstrate the effectiveness of LogitGap.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n\n1. The proposed method is conceptually simple and easy to understand, making it broadly applicable across various domains.\n\n2. The authors support their approach with both comprehensive theoretical analysis and extensive experimental validation, which strengthens the credibility of the method.\n\n3. The paper is well-organized and clearly written, facilitating accessibility and readability for a wide range of readers.\n\nWeaknesses:\n\n1. The theoretical motivation of the proposed method relies on certain assumptions that are not fully verified or supported. If these assumptions are trivial or well-established, proper references should be provided to increase the logical soundness. For instance, Figure 2 lacks critical context — it is unclear under what setting it was generated, whether it represents a single data point or an average over multiple samples.\n\n2. The applicability of the method is demonstrated solely on vision classification tasks, which limits the scope of the contribution. Given the simplicity and generality of the proposed formulation, it would be more compelling to show its effectiveness across a broader set of domains or tasks.\n\n3. The theoretical explanation is overly reliant on the supplementary material. While it is acceptable to place extended proofs in the appendix, the main text should still provide enough intuition and explanation for readers to understand the formulation without constantly referring to the supplement. As it stands, readers may find it difficult to follow the logic behind the proposed metric.\n\nQUESTIONS:\n1. Why this specific LogitGap formulation?\nThe motivation to combine the strengths of MaxLogit and MCM is clearly explained, and the experimental results support the effectiveness of LogitGap. However, the specific choice of computing the difference between the maximum logit and the average of the remaining logits requires further justification. Why is this particular subtraction-based formulation considered the most effective? For example, were alternative designs such as weighting the MCM score by the max logit or using other aggregation strategies considered? Providing experimental comparisons or ablation study results on these alternatives would help strengthen the justification for the robustness or optimality of the proposed formulation.\n\n2. Sensitivity to N (number of logits averaged):\nThe sensitivity of LogitGap to hyperparameter N is a significant concern for practical applications. While N averages non-maximum logits, low-ranked classes likely have consistently low values, regardless of whether they are in-distribution (ID) or out-of-distribution (OOD). This could reduce the average of the subtraction term, making ID/OOD distinction difficult. This challenge also affects MCM. We anticipate MCM would perform better if evaluated with an N value derived similarly to LogitGap's N. A comparison under such conditions would ensure a fair evaluation. We recommend conducting this experiment and sharing the results.\n\nLIMITATIONS:\n1. **Motivation Example in Section 4.1 may be unintuitive**:\n   In the provided example, 𝑧₂ shows high logits for two classes and is described as likely being an in-distribution sample, while 𝑧₁ lacks a dominant logit. While this may be intuitive from an OOD detection perspective, it contradicts typical classification expectations, where such ambiguity often indicates a confusing or hard-to-classify input. This could lead to confusion for readers. If a better illustrative example is not available, the current description should be revised for clarity. For instance, instead of:\n   \"Despite the significant difference in magnitude and sharpness...\"\n   a more intuitive phrasing might be:\n   \"Since OOD samples tend not to show confident alignment with any known class, a sample like 𝑧₂, which activates multiple classes relatively strongly, may indicate a higher likelihood of being in-distribution.\"\n\n2. **Excessive reliance on supplementary material for theoretical explanation**:\n   As previously mentioned, key derivations—especially Theorem 4.1—are presented only in the supplementary material. This is problematic given the central importance of this result. The main text does not provide sufficient intuition or explanation about why the theorem holds or how it contributes to the final formulation. Without such context, readers may struggle to assess the theoretical soundness of the method. Including a brief, self-contained summary or intuition for the theorem in the main paper would significantly improve clarity.\n\n3. **Limited generalization beyond vision classification tasks**:\n   Although the method is theoretically general, its evaluation is confined solely to vision classification benchmarks. Given the simplicity of the formulation and its potential for cross-domain applicability, including at least one experimental setting beyond vision classification would strengthen the paper. This would help demonstrate the broader applicability and robustness of the method across different domains.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nLogitGap is a simple post-training method for differentiating between in-distribution (ID) and out-of-distribution (OOD) samples during inference. LogitGap uses a scoring function that explicitly measures the average gap between a sample’s largest logit and its remaining logits, exploiting the observation that ID samples have a much larger 'logit gap' than OOD samples. Using the logits from all K classes in the scoring function can increase noise in the score as many classes that are semantically far from the sample have negligible logit values. Hence, the authors compute the gap only over the top-N logits. N is chosen automatically from a handful of ID samples to maximize ID/OOD separation. A theoretical analysis links LogitGap to softmax–based scores such as MCM and proves it yields lower false-positive rates when the softmax temperature is high, while extensive experiments confirm the practical gains. On zero-shot CLIP ViT-B/16, LogitGap cuts FPR95 by up to 5.8% versus the previous best logit method, and when paired with few-shot prompt-tuning or training-time OOD schemes it delivers consistent extra improvements across semantic-shift and covariate-shift benchmarks.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths\n\n- The scoring function is simple to compute given the logits and does not require additional training.\n- LogitGap scoring function uses information from all logits as opposed to prior methods that may use only the maximum value.\n- The false-positive rate has a minimum when N lies between 20% and 50% of classes, thereby simplifying selection of N for image classification tasks.\n\nWeaknesses\n\n- We must perform the computation for all DNN layers in order to obtain the logits to determine whether a sample is ID or OOD. \n- Determining the correct N is not entirely free as LogitGap requires access to a small ID validation set, which may be unrealistic in zero-sample deployment scenarios\n- Relies on synthetic OOD samples generated by noise/interpolation, whose representativeness for real-world OOD samples is unclear and not justified.\n\nQUESTIONS:\n- Can LogitGap be applied to features at an earlier layer in the DNN in order to detect an OOD sample wtihout performing the full inference computation? This would further reduce the overhead of OOD inference.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 4, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-14"}