Long-Tailed Out-of-Distribution Detection via Normalized Outlier Distribution Adaptation
Authors: Wenjun Miao
Guansong Pang
Jin Zheng
Xiao Bai
Venue: NeurIPS 2024 poster (2024)
Keywords: out-of-distribution detection
Primary area: safety_in_machine_learning

Abstract: One key challenge in Out-of-Distribution (OOD) detection is the absence of ground-truth OOD samples during training. One principled approach to address this issue is to use samples from external datasets as outliers ($\textit{i.e.}$, pseudo OOD samples) to train OOD detectors.
  However, we find empirically that the outlier samples often present a distribution shift compared to the true OOD samples, especially in Long-Tailed Recognition (LTR) scenarios, where ID classes are heavily imbalanced, $\textit{i.e.}$, the true OOD samples exhibit very different probability distribution to the head and tailed ID classes from the outliers.
  In this work, we propose a novel approach, namely $\textit{normalized outlier distribution adaptation}$ (AdaptOD), to tackle this distribution shift problem.
  One of its key components is $\textit{dynamic outlier distribution adaptation}$ that effectively adapts a vanilla outlier distribution based on the outlier samples to the true OOD distribution by utilizing the OOD knowledge in the predicted OOD samples during inference.
  Further, to obtain a more reliable set of predicted OOD samples on long-tailed ID data, a novel $\textit{dual-normalized energy loss}$ is introduced in AdaptOD, which leverages class- and sample-wise normalized energy to enforce a more balanced prediction energy on imbalanced ID samples. This helps avoid bias toward the head samples and learn a substantially better vanilla outlier distribution than existing energy losses during training. It also eliminates the need of manually tuning the sensitive margin hyperparameters in energy losses.
  Empirical results on three popular benchmarks for OOD detection in LTR show the superior performance of AdaptOD over state-of-the-art methods.
Code is available at https://github.com/mala-lab/AdaptOD.

Long-Tailed Out-of-Distribution Detection via Normalized Outlier Distribution Adaptation
Wenjun Miao Beihang University, China miaowenj junbuua.. edu. en
Guansong Pang Singapore Management University, Singapore gspangOsmu. edu.
Jin Zheng*i Beihang University, China edu.. edu. cn
Xiao Bai*
Beihang University, China baixiao@buaa edu. cn
j jinzheng@buaa.
Abstract
One key challenge in Out-of-Distribution (OOD) detection is the absence of ground- truth OOD samples during training. One principled approach address this issue is to use samples from external datasets as outliers (i.e., pscudo OOD samples) to train OOD detectors. However, we find empirically that the outlier samples often present a distribution shift compared to the true OOD samples, especially in Long- Tailed Recognition (LTR) scenarios, where ID classes are heavily imbalanced, i.e. the true OOD samples exhibit very different probability distribution to the head and tailed ID classes from the outliers. In this work, we propose a novel approach, namely normalized ouilier distribution adaptation (AdaptOD), to tackle this distribution shift problem. One of its key components dynamic outlier distribution adaptation that effectively adapts vanilla outlier distribution based on the outlier samples the true OOD distribution by utilizing the OOD knowledge in the predicted OOD samples during inference. Further, to obtain more reliable set of predicted OOD samples on long-tailed data, a novel dual-normalized energy loss is introduced in AdaptOD, which leverages class- and sample- wise normalized energy to enforce more balanced prediction energy on imbalanced ID samples. This helps avoid bias toward the head samples and learn substantially better vanilla outlier distribution than existing energy losses during training. It also eliminates the need of manually tuning the sensitive margin hyperparameters in energy losses. Empirical results on three popular benchmarks for OOD detection in LTR show the superior performance of AdaptOI) over state-of-the-art methods. Code is available at https://github. wttptttuuuboooooooooommeooeeee hta///mllllaaa labaall-ab/AdapOOO
1 Introduction
Deep neural networks (DNNs) are widely known to be overconfident about what they do not know when applying them real-world scenarios in open environments [15,42], such autonomous driving [18] and medical diagnosis [21]. Consequently, the high-contien nigh-confidence predictions can misclassify out-of-disttibuttin (OOD) samples from unknown classes as one of the known or in-distribution (ID) classes [24,48]. This issue is further amplified when ID samples exhibit a class-imil insalaiebde distribution in Long-Tailes Recognition (LTR) scenarios. This because head samples often receive similarly high-confident prediction as OOD samples, while the tail samples receive substantially
"Beihang University and Beihang Jiangxi Research Institute Cooresponndig authors: G. Pang and J. Zheng
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

LSUN
CRAARD
Class-wiis Enargy
B '8 80
Adaytes outlees
Outfiers
Class-wice Energy
Outllers
6002
6.0c0
Enegyde
Class-wise
Thature
SVNN
3
20.005
0.02
0005
V
60
26
Head <t- Class ->>
Tail
Head <c.. Class Tall
Head < Class -> Tail
Head <-. Class Tbil
(a)
(b)
(c)
(d)
Figure I: (a) Mean energy distribution on six OOD datasets with CIFARIO0-LT [3] as ID data. (b)
The results of the energy distribution of the OOD dataset SVHN [3]] using our proposed dynamic outlier distribution udaptation (DODA) and an existing energy loss EnergyOE [24], where the outlier
data is taken from ImnImaaeS80M [38]. (c) The results of using both of our proposed DODA and
dual-normalized energy loss (DNE). (d) The ratio of the energy of each ID) class to the aggregated energy of all ID classes.
lower-confident prediction, leading to an indistinguisaability between OOD and head samples and the tendency of wrongly detecting tail samples OOD samples [30, 40,.99. We address the problem of long-tailed OOD detection, aiming at ensuring LTR accuracy while rejecting unknown samples.
One notorious challenge in OOD detection is the lack of ground-truth information OOD samples, as they can be drawn from any unknown distribution. One popular solution to tackle this challenge is to use samples from external datasets outliers (i.e., samples that do not overlap with ID and OOD samples, also known as pseudo OOD samples) to train OOD detectors [4, 12,124,, 2,33.. This approach can be implemented by fitting the prediction probability of the outlier data prior distribution over the ID classes [17] or a margin-based global energy function [4]. Despite showing good performance on various benchmarks, all of these methods assume that the distribution of the outliers is well aligned with that of the true OOD samples in the target data. However, the outliers often present distribution shift compared the true OOD samples, especially in LTR scenarios [49.56] 56), i.e., the true OOD samples exhibit very different probability distribution to the head and tailed ID classes from the outliers. Due to the bias toward head classes, the distribution shift particularly severe w.r.t. the head samples. For example, as shown in Fig. 1a, the energy distribution of six popular OOD datasets differs significantly from each other, where CIFARIO0-LT [3] is used as the ID dataset. This implies that any of these OOD datasets used as outlier data source can largely mismatch the distribution of the true OOD data if the other five datasets are used as the true OOD data. Such distribution shift can largely mislead the training of detection models, leading to downgraded detection performance.
To tackle this problem, in this work, we propose a novel approach for OOD detection in LTR, namely Normalized Outlier Distribution Adaptation (A (AdaptOD). Dynamic Outlier Distribution Adaptation (DODA) a key component of AdaptOD. Given a vanilla outlier distribution, DODA performs test- time adaptation (TTA) to dynamically adapt the outlier distribution to the true OOD distribution by utilizing the OOD knowledge embedded in the predicted OOD samples. This reduces the distribution gap between the outlier and the OOD distributions, enabling a more accurate estimation of OOD scores. As illustrated in Fig. 1b, large gap exists between the energy distribution of the outlier data (Tinylmages80M [38]) and the true OOD data (SVHN [31]). By contrast, our adapted outlier distribution is better aligned to the OOD distribution. Importantly, the ground truth of the test data is assumed to be unavailable during TTA, and as we will show in the experiments (see Table 5), DODA based on the predicted OOD samples can well approximate the upper-bound performance obtained when DODA can get access to the ground truth of the test data to perform TTA (i.e., an oracle model). There have been a few TTA methods for OOD detection, but they require online model retraining [49] or feature memory augmentation [56]. By contrast, DODA focuses on the calibration of the outlier distribution, effectively eliminating the retraining or memory overheads.
On the other hand, training OOD detectors using energy loss functions [4,24] is a principled approach to learn the vanilla outlier distribution. However, existing energy losses can underestimate the tail class distribution and involve sensitive hyperparameters on energy margins. As result. the vanilla outlier distribution learned by using these losses often misclassifies tail samples as OOD samples during TTA. This can significantly affect the distribution adaptation in DODA, leading to still& a relatively large gap between the adapted outlier distribution and the OOD distribution, as shown in Fig. lb. Therefore, AdaptOD introduces a novel Oual-Normalized Energy loss (DNE) to

---

balance energy prediction for imbalanced ID samples and learn a better vanilla outlier distribution for subsequent DODA. Unlike existing energy losses that are focused on sample-wise energy estimation. DNE utilizes both class-wise and sample-wise normalized energy. This helps obtain more balanced prediction energy on the head and tail samples, transferring the energy from the head samples to the tail samples, thereby avoiding the bias toward the head classes (see Fig. ld). In doing so, DNE is also free of energy margin hyperparameters and enables the learning of a better vanilla outlier distribution. This guarantees better starting point for the outlier distribution adaptation and the accuracy of the predicted OOD samples at testing time in DODA, and thus yielding substantially better aligned outlier distribution (see Fig. Ic vs. Fig. lb). Our main contributions are follows:
We propose the novel approach AdaptOD for OOD detection in LTR. To our best knowledge, it is the first approach for adapting the outlier distribution to the true OOD distribution from both the training and inference stages.
In AdaptOD, we introduce two new components, DODA and DNE, to reduce the gap between the learned outlier distribution and the true OOD distribution in the presence of long-tailed ID data. DODA builds upon a vanilla outlier distribution and then dynamically adapts this distribution to the true OOD distribution with the OOD knowledge obtained at testing time. DNE is designed to perform class- and sample-wise normalized energy training, which enforces more balanced prediction energy for imbalanced ID samples, enabling the learning of largely enhanced vanilla outlier distribution for more effective DODA.
Extensive empirical results on three LTR benchmarks CIFARIO-LT, CIFAR I00-LT, and ImageNet-LT using six popular OOD datasets demonstrate that AdaptOD substantially outperforms the state-of-the-ar (SOTA) OOD detection methods in various LTR scenarios.
2
Related Work
OOD Detection in Long- Recognition (LTR). In recent years, OOD detection and LTR have been extensively developed. The former determines whether a given input sample belongs to known classes (in-distribution) or unknown classes -futrof-distribution)) [22, 23, 25, 35, 37, 43, 43,44,52 $2,55,,57 while the latter expects to train on class- imbalanced datasets [1, 9, 13, 33, 34, 36]. PASCL [40] reveals the difficulty of the OOD detection problem in LTR, and establishes performance benchmarks for OOD detection in LTR based on the SC-OOD benchmark [48]. This setting is also extended to medical image analysis (28,50), which utilizes a strong data augmentation to discriminate II) data and OOD data. Recent studies [4, 17] find that fitting the prediction probability of outlier data to a long-tailed distribution is more effective than using a uniform distribution. They specify this distribution based the number of samples in ID classes or pre-trained ID model to learn this outlier distribution. However, it is difficult to obtain such an accurate distribution for outliers in LTR. Other studies [30,45] attempt to learn an extra outlier class to overcome the need for learning lailed distributions of outliers. But they need a more complex model design. More importantly, all these methods assume that the outlier samples can well represent the distribution of the true OOD data, but this often does not hold in practice since OOD data can be sampled from highly different unknown distributions in different application scenarios. Our approach tackles this problem by adapting the outlier distribution to that of the true OOD data.
Test-Time Adaptation (TTA) for OOD Detection. Recently, TTA [8, 14,53] has been introduced to OOD detection, in which unlabeled test data that can be seen only once are used to perform online updating pre-trained DNNs for enhancing task performance and quickly adapting real- scenarios. There are two primary approaches for TTA in other tasks: retraining the model based on unsupervised objectives 59,41,54 and updating the feature memory for each class (16.47,55) However, unlike these TTA methods that generalize training data to test data and maintain the same label space between them, TTA for OOD detection [7,49, 9,56] addresses the challenge of identifying unknown classes in test data. While training data includes ID data and outlier data, test data comprises not only ID data but also true OOD data consisting of unknown classes that do not overlap with the ID and outlier data. In particular, AUTO [49] is recent method that attempts to assign pseudo labels to unlabeled test data, and then directly uses these pseudo-labels and test data to retrain the model online through Outlier Exposure [12]. AdaOOD [56] utilizes a memory bank to store feature memories of ID data, then updates these memories online during inference, and lastly employs a k-Ned distance method to detect OOD samples. However, they fail to work well in the LTR scenarios due
3

---

Unlabeled Test Data Stream
Adapted Outlier
G"(r)
Training
Distribution
a Calibrate
Class
Cless
Closs
Norman malizationn
Energy
Single Test Image
Head
Tail
Updale
Online
G (8)
samples
Balloon
Cat
Suftmu
Egergy
Energs g
OOD Filter
Outlier
Outlier
Batth Eaver
Extraction
(()
OOD
Legit Ourput
M4
(a) R Predicted OOD sample
Outlier samples
Energy
Dynamic Outlier Distribution Adaptation (DODA) Dual-Normalized Energy Loss (DNE)
Sample-wise Loss
Overview of AdaptOD, which consists of two components, DODA (Left) and DNE (Right). Left: Each test sample is assigned a global energy-based OOD score G(x) to adapt the outlier distribution pout, DODA then uses the adapted outlier distribution pout to calibrate the global energy score Gzx). obtaining the calibrated global energy score G (ar) as the OOD score. Right: For each iteration, DNE first applies Batch Energy Normalization on logit output to obtain the normalized energy, and then utilizes this energy to optimize a dual energy loss function at both the class and sample levels.
Figure 2:
to the large variation the heavily imbalanced training data. Moreover, these methods require retraining or additional memory overheads. ETLT [7] attempts to calibrate OOD scores by linear regression of its input feature but requires a batch-wise inference to obtain sufficient test samples for the regression. DODA instead utilizes the dynamically adapted outlier distribution to calibrate the prediction output of test data during inference without any retraining or memory overhead.
3 Approach
Preliminaries. Let Xón denote the input space of the ID data and Yin = set of & imbalanced ID classes the label space. We have genuine OOD data Xtrue_out that is different from X It is normally assumed that genuine OOD data X trreeouu are not available during training since OOD samples are unknown instances. However, we can obtain auxiliary OOD data from external datasets, which can be used as outliers X Yaur_out to act surrogate OOD data for traiiing/fin-ttningg LTR models. That is, Xauæ_out is still different from Xtrae_out, but both of them are OOD w.r.t. X There is class overlapping among ID data X genuine OOD data X Xtrue_out and outlier data X _aut_oott Then the training and test sets can be respectively denoted as: Xtrain = X U X yaur_ott and Xeet = X U X trree_out
k} be the
OOD detection in LTR is to learn classifier with training data Xtrain so that for any test data E pestt if z is drawn from X (from either head or tail classes), then f can classify x into the correct ID class, whereas if is drawn from Xtrue_out, then ff can detect xr as OOD data.
TTA for OOD detection in LTR is to online update the above pre-trained classifier f with test data Plest during the inference stage, in which for any unlabeled single test sample x E Xtest utilizing pre-trained classifier f to predict whether x belongs to ID or OOD data at the current iteration, then using the predicted label and the test sample updated classifier f is used to identify a new test sample and continuously update the classifier f. Notably, each sample can only be seen by f once during inference.
update the classifier f. At the next iteration, the
3.1 Overview of AdaptOD
The proposed AdaptOD approach designed tackle the aforementioned distribution shift issue for OOD detection in LTR. As shown in Fig. 2, AdaptOD consists of two components, namely Dynamic Outlier Distribution Adaptation (DODA) and Dual-Normalized Energy Loss (DNE). DODA dynamically adapts the learned outlier distribution to the true OOD distribution during inference to reduce the distribution gap between them. DNE designed to perform both class-wise and sample-wise normalized energy training to obtain more balanced prediction energy on imbalanced ID samples, thereby yielding an enhanced vanilla outlier distribution and enabling better distribution adaptation in DODA. Below we introduce each component in detail.

---

3.2 DODA: Dynamic Outlier Distribution Adaptation
Previous OOD detection methods in LTT suffer from a distribution shift between outlier data and true OOD data. This issue can largely limit the performance of these OOD detectors. Therefore, we propose dynamically adapt the outlier distribution the true OOD distribution and further use it to calibrate the prediction output of test samples at the inference stage.
Dynamic Distribution Adaptation with Predicted OOD Samples. Recently, energy- -aargy- meth- ods [4, 24], which use a global energy score over the ID classes as an OOD score for each test sample, have achieved SOTA performance for OOD detection in LTR. Motivated by this success, we learn and adapt the vanilla outlier distribution pout, which is initialized by the global energy from the LTR model predictions on the outlier data, that of the true OOD data, and then use poottosse calibbrated global energy score as the OOD score. Specifically, given a set of l ID classes, for any test sample Prrest, its global energy score G(.) is defined as [24]:
G(x) Σ c06,)
(1)
where fz(x) is the logit output of sample in class j.j 21.2...., k3. Let pout R be an initial outlier distribution. DODA performs test-time adaptation to dynamically adapt the outlier distribution pout to the true OOD distribution based on the OOD knowledge from the samples predicted OOD during inference. To this end, we designed an OOD filter using training data to identify OOD samples. Since is easy to obtain the distribution of global energy score for training ID samples, we use an offline method to determine threshold for filtering OOD samples based on this energy distribution. This avoids adverse effects on the adaptation speed during inference. Formally, given ID examples from training data x {01, ..,,.., Zn ). where x E X and is the number of training ID samples, we estimate the mean Min and standard deviation din of the global energy distribution by:
G(x:)
(G(x:) pan)
pin
Oin
n
n
We then utilize Z-score-based method to implement the OOD filter, with the Z-score defined as:
=
Pin. Oini
where a is a hyperparameter. a is used by default during the inference stage, and this setting works well throughout our experiments. More discussion about a is described in Appendix D.3.3,
To adapt the outlier distribution pout, DODA utilizes the predicted OOD samples by the OOD filter to perform a momentum update of pout during inference, so that Pout will represent the mean of energy distribution for the predicted OOD samples. The entries in the vanilla outlier distribution pout are initialized from the mean energy distribution of the outlier data, and they are updated in an online fashion. Specifically, when the OOD filter detects the t-th test sample x as OOD sample (i.e,, its global energy G(x) < R), DODA performs an update of pout as follows:
G(x) R
pou' (t 1)
M41
(pot
(t),
G(r) 2 R
where DODA only keep the number of predicted OOD samples M and current pout during inference.
Calibrated OOD Score based the Adapted Outlier Distribution. After obtaining the adapted
pout, we use it to calibrate the global energy score G(-) and define the OOD score as follows:
ef(()
GP (x) Σ 1+ Poat'
(5)
where & E xlest and (.) denotes the calibrated global energy score with the adapted outlier distribution. This way helps reduce the energy proportion of the head classes that true OOD distribution leans toward in the original global energy score G(-). In doing so, the distribution gap between the outliers and true OOD is effectively reduced in final OOD score GP()). enabling more
5

---

accurate estimation of OOD scores in heavily imbalanced ID data without incurring any retraining cost or additional memory expense.
3.3 DNE: Dual-Normalized Energy Loss
When using existing global energy loss to obtain the vanilla outlier distribution pauu, the distribution of tail samples is indistinguishable from that of OOD samples due to the underestimating of tail samples. We are also required to manually tune the sensitive hyperparameters on energy margins under complex class imbalance. These can lead an inaccurate OOD filter used in Eq. 3 and subsequently affect the distribution adaptation in DODA. To tackle these issues, we propose a Dual- Dual-Normalized Energy Loss (DNE), which consists of two novel components, namely dlass-wise normalized energy loss (DNE-C) and sample wise normalized energy loss (DNE-S). DNE-C is class-wise training loss for balancing the sum of energy on all ID samples for each ID class, whereas DNE-S is a sample-wise training loss for balancing the sum of energy on all ID classes for each ID sample. DNE learns balanced prediction energy distribution on imbalanced ID samples, which helps further reduce the bias toward the head classes in Eq. 5, thereby improving vanilla outlier distribution for a better OOD filter in Eq. 3 and a better vanilla outlier distribution pour in Eq. 4. It also provides stable energy margins, eliminating the need of manual tuning of these margins.
Batch Energy Normalization. To this end, we first propose novel batch energy normalization method, which conducts energy normalization on the logit output of each class for a batch of training samples. In doing so, the energy of each sample is dependent on the energy of other ID samples and OOD samples relative to the same class. This helps transfer the energy knowledge from the head samples to the tail samples, enabling a better estimation for the energy distribution of tail samples.
Formally, let x E X be one training batch of ID data, with x = {xft", ,,="",,,, ,..., ..f.) and 3im be its batch size, and out E X Xaturout be a set of outlier data in a training batch, with x tuqut .rgut ..... &gour) } whose set size is bout, then the batch energy normalization (2, for a sample
rr E x ou in class j E 21,2, k} with classifier ʃ is defined as: efg(a.)
Fz( = edi(at") +... + (
(6)
+ (fs(()))) + + e (
where [x)) is the logit output of sample x in class j. Essentially, we use the logit output of all samples in training batch on class to normalize the energy prediction of sample x. This largely reduces the energy prediction bias toward the head samples. The energy of the outlier data is included as calibration modulation. Then, those normalized energy scores are used for the dual-normalized energy losses, DNE-C and DNE-S, to better balance the prediction energy of long-tailed ID samples.
Additionally, compared to the current energy based method [4, 24] for OOD detection in LTR that requires manually designed energy margin hyperparameters, batch energy normalization adjusts the energy of the batch samples on each class to the same scale, so it can provide stable energy margins for the balanced training without relying on the training dataset and/or the class imbalance factor, without the need of manually tuning them.
Class-wise Normalized Energy Loss (DNE-C). DNE-C independently regularizes the energy for each class to enhance the normalized energy of ID samples for more class-wise balanced energy. Formally, let Din, = (Xin, yin) and Dont X Xaur_out then we can independently minimize the class energy on each class as follows:
Lo ( m. C,(x))) + Ex [(ma Cx(x) )D,, (7)
where m. =1 1 and m = 0 are the default margin hyperparameter settings without the need of manual tuning on different datasets (see Appendix C for more details). The class-wise normalized energy C,(x).j E 21,2... ..., k} is defined as:
Cx(x) Σ (xi))
(8)
==1

---

Table 1: Comparison of AdaptOD with EnergyOE and COCL on six OOD datasets.
OOD
ID Dataset:
CIFARIO-LT
ID Dataset:
CIFARTO0-1
Dataset
Method
AUC
AP-imt
AP-outt
FPR.
AUC1
AP-in
AP-out
FPR.
EnergyOE [24]
95.53
97.42
92.93
18.44
79.56
86.03
70.88
79.45
Texture [5]
COCL [30]
96.81
98.21
93.86
14.65
81.99
88.05
74.38
59.79
(OursOD(Ours)
98.22
98.81
94.91
11.60
83.88
89.43
76.47
58.47
EnergyOE [24]
96.63
92.33
98.46
14.37
86.19
81.42
91.74
34.36
SVHN [31]
COCL [30]
96.98
93.25
98.61
12.59
89.20
81.57
94.21
54,46
OOuss
98.13
94.34
99.11
10,33
93.09
91.32
96,86
17.63
EnergyOF [24]
84.44
85.74
84.63
61.73
61.15
67.12
56.66
91.42
CIFAR [19]
COCL [30]
86.63
86.66
86.28
52.21
62.05
66.14
56.82
93.88
(dursOD(Ours)
89.05
89.93
88.22
45.51
72.77
76.37
70.58
86.04
EnergyOE [24]
88.40
91.65
84,95
46.23
70,78
79.40
55,90
90.74
TIN [20]
COCL [30]
90.43
92.52
87.03
46.12
71.87
81.89
57.12
83.93
AdursOD(Ours))
91.40
93.85
88.18
42.77
72.87
82.06
58.92
88.24
EnergyOE [24]
94.00
94.78
93.70
28,42
81.61
86.57
69.16
80.57
LSUN [51]
COCL [30]
94.85
95.43
93.98
27.48
84.10
89.89
69.80
74.67
(Ours)
96.16
96.84
95.86
24.12
85,70
90.55
72.70
70.20
EnergyOF: [24]
92.51
84.26
97.14
33.63
79.12
63.38
89.09
81.43
Place365 [58]
COCL [30]
93.97
87.36
97.56
32.25
80.30
68.65
89.16
77.83
AdartOD(Ours)
95.19
89.56
98.44
29.22
83.27
68.82
91.44
71.63
where b is the batch size of the batch (if x is x that is h and is xou that b is Bou)), and I; is the i-th sample in the batch x. Notably, even if some classes do not have the corresponding ID samples in certain training batch, this loss also work well. This is because there is less distribution shift among classes in the ID data compared to the OOD data. Therefore, the output of ID samples on incorrect ID classes should also be higher than the OOD samples. DNE-C balances the sum of energy on all ID samples for each ID class and distinguishes outlier samples from ID samples, especially for the underestimated tail classes in class-wise manner.
Sample-wise Normalized Energy Loss (DNE-S). DNE-S independently regularizes the energy for each sample to enhance the energy of ID samples for sample-wise balanced energy. Formally, we minimize the global energy all classes of each sample as follows:
Ls ((,,))D,, [ma m S (zz))) + EL-D...
[mar S(x) (
m 1),,))
where i Ux xout, mf, = प. and m are the default margin hyperparameter settings that can also work stably regardless of the ID/OOD datasets (see Appendix C). Then the sample-wise normalized energy S(x) can be defined as:
S(a) Σ (x))
(10)
After doing this, we can regularize the global energy of the ID data, particularly the low global energy for tail samples. DNE-S efficiently balances the energy between head and tail samples. As result, the combination of DNE-C and DNE-S can learn substantially more balanced prediction energy of ID samples, facilitating DODA solve the distribution shift problems.
Overall Training Objective. Overall, we utilize the cross-entropy loss, together with our two normalized energy losses, to train our model. The final objective of our DNE training is as follows:
Ltotal Br, [+([())))) Lanes
(11)
where l is a cross-entropy loss, along with the two normalized energy losses:
(m Lane =ss + Lc = B ((max(00 min - S(x))) Ez~D.n. (maan(0, S(x) - m)
(12)
Σ (Ex. (maz m,, C,(x))) E [(maz(0, C,(x) m)
where Le is as defined in Fq. and Ls as defined in Fq. The algorithm of AdaptOD described in Appendix B.
7

---

Table 2: Comparison to different long- OOD detection methods.
ID Dataset:
CIFARIO-LT
ID Dataset:
CIFARI00-L1
Method
AUC1
AP-int
AP-out
EPR]
ACC1
AUC1
AP-in
AP-outt
IPR.
ACC1
OE [12]
89.76
89,45
87.22
53.19
73.59
73.52
75.06
67.27
86,30
39.42
EnergyOF [24)
91.92
91.03
91.97
33.80
74.57
76.40
77.32
72.24
76.33
41.32
PASCL [40]
90.99
90.56
89.24
42,90
77.08
73.32
74.84
67.18
79.38
43.10
EAT [45]
92.87
91.76
92.40
32.42
81.31
75.45
76.02
70.87
77.83
46.23
Class Prior [17]
92.08
91.17
90.86
34.42
74.33
76.03
77.31
72.26
76.43
40.77
BERI, [4]
92.56
91.41
91.94
32.83
81.37
77.75
78.61
73.10
74.86
45.88
COCL [30]
93.28
92.24
92.89
30.88
81.56
78.25
79.37
73.58
74.09
46.41
OE (1]+=DODA(Our))
91.62
90.55
89.39
49.02
73.59
75.46
77.14
69.88
83.67
39.42
EnergyOE (24]+DODA((Urs)
93.36
92.17
92.97
30.82
74.57
79.40
80.89
76.54
72.63
41.32
BERI. (4)+DODA(Our))
93.77
92.62
93.15
29.41
81.37
79.45
81.15
75.52
70.51
45.88
COCL )
93,89
93.06
93.39
29.25
81.56
79.81
81.26
75.93
70.14
46.41
AduptOD(OOrr)
94.69
93.89
94.12
27.26
82.27
81.93
83.09
77.83
67.37
47.91
Table 3: Comparison to different TTA-based OOD detection methods,
Training
TTA
ID Dataset:
CIFARIO-LT
ID Dutuset: CIFARIO0-LT
Method
Method
AUCT
AP-int
AP-nult
HPR.
AUCT
AP-in
AP-out
HPR]
w/o TTA
89.76
89,45
87.22amm
53.19
73.52
75.06
67.27
86.30sas
AUTO [49]
90.49 e22s
89.83
87.45
S2.63.a4
73.93
75.98
67.74.00
a5.7
OE [12]
AdaODD [56]
90.89.aax
90.17
87.88
a1.44_a*
24467
76.53
67.89
85.34
DODA(Ours)
91.62-+2>
90.55
89.39
49.02-a4
75.46-a37
77.14 17.14
69.88-ex0
83,67
w/o TTA
91.922eee
91.03
91.97
33.80
77.32sm. 76.40=a1
72.24
76.33
AUTO [49]
92.48=1a3
91,43aass
92.44
31.99
77.65
78. 1
74, 74
74.66
EnergyOE [24]
AdaODD [56]
91.63aa
91.73
77.67 _2.83
24.05
74.86
(Our()urs))
93.36
97.17
92.97
30.82=a51
79.40
80.8 30.89
76.54.00
72.632044
w/o TTA
92.56
:440
91.41
6.78 32.83 91.944.0a
77.75 0:17
78.61 ins:
73.10
74.86
AUTO [49]
92.41
91.73
92.42
=1.91
77.99 =2.9
78. 50
73.50=eam
s4.0
BERI [4]
AdaODD [56]
92.68
91.79
92. s2
31.41=a21
a8.26
78. s.
73.61
73.76
DODA(Ours)
93.77=a2
92. 62.62
93. 15
29.41=a9
79.45 -3.4
15
75.52=00
70.51
w/o TTA
93.28 8300
92.24
92, 1077
30.88
78.25
79.37 soos
73.58
74.09
COCL [30]
AUTO [49]
93.62
92.74
93. 10
30.41
78.85
79. .9.999mm
=4.01
72.75
AdaODD [56]
93
92.60anss
93.05
30.79
79.07
80.00
74.60
23,09
(Our))
93.89 ex
93.39 93.061a5
29.25 0.40
79.81 jass
81.26109
75.93 10722
14.14
w/o TTA
92.77=04
12.18
22.62
31.48=a..
77.92
78.97
73.92=021
74.44
DNE (Ours)
AUTO [49]
92.89=a4
92.69zass
92.
30.85 a00o
a9.
80.19
14.8
72.10
AdaODD [56]
93.39_a.
92.27
92.92:a0
30.78
80,26
81.72
75.62:200
71.962as5
(Our))
94.69=112
93,89
94.12
27.26
81.93-a31
83,09:a04
77.83=
67.37
4 Experiments
4.1 Experiment Settings
Datasets.
Following [30, 40, 45], we use three popular long-tailed datasets CIFARIO-LT [3].
CIFAR100-LT [3] and ImageNettLL [26] as ID data X The default imbalance ratio is set to P 100
on CIFAR10/100-11 TinyImageS80M [38] is used as the outlier data Xaaz_autt for CIFAR10//00-LT
and ImageNet-Fxtra [40] is used as outlier data for ImageNet-L.T. We use six datasets CIFAR [19],
Texture [5], SVHN [31], LSUN ], Places365 [58] and TinyImageNet [20), all of which are
introduced in the SC-OOD benchmark [48] the OOD test set for CIFAR10/100-1) and ImageNet-
lk-OOD [40] as the OOD test set for ImageNet-LT. More details about the datasets are presented in Appendix A.I.
Implementation Details. Our AdaptOD is compared with seven SOTA OOD detection methods on long-tailed data, including two popular methods: OE [12] and EnergyOE [24], and five recent
methods: PASCL [40], EAT [45], Class Prior [17], BERL [4], and COCL [30]. Further, we also
compare AdaptOD with two SOTA TTA methods for OOD detection, including AUTO [49] and
AdaOOD [56]. We use ResNet18 [11] as our backbone on CIFAR10/100-13 and ResNet50 [11] on ImageNet-LT. Following fine-tuning-bast based methods OE [12], EnergyOE [24], and BERL [4], our approach AdaptOD employs a similar training strategy to them that we obtain a pre-trained model with only ID data and fine-tune this model with both ID data and outlier data. The reported results are averaged over six independent runs. More details about the implementation details are presented in Appendix A.2.
8

---

Table 4: Comparison results on the large-scale ID dataset ImageNet-LT,
Training
AUCT
AP.in
AP-our]
FPR]
Method
ACC:
AP-ial
AP-out
FPR_
ACC1
wĀo FIA
71. 16
45, 37.97
85 13.03
76.98
EnergOOB [24]
OF [12]
68.13
69.43
45.12
43.87
82.54
84.75
90.98
76.89
44,00
44,42
BERL [4]
AdaODD [56]
(49)
71.80
71.06amm
46.47auo
46.58
86.05
85.56
77.36.1
26.09
FAT [45]
69.84
43.15
81.32
80.97
46.79
DODA(OurS)
73.12
13334
74.92
PASCL [40]
68.00
43.32
82.69
82.28
47.29
wão TTA
71.85a115
46.
26.2
75.60
75.60
Class Prior [17]
DERL [4]
71.16
70.43
45.26
45.97
84.82
85.63
76.98
77.63
50.42
46,83
COCL [30]
AdaODD [56]
AUTOO (49]
71.79amm
72.35
46.84as
47.
47.20
86.89
86.89 uss
15.06
COCL [30]
71.85
46.76
$6.21
75.60
51.11
(ODA(Ourr)
13.27
47.98.ue
87.77
74.71
BERL DODA
73.12
47.34
86.95
74,92
30.42
wlo TTA
72.04400
46.53aas
86. a6ai0
75.82
COCL [30]+DDDA
73.27
47.98
57.77
74.71
51.11
DNE (Ours)
AUTO [49]
73
47.26kr
87.1 1ar in
74.00
AdaptOOKOOurs)
74.32
49.02
88.63
72.91
31.67
AdaODD [56]
73.10
46.83
14.64
DODA(Ours)
74.32
49.02
88.63 LKD
12.91
Evaluation Measures. Following [30,48], we use the below common metrics for OOD detection and ID classification: (1) FPR is the false positive rate of OOD examples when the true positive rate of ID examples is at 95%, (2) AUC computes the area under the receiver operating characteristic curve of detecting OOD samples, (3) AP measures the area under the precision- recall curve, which can be either AP-in in which ID samples are treated as positive or AP-out in which OOD samples are regarded as positive, and (4) ACC calculates the classification accuracy of the long-tailed IDD data. The reported results are averaged over six independent runs with different random seeds by default.
4.2 Empirical Results
AdaptOD vs. Other OOD Detection Methods in LTR. Table I presents the comparison of our AdaptOD with two SOTA OOD detectors in LTR (EnergyOE [24], COCL [30]) on CIFARI0/100-LT using six OOD test datasets. These fine-grained results are not available for the other competing methods and thus they are omitted in this table. AdaptOD shows the best performance in all four metrics on each of the six OOD datasets. Table 2 shows the comparison of our AdaptOD with SOTA OOD detectors in LTR on CIFARI0/1000-E which is the average performance over six OOD test datasets. Following the previous methods [4.40,45], we report our accuracy with AdjLogit [29] for fair comparison. AdaptOD is also the best performers in the averaged results when comparing to all seven competing methods. This consistent improvement and SOTA performance of AdaptOI on both ID and OOD data indicate that the distribution gap between the outlier samples and the true OOD samples is effectively reduced by AdaptOD. Notably, the improvement is large on the near OOD dataset CIFAR [19], which cannot be achieved by previous SOTA methods [4, 30].
DODA as an Enabler to Existing Methods. Table also presents the results of our proposed component DODA in using as a plug-in to tackle the distribution shift problem in four SOTA methods (OE, EnergyOE, BERL, and COCL) on CIFAR 10/00L0/ It shows that DODA can consistently enhance the OOD detectors in all four metrics, demonstrating the strong capability of DODA in reducing the learned outlier distribution gap to the distribution of the true OOD data (see Appendix indicates
D for more details). The consistent improvement of having DODA as plug-and-plass the presence of the distribution shift problem encountered by existing SOTA detectors and the universal effectiveness of DODA in tackling the problem. Note that AdaptOD as a whole achieves consistent and substantial improvement over the four DODA-enabled models, showcasing that the other component of AdaptOI,, DNE, helps learn balanced IDD prediction energy and better align the adapted outlier distribution to the true OOD one.
AdaptOD vs. Other TTA Methods for OOD Detection. Table 3 shows the comparison of AdaptOD with two SOTA TTA methods AUTO and AdaODD for OOD detection on CIFAR10/100- LI. To have a straightforward and extensive comparison, we compare DODA with the two TTA methods, all of which are added on top of the same training method. In the experiments, we use five training methods, including four SOTA long-tailed OOD detection methods and our proposed DNE method. It is impressive that our DODA component consistently remains the best performer when the TTA methods are combined with all five different training methods on both ID datasets. DODA achieves better performance all four OOD detection metrics across five OOD training methods, indicating that DODA is a stronger and more generic TTA method for different OOD detectors. Moreover, the combination of our training method DNE and TTA method DODA, which is our approach AdaptOD as whole, achieves the best performance across all 20 possible combinations.

---

Table 5: Ablation study results CIFARIO-LT, CIFARIOO-LT and ImageNet-LT.
DODA DNE-C DNES
Dataset:
CIFARIO
ID Dataset:
CIFARIOD-LT
ID Dataset:
ImageNet-LL
ADCT
AP-InT
AP-ouuf
FPR.
AUCT
AF-in
A.-our
FPR.
AUICT
AP-mn
AP-ruff
FPR].
Baseline (Energy OB [24])
91.92
91.03
91.97
33.80
76.40
77.32
72.24
76.33
69.43
45.12
84.75
76.89
x
x
80.35
81.46
77.02
78.71
67.42
68.29
63.86
85.44
58.33
38.40
77.61
89.73
92.63
92.05
92.46
30.17
78.10
80.22
74.17
71.65
71.71
45.99
86.37
74.31
/
X
92.12
91.54
92.33
31.85
76.89
77.94
72.76
74.97
71.11
45.59
85.77
76.83
91.92
77.46
72.55
/
/
92.77
92.18
92.62
31.48
77.92
78.97
73.92
74.44
72.04
46.53
86,06
75.82
/
X
93.81
93.32
93.53
28.69
80:07
82.13
75.73
68.64
73.14
47.61
87.19
73.67
92.98
93.02
29.52
79.76
81.89
75.31
69.10 19
47.32
86.83
74,48
V
94.69
93.89
94,12
27.26
81.93
83.09
77.83
67.37
74.32
49.02
88.63
72.91
Oracle Modell
95.33
94.75
94.96
25.02
83.60
85.09
78.85
65.37
75.84
50.20
89.97
70.71
Performance on Large-scale ID Data. To demonstrate the scalability of our approach, we also perform experiments the large-scall ID dataset ImageNet-LT. The empirical results are presented in Table 4, which shows that our approach AdaptOD also achieves the SOTA performance in both the OOD detection performance and the ID classification aceuracy.
4.3 Further Analysis of AdaptOD
Ablation Study. The effectiveness of our two pro- posed components, DODA and DNE, have been justi- fied in Table 3. Here we provide more fine-grained analysis of DODA and its combination
83
two im- proved energy losses used in DNE, Le (Eq. 7, de- noted DNE-C) and L (Eq. 9, denoted as DNE-S), in Table 5, with EnergyOE [24] used as baseline. The results show the important contribution of each com- ponent to the overall superior performance of the full model AdaptOD. Further, we compare AdaptOD to an oracle model that utilizes the ground true OOD data to update the outlier distribution pout in DODA. It shows that AdaptOD has only small performance gap to the oracle model, indicating that AdaptOD well approximate the true OOD distribution by the predicted labels of the OOD samples, without involving any ground truth during TTA.
81
79
75
AUTO
Ada0OD
73
AdaptOD (Ours)
71
0%
20%
40%
60% 80% 100%
OOD Samples
Figure 3: The average performance over six OOD datasets on CIFAR100-LE with an in- creasing percentage of true OOD samples fed to TTA methods.
OOD Data Exploitation in TTA. To independently evaluate the effectiveness of exploiting OOD data to adapt the outlier distribution, we report the performance of three TTA methods with an increasing number of labeled OOD samples based on our DNE Fig. All three TTA methods achieve increasing performance for OOD detection in LTR with more and more true OOD data used for the adaptation. However, AUTO and AdaOOD struggle with the difference between training and testing ID data at the early stage of inference, while AdaptOD can utilize the adapted outlier distribution to quickly adapt to the true OOD distribution and achieve significantly improved performance.
Conclusion
To address the distribution shift problem in long-tailed OOD detection, we propose novel approach called AdaptOD. It utilizes a novel normalized energy-based loss dual-normalized energy loss (DNE) to learn balanced prediction energy on imbalanced ID samples and enhanced vanilla outlier distribution, then uses dynamic outlier distribution adaptation (DODA) adapt the outlier distribution to the true OOD distribution. DODA is shown to be a significantly improved TTA method than existing TTA methods for OOD detection. We also show that DNE can used to support DODA with its specially designed energy training for better test-time distribution adaptation. Experiments on three popular benchmarks demonstrated that AdaptOD significantly enhances the performance of both OOD detection and long-tailed classification.
10

---

Acknowledgments
The participation of W. Miao, Zheng, and X. Bai in this work was supported by National Natural Science Foundation of China (No. 62372029 and No. 62276016), while the participation of G. Pang was supported in part by Lee Kong Chian Fellowship.
References
[1] Alshammari, S., Wang, Y.X., Ramanan, D.. Kong, S.: Long-tailed recognition via weight
balancing. In: Proceedings of the IEEE/CVF Conference Computer Vision and Pattern
Recognition, pp. 6897-6907 (2022) 3, 15
[2] Bai, J., Liu, Z., Wang, H., Hao, J., Feng, Y., Chu, H., Hu, H.: On the effectiveness of out-of- distribution data in self-supervised long-tail learning. arXiv preprint arXiv:2306.04954 (2023)
3
[3] Cao, K., Wei, C.,, Gaidon, A., Arechiga, N., Ma, T.: Learning imbalanced datasets with label- distribution-aware margin loss. Advances in neural information processing systems 32 (2019)
8, 15
[4] Choi, H., Jeong, H., Choi, J.: Balanced energy regularization loss for out-of-distribution detection. In: Proceedings of the IEEE/CVE Conference Computer Vision and Pattern Recognition, pp. 15691-15700 (2023) 3, 5, 6, 8, 9, 15, 20
[5] Cimpoi, M.. Maji, S., Kokkinos, I., Mohamed, S., Vedaldi, A.: Describing textures in the wild. In: Proceedings the IEEE conference on computer vision and pattern recognition, pp. 3606-3613 (2014) 7, 8, 15, 18
[6] Deng, J., Dong, W., Socher, R., Li, L.J. Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. 2009 IEEE conference on computer vision and pattern recognition, pp.
248-255. leee (2009) 15
[7] Fan, K., Wang, Y., Yu, Q., Li, D., Fu, Y:: A simple test-time method for out-of-distribution detection, arXiv preprint wtXiv:2207.08222 (2022) 3,
[8] Gao, Z., Yan, S., He, X.: Atta: Anomaly-aware test-time adaptation for out-of-distribution detection in segmentation. Advances in Neural Information Processing Systems 36 (2024) 3
[9] Gou, Y., Hu, P., J., Zhu, H., Peng, X.: Rethinking image super resolution from long-tailed distribution learning perspective. In: Proceedings of the IEEE/CVF Conference on Computer
Vision and Pattern Recognition. pp. 14327-14336 (2023) 3
[10] He, H., Garcia, E.A.: Learning from imbalanced data. IEEE Transactions on knowledge and data engineering 21(9), 1263-1284 (2009) 15
[11] He, K., Zhang. X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceed- ings of the IEEE conference computer vision and pattern recognition. pp. 770-778 (2016) 8, 15
[12] Hendrycks, D., Mazeika, M., Dietterich, T.: Deep anomaly detection with outlier exposure.
arXiv preprint arXiv: 181.04466 (2018) 3, 8, 9, 17, 18
[13] Hong, Yao, J., Zhou, Z., Zhang, Y., Wang. Y.: Long-tailed partial label learning via dynamic rebalancing. arXiv preprint arXiv:2302.05088 (2023) 3
[14] Hu, X., Uzunbas, G., Chen, S., Wang. R., Shah, A., Nevatia, R., Lim, S.N.: Mixnorm: Test-time adaptation through online normalization estimation. arXiv preprint arXiv:21 10. 1478 (2021)
[15] Huang, R., Li, Y.: Mos: Towards scaling duitt--distribution detection for large semantic space. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 8710-8719 (2021)
[16] Iwasawa, Y., Matsuo, Y.: Test-time classifier adjustment module for model-agnostic domain generalization. Advances in Neural Information Processing Systems 34, 2427-2440 (2021) 3
[17] Jiang, X., Liu, F., Fang, 7., Chen, H., Liu, T., Zheng, F., Han, B.: Detecting out-of-distribution data through in-distribution class prior. In: International Conference on Machine Learning. PMLR (2023) 2, 8,9

---

[18] Kendall, A., Gal, Y.: What uncertainties do we need in bayesian deep learning for computer
vision? Advances in neural information processing systems 30 (2017)
[19] Krizhevsky, A., Hinton, G., et al.: Learning multiple layers of features from tiny images (2009)
7. 8. 9, 15, 18
[20] Le, Y., Yang, X.: Tiny imagenet visual recognition challenge. CS 231N 7(7), 3 (2015) 8, 15,
18
[21] Leibig, Allken, V., Ayhan, M.S., Berens, P., Wahl, S.: Leveraging uncertainty information from deep neural networks for disease detection. Scientific reports 7(1), 17816 (2017) I
[22] Li, J., Chen, P., He, Z., Yu, S., Liu, S., Jia, J.: Rethinking out-of-distribution (ood) detection: Masked image modeling is all you need. In: Proceedings of the IEEE/CVF Conference
Computer Vision and Pattern Recognition. pp. 11578-11589 (2023) 3
[23] Li, T. Pang, G., Bai, X., Zheng, J., Zhou, L., Ning. X.: Learning adversarial semantic em- beddings for zero-shot recognition in open worlds. Pattern Recognition 149, I 110258 (2024)
3
[24] Liu, W., Wang, X., Owens, J., Li, Y:: Energy-based out-of-distributioo detection. Advances in neural information processing systems 33, 21464 21475 (2020) 1, 2, 5, 6, 7, 8, 9, 10, 15, 17,
18, 19, 20
[25] Liu, Y., Ding, C., Tian, Y., Pang, G., Belagiannis, V., Reid, I., Carneiro, G.: Residual pattern learning for pixel-wise out-of-distribution detection in semantic segmentation. In: Proceedings of the IEEE/CVF International Conference on Computer Vision. pp. 1151-1161 (2023) 3
[26] Liu, Z., Miao, Z., Zhan, X., Wang, J., Gong, B., Yu, S.X.: Large-scale long-tailed recognition in an open world. In: Proceedings of the IEEE/CVF conference computer vision and pattern
recognition. pp. 2537-2546 (2019)
[27] Loshchilov, Hutter, F:: Sgdr: Stochastic gradient descent with warm restarts. arXiv preprint
arXiv:1608 03983 (2016) 15
[28] Mehta, D., Gal, Y., Bowling, A., Bonnington, P., Ge, Z.:
detection for
long-tailed and fine-grained skin lesion images. In: International Conference on Medical Image
Computing and Computer-Assistie Intervention. pp. 732-742. Springer (2022) 3
[29] Menon, A.K., Jayasumana, S., Rawat, A.S Jain, H., Veit, A., Kumar, S.: Long-tail learning via logit adjustment. arXiv preprint arXiv:2007.00 .0300700.33444 (2020) 9
[30] Miao, W., Pang. G., Li, T., Bai, X., Zheng, J.: Out-of-distribution detection in long-tailed recognition with calibrated outlier class learning. In: Proceedings of the 38th Annual AAAI Conference on Artificial Intelligence (2024) 2, 3, 7, 8, 9, 15, 17, 18, 19, 20
[31] Netzer, Y., Wang, T., Coates, A., Bissacco, A., Wu, B., Ng, A.Y.: Reading digits in natural images with unsupervised feature learning (2011) 2, 7, 8, 15, 18
[32] Ridnik, T., Ben-Baruch Baruch, E., Noy, A., Zelnik-Manor, L..: Imagenet-21k pretraining for the masses.
arXiv preprint arXiv:2 104.. 10972 (2021) 15
[33] Shi, J.X., Wei, T., Xiang, Y., Li, Y.F.: How re-sampling helps for long-tail learning? Advances in Neural Information Processing Systems 36 (2023) 3
[34] Shi, J.X., Wei, T., Zhou, Z., Shao, J.J., Han, X. Y., Li, Y.F.: Long-tail tail learning with foundation model: Heavy fine-tuning hurts. In: Forty-first International Conference Machine Learning (2024)3 3
[35] Sun, Y., Guo, C., Li. Y.: React:
detection with rectified activations. Advances in Neural Information Processing Systems 34, 144-157 (2021)* 3
[36] Tang, K., Tao, M., Qi, J., Liu, Z., Zhang, H.: Invariant feature learning for generalized long-
tailed classification. In: European Conference Computer Vision. pp. 709-726. Springer (2022) 3
[37] Tian, Y., Liu, Y., Pang, G., Liu, F., Chen, Y., Carneiro, G.: Pixel-wise energy- abstention learning for anomaly segmentation on complex urban driving scenes. In: European Conference on Computer Vision. pp. 246-263. Springer (2022)* 3
[38] Torralba, A., Fergus, R., Freeman, W.T.: 80 million tiny images: A large data set for non- parametric object and scene recognition. IEEE transactions on pattern analysis and machine intelligence 30(11), 1958-1970 (2008) 2, 8, 15
12

---

[39] Wang, D., Shelhamer, E., Liu, S., Olshausen, B., Darrell, T.: Tent: Fully test-time adaptation by entropy minimization. arXiv preprint arXiv:2006. (2020)
[40] Wang, H., Zhang, A., Zhu, Y., Zheng. S., Li, M.. Smola, A.J., Wang, Z.: Partial and asymmetric contrastive learning for out-of-distribution detection in long-tailed recognition. In: International
Conference on Machine Learning. pp. 23446-2333 23458.. PMLR (2022) 2. 3, 8, 15 15
[41] Wang, Q.. Fink, O., Van Gool, L., Dai, D.: Continual test-time domain adaptation. In: Proceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recogmition, pp. 7201-7211 (2022) 3
[42] Wang, X., Lian, I.., Miao, Z., Liu, Z., Yu, S.X.: Long-tailed recognition by routing diverse distribution-awen aware experts, arXiv preprint arXiv:2010.01800 (2020)
[43] Wang, 2., Li, Y., Chen, X., Lim, S.N., Torralba, A., Zhao, H., Wang. S.: Detecting everything
in the open world: Towards universal object detection. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 11433-11449 (2023)
[44] Wei, H., Xie, R., Cheng, H., Feng. L., An, B., Li, Y:: Mitigating neural network overconfidence with logit normalization. In: International Conference Machine Learning, pp. 23631-23644
PMLR (2022)
[45] Wei, T., Wang, B.L.,, Zhang. M.L.: Eat: Towards long-tailed out-of-distribution detection. arXiv preprint (.Xi::2312.08922 (2023) 3, 8,
[46] Wu, Y., Chen, Y., Wang, L., Ye, Y., Liu, Z. Guo, Y., Fu, Y.: Large scale incremental learning. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp. (2019) 15
[47] Xiao, Z., Zhen, X., Shao, L., Snoek, C.G.: Learning to generalize across domains on single test samples. arXiv preprint arXiv:2202.00
(2022) 3
[48] Yang, J., Wang, H., Feng, L., Yan, X., Zheng, H., Zhang, W., Liu, Z.: Semantically coherent out-of-distribution detection. In: Proceedings of the IEEE/CVF International Conference on
Computer Vision. pp. 8301-8309 (2021) 1, 8, 9, 15
[49] Yang. P., Liang, J., Cao. J., He, R.: Auto: Adaptive outlier optimization for online test-time ood detection. arXiv preprint arXiv:2303. 12267 (2023) 2, 3, 8, 9, 19
[50] Ye, Y., Chen, S., Ni, D., Huang, R.: Triaug: Out-of cittof-distribution detection for robust classification
of imbalanced breast lesion ultrasound. arXiv preprint arXiv:2402.07452 (2024) 3
[51] Yu, F., Seff, A., Zhang, Y., Song. S., Funkhouser, T., Xiao, J.: Lsun: Construction of large-scale image dataset using deep learning with humans in the loop. arXiv preprint arXiv: 1ww.1036603335 (2015) 7, 15, 18
[52] Yu, Y., Shin, S., Lee, S., Jun, C., Lee, K.: Block selection method for using feature norm in out-of-distribution detection. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 15701-15711 (2023)* 3
[53] Zhang, M., Levine, S., Finn, C.: Memo: Test time robustness via adaptation and augmentation. Advances in Neural Information Processing Systems 35, 38629-38642 (2022) 3
[54] Zhang, Y.F., Zhang, H., Wang. J., Zhang, Z., Yu, B., Wang. L.., Tao, D., Xie, X.: Domain-specific risk minimization. arXiv preprint ar arX:2:2208 00660080.8661111 (2022)
[55] Zhang. Y., Wang. X., Jin, K., Yuan, K., Zhang, Z., Wang, L., Jin, R., Tan, T.: Adanpe: Exploring non non-parann con-parametric classifier for test-time adaptation. In: International Conference on Machine Learning- pp. 41647-41676. PMLR (2023) 3
[56] Zhang, Y., Wang, X., Zhou, T., Yuan, K., Zhang, Z., Wang, L., Jin, R., Tan, T.: Model-free test time adaptation for diittriiittiitttinn detection. arXiv preprint arXiv:2311.16420 (2023) 2, 3,
8, 9, 19
[57] Zhang. Z., Xiang, X.: Decoupling maxlogit for nut-of-distributio detection. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 3388-3397 (2023) 3
[58] Zhou, B., Lapedriza, A., Khosla, A., Oliva, A., Torralba, A.: Places: A 10 million image database for scene recognition. IEEE transactions on pattern analysis and machine intelligence 40(6), 1452-1464 (2017) 7, 8, 15, 18
13

---

[59] Zhu, F., Cheng, Z., Zhang, X.Y., Liu, C.L.: Openmix: Exploring outlier samples for misclassifi- cation detection. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 12074- 12083 (2023) 2
14

---

A More Experiment Settings
A.1 Datasets
For ID datasets, the original version of CIFARIO [10] and CIFARIO0 [10] contains 50, 000 training images and 10, 000 validation images of size 32 x 32 with 10 and 100 classes, respectively. CIFARIO- LT and CIFARIO0 100-LT are the imbalanced version of them, which reduce the number of training examples per class and keep the validation set unchanged. The imbalance ratio p denotes the ratio between sample sizes of the most frequent class and least frequent class. Following [3], we utilize an exponential decay in sample sizes across different classes.
ImageNet-LT [46] is lcrge-scale dataset in long-tail recognition, which truncates the balanced version ImageNet [6]. ImageNet-LT has 1, 000 classes, which contain 115, 846 training images with the number of per-class training data ranging from 5 to 1, 280, and 20, 000 validation images with balanced class size.
For outlier data, Tinylmages80M [38] contains 80 million images with a size of 3232. We use subset of random 30K images as the outlier data for CIFARIO-LT and CIFARI00-LT [4, 40]. We use ImageNet-Extra [40] that contains 517, 711 images belonging to 500 classes from ImageNet-22k [6] but having not overlapping with the 1, 000 in-distribution classes in ImageNet-LT [4,40].
For OOD datasets, we use SC-OOD benchmark [48] as true OOD data for CIFARIO-LT and
CIFARI 40] following 403. The SC-OOD benchmark contains six datasets: CIFAR [19]
with 10, 000 images, Texture [5] with 5, 640 images, SVHN [31] with 26, 032 images, LSUN [51]
with 9, 998 images for CIFAR10-LT and 7, 571 images for CIFAR PAARR0OK 00-LT, Places365 [58] with
35, 195 images for CIFARIO-LT and 33, 773 images for CIFAR
and TinyImageNe [20]
with 8, 793 images for CIFARIO-LT and 7, 498 images for CIFAR 000 Following [30, 40), we
use ImageNet-Ik-OOD [40] that contains 50, 000 images belonging to 1, 000 classes evenly from
ImageNeet222K, which have not overlapping with the 1, 000 ID classes in ImageNet-I.T and the 500
outlier classes in ImageNet-Fxtra, A summary of the ID and OOD datasets is presented in Table 6.
Table 6: Key statistics of the ID and OOD datasets used.
CIFARTO-LT
CIFARTDO-LT
ImageNet-LT
Benchmark
Dataset
Images
Class
Dataset
Images
Class
Dataset
Images
Class
data (Training)
CIFARIO-LT
10
CIFARIOO-LT
100
ImageNet-LT
115,846
1,000
ID data (Testing)
CIFARIO-1T
10,000
10
CIFARIOD-IT
10,000
100
ImageNet-IT
20,000
1,000
Outlier data
Tiny Images80M
30,000
Tiny ImngesagS00MM
50,000
ImagoNot-Extra 517,711
500
CIFARI00
10,000
100
CIFARIO
10,000
10
Texture
5,640
Texture
5.640
47
0DD data
SVHN
26,052
10
SVHN
26.032
10
ImageNet- Ik-0OD
50,000
1000
LSUN
9,998
LSUN
7,571
Places365
35,195
Places365
33,773
Net
8,793
TinyImageNNet
7,498
A.2 Implementation Details
For experiments on CIFARIO-LT [3] and CIFARI00-LT [3], we pre-train our model based on ResNet18 [11] for 320 epochs with an initial learning rate 0.01 [1, 4] using only cross-entropy loss and fine-tune the linear classifier of this model for 20 epochs with an initial learning rate 0.001 [4, 24]. The batch size is 64 for ID data at the pre-training stage, 128 for ID data at the fine-tuning stage, and 256 for outlier data at the fine-tuning stage [4,30, 40]. Our outlier dataset is a subset of IInyImageS80M [38] with 30K images [4, 40..
For large-scale dataset ImageNet-LT, which contains 1 115,846 images of 1,000 classes, we train our model based on ResNet50 [11] for 100 epochs with an initial learning rate of 0.1 [30] using only cross-entropy loss and also fine-tune the linear classifier of this model for 20 epochs with an initial learning rate 0.01. Our auxiliary dataset is a subset of ImageNet22k [32] with 516K images, following [30.40].
All experiments use SGD optimizer and decay the learning rate to zero using a cosine annealing learning rate scheduler [27]. All experiences are performed with 8 NVIDIA RTX 3090.
15

---

B
The AdaptOD Algorithm
The full steps of the training and inference in AdaptOD are given in Algorithm below.
Algorithm 1: AdaptOD
Training
Input: Pre-trained model f
Data: Training dataset Dirain Auxiliary dataset D
for each iteration do
2: Sample mini-batch of ID training data: {(=]", yi) (=1 from Dtrain
3: Sample mini-batch of OOD auxiliary data: {(ggu) )=== from Derain
4: Perform batch energy normalization based on Eq. 6
5: Perform gradient descent model f with fiotel based on Eq. 11
6: end for
Inference
Input: Outlier distribution pout; Fine-tuned model ʃ
Data: Test dataset D
1: for each sample æ in dataset inUJout do
2: Adapt the outlier distribution pout with sample x and model f based Eq. 4
3: Obtain calibrated global energy score G (xx) for sample æ as OOD score using the outlier distribution pout based Eq.
4: end for
C Discussion of Stable Margin Hyperparameters in DNE
C.1 Margin Hyperparameters in DNE-C
To identify OOD samples, we expect ID samples to have high energy, while OOD samples to have low energy. Formally, let x = {af" . ..... Tipn 1 E X be one training batch of ID) data, with 4d be its batch size, and a corresponding batch of outlier data x {xqut .:22 ..... &oou whose batch size is baut, the sum of flas-normallzer energy Fy (x) for all training samples in one batch (ID samples and outlier samples in one batch) in each class j would be one:
5m Cy(x²") + Cx(xxut)= Fz("")
Doue
Σ Fz(2u0)) =
(13)
DNE-C class-wisel, constrains the energy that optimizes the class energy of ID samples to be large for each class, while the class energy of outlier samples is small for each class. Therefore, the expected class-wise normalized energy for the batch of outlier samples C,(x""), E (1,2, k} would be on all classes, while the expected class-wise normalized energy for the batch II) samples (xux)))), ).j {1,2, k} would be zero on all classes:
(x 1,
(14)
Cx -0
To this end, we m and m for each class margin in Eq. which optimizes the wlass-wise normalized energy for the batch of outlier samples Cx(x")) on each class j towards one, while at the same time optimizing the class-wise normalized energy for the batch of ID samples C, (xou)) each class j towards zero. In this way, these energy margin hyperparameters do not rely on the training dataset and/or the imbalance factor.
C.2 Margin Hyperparameters in DNE-S
Similarly, the sum of class-normalized energy F; (x)) for all training samples in one batch (ID samples and outlier samples in one batch) each class j would be one. Furthermore, the sum of class- normalized energy (x) for all training samples in one batch over all classes would be & since there
16

---

are k categories in the ID data:
6
8
k
bone A Pa(2t()) k. =1
Σ ((a)")
Σ S(a?t))
Σ [=1 ==1
Pa(a
(15)
DNE-S sample-wisely constrains the energy that optimizes the global energy of each ID sample to be large, while being well-balanced between head samples and tail samples. Therefore, the expected sample-wise normalized energy S(x), x E x would be k for each ID sample, which is evenly divided the same scale to each ID sample (for either head samples or tail samples). And the expected sample-wise normalized energy S(x), x E xout would be zero for each outlier sample:
( 7 bin I E Ex" ((x) + 0, x E xout.
(16)
& Therefore, we set mi, = ges and mout = O, which optimizes the sum -lass-normllized energy on & all classes (ax) for each ID sample to 5,m and optimizes the sum of class-normalized energy on all classes (zx) for each outlier sample zero. After doing this, we can regularize the global energy of the ID data, particularly the low global energy for tail samples, reducing the over-confident prediction of head samples. The same as the DNE-C loss, these specified margin parameters for training in DNE-S also do not rely on the training dataset and/or the imbalance factor.
D More Experimental Results
D.1 More Results for DODA
Table 7 presents the results of our proposed component DODA in enabling two popular baselines OF [12] and EnergyOE [24] on CIFAR 10/100-IT on the six OOD test datasets. shows that DODA consistently enhance the OOD detection for both basclines in all four metrics across all six datasets, demonstrating the strong capability of DODA in reducing the learned outlier distribution gap to the true OOD. Nevertheless, these DODA-enabled baselines underperform AdaptOD, indicating that the other component of AdaptOD (iee, DNE) helps to produce a largely enhanced vanilla outlier distribution for DODA.
D.2 Differentiating OOD Data from Head and Tail Samples.
To evaluate the effectiveness distinguishing OOD data from head and tail samples, we perform two particular inference settings: with only tail samples and OOD samples, and another with only head samples and OOD samples. Table shows the averaged results over the six OOD test datasets on CIFAR 10/100-LT of the baseline EnergyOß [24], previous SOTA model COCL [30], and our AdaptOD. It can be observed that AdapIOD does a better job than the two methods both scenarios, resulting in significantly enhanced overall detection performance.
D.3 More Ablation Study
D.3.1 Imbalance Ratio
In the Experiments section, we use the default imbalance ratio p = 100 on both CIFARIO-LT and CIFAR100-LTT In this section, we show that our method can work well under different imbalance ratios. Table 9 shows the comparison of AdaptOD with two SOTA long-tailed OOD detection methods EnergyOE [24] and COCL [30] CIFARIO-LE with p 50 and م 10. Our approach can significantly outperform these baselines in not only OOD detection performance but also ID classification accuracy by considerable margin with different imbalance ratios. Furthermore, our approach AdaptOD performs better in more imbalanced datasets, indicating the superiority of AdaptOD for OOD detection in long-tail tail recognition.
Table 10 shows the comparison our approach AdaptOD with two SOTA TTA methods AUTO and AdaODD for OOD detection CIFAR 10-LT with 50. To have straightforward and extensive comparison, we compare DODA (the component of AdaptOD) with the two TTA methods, all of which are added on top of the same training method. In the experiments, use three training
17

---

Table 7: Results of original and DODA- OE and Energy OE. and AdapiOD.
OOD
ID Dataset:
CIFARTO-LT
ID Dataset: CIFARTOO-LT
Dataset
Method
AUCF
AF-in
AF-out
FPRI
AUCT
AF-int
AP-outt
FPR
92.30
96.01
82,57
48.65
76.01
85.28
57.47
87.45
OF (1Ors)
95.02
96.97
84.40
46,99
77.93
86.51
62.48
82.75
Texture
EnergyOE [2]]
95.53
97.42
92.93
18.44
79.56
86.03
70.88
79.45
EnergyOE (2:) (20)A
97.32
97.88
93.65
16.28
80.85
87.34
75.52
77.00
AdapsOD(Ours)
98.22
98.81
94.91
11.60
83.88
89.43
76.47
58.47
(12
94.86
91.59
97.00
29.11
81.82
73.25
89.10
80,98
OF (12)=-DDAAOOrs
95.95
92.01
98.16
25.75
84.20
75.78
91.66
74.86
SVHN [31]
EnergyOE [24]
96.63
92.33
98.46
14.37
86.19
81.42
91.74
34.36
EnergyOE DODD((OSS)
97.24
92.88
98.73
12.86
90.26
88.13
95.30
21.73
AdaptOD(Ours)
98.13
94.34
99.11
10.33
93.099
91.32
96.86
17,63
OE [12]
83.32
84.06
80.83
65.82
62.60
66.16
$7.77
93.53
(12)=DOODA(Oum
85.52
86.03
83.15
60.99
66.02
72.11
62.03
90.85
CIFAR [19]
EnergyOE 1 [24]
84.44
85.74
84.63
61.73
61.15
67.12
56,66
91.42
EnergyOE DO)A
86.71
87
87.01
54.33
70.42
76.10
68.66
89.87
Odap
89,05
89.93
88.22
45.51
72.77
76.37
70.58
86,04
OE [12]
86.35
89 88
79.30
64.50
688
79.36
51.82
88 54
[12]-DODA(Ours]]
88.39
90.88
82.70
61.40
70.36
79.72
53.44
$8.38
TIN [20]
EnergyOE [2
88.40
91.65
84,95
46.23
70.78
79.40
35,90
90.74
EnergyOE DOODA
89.93
92.46
86.
44.02
71.25
79.80
57.91
89.42
Adops)
91.40
93.85
88.18
42.77
72.87
82.06
58.92
88,24
OE
91.37
93.06
88.37
53.99
76.81
85.33
60.94
83.79
OF (I2)) (Ours)
93.09
93.42
91.30
48.39
77.83
86 24
62. 10
%2.44
LSUN [51]
EnergyOE (2]
94.00
94.78
93.70
28.42
81.61
86.57
69.16
80.57
EnergyOE: [24 DODAA
94.92
95.77
94.56
26.17
82.54
88.12
70.88
77.68
AdaptOD(OOurs)
96.16
96.84
95.86
24.12
85.70
90.55
72.70
70.20
OE [12]
90.20
82.09
95.24
57.06
75.68
60.99
86.31
83.55
OF (EL11)))
91.74
83.99
96.64
52.57
76.39
62.48
87.52
82.72
Place365 [58]
EnergyOE OE 24]
92.51
84.26
97.14
33.63
79.12
63.38
89.09
81.43
EnergyOE (OO)ss
94.03
86.15
97.75
31.23
81.08
65.85 85
90.94
80.09
AdaptOD(Ours)
95.19
89.56
98.44
29.22
83.27
68.82
91.44
71.63
Table 8: Comparison results on separating head/tail samples from OOD samples.
Head Samples
Tail Samples
ID dataset
method
AUC²
AP-in
AP-oult
FPR.
AUC1
AP-int
AP-outt
FPR.]
EnergyOR [24]
95.88
89.67
98.31
23.06
83.45
61.07
93.37
58.61
CIFARIO-LT
COCL [30]
96.34
93.34
98.67
19.59
91.91
76.98
97.15
34,30
AduptOD(Ours)
98.20
96.80
99.00
11.20
93.40
80.58
98.27
30.70
EnergyOR [24]
84.22
69.70
92.81
69.42
67.63
35.85
85.96
81.77
CIFARIO0-LT
COCL [30]
87.73
73.84
93.94
66.01
74.85
47.76
87.59
77.01
AdaptOD(Ours)
91.81
80.42
96.43
58.49
78.34
56.67
91.15
70.82
Table 9: Comparison results of imbalance ratio among EnergyOE [24], COCL [30], and our approach
AdaptOD on CIFARIO-LT.
Imbalance Ratio
Method
AUCF
AP-inf
AP-out"
FPR.
ACC7
EnergyOE [24]
91.92
91.03
91.97
33.80
74.57
100
COCL [30]
93.28
92.24
92.89
30.88
81.56
(daptOD(Ours)
94.69
93.89
94.12
27.26
82.27
EnergyOE [24]
93.48
92.68
93.05
29.74
81.23
p
COCL [30]
94.30
93.85
93.31
26.98
84.89
AdaptOD(Ours)
95.14
94.53
94,66
24.43
85.77
EnergyOB [24]
95.03
94.34
94.83
25.26
88.47
10
COCL [30]
95.71
95.12
95.33
20.91
89.65
OdurtOD(OOrs)
96.34
95.72
95.86
18.33
90.24
methods, including two SOTA loigetailed OOD detection methods and our proposed DNE-based training method. impressive that our DODA component consistently remains the best performer when the TTA methods are combined with all three different training methods on the CIFAR10-LT7 datasets.
D.3.2 Network Architectures
In the Experiments section, we use the standard ResNer18 as the backbone model on both CIFAR 10- LT and CIFAR100-LIT To show the generality of our method, we also perform long-tailed OOD detection experiment using both ResNet34 and ResNet18. The results are shown in Table 11 and
18

---

Table 10: Comparison different TTA-based OOD detection methods on CIFARIO-LT with p 50.
Training
Test
AUC1
AP-in
AP-oul7
FPR,
w/o TTA
93.48=0.25
92.68=0.33
93.05=0.30
29.74:4022
AUTO [49]
93.85+0.29
92.84++025
93.34++099
19.10+-0.35
EnergyOB [24]
AdaODD [56]
94.14:0.33
92.92=0.32
93.60=0.33
29.01=0:20
(Our(
94.60+00228
93.46+0.36
93.91=0.26
28.42++0.24
w/o TTA
94.3010..25
93.85-10.25
93.31=0.44
26.9811.228
AUTO [49]
94.62++0.11
93.91=0.33
93.52=0.40
26.49::0.37
COCL [30]
AdaODD [56]
94.41+0.29
93.84=0.31
93.35=0.46
26.67-40.35
(OuA(Ours)
94.8240.24
94.13-10.29
94.21-0.36
26.02.10.30
TTA
93.85+0.38
9343++028
03.62-0.39
27,47
DNE (Ours)
AUTO [49]
94.5911.43
93.68.10.3
93.98
0..98
26.50.10.35
Ad [56]
94.7510.42
93.78110.2
9..22-0.44
25.64.10,37
(Our)
95.14+0.41
94,53
94.66 0.36
24.43+0.32
Table 11: Comparison results of model structure among EnergyOE [24], COCL [30], and our
approach AdaptOD on CIFAR I0-LT.
Model
Method
AUCT
AP-in
AP-out
FPR.
ACCT
EnergyOE [24]
91.92
91.03
91.97
33,80
74.57
R8sNer18
COCL. [30]
93.28
92.24
92.89
30.88
81.56
AdaptOD(Ours
94.69
93.89
94.12
27.26
82.27
EnergyOE [24]
92.25
91.37
92.31
32.44
74.89
ResNet34
COCL [30]
93.52
92.93
92.83
30.74
81.75
AduptOD(Ours)
94.98
94.33
94.52
26.61
83.47
Table 12:
Comparison to different based OOD detection methods on CIFARIO-LT using ResNet34. The results are averaged over the six OOD test datasets in the SC-OOD benchmark.
Training
Test
AUC1
AP-int
AP-outf
[PRR
TTA
92.25+0.32
91.37=0331
92.31+0.28
32.47:0.377
AUTO [49]
92.98+0.40
91.93-0.26
92.60+0.38
32.03-+0.38
EnergyOF [24]
AdaODD [56]
93.16+0.36
92.10=0.50
92.90-10.46
31.87=20.47
(Ours)
93.81+0.32
92.68-0.28
93.27400.39
29.65++0.36
w/o TTA
93.52.10.66
92.93-0.48
92.83:10.27
30.74.10.38
AUTO [49]
93.73::0.45
93.04=0.40
93.26:10.31
29,60:10.40
COCL [30]
AdaODD [56]
93.90.10.47
93.19-0.35
93.46.10.44
29.31
(Our(
94.27-10.99
93.57=0.38
93.82.00
28.78.10.34
w/o I'AA
93.28+0.30
92.44-025
92.95+0.32
31.18+0.33
DNE (Ours)
AUTO [49]
93.77.10.3
92.83
93.36.10.45
29.99
AdaODD [56]
93.8411042
93.04-0.39
93.6110.37
29.53110:42
(Our))
94.98+0.35
94.33 0.33
94.52+0.40
26.61+0037
Table 12. Table 11 shows the comparison of AdaptOD with two SOTA long-tailed OOD detection methods EnergyOE [24] and COCL [30] on CIFAR 10-LT using different backbone models. Table 12 shows the comparison of AdaptOD with two SOTA TTA methods AUTO and AdaODD for OOD detection on CIFARIO-LT using ResNet34. AdaptOD maintains its superiority with different network architectures.
D.3.3 Sensitivity
In the Approach section, we utilize a 7-score-hased method based on training ID data to implement the OOD filter, which predicts true OOD samples for adapting outlier distribution in DODA. The threshold R in the OOD filter is calculated with only training ID data and can be directly used during inference. is hyperparameter large number of ID samples as OOD samples. On the other hand, too low value of ft will filter out too many true OOD samples. In both cases, the outlier distribution adaptation becomes ineffective. Fig. 4 shows the sensitivity of AdaptOD with respect to in Eq. 3, showing that the performance of AdaptOD is relatively stable with a relatively large range of a values. e.g., 3.5).
adjust the threshold R. too high value of R can misclassify
D.3.4 Computational Overhead
AdaptOD performs normalization on the logit output of each class for each batch of training samples before energy training. Fig. shows the computational overhead of AdaptOD compared to the baseline EnergyOE and the previous SOTA method BERL using the same backbone on CIFAR100-LT
19

---

78
75
AdaptOD (Ours)
72
Baseline
69
66
63
3 Numerical Value of
Figure 4: Average performance of AdaptOD w.r.t. over six OOD datasets CIFARIOO-LT.
with single NVIDIA RTX 3090, in which they are all fine-tuning- and energy- basedd methods. shows that the training speed AdaptOD is similar with the previous methods on both ResNet18 backbone and ResNet34 backbones.
Table 13: Comparison results training time (seconds) on CIFARIOO-LT.
Training Time (scconds)
Model
EnergyOE [24]
BERL [4]
AdaptOL(OOrs)
ResNe118
8.56
9.12
8.84
ResNet34
12.65
13.12
12.89
Table 14: Comparison results synthetic OOD datasets with CIFAR10-1 10-LT.
Dataset
Method
AUC
AP-in
AP-ourT
FPR.
EnergyOB [24]
99.74
99.76
99.33
1.96
BERL [4]
99.76
99.34
99.16
0.49
Gaussian
COCL [30]
99.68
99.79
99.39
0.02
AdaptOD (Our))
99.83
99.87
99.58
0.08
EnergyOR [24]
99.13
99.25
97.16
2.32
BERI. [4]
99.00
99.06
96.26
1.42
Rademacher
COCL [30]
99.76
99.84
99.56
0.01
AdaptOD(Ours)
99.78
99.65
99.61
0.04
EnergyOB [24]
90.16
93.25
85.39
9.44
BERL [4]
93.18
96.87
89.34
6.54
Blubs
COCL [30]
98.75
99.17
97.49
1.04
AdaptOD (urr))
99.12
99.43
98.62
0.53
EnergyOB [24]
96.34
97.42
93.96
4.57
BERL [4]
97.32
98.42
94.92
2.81
Average
COCL [30]
99.40
99.60
98. 98.81
0.35
AdaptOD(Ours)
99.58
99.65
99.27
0.22
D.4 Experiment Results on Synthetic OOD Datasets
To demonstrate the superiority of our approach AdaptOD on diverse OOD datasets, we also evaluate our approach AdaptOD with three synthetic OOD datasets on CIFARIO-LT, including Gaussian, Rademacher, and Blobs. Specifically, Gaussian noises have each dimension sampled from an isotropic Gaussian distribution. Rademacher noises are images where each dimension is or 1 with equal probability, so each dimension is sampled from a symmetric Rademacher distribution. Blobs noises consist of algorithmically generated amorphous shapes with definite edges. We use three SOTA methods for comparison, including EnergyOE [24], BERL [4], and COCL. [30]. As in Table 14, our approach AdaptOD achieves similarly significant improvement over these methods on these synthetic OOD datasets the other OOD datasets.
20

---

E
Limitation and Broader Impacts
E.1
Limitation
While AdaptOD offers a straightforward and competitive solution for out-of-distribution detection in long-tailed recognition, necessitates the incorporation additional outier data to learn an enhanced vanilla outlier distribution, increasing the difficulty of applying it to real-world scenarios. The DODA component in AdaptOD is an attempt that utilizes the detected OOD samples during the inference stage to improve OOD detection performance. This requires online updating of the learned outlier distribution. An alternative way is to optimize the outlier distribution and reduce its gap the OOD distribution during training, eliminating the online updating step. The lack of true OOD data remains as major challenge in such approaches. We leave for future work.
E.2 Broader Impacts
OOD detection is a branch of anomaly detection that typically plays positive role in enhancing model security in various safety-critich applications such as autonomous driving. When applying our methods, we need to ensure that the models are only used for the purpose of enhancing the safety of deep learning models real-world environments and do infringe on human privacy.
21

---

NeurIPS Paper Checklist
1. Claims
Question:
Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification:
Abstract and introduction accurately reflect the paper's contributions and
scope.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification:
The limitations of the work discussed in Appendix E.1.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
the paper. The authors should use their best
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
22

---

Justification:
There is no theoretical result in this paper.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: Our paper fully discloses the information needed to reproduce the main experimental results.
Guidelines:
The answer NA means that the paper does not include experiments. If the paper includes experiments, No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm. (b) If the contribution is primarily
model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome deseribe the particular way they provide for reproducibility, In the case of closed-source models, it may be that access to the model is limited in some way (e.g., registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
23

---

Answer: [Yes]
Justification:
Our paper provides open access to the data and code.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (c.g., for a new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https: /Codeeuomiisseeeeeeee
/nips. wwtpccccpppcccaecccccccccccccclt /ondei/eeddeeeeeeeuuiiiiiiii ssisssolicyy for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible supplemental material (appended the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental Setting/Details
Does the paper specify all the training and test details (c.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary understand the results?
Question:
Answer: [Yes]
Justification: Our paper specifies all the training and testing details
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core the paper to a level of detail that is necessary appreciate the results and make sense of them.
The full details can be provided either with the code, appendix, as supplemental material.
7. Experiment Statistical Significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
Justification: Our paper shows average results six runs with different random seeds and report the variance for our metrics.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the enror bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).

---

It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures tables in the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: Our paper provides sufficient information on the computer resources.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).
9. Code Of Ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https: http///eerrppe.eeeeeeeee httpp//rrs..cppppli////////c///t oe ctw.ieelslllllleccccccccccc
ines?
Answer: [Yes]
Justification:
Our research complies with the NeurIPS Code of Ethics.
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws or regulations in their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification:
The societal impacts of the work are discussed in Appendix E.2.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no socictal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
25

---

The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: The paper poses no such risk.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g.. code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: Our paper gives proper acknowledgement the original papers that produced the code packages datasets.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
26

---

If this information is not available online, the authors are encouraged to reach out to the asset's 's creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification: Our paper does not release new assets.
Guidelines:
The answer NA means that the paper does not release new assets,
Researchers should communicate the details of the dataset/code/moodl as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: Our paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According the NeurIPS Code of Ethies, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: Our paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
27


Reviews:
[{"review_id": "U3G85bJKYc", "reviewer": "Reviewer_KD4x", "summary": "This paper introduced the normalized outlier distribution adaptation (AdaptOD) which adapts the outlier distribution to the true OOD distribution from both the training and inference stages. Such AdaptOD includes two components: one is dynamic outlier distribution adaption (DODA), which adapts the outlier distribution to the true OOD distribution. The other is dual-normalized energy loss (DNE), which includes class-level and sample-level normalized energy to enforce more balanced prediction energy for imbalanced ID samples. The experiments are conducted on three OOD LTR benchmarks.", "questions": "1. The experimental comparisons for Tab. 1 seem unfair. The COCL and EnergyOE can not access the true OOD data, while the proposed method AdaptOD updates the global energy score with true OOD data in the test phrase. \n\n2. For Tab. 2 and Tab. 3, the previous methods with DODA modules obeying the OOD setting should be considered.", "limitations": "The authors have discussed the limitations of this paper, and there is no negative societal impact.", "rating": 5, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "1. The introduction of DODA and DNE is novel and addresses the key issue of distribution shift in OOD detection, especially in LTR scenarios.\n\n2. The combination of DODA and DNE demonstrates significant improvements over existing methods, both individually and collectively, as evidenced by the ablation studies.\n\n3. The paper is exemplarily structured and articulated. It offers a lucid and comprehensive elucidation of the AdaptOD methodology, its constituent elements, and the experimental framework, rendering it accessible to readers.", "weaknesses": "The motivation and mechanism underlying dynamic outlier distribution adaption (DODA) are unclear and lack a theoretical foundation.\n(1)\tMy primary concern is on the setting. The proposed AdaptOD with the introduction of test time adaptation (TTA) seems unfair. In previous work on OOD, the model's update process could not access the true OOD data and adopted the outlier as an alternative, while in this paper, the true OOD data is used to update the outlier distribution in the module DODA. As described in Tab. 5, without the DODA modules, the AUROC is 72.04 on ImageNet-LT, and such an increase is relatively small compared to SOTA. That indicates the main improvement depends on the knowledge leakage of true OOD samples during TTA.\n\n(2)\tI am confused about the motivation of DODA under the particular OOD setting, which adapts the outlier distribution to true OOD distribution. Suppose we could access the true OOD distribution in the test phase and update the outlier distribution with the predicted OOD sample. Why don't we use the true OOD distribution to calibrate the scores?  The DODA provides access to the true OOD samples and provides the perfect solution for auxiliary OOD data that directly uses the predicted OOD sample as auxiliary OOD data. Moreover, as described in Fig. 1, the curves of adapted outliers almost coincided with the true OOD. The authors would better give more interpretation.\n\n\n(3)\tI am confused about the mechanism of DODA. Whether the DODA would adapt the outlier distribution to the true OOD distribution, as the initialized model detects the OOD sample at the beginning, the ID samples would be wrongly predicted as OOD, and that would lead to the optimization of the outlier distribution to the ID data distribution rather than the OOD distribution.  The direction of the update of outlier distribution is uncertain how the DODA guarantees the direction to True OOD distribution. The authors would better give more theoretical analysis and support.\n\n(4)\tIs adapting the outlier distribution to true OOD distribution with specified OOD samples meaningful for OOD detection? As the OOD samples can be drawn from any unknown distribution, the benchmarks are just built to evaluate the models’ ability of unknown detection. The specified adaption of outlier distribution would improve the performance on corresponding benchmarks. However, the specified adaption can not generalized to other benchmarks and would be useless for the OOD sample from an unknown distribution.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for recognizing the novelty of our method and its improvement over existing methods. On the other hand, there might be some misunderstanding regarding the setting, which we will clarify in the following responses and further improve our writing.\n\n> **W1: concern for the setting**\n\n- **Test-time adaptation (TTA) for OOD detection**\n\nTTA is a widely used technique that utilizes test data (without knowing their class labels) to help models improve performance and adapt to real-world scenarios. Its application to OOD detection is relatively less explored, but it can help quickly adapt to new OOD scenarios and improve OOD performance due to the **possibility** of utilizing real OOD samples. This has been demonstrated in recent studies, AUTO and AdaOOD, and more recently published RTL at CVPR2024 [Ref1]. These justify the importance of designing TTA methods to enhance OOD methods. However, please note that the TTA module is assumed NOT to be able to get access to the ground truth of the test data in all these methods, including ours, meaning that it handles unlabeled test data. Ground truth of test data is used in detection performance evaluation only. Thus, there is no data (label) leakage issue.\n  \n- **Unfair comparison to methods that could not access the true OOD data**\n\nIt is true that our method and other TTA-driven methods have the advantages of accessing the samples drawn from true OOD distribution. To ensure a fair, comprehensive comparison, in Tab. 2, we compare our full method AdaptOD with DODA-enabled four SOTA, non-TTA methods. Furthermore, in Tab. 3, we compare our DODA with the two existing TTA methods for OOD detection, where each of them is respectively combined with different current OOD detection methods. Additionally, we further compare our AdaptOD with RTL [Ref1] in the general response. All of these experiments demonstrate the superiority of AdaptOD in long-tailed OOD detection.\n\n- **Small AUC improvement on ImageNet-LT compared to SOTA without the DODA module**\n\nBoth our modules, DODA and DNE, make major contribution to the overall detection performance of our method AdaptOD. We agree that the DNE module leads to relatively small AUC improvement on ImageNet-LT, but it can consistently improve not only all OOD detection metrics but also ID classification accuracy across all three benchmarks. Additionally, DNE also provides stable energy margins on different datasets, eliminating the need of manual tuning of these margins.\n\n[Ref1] Fan, Ke, et al. \"Test-Time Linear Out-of-Distribution Detection.\" Proceedings of CVPR, 2024.\n\n> **W2: the motivation of DODA**\n\n- **Why don't we use the true OOD distribution to calibrate the scores?**\n\nThere may be a setting misunderstanding. Although we can access the test data that contains the true OOD data, we do not know the true OOD distribution since we cannot access the test labels. Therefore, DODA assigns the pseudo labels to test data at first, and then uses the predicted OOD samples (maybe involving false predictions) to gradually estimate the true OOD distribution. \n\n- **The curves of the adapted outliers almost coincided with the true OOD**\n\nThis occurs because of highly effective outlier distribution adaptation in our method. This is also justified by the results in Tab. 5, where AdaptOD has only a small performance gap to the Oracle model that utilizes the ground truth labels of the OOD test data to update the outlier distribution. This indicates that AdaptOD can well approximate the true OOD distribution by the predicted labels of the OOD samples.\n\n> **W3: the mechanism of DODA. Wrongly predictions may lead to bad optimization of outlier distribution**\n\nYes, it is true that the wrongly predicted OOD samples would lead to bad outlier distribution during TTA, as shown in Fig. 3.\nTherefore, our DODA designs a Z-score-based method in Eq. 3 to implement an OOD filter based on training ID data, where $\\alpha=3$ (corresponding to 99\\% confidence interval) is set so as to utilize a predicted OOD sample as true OODs with very high confidence. There may be a few wrongly predicted OOD samples, but their influence is limited in DODA as they are often similar to true OOD data, and moreover, the optimization of the outlier distribution would be dominated by the most correctly predicted OOD samples. As a result, our AdaptOD can quickly adapt to the outlier distribution and perform very stably thereafter (see Fig. 3).\n\n> **W4: the meaning of adapting the outlier distribution to true OOD distribution with specified OOD samples**\n\nYes, OOD samples can be drawn from any unknown distribution in different deployment stages and/or applications. This is exactly the main motivation of our work that we aim to continuously adapt the outlier distribution by utilizing dynamic, unknown OOD samples in the target application scenarios to tackle this challenge. Our comprehensive results with different OOD data across three ID data benchmarks show that AdaptOD can generalize well regardless of the difference in ID or OOD datasets.\n\n> **Q1: The experimental comparisons for Tab. 1 seem unfair**\n\nCOCL and EnergyOE are previous SOTA methods, so we directly compare them in Tab. 1 to show the significant improvement of AdaptOD as a whole. For a fairer comparison, in Tab. 2 and Tab. 3, we report the average performance across six OOD datasets of Tab. 1 which combine COCL and EnergyOE with DODA and other TTA methods, so COCL and EnergyOE also can access the test-time true OOD data.\n\n> **Q2: for Tab. 2 and Tab. 3, the previous methods with DODA modules obeying the OOD setting should be considered**\n\nDODA is a TTA method that can be combined with previous OOD methods to utilize the unlabeled test data for tackling the distribution shift problem. For a fair comparison, we have compared AdaptOD with the previous OOD detection methods enabled by DODA in Tab. 2, and we have also compared DODA with other TTA methods in Tab. 3 and the table above with the recent RTL.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "FOgjIOhvPW", "reviewer": "Reviewer_t7zA", "summary": "This paper addresses the challenge of OOD detection in LTR scenarios, where the distribution of classes is heavily imbalanced. The key issue highlighted is the absence of true OOD samples during training, which hampers the effectiveness of OOD detectors, especially the significant distribution shift between outlier samples and true OOD samples in LTR scenarios. The authors propose a novel approach called AdaptOD to tackle this problem by reducing the distribution gap between the outlier samples used for training and the true OOD samples encountered at inference time. \n\nAdaptOD introduces two main components: DODA performs test-time adaptation to dynamically adjust the outlier distribution to better align with the true OOD distribution using the OOD knowledge from predicted OOD samples during inference. And DNE is designed to balance the prediction energy for imbalanced ID samples during training, which helps in learning a more effective vanilla outlier distribution that is crucial for DODA's adaptation process.\n\nThe authors demonstrate the effectiveness of AdaptOD through extensive experiments on CIFAR10/100-LT and ImageNet-LT. The results indicate its superiority in handling the long-tailed OOD detection problem.", "questions": "1. DNE is designed for more effective DODA, what's the most important factor in applying TTA methods on OOD detection in LTR during training stage?\n2. The updated form of the outlier distribution in Eq.4 is sample-wise, how does it compare to batch-wise updates?", "limitations": "The limitations have been discussed in Appendix E.", "rating": 7, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. Exploring OOD detection in the context of long-tailed recognition has practical value, and the discovery of strong distribution shift between outlier samples and true OOD samples in LTR scenarios, as shown in the Fig. 1a, is meaningful.\n2. The motivation is clear. AdaptOD utilizes the DNE to learning a better vanilla outlier distribution first, and then performs DODA to dynamically adapt the outlier distribution to the true OOD distribution.\n3. The method is elegant. DODA only calibrates the outlier distribution, effectively eliminating the retraining or memory overheads, and well approximating the upper-bound performance. DNE effectively adapt the standard energy loss in LRT scenarios with batch energy normalization, and further eliminated the dependence of energy bar on imbalance factors and training datasets.\n4. This paper is well-written, and the empirical evaluation is comprehensive.", "weaknesses": "1. The goal of the CET component is to \"enforce more balanced prediction energy for imbalanced ID samples\" as line 88, which is similar to BERL[1]. The author should further clarify their differences.\n2. AdaptOD performs better in near OOD dataset CIFAR, which cannot be achieved by previous methods. It would be better to further discuss the reason.\n\n[1] Choi, H., Jeong, H., Choi, J.Y.: Balanced energy regularization loss for out-of-distribution detection. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 15691–15700 (2023)", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for appreciating our contribution and for your thoughtful and detailed feedback. Please find our responses to your concerns below:\n\n> **W1: The goal of the DNE component is to ``enforce more balanced prediction energy for imbalanced ID samples\" as line 88, which is similar to BERL**\n\nBERL only balances the prediction energy at the sample level, while our DNE balances not only the sample level but also the class level, with the support of the novel batch energy normalization. This difference allows AdaptOD to achieve a much better energy balance and thus a better OOD detection and ID classification accuracy, as shown in Tabs. 2 and 3. It also provides stable energy margins across different datasets, eliminating the need of manual tuning of these margins as in BERL, helping a better adaptation during DODA.\n\n> **W2: AdaptOD performs better in the near-OOD dataset CIFAR, which cannot be achieved by previous methods. It would be better to discuss the reason in further detail**\n\nNear OOD dataset CIFAR is similar to the training ID data, so previous methods are always confused between them. However, DODA can effectively utilize the test-time OOD data during inference to effectively adapt the outlier distribution and subsequently calibrate the global energy score. As a result, it helps AdaptOD discriminate near OOD samples from the ID samples better than previous methods. This is supported by the results in Fig. 1 in the attached PDF in the general response, where the performance of three TTA methods, including ours, increases significantly with an increasing number of OOD samples from CIFAR involved in the adaptation.\n\n> **Q1: DNE is designed for more effective DODA, what's the most important factor in applying TTA methods on OOD detection in LTR during the training stage?**\n\nEnergy regularization is a mainstream method to improve OOD performance, and in long-tailed OOD detection, it is used to alleviate the bias of the global energy toward head samples in existing studies such as BERL. But we observe that there is a class-level bias toward head classes, in addition to the sample-level bias. Thus, one major contribution from DNE is to enable more balanced energy predictions overhead and tail samples due to its sample- and class-level energy debiasing. Another important contribution lies at its Batch Energy Normalization, which addresses a largely ignored issue in long-tailed OOD detection that requires careful manual tuning of energy margins to work well on different ID/OOD datasets. DNE provides stable energy margins for different long-tailed ID/OOD datasets, alleviating the aforementioned biases toward head classes without relying on the data-specific manual margin tuning.\n\n> **Q2: The updated form of the outlier distribution in Eq.4 is sample-wise, how does it compare to batch-wise updates?**\n\nIn DODA we utilize the statistic of training ID data to guarantee the optimization of outlier distribution in Eq. 3 during the test phase, and the assigned pseudo label for each unlabeled test sample only depends on this statistic. Therefore, our predicted OOD samples used to adapt the outlier distribution once is sufficient for our method to work well. If batch-wise updates are used, it can help the optimization of outlier distribution become more stable at the beginning of DODA, with only minor difference in detection effectiveness, as shown in Tab. C and Tab. D below. Sample-wise dynamic updates as in Eq. 4 are more desired than batch-wise approaches in the pursuit of instant OOD detection, so we focused on the former one in this work.\n\nTable C: Result of sample-wise and batch-wise update on CIFAR10-LT with AdaptOD. \n\n|             | AUC $\\uparrow$ | AP-in $\\uparrow$ | AP-out $\\uparrow$ | FPR $\\downarrow$ |\n| :---------: | :------------: | :--------------: | :---------------: | :--------------: |\n| Sample-wise |     94.69      |      93.89       |       94.12       |      27.26       |\n| Batch-wise  |     94.74      |      93.95       |       94.18       |      27.22       |\n\nTable D: Result of sample-wise and batch-wise update on CIFAR100-LT with AdaptOD. \n\n|             | AUC $\\uparrow$ | AP-in $\\uparrow$ | AP-out $\\uparrow$ | FPR $\\downarrow$ |\n| :---------: | :------------: | :--------------: | :---------------: | :--------------: |\n| Sample-wise |     81.93      |      83.09       |       77.83       |      67.37       |\n| Batch-wise  |     82.06      |      83.14       |       77.88       |      67.32       |", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "K8EOsvKWuL", "reviewer": "Reviewer_eeVQ", "summary": "This paper addresses the issue of true OOD samples having distribution shifts in scenarios of Long-tailed Recognition (LTR) with heavy imbalance. To cope with this, the paper proposes normalized outlier distribution adaptation (AdaptOD) with two key components: Dynamic Outlier Distribution Adaptation (DODA) and dual-normalized energy (DNE) loss. DODA dynamically adapts the outlier distribution during inference to align better with the true OOD distribution, while DNE helps achieve balanced prediction energy across imbalanced ID classes. This method was tested on various benchmark datasets, including both low- and high-resolution datasets.", "questions": "N/A", "limitations": "see weakness", "rating": 5, "confidence": 3, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "- The proposed method, reducing distribution shift of OOD on training and inference stage, is novel.\n- The paper includes a detailed ablation study that demonstrates the generality of the proposed methodology.\n- The paper's class-wise and sample-wise normalization techniques improve the balance in prediction energy for imbalanced ID samples, enhancing overall performance compared to existing methods.\n- This framework shows good experimental results compare to several baselines.", "weaknesses": "- The overview of Figure 2 is difficult to understand. Simplifying and highlighting only the necessary information would be beneficial. There are too many redundant details.\n- While recent methodologies improve long-tailed OOD detection performance using synthetic outliers without real outliers, this approach still relies on auxiliary outliers. This dependency can limit the practical applicability of the method.\n- The method's effectiveness is questionable when handling distribution shifts between training auxiliary outliers and actual real-world outliers, potentially compromising its performance if the shift is large.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your constructive feedback and appreciation of the novelty of our work and its empirical justification. We provide our response to your concerns one by one in the following.\n\n> **W1: The overview of Figure 2 is difficult to understand. Simplifying and highlighting only the necessary information would be beneficial.** \n\nThanks for your advice, we have provided a revised Figure 2 in the attached PDF and simplified the overview of Figure 2 as follows. \n\n- we will rewrite the DODA part as: Each test sample is assigned a global energy-based OOD score $\\mathbb G(x)$ to adapt the outlier distribution $\\mathcal{P}^{out}$. DODA then uses the adapted outlier distribution $\\mathcal{P}^{out}$ to calibrate the global energy score $\\mathbb G(x)$, obtaining the calibrated global energy score $\\mathbb G^{\\mathcal{P}}(x)$ as the OOD score. \n\n- we will also rewrite the DNE part as: For each iteration, DNE first applies Batch Energy Normalization on logit output to obtain the normalized energy, and then utilizes this energy to optimize a dual energy loss function at both the class and sample levels.\n\n> **W2: While recent methodologies improve long-tailed OOD detection performance using synthetic outliers without real outliers, this approach still relies on auxiliary outliers. This dependency can limit the practical applicability of the method.**\n\nAs far as we know, existing SOTA long-tailed OOD detection methods all take the outlier exposure (OE) approach, relying on the availability outlier data; non-OE-based methods still cannot work well on long-tailed ID data due to the heavy class imbalance. However, all these existing OE-based methods are challenged by the distribution gap between the distributions of outlier data and true OOD data. The proposed AdaptOD effectively tackles this common, crucial challenge by the novel DODA and DNE modules. We agree that there might be application scenarios where no auxiliary outlier data may be available. We will explore to extend our method to this setting by having the initial outlier distribution trained on synthetic outlier data, or even randomly initializing the outlier distribution. Specifically, we attempt to randomly Gaussian initializing which has each dimension sampled from an isotropic Gaussian distribution as the auxiliary outliers to train the OOD models as shown in Tab. E and Tab. F below.\n\nTable E： Results on CIFAR10-LT using randomly Gaussian initializing as auxiliary outliers\n\n|          | AUC $\\uparrow$ | AP-in $\\uparrow$ | AP-out $\\uparrow$ | FPR $\\downarrow$ |\n| :------: | :------------: | :--------------: | :---------------: | :--------------: |\n| EnergyOE |     84.65      |      85.27       |       82.66       |      52.70       |\n|   COCL   |     90.53      |      89.44       |       88.92       |      37.89       |\n| AdaptOD  |     93.06      |      92.75       |       92.87       |      29.31       |\n\nTable F： Results on CIFAR100-LT using randomly Gaussian initializing as auxiliary outliers\n\n|          | AUC $\\uparrow$ | AP-in $\\uparrow$ | AP-out $\\uparrow$ | FPR $\\downarrow$ |\n| :------: | :------------: | :--------------: | :---------------: | :--------------: |\n| EnergyOE |     71.85      |      72.37       |       68.61       |      80.27       |\n|   COCL   |     76.07      |      76.74       |       69.58       |      78.62       |\n| AdaptOD  |     79.21      |      81.25       |       74.89       |      70.48       |\n\n\n> **W3: The method's effectiveness is questionable when handling distribution shifts between training auxiliary outliers and actual real-world outliers, potentially compromising its performance if the shift is large.**\n\nTo mitigate the influence of a large distribution shift between training auxiliary outliers and actual real-world outliers during DODA, we utilize the statistic of training ID data to guarantee the effective optimization of outlier distribution in Eq. 3 during the testing phase, which effectively alleviates the reliance on the prior from the training auxiliary outlier data during the adaptation of DODA, thereby reducing the large adverse effects brought by the auxiliary outlier data that is highly different from the true OOD data. As a result, our AdaptOD can achieve SOTA performance over six large diverse OOD test datasets (including both the near-OOD dataset and the far-OOD dataset) and three synthetic OOD datasets as well on the CIFAR10/100-LT-based ID dataset benchmarks.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "3M4sunR6F7", "reviewer": "Reviewer_udMg", "summary": "The paper proposed a novel approach, namely normalized outlier distribution adaptation (AdaptOD), to tackle this distribution shift problem, where ID classes are heavily imbalanced, i.e., the true OOD samples exhibit very different probability distribution to the head and tailed ID classes from the outliers.  One of its key components is dynamic outlier distribution adaptation that effectively adapts a vanilla outlier distribution based on the outlier samples to the true OOD distribution by utilizing the OOD knowledge in the predicted OOD samples during inference. Further, to obtain a more reliable set of predicted OOD samples on long-tailed ID data, a novel dual-normalized energy loss is introduced in AdaptOD, which leverages class- and sample-wise normalized energy to enforce a more balanced prediction energy on imbalanced ID samples.", "questions": "see above", "limitations": "n/a", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The paper is written well and is easy to understand.\n\n2. The studied problem is very important.\n\n3. The results seem to outperform state-of-the-art.", "weaknesses": "1. It would comprehensive if the accuracy is reported on ID data\n\n2. I am curious why there is not a weight coefficient before DNE loss\n\n3. It is better to ablate on the number of outlier data used during adaptation", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your appreciation of the novelty of our approach, empirical justification, problem setting, paper clarity, as your invaluable feedback on further enhancing the work. Please find our one-by-one responses to your concerns below:\n\n> **W1: It would be comprehensive if the accuracy is reported on ID data**\n\nWe have reported the ID accuracy of all methods on CIFAR10-LT and CIFAR100-LT in Tab. 2 and on ImageNet-LT in Tab. 4. It is impressive that our AdaptOD achieves not only consistently substantial improvement over current SOTA methods on OOD detection metrics but also consistently better ID classification accuracy on all three widely-used ID classification benchmarks.\n\n\n\n> **W2: I am curious why there is not a weight coefficient before DNE loss**\n\nThis is mainly because the DNE loss works stably with the cross-entropy loss in Eq. 11. We provide some ablation study results of having a coefficient hyperparameter for regularizing the DNE loss in Tab. G below, from which it is clear that the DNE loss can work well within its importance hyperparameter set in [0.5, 2.0].\n\nTable G: Averaged AUC results of OOD detection on CIFAR10/100-LT for AdaptOD with different importance weights of the DNE loss\n\n|   Weight    |  0.5  |  1.0  |  1.5  |  2.0  |\n| :---------: | :---: | :---: | :---: | :---: |\n| CIFAR10-LT  | 94.51 | 94.69 | 94.65 | 94.56 |\n| CIFAR100-LT | 81.85 | 81.93 | 81.90 | 81.86 |\n\n\n\n> **W3: It is better to ablate on the number of outlier data used during adaptation**\n\nIf we understand your question correctly, the results in Fig. 3 should be the results you are looking for, where we report the performance of our proposed DODA and two existing TTA methods with an increasing number of OOD samples. It shows that our AdaptOD can utilize the test-time OOD data to adapt the true OOD distribution much faster and substantially more effective.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
We sincerely appreciate the reviewers' time and invaluable feedback. We are encouraged that there is a unanimous consensus among all reviewers highlighting our method's effectiveness through extensive experiments. The reviewers also appreciate the importance of the addressed problem (KD4x, t7zA, udMg) and the novelty of our method (KD4x, t7zA, eeVQ). Additionally, we are pleased that the reviewers found the paper clear and easy-to-understand (KD4x, t7zA, udMg)

We respond to each reviewer's comments in detail below. We will incorporate the reviewers' suggestions into the manuscript revisions, which we believe will significantly enhance the paper's quality. We also include an additional PDF containing a revised Figure 2, and visualization results on the near OOD dataset CIFAR .

To further demonstrate the improvement of our AdaptOD, we conduct extra experiments compared with RTL [Ref1]. We present the new results in Tab. A and Tab. B.

  Table A: Result of AdaptOD and RTL on CIFAR10-LT. 

  |         | AUC $\uparrow$ | AP-in $\uparrow$ | AP-out $\uparrow$ | FPR $\downarrow$ |
  | :-------: | :--------------: | :------: | :-------: | :----: |
  | AdaptOD | 94.69 | 93.89 | 94.12 | 27.26 |
  | RTL  | 92.69 | 92.73 | 92.34 | 30.68 |

  Table B: Result of AdaptOD and RTL on CIFAR100-LT.

  |         | AUC $\uparrow$ | AP-in $\uparrow$ | AP-out $\uparrow$ | FPR $\downarrow$ |
| :-------: | :--------------: | :----------------: | :-----------------: | :----------------: |
  | AdaptOD | 81.93          | 83.09            | 77.83             | 67.37            |
  | RTL | 79.58          | 81.06            | 75.04             | 72.34            |

**References**

- [Ref1] Fan, Ke, et al. "Test-Time Linear Out-of-Distribution Detection." Proceedings of CVPR, 2024.

Meta-review: This paper offers a novel approach to handling the significant challenge of distribution shifts in out-of-distribution (OOD) detection, especially under long-tailed recognition scenarios. The introduction of Dynamic Outlier Distribution Adaptation (DODA) and Dual-Normalized Energy Loss (DNE) is a commendable attempt to refine OOD detection mechanisms, supported by substantial experimental validation showing improved performance over existing methods.

However, the reviews and discussions reveal a mix of appreciation for the novelty and effectiveness of the proposed methods and concerns regarding certain aspects like the theoretical foundation of DODA and the practical applicability of the approach given its dependency on auxiliary outliers. The authors have responded thoroughly, attempting to clarify misunderstandings and expand on the methodology. Given the overall positive impact of the method demonstrated through rigorous experimentation and its potential to advance OOD detection practices, I recommend accepting this paper with necessary revisions.
Decision: Accept (poster)