Typicalness-Aware Learning for Failure Detection
Authors: Yijun Liu
Jiequan Cui
Zhuotao Tian
Senqiao Yang
Qingdong He
wangxiaoling
Jingyong Su
Venue: NeurIPS 2024 poster (2024)
Keywords: failure detection
overconfidence
typicalness-aware training
logit norm
Primary area: machine_vision
TL;DR: This paper introduces a novel typicalness-aware training approach for deep neural networks that helps mitigate overconfidence  in Failure Detection.

Abstract: Deep neural networks (DNNs) often suffer from the overconfidence issue, where incorrect predictions are made with high confidence scores, hindering the applications in critical systems. In this paper, we propose a novel approach called Typicalness-Aware Learning (TAL) to address this issue and improve failure detection performance. 
We observe that, with the cross-entropy loss, model predictions are optimized to align with the corresponding labels via increasing logit magnitude or refining logit direction. However, regarding atypical samples, the image content and their labels may exhibit disparities. This discrepancy can lead to overfitting on atypical samples, ultimately resulting in the overconfidence issue that we aim to address.
To address this issue, we have devised a metric that quantifies the typicalness of each sample, enabling the dynamic adjustment of the logit magnitude during the training process. By allowing relatively atypical samples to be adequately fitted while preserving reliable logit direction, the problem of overconfidence can be mitigated. TAL has been extensively evaluated on benchmark datasets, and the results demonstrate its superiority over existing failure detection methods. Specifically, TAL achieves a more than 5\% improvement on CIFAR100 in terms of the Area Under the Risk-Coverage Curve (AURC) compared to the state-of-the-art. Code is available at https://github.com/liuyijungoon/TAL.

Typicalness-Aware Learning for Failure Detection
Yijun Liu'
Jiequan Cui Zhuotao Tian 1D4
Senqiao Yang²
Qingdong He Xiaoling Wang' Jingyong Su L
(liuyijun)ssten hit.edu.cc.n
Harbin Institute of Technology (Shenzhen) "Nanyang Technological University
3he Chinese University of Hong Kong *Tencent Youtu Lab
Abstract
Deep neural networks (DNNs) often suffer from the overconfidence issue, where incorrect predictions are made with high confidence scores, hindering the appli- cations in critical systems. In this paper, we propose novel approach called Typicalness-Aware Learning (TAL) to address this issuc and improve failure detec- tion performance. We observe that, with the cross-entropy loss, model predictions are optimized to align with the corresponding labels via increasing logit magnitude refining logit direction. However, regarding atypical samples, the image content and their labels may exhibit disparities. This discrepancy can lead to overfitting atypical samples, ultimately resulting in the overconfidence issue that we aim to address. To tackle the problem, we have devised a metric that quantifies the typicalness of each sample, enabling the dynamic adjustment of the logit magnitude during the training process. By allowing atypical samples to be adequately fitted while preserving reliable logit direction, the problem of overconfidence can be mit- igated. TAL has been extensively evaluated on benchmark datasets, and the results demonstrate its superiority over existing failure detection methods. Specifically, TAL achieves more than 5% improvement CIFAR 100 terms of the Area Under the Risk-Coverage Curve (AURC) compared to the state-of-the-arr. Code is available hutp://gihib..ommiiiiinggg/////
Introduction
Failure detection plays vital role in machine learning applications, particularly in high-risk domains where the reliability and trustworthiness of predictions are crucial. Applications such as medical
diagnosis 8 autonomous driving I99 42 43], and other visual perception tasks [2 38
32
25| 35
30
37
29
36 require accurate assessments of prediction confidence before making critical decisions.
The goal of failure detection is to enhance the reliability and trustworthiness of predictions, ensuring that high-confidence predictions are relied upon while low-confidence predictions appropriately rejected (26 47). This is essential for maintaining the safety and effectiveness of these applications.
Indeed, deep neural networks (DNNs) trained using the cross-entropy loss often suffer from the issue of overconfidence. This leads unreliable confidence scores, which in turn hinder the effectiveness of failure detection methods. It is common for models to make incorrect predictions with high confidence scores, sometimes even close to 1.0. A recent study called LogitNorm (40) has shed light on this problem. reveals that the softmax cross-entropy loss can cause the magnitude of the logit vector continue increasing, even when most training examples correctly classified. This phenomenon contributes to the model's overconfidence.
To alleviate the overconfidence problem, LogitNorm 40 proposes assigning constant magnitude decouple the influence of the magnitude during optimization. The cross-entropy loss with the
X Corresponding Authors. Email: sanrhhuttao@hi..chu..oo sujingyon@@hit.cdu.com
38th Conference on Neural Information Processing Systems (NeurIPS 2024).

---

Alypical Sample
Deep Noural Network
Model Prediction
Direction towands Target
Human
Ideal direction (Humna Pescoption)
Canegory: Horse
Confidence: 0.95
NV
clf
a a=< t.<.ff>>>> I @e
(()()));)))))
log
41
el
Hoose
Figure 1: Illustration the motivation. We observe that directly aligning the predictions of atypical samples to the target label is not appropriate, causing overconfidence (horse with 95% confidence). Instead, the confidence should be aligned with the human perception. During training, the cross- entropy loss increases the magnitude [|] and adjusts their direction towards the target (represented by the angle ai.. Consider this example where an image of human body with a horse head is presented, the loss may optimize towards f2 in the blue box, which is not the ideal outcome direction. Instead, would be better to optimize towards f 1. rather than being biased towards either one, ensuring more balanced and unbiased representation and allowing for a more accurate estimation of confidence.
decoupled logit vector f = f(;; b) is defined as what follows:
ellfll.f
(CE(f.y) - log
(I)
E ellfll-A.
where θ is the parameters of the DNN model, x: is the input image with label y. the logit vector f is decoupled into the magnitude I[ and the directions f. Based on the decomposition of the logit vector in Eq. ( we can observe that the overconfidence issue could stem from either increasing [[ or decreasing (the angle between the directions of prediction and label) during training.
Key observation & Motivation. Following the exclusion of the logit magnitude's 's impact by LogitNorm [4], we posit that the risk of the overconfidence issue still arises from logit directions. Typical samples, which have clear contextual information, help models generalize well. However, optimizing the direction for ambiguous atypical samples can still cause overconfidence. In these cases, the labels do not match the image context well. Aligning the logit direction of atypical samples may still lead to high softmax scores near 1.0 which worsens the overconfidence problem.
According to previous work 51 44 the definitions of typical and atypical samples are based on their semantic similarity to most samples and the ease with which the model learns them. Specifically:
Typical samples are those that exhibit similarity to a majority of other samples at the semantic level. These samples possess typical features that are easier for deep neural networks to learn and generalize.
Atypical samples, on the other hand, differ significantly from other samples the semantic level. They pose a challenge for the model to generalize due to their uniqueness. These samples are often located near the decision boundary, causing the model to have higher uncertainty in making predictions for them.
In Fig.[] [ we present atypical example to illustrate the issue at hand. Despite the ground-truth label being a horse, the image depicts a horse with human body, which could reasonably be predicted as either human or a horse with a confidence score of around 50%. However, the model incorrectly predicts the image as horse with an excessively high confidence score of 95%. Upon examining
Eq. [ we observe that the confidence score is determined by two crucial factors: magnitude and direction. This prompts an important question regarding the decoupling of these factors determine which one is more reliable in accurately approximating real confidence. Addressing this inguiry is essential for effective failure detection.
Our approach. Based on the aforementioned observations, we propose a novel approach called Typicalness-Awar Learning (TAL). TAL dynamically adjusts the magnitudes of logits based on the typicalness of the samples, allowing for differentiated treatments of typical and atypical samples. By doing TAL aims to mitigate the adverse effects caused by atypical samples and emphasizes that the direction logits serves as a more reliable indicator of model confidence. In the blue dashed box of Fig. []
we provide an example that illustrates the impact of TAL on an atypical sample. The logit
vector could be changed from fa fI, indicating that the scores obtained with f for both "horse"

---

and "human" become nearly equal. This alignment better aligns with human perception, highlighting the effectiveness of TAL in improving model confidence estimation.
The proposed TAL approach is model-agnostic, making easily applicable to models with various architectures, such as CNN [1] and ViT [6]. Experimental results on benchmark datasets, including CIFARIO, CIFAR 100, and ImageNet, demonstrate the superiority of TAL. over existing failure detection methods. Specifically,, on CIFARION our method achieves significant improvement of more than 5% in terms of the Area Under the Risk Cisk-Coverege Curve (AURC) compared to the faat--ffthhe-att method 49).
In summary, the main contributions of this paper are as follows:
We propose a new insight that the overconfidence might stem from the presence of atypical samples, whose labels fail to accurately describe the images. This forces the models to conform to these imperfect labels during training, resulting in unreliable confidence scores.
In order to mitigate the issue of overfitring atypical samples, we introduce the Typicalness- Aware Learning (TAL.), which enables the identification and separate optimization of typical and atypical samples, thereby alleviating the problem of overconfidence.
Extensive experiments demonstrate the effectiveness and robustness of TAL. Besides, TAL has no structural constraints to the target model and is complementary to other existing failure detection methods.
2 Background and Preliminary
Prior to introducing our method, we present the background of Failure Detection (FD). Additionally, we highlight the distinctions between failure detection and two closely related concepts: Confidence Calibration (CC) and suttof-Distribution detection (OoD-D).
Failure Detection. Failure detection (FD) [8 26 49 50 aims to differentiate between correct and incorrect predictions by utilizing the ranking of their confidence levels. In particular, a confidence- rate function k(-) is employed to assess the confidence level of each prediction. High-confidence predictions are accepted, while low-confidence predictions are rejected. By using a predetermined threshold 5 E IR+ users can make informed decisions based on the following function g:
accept if <(x) 2 8,
g g
reject otherwise.
where x(-) denotes confidence-rate function, such as the maximum softmax probability.
Failure Detection vs. Confidence Calibration. Confidence calibration (CC) 33 23 27 45
primarily emphasizes the aligmment of predicted probabilities with the actual likelihood of correctness, rather than explicitly detecting failed predictions as in FD. The goal of CC is to ensure that the predictive confidence is indicative of the true probability of correctness:
P - --p") = p", Vp" E [0, 1].
This implies that when a model predicts a set of inputs x to belong to class y with a probability p", we would expect approximately p" of those inputs to truly belong to class y.
However, as observed by [49]. models calibrated with CC algorithms do not perform well in FD. Traditional metrics used to evaluate CC, such as the Expected Calibration Error (ECE (28)., do not accurately reflect performance in FD scenarios. Instead, alternative metrics like the Area Under the Risk-Coverage Curve (AURC 7 and the Area Under the Receiver Operating Characteristic Curve (AUROC (2)
are recommended for assessing FD performance.
Failure Detection vs. Dut-rf--istributtin Detection. While both -uitr--Distributionn Detection (OoD-D) and failure detection tasks aim to enhance confidence reliability, they have distinct objectives, as depicted in Fig. 2 OoD-D focuses on rejecting predictions of semantic shift while accepting in-distribution predictions. However, it does not explicitly address the rejection of cases affected by covariate shifts. Additionally, through empirical observations in Sec. 4 find that OoD-D methods are not well-suited for the Failure Detection task.
Traditional Failure Detection (Old FD) vs. New Failure Detection (New FD). Traditional failure detection methods [49] 50, 26 primarily focus on assessing the accuracy of predictions for in- distribution data. They also evaluate the discriminative performance of distinguishing correct and
3

---

OoD-D
Old setting FD
Purpose: New setting
Semantic shift
Semanntc shift
shift
shift
GOD-D
Old HDD
ID
New FD
ID
ID
Covariste Shift
Covariate spplt
Roject
Accept
Reject Fallure
Accept Success
Accept
Fallure
Success
In-Distribution test cases
Covariate Shifts test cases
Semantic Shifts tast cases
29
Examples
GT/Pred.
Airalane/Bird
Horse/Worse
Airp /Bird
Horse/Norse
X/Robot
X/Number
0OD-D
Accept
Accept
Reject
Reject
Old FD
Reject
Accopt
Reject
Accept
Accept
Reject
Accept
Reject
Reject
Incomee:
Decision Boundary OnD/ old FD new FD
Accept (keep) OcD / old FD/ From FD label
Reject (fiter) GoD / eld new ED label
ambiguous OoD TD new FD label
Unseen
Prediction
class
The differences between closely related tasks. The blue curve represents the decision boundary, and the shaded area in the figure indicates incorrect predictions. (a) illustrates the objective of OoD-D tasks to reject predictions with semantic shifts and accept in-distribution predictions, without concern for predictions with covariate shifts. (b) shows the old setting of FD tasks, accepting correct in-distribution predictions and rejecting incorrect out-of-distribution predictions. (c) displays the new setting of FD tasks, accepting correct in-distribution predictions and correct predictions with covariate shifts, while rejecting incorrect in-distribution predictions, incorrect predictions with covariate shifts, and predictions with semantic shifts. (d) illustrates examples of OoD-D, Old FD, and New FD tasks. A classifier trained on CIFARIO 21 is evaluated on 6 images under a whole range of relevant distribution shifts: For instance, the 3rd and the 4th images in grayscale depict an airplane and horse which encounter covariate shifts from that in the original CIFARIO. The 5th and the 6th images depict samples belonging to unseen categories with semantic shifts.
Figure 2:
incorrect predictions for covariate shift data based on confidence scores. While these approaches address certain aspects of distribution shifts, they overlook the semantic shifts.
To address the limitations of Oft-of-Dee Distribution Detection (OoD-D) and traditional failure detection (Old FD) methods, [18] proposes a new setting called New FD. The objective of New FD is to accept correct predictions for both in-distribution and covariate shift samples. while rejecting incorrect predictions for all possible failures, including in-distribution, covariate shift, and semantic shift samples. Compared to OoD-D and Old FD, it enables more effective decision-making in real world.
Method
In this section, present our proposed strategy called Typicalness-Aware Learning (TAL), as shown in Fig. First, in Sec. 3.1 I. Sec.
we address the shortcomings of existing training objectives and identify potential cause of overconfidence. Next, in Sec. 3.2 outline the methodology used to calculate the "typicalness" " of samples, enabling selective optimization and mitigating the negative impact of atypical samples. Finally, in Sec. 3.3 we introduce the TAL strategy, which incorporates the computed typicalness values for individual samples, resulting in improved performance for the failure detection task.
overfitting of atypical samples
3.1 Revisit the Cross-entropy Loss
In Eq. (T). the optimization the cross- cross-entropy loss involves either increasing the magnitude of the logits or aligning them better with the labels. However, LogitNorm [40] researchers have observed that as the training progresses and the model becomes more accurate classifying samples, it tends to generate significantly larger logit magnitudes, leading to overconfidence. To address this issue,

---

(p-10)11)
Features Queue (FQQ)) )
HFQ Updating
pop
update
My.oy oy
Prediction
Contidence: Cosine
Encoder
Feature
MILP
Logit
Iaput
P4.
Cu -haa TiCcrtff.s)
Typicalness Measure
vi. ri.ppical streate spsinats derction
train
Ti aipai, weakle spdinias dircctin
T-wware
Learning
(4-2-)--1)
(4-1-9/-1)
HFQ
The framework of TAL. During training, statistical information (mean H4j and variance a,) of features from correct predictions updates the Historical Features Queue (HFQ) time-step t. The typicalness measure is calculated by comparing these statistics between the current batch and the HFQ. This 7 influences the overall loss calculation, guiding the model to differentiate between atypical and typical samples. In the inference phase, TAL operates similarly to a model trained with conventional cross-entropy. Confidence is derived from the cosine similarity of the predicted logit direction, emphasizing our approach of using direction as a more reliable confidence metric. The framework distinguishes between typical (high and atypical (low r) samples, influencing the optimization process accordingly.
Figure 3:
LogitNorm introduces constant magnitude T in Eq. mitigate the problem:
Diogit (f.v) log
(4)
Σ efar
By keeping the magnitude constant in Eq. 4 the model places greater emphasis producing features that align more closely with the target label in terms direction, in order to minimize training loss. However, as shown in Fig. [ the presence of atypical samples with ambiguous content and labels may still cause overconfidence. Hence, it is imperative devise a method that allows for the differentiation and separate optimization of typical and atypical samples.
Drawing inspiration from the cognitive process of human decision-making making, it is reasonable to distinguish between typical and atypical samples by leveraging the knowledge acquired during the training phase. This approach allows us to effectively mitigate the negative effects of atypical samples, thereby preventing the occurrence of erroneous overconfidence.
3.2 Distinguish Typical and Atypical Samples
To differentiate typical samples from atypical ones, we introduce a method for evaluating typicalness and implement typicalness-aware learning (TAL). This approach entails calculating the mean and variance of the feature representations. Specifically, we calculate the mean and variance of each sample's feature channels based on insights from CORES (31). The insight stems from the observation that in-distribution samples show larger magnitudes (mean) and variations (variance) in convolutional responses across channels compared to OoD samples, which are a type of atypical sample. The mean response of OOD samples smaller than correct ID samples, as shown in Fig. 5 a). These statistical characteristies are subsequently compared to a set of historical data, representing typical samples, stored in structured queue known the "Historical Feature Queue" (HFQ). By comparing the statistical features of the input samples to those in the HFQ, we can quantify their typicalness.
Initialize and update HFQ. We commence the process by initializing the HFQ, denoted Q, with a predetermined size equivalent to the number of samples in the training dataset. This structured queue is responsible for retaining the mean and variance of feature representations for typical samples identified throughout the training phase.
To establish the initial state of the queue, we do not adopt the model trained from scratch. Instead, employ model that has been trained for few epochs, corresponding the total training epochs. This approach ensures the quality of the queue during the early stages of training. In this study, set À 0.05, which corresponds to 5% of the total training duration.
small portion (A) of
5

---

During this initialization phase, for each batch of data, we calculate the mean (f) and variance (a²) of the feature vectors for each correctly predicted sample. Each sample in the queue stores its statistical features, denoted as follows, given prediction y and the ground truth label y:
Q {(r, 0() Uss = y)
(5)
where μ4 and of represent the mean and variance of the feature vectors of the i-th sample, respectively.
Once initialized, the statistics (mean and variance) of accurately predicted samples in each batch are directly added the queue, shown Fig. 3 The queue has a fixed length of 20,000, and the ablation study is provided in Sec 4.3 The queue is updated using a First-In-Firs--Or (FIFO) approach, guaranteeing that it preserves representative assortment of typical samples observed throughout the training process, while also adapting to the evolving data distributions. We empirically find this simple strategy works well in experiments.
Typicalness assessment. To evaluate the typicalness T of a new sample, we first calculate the mean (Nnow) and variance (o of its features f. Subsequently, we compute the L2 distance d between the feature distribution the new sample, represented by Mnow and o2 news and the distributions of the features stored in the HFQ, denoted as v.) E Q. Finally, we normalize the resulting distance using min-max normalization to obtain the typicalness T.
d = min (1y.05))Q
W W(Hnews
(eenew knew), (1y;
(6)
dmnn
T=1 dmaz dmii
Where dmin and dmaa represent the minimum and maximum distances of samples in the batch. Eq. (7) normalizes the value of within the range of [0. 1]. then can serve as a indicator of sample typicalness. A high value suggests that the sample is highly typical compared to the historical data. Conversely, a low I value indicates an atypical or anomalous sample.
3.3 Typicalness-Aware Learning
Sec. 3.J highlights the potential negative impact of atypical samples on the training process. Building upon the insights provided in Sec. 3.2, now introduce Sypicalness-Awan Learning (TAL) in this section. TAL leverages the typicalncss to distinguish between typical and atypical samples during the optimization process. This approach aims to mitigate the issue of overconfidence that arises from the presence of atypical samples.
The training objective of TAL. The training objective of TAL is defined by incorporating an additional loss term LTAL. This is achieved by modifying the LogitNorm equation, denoted as Eq. to Eq. (8) where the samples æ are assigned with dynamic magnitudes T(T) based on typicalness т.
CF (f.z) = - log E G.TG)
(8)
Dynamic magnitude T(r). Given the upper bound Tiux and the lower bound Tmins the dynamic magnitude (T)) can be obtained via:
T(T) = Tmin + (1 - r) x (Tmax Tmin):
(9)
where we empirically set Tiux and Tiin to and 100, and they perform well on different benchmarks. ablationstudy study on different values is shown in Sec.
The ablation
Specifically, in Eq. 9 smaller magnitude T(T) will be assigned to typical samples with large t, and larger magnitude T(T) will be assigned to less typical samples with smaller T, enabling different treatments for typical/atypical samples. In this manner, for atypical samples, a higher value of TT)) reduces the influence that pulls them towards the label direction. This helps prevent their logit directions from being excessively optimized.
In other words, the inverse proportionality between T(r) and encourages the model to yield directions of f that are well-aligned with the labels for the typical samples with large r by assigning a small magnitude T [r)). Conversely, for atypical samples with small т, the directions are not required

---

to be as precise as the typical ones as the current T(r) is large, to mitigate the adverse impacts brought by the ambiguous label. To this end, the direction f can serve more reliable indicator of the model confidence.
The overall optimization. Fig. 3 iillustrates the TAL framework, showing both training and inference processes. During the training process, we utilize both the proposed TAL loss LIAL and cross- eross-entropy loss Lce as it exhibits stronger feature extraction capabilities than LogitNorm [40]. The overall loss is:
aal = 7 TAL (f,9) + (CBT))cB(f.2)
(10)
The TAL loss, denoted LTAL. is utilized to optimize the directions of reliable typical samples with large typicalness T, as well as potentially some atypical samples with small т.
The CE loss (not only relying on the CE loss, as it is regulated by 7) for atypical samples enables the optimization of both direction and magnitude. This may help reduce the adverse effects of atypical samples on the direction, enhancing the reliability direction as a confidence indicator. This ensures that the optimization force on the logit directions of atypical samples is weaker compared to that of typical samples. The inference process does not involve the calculation typicalness, and the only difference from the normal inference process is that our method uses Cosine as the confidence score.
To summarize, our proposed approach, indicated in Eq. (TO), enables models to selectively and adaptively optimize typical and atypical samples according to their typicalness values. This strategy enhances the reliability of feature directions indicators of model confidence, ultimately improving the performance on failure detection task.
4 Experiments
To evaluate the effectivvneess of the proposed Typicalness-Awart Learning (TAL) strategy, we conduct extensive experiments on various datasets, network architectures, and failure detection (FD) settings. More details such as the training configuration can be found in Appendix [A]
Datasets and models. We first evaluate the small-scale CIFAR-100 100 [2 dataset with SVHN [I] as its out-of oi-diitribttiinn (OOD) test set. To demonstrate scalability, we further conduct experiments
on large-scale ImageNet [5] using ResNet-50, with Textures R and WILDS (20) serving OOD data. For CIFAR- 100, we verify TAL's effectiveness across diverse architectures including ResNet [4)
WRNet 46). DenseNet (16), and the transformer-based DeiT-Small (39 Detailed experimental
results are provided in Appendix C
Three settings. We evaluate TAL under three different settings: Old FD setting, OOD detection setting, and New FD setting (detailed in Section 2) While Old FD distinguishes between correct and incorrect in-distribution predictions, and OOD detection identifies diit--f--isttittioonn samples, New FD setting aims to separate correctly predicted in-distribution samples from both mis- classified and -itoo-ddistribution samples. We maintain a 1:1 ratio between in-distribution and -utto--dist samples in testing, and also report results for Old FD and OOD detection settings for completeness.
Baselines. We compare our proposed TAL method against classical Maximum Softmax Probability (MSP), MaxLogit[[3], Cosine 48 Energy
24 Entropy (34]. Mahalanobis 4. Gradnorm [T7]. SIRC 41 and recent LogitNorm 44), OpenMix (50) and (FMFP) 49]. It is worth noting that FMFP
focuses on improving accuracy for failure detection.
Evaluation metrics. To comprehensively assess the performance of TAL in failure detection, we 49 9. including Area Under the Risk Risk-Coverage Curve (AURC), Area Under the Receiver Operating Characteristic Curve (AUROC), False Positive Rate at 95% True Positive Rate (FPR95).
adopt three widely recognized evaluation metrics [1]
4.1 Comparisions with the Slate-of-the-ard on CIFAR
Evaluation with CNN-based architectures. As shown in Tab. D our TAL strategy outperforms existing methods in New FD settings. Here are the key observations: 1) OoD methods like Energy and LogitNorm do not achieve satisfactory performance in the FD task. Please refer to Appendix B for explanation. 2) TAL consistently surpasses the baseline MSP and MaxLogit across various network architectures by a large margin. 3) In terms of comparison with FMFP, the prior SOTA
7

---

method in the field of FD, our analysis reveals that TAL complementary to the FMFP method and exhibits superior performance when combined with it. 4) Regarding Openmix, the metries presented demonstrate that it falls short when compared to TAL.
Table Evaluation results of the proposed TAL on CIFARI00.
Architecture
Method
Old setting FD
OOD Detection
New setting
ID-ACC
AURC,
FPR95
AUROCT
AURCE
FPRES,
AUROCT
AURCI
FPR95.
AUROCI
MSP
99.83
67.49
84.07
293.64
33.41
74.55
376.42
66.92
$4.00
72,01
Cosine
+8
96.53
65.15
84.42
271.13
78.50
79.31
361 87
56.23
86.93
72.01
Energy
24
135.85
74,66
77.20
275.
83.18
77.78
387.65 44
66.96
83.21
72.01
MasLogii 15
133.19
72.33
77.96
275.85
82.53
77.73
385.81
65.08
83.56
72.01
Eatropy
100.05
66.28
84.12
287.62
373.49
61.33
Mahulanobss
4
Gradnorm
369.86
35.30
490..1
98.17
49.26
679.48
42.76
72.01
ResNe:100 10110000 14]
SIRC
LagitNorm 30
100:56
125.59
66.37
72.87
84.01
79.31
287.95
235.50
73.25
81.03
$3.35
75.90
356.88
374. 12
55.30
61.22
87.50
84.65
72.01
70.34
OpenMix
30
85.66
63.82
85.25
542.16
87.03
69.21
406 80
70.37
80.25
73.68
85.36
87:15
TAL FMFF
73.16
64.82
85.51
215.52
78.61
81.59
320.73
55:22
$5.18
75.59
MSP MS
$7.60
60:13
87.47
264.67
80.04
79.41
321. 39
57.26
X7.23
78.55
Cosine 48
56.68
58.65
87.56
279.53
81.60
77.91
334.04
$7.96
86.08
78.55
Energ
24
66.33
65.64
84,73
257.78
78.93
80.83
323 27
58.70
87.07
78.55
MaxLaggi
64.85
62.70
85.
Eatropy
260.92
Mahalamobs
76,43
72.99
81.44
231.16
61.11
85.74
312.53
30.49
88.73
78.55
Godnorm
1+
134,85
76.20
68.51
345 28
75.61
75.57
405 68
65.55
77.46
78.55
WRNe
46
LogitNorm 20
SIRC []
59.57
78.86
65.56
63.38
86,70
82.68
2045445
259.93
77.99
58.98
89.10
80.50
291.87
320.35
+1.SS
56.93
92.03
87.45
78.55
77.15
50
50.05
245.36
56.65
89:50
245.10 10
75.71
81,42
250.23
TAL, w/FFF
44.21
58.77
88.40
224 55
71:32
85:31
27% 2X
47.14
91.05
80.98
MSP
79.06
63.62
85.
257 69
77.86
79.77
333 86
57.58
87.64
74.62
Cosine
18
82.36
63.65
84.72
250.14
77.01
81.48
332. 18
53.82
88.25
74.62
Enengy
80
78.
MaxLogn
240.06
Entropy
79.91
65.11
85.16
250.58
75.54
81.34
330.88
53.97
38.30
74.62
Mahulambis
4
159.95
95.30
64.54
208.3
$4.83
89.23
35% 90
63.22
84.93
74.62
Gradnorm
17
249.85
94.23
49.58
310.31
$0.45
73,08
483.52
83.21
68.79
74.62
DeaseNet 16
LogitNorm
SIRC
11
109.80
80.21
65.41
73.28
78.90
85.37
219.73
216.63
60.21
74.48
87.27
81.58
330.63
330.07
53.32
$8.36
75.80
74.62
OpenMix
59.52
87.29
TAL
73.28
60.90
86.51
250.78
75,46
81.35
325.70
52.05
88.73
75.14
FMFP 40
56.67
62.45
87:72
258.04
77.11
79.31
314.57
59.81
87.37
77.96
TAL w/ EMFF
60.52
63.65
86.57
224.62
63:43
$6.01
296.75
45.10
90.93
77.85
4.2
Comparisions with the Baseline on ImageNet.
Table 2: Evaluation results of the proposed TAL on ImageNet.
setting
setting
Archilecture
Method
IDACC
AURCL
FPR951
AUROCI
AURCJ
FPR95
AUROCH
AURC|
FPROS
AUROCS
Imagenel vs. Textures
MSP 15
72.73
63.95
18
301.27
46.01
87.21
351.26
49.64
86.99
76.13
Cosine 4v
50.64
Euergy
MaxLoppi 1
113.35
72.11
77.29
278.
34.1
90.57
349.3
41.59
88.1
76.13
ResNet50 13
Mahalanbis
Entmpy 34
4
208.22
74.51
96.19
67.07
54.23
K5.45
292
288.17
57.61
38.3
88.92
36.51
397.18
344.73
43.95
65.34
88.27
B0.22
76.13
76.13
Residul [
258.18
97.01
49.0
316.1
57.77
83.
431.12
65.55
77.12
76.13
Gradrom
63.67
346.42
TAL
61.66
61.93
87.11
290.5
17.66
87.51
338.45
50.11
88.29
76.13
TAL-SIRC
64.55
63.86
87.15
288. 23
46.91
87. 88
336 56
49.68
88.35
76.43
Imaggenet
WILDS
MSP
72.73
63.95
86.18
272.93
59.27
87.72
326 25
60.19
87.42
76.13
326.99
Energy M
118.66
76.33
75.81
235.88
37.67
94.22
318.89
45.27
90.60
76.13
Maxi. nggr
113.35
72.11
77.29
237.28
38.93
93.97
317.46
45.45
90.69
76.13
ResNet50 L3
Entropy 34
Mahalanoiis
208.22
74.61
96.19
67.07
54.23
85.48
261.11
259.01
$1.20
77.17
$5.20
90.44
316.26
382.46
$4.32
80.91
81.51
$9.47
76.13
76.13
Resichal
258.18
282.44
41.30
85.09
84.39
77.99
SIRC
41
72.91
63.67
86.11
267.33
52.17
$9.05
322.41
$4.43
88.46
76.13
TAL (ours)
64.55
61.93
$7.11
232.11
10.97
91.28
288.67
45.55
92.91
76.13

---

175
127
150
SAUNC Guroo Langen
150
125
120
113
100
102
-9
17.5
10
12.5
10.0
10
50
5.0
33.80
19600
10003
4030
3009
vs. Ti..
Tiue T...
(4) EAURCC vs. Quene Langth
Figure 4: (a) and (b) is the ablation study of Tmin. Tmax- And (c) is the ablation study on the length of the Historical Feature Queue.
To showcase the scalability of our approach, we present the results ImageNet in Table 2 It is obvious our TAL strategy consistently enhances the failure detection performance method, significantly improving the reliability of confidence. Notably, TAL. reduces the AURC by 3.7 and 11.6 points, indicating a better overall performance distinguishing between correct and incorrect predictions. It is worth noting that TAL achieves these impressive improvements while maintaining comparable accuracy to the MSP baseline.
baseline
Additionally, we present the Risk Risk-Coveran -overace (RC) curves (Fig. ((c) for both old and new FD task settings on ImageNet. The comparison between TAL and baseline RC curves demonstrates the effectiveness of our method. Fig. d further visualizes typical and atypical data examples. For the fish category in ImageNet, typical data includes common fish images, while atypical data comprises both rare fish images from ImageNet and out-of-distribution samples.
4.3 Ablation Study
The ablation study of key components. We conduct experimental ablations on the components of our TAL. loss with CIFAR The results are summarized in Table 3 With the dynamic magnitude T(T) strategy, achieve substantial enhancements in Failure Detection performance, which manifests the effectiveness of integrating typicalness-awes strategies into training approaches.
Table 3: Ablation of the key components. Best are bolded and second best are underlined. AURC and EAURC are multiplied by 10a the remaining metrics are percentages except ACC. "Fixed T" means the dynamic magnitude T(T) in TAL is not adopted.
Method
Setting
AURC
EAURC
AUROC
FPR85
TNR95
AUPR Success
AUPR Emur
ACC1
Fixed
108.46
83.87
92.21
68.35
Fixed T' Cross entropy
New FD
362.77
99.60
85.41
86.66
57.00
80.57
91.10
TAI, loes (Dynamic T) + Cross entmpy
New FD
Old FD
351.49
94.33
72.69
49.43
88.92
85.58
61.24
47.44
$2.46
38.69
93.56
83.03
92.86
68.70
0.71
0.72
The impacts of min and Tmxx. We perform an ablation study using ResNetl on the CIFARIO and CIFAR 100 datasets to examine the impact of Tmin and T Tmax failure detection performance. Fig. 4 (a) and Fig. (b) present the experimental results of the failure detection metric EAURC for 10 and CIFAR 100, respectively. Darker regions in the figures correspond to lower values of the metric, indicating superior failure detection performance. The findings suggest that while Tuin should not be set too small, moderate increase in T Tmax can enhance failure detection capabilities.
The effects of the length of Historical Feature Queue. We conduct ablation study the CIFAR 100 dataset to examine the impact of queue length on failure detection performance. The original C0F dataset consists of 50,000 training images, with 5,000 images reserved for validation and the remaining 45,000 images used for training. The results, depicted in Figure 4 (c), demonstrate that queue lengths ranging from 10,000 to 50,000 yield similar failure detection performance. However, when the queue length exceeds 50,000, there is a noticcable decline in failure detection performance.

---

AURC caree curinn the traerr
(a) Mean of features
(b) Typicalness mcasurement
Nykk Coverage Curse Old Setting
Byyk Coveroge Curse Now Setiing
Zuaiina
T4L
Fin
Coverage
Coverage
Cowera Paat
Sanado Tooue
(d) Typical and atypical examples Figure 5: (a) Comparison of the Mean of Features between ID and OOD; (b) Comparison different methods for measuring typicality; (c) The Risk-Coverage curves old and new setting FD tasks; (d) Examples of typical and atypical examples.
(c) curves
Ablation of Typicality Measures. As depicted in Fig. (b), we have conduced extra ablation experiments with K-nearest neighbor (KNN) distance and Gaussian Mixture Models (GMM) to assess typicality. These alternative measures did enhance performance (lower AURC preferable), thereby reinforcing the validity of our selection of mean/variance criteria.
Concluding Remarks
Summary. This paper introduces Typicalness-Awan Learning (TAL), a novel approach for mitigating overconfidence DNNs and improving failure detection performance. The effectiveness of TAL be attributed to crucial insight: overconfidence in deep neural networks (DNNs) may arise when models are compelled to conform to labels that inadequately describe the image content of atypical samples. To address this issue, TAL leverages the concept of typicalness to differentiate the optimization of typical and atypical samples, thereby enhancing the reliability of confidence scores. Extensive experiments have been conducted to validate the effectiveness and robustness of TAL. We hope TAL can inspire new ideas for further enhancing the trustworthiness of deep learning models.
The main contribution of TAL lies in recognizing the issue of overfitting atypical samples cause of overconfidence and proposing comprehensive framework tackle this problem. Given that the methods adopted in this work are simple yet effective, there is still potential for further improvement by incorporating calculation and the dynamic magnitude generation. These are left as future work to be explored.
Limitations.
advanced designs, such the methods for typicalness
Broader impacts. As deep learning models become increasingly integrated into critical systems, from autonomous vehicles to medical diagnostics, the need for accurate and reliable confidence scores is paramount. TAL's ability to improve failure detection performance directly addresses this need, potentially leading to safer and more dependable AI systems.
Acknowledgments
This work was supported by National Natural Science Foundation China (grant No. 62376068, grant No. 62350710797), by Guangdong Basic and Applied Basic Research Foundation (grant No. 2023B 1515120065, by Shenzhen Science and Technology Innovation Program (grant No. JCYJ202208 18102414031)
10

---

References
[1] Kendrick Boyd, Kevin H Eng. and C David Page. Area under the precision-recall curve: point estimates and confidence intervals. In Proceedings of the European Conference Machine Learning and Principles and Practice of Knowledge Discovery in Databases (ECML-PKDD), pages 451 456-466. Springer, 2013.
[2] Andrew P Bradley. The use of the area under the roc curve in the evaluation of machine learning algorithms. Pattern Recognition (PR), (0(7):1145-11555 1997.
[3] Mircea Cimpoi, Subhransu Maji, Iasonas Kokkinos, Sammy Mohamed. and Andrea Vedaldi. Describing textures in the wild. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 3606-3613, 2014.
[4] Roy De Maesschalck, Delphine mnaanRRibbuud, and Désiré L. Massart. The mahalanobis distance. Chemometrics and intelligent laboratory systems, 50(1):1-18, 2000.
[5] Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition (CVPR), pages 248-255, Icee, 2009.
[6] Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929 2020.
[7] Ran El- Yaniv et al. On the foundations of noise-free selective classification. Journal of Machine Learning Research (JMLR), 11(5), 2010.
[8] Andre Esteva, Brett Kuprel, Roberto A Novoa, Justin Ko, Susan M Swetter, Helen M Blau, and Sehastian Thrun. Dermatologist-level classification of skin cancer with deep neural networks. Nature, 542 2017.
[9] Ido Galil, Mohammed Dabbah, and Ran Yaniv. What can learn from the selective prediction and uncertainty estimation performance of 523 imagenet classifier? In Proceedings of the International Conference on Learning Representations (ICLR), 2023.
[10] Yonatan Geifman, Guy Uziel, and Ran El- Yaniv. Bias-reduced uncertainty estimation for deep neural classifiers. arXiv preprint 1805.
2018.
[11] Ian J Goodfellow, Yaroslav Bulatov, Julian Ibarz. Sacha Arnoud, and Vinay Shet. Multi-digit number recognition from street view imagery using deep convolutional neural networks. arXiv preprint arXiv: 1312.6082. 2013.
[12] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2016.
[13] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770-778, 2016.
[14] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Identity Mappings in Deep Residual Networks. In Proceedings of the Europeon Conference on Computer Vision (ECCV), volume 9908. pages 630-645. Springer International Publishing, Cham, 2016. Series Title: Lecture Notes in Computer Science
[15] Dan Hendrycks and Kevin Gimpel. A baseline for detecting misclassified and dut-o--distributtin examples neural networks. arXiv preprint arXiv: 16.0.0100222300 136, 2016.
[16] Gao Huang, Zhuang Liu, Laurens Van Der Maaten, and Kilian Q. Weinberger. Densely Connected Convolutional Networks. In Proceedings of the IEEE/CVE Conference on Computer Vision and Pattern Recognition (CVPR), pages 2261-2269, Honolulu, HI, July 2017. IEEE
11

---

[17] Rui Huang, Andrew Geng, and Yixuan Li. On the importance of gradients for detecting distributional shifts in the wild. Advances in Neural Information Processing Systems, 34:677- 689, 2021.
[18] Paul Jaeger, Carsten Tim Lüth, Lukas Klein, and Till J. Bungert. A call to reflect on evaluation practices for failure detection in image classification. In Proceedings of the International Conference on Learning Representations (ICLR), 2023.
[19] Joel Janai, Fatma Güney, Aseem Behl, Andreas Geiger, et al. Computer vision for autonomous vehicles: Problems, datasets and state of the art. Foundations and Trends@) in Computer Graphics and Vision (FOUND TRENDS COMPUT), 12(1-3):1-308, 2020.
[20] Pang Wei Koh, Shiori Sagawa, Henrik Marklund, Sang Michael Xie, Marvin Zhang, Akshay Balsubramani, Weihua Hu, Michihiro Yasunaga, Richard Lanas Phillips, Irena Gao, et al. Wilds: A benchmark of in-the-wild distribution shifts. In International conference on machine learning, pages 5637-5664. PMLR, 2021.
[21] A. Krizhevsky and G. Hinton. Learning multiple layers of features from tiny images. Master's thesis, Department of Computer Science, University of Toronto, 2009.
[22] Xin Lai, Zhuotao Tian, Yukang Chen, Yanwei Li, Yuhui Yuan, Shu Liu, and Jiaya Jia. Lisa: Reasoning segmentation via large language model. arXiv preprint arXiv:2308. 00692, 2023.
[23] Tsung-Yo Yi Lin, Priya Goyal, Ross Girshick, Kaiming He, and Piotr Dollár. Focal loss for dense object detection. In Proceedings of the IEEE International Conference on Computer Vision (ICCV), pages 2980-2988, 2017.
[24] Weitang Liu, Xiaoyun Wang, John Owens, and Yixuan Li. Energy-based Out-of-diitribution Detection. In Proceedings of the Advances in Neural Information Processing Systems (NeurIPS), volume 33, pages 21464-21475. Curran Associates, Inc., 2020.
[25] Xiaoliu Luo, Zhuotao Tian, Taiping Zhang, Bei Yu, Yuan Yan Tang, and Jiaya Jia. Pfenet++: Boosting few-shot semantic segmentation with the noise-filtered context-aware prior mask. IEEE Transactions on Pattern Analysis and Machine Intelligence, 46(2):1273-1289) 2024.
[26] Jooyoung Moon, Jihyo Kim, Younghak Shin, and Sangheum Hwang. Confidence- Aware Learning for Deep Neural Networks. In Proceedings of the International Conference on Machine Learning (ICML). pages 7034-7044. PMLR, November 2020, ISSN: 2640-3498.
[27] Rafael Müller, Simon Korbblith, and Geoffrey E Hinton. When does label smoothing help? Proceedings of the Advances in Neural Information Processing Systems (NeurIPS), 32, 2019.
[28] Mahdi Pakdaman Nacini, Gregory Cooper, and Milos Hauskrecht. Obtaining well calibrated probabilities using bayesian binning. In Proceedings of the AAAI conference on artificial intelligence (AAAT), volume 29, 2015.
[29] Bohao Peng, Zhuotao Tian, Xiaoyang Wu, Chenyao Wang, Shu Liu, Jingyong Su, and Jiaya Jia. Hierarchical dense correlation distillation for few-shot segmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2023.
[30] Bohao Peng, Xiaoyang Wu, Li Jiang, Yukang Chen, Hengshuang Zhao, Zhuotao Tian, and Jiaya
Jia. Oa-cnns: Omni-adaptive sparse cnns for 3d semantic segmentation. In CVPR, 2024.
[31] Keke Tang, Chao Hou, Weilong Peng, Runnan Chen, Peican Zhu, Wenping Wang, and Zhi- hong Tian. Cores: Convolutional response-based score for -fiss--dittiibttion detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10916-10925, 2024.
[32] Longxiang Tang, Zhuotao Tian, Kai Li, Chunming He, Hantao Zhou, Hengshuang Zhao, Xiu Li, and Jiaya Jia. Mind the interference: Retaining pre-trained knowledge in parameter efficient continual learning of vision- models. arXiv preprint arXiv:2407.05342 2024.
[33] Sunil Thulasidasan, Gopinath Chennupati, Jeff A Bilmes, Tanmoy Bhattacharya, and Sarah Michalak. On mixup training: Improved calibration and predictive uncertainty for deep neural networks. Proceedings of the Advances in Neural Information Processing Systems (NeurIPS), 32, 2019.
12

---

[34] Yu Tian, Yuyuan Liu, Guansong Pang, Fengbei Liu, Yuanhong Chen, and Gustavo Cameiro. Pixel-wise energy energy-biased abstention learning for anomaly segmentation on complex urban driving scenes. In Proceedings of the Europeon Conference on Computer Vision (ECCV), pages 246-263 Springer, 2022.
[35] Zhuotao Tian, Jiequan Cui, Li Jiang, Xiaojuan Qi, Xin Lai, Yixin Chen, Shu Liu, and Jiaya Jia. Learning context-aware classifier for semantic segmentation. In Proceedings of the Thirty- Seventh AAAI Conference on Artificial Intelligence, 2023.
[36] Zhuotao Tian, Xin Lai, Li Jiang, Shu Liu, Michelle Shu, Hengshuang Zhao. and Jiaya Jia. Generalized few-shot semantic segmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2022.
[37] Zhuotao Tian, Michelle Shu, Pengyuan Lyu, Ruiyu Li, Chao Zhou, Xiaoyong Shen, and Jiaya Jia. Learning shape-aware embedding for scene text detection. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition (CVPR), June 2019.
[38] Zhuotao Tian, Hengshuang Zhao, Michelle Shu, Zhicheng Yang. Ruiyu Li, and Jiaya Jia. Prior guided feature enrichment network for few-shot segmentation. TPAMI, 2020.
[39] Hugo Touvron, Matthieu Cord, Matthijs Douze, Francisco Massa, Alexandre Sablayrolles, and Hervé Jégou. Training data-efficient image transformers & distillation through attention, January 2021. arXiv:2012. 12877 [cs].
[40] Hongxin Wei, Renchunzi Xie, Hao Cheng, Lei Feng, Bo An, and Yixuan Li. Mitigating Neural Network Overconfidence with Logit Normalization. In Proceedings of the International Conference on Machine Learning (ICML), pages 23631-23644. PMLR, June 2022. ISSN: 2640-3498.
[41] Guoxuan Xia and Christos-Save -Svvaas Bouganis. Augmenting softmax information for selective classification with out-of-disrribttion data. In Proceedings of the Asian Conference on Computer Vision, pages 1995-2012, 2022.
[42] Senqiao Yang, Jiamiig Liu, Ray Zhang, Mingjie Pan, Zoey Guo, Xiaoqi Li, Zehui Chen, Peng Gao, Yandong Guo, and Shanghang Zhang. Lidar-llm: Exploring the potential of large language models for 3d lidar understanding. arXiv preprint arXiv:2312. 14074, 2023.
[43] Senqiao Yang, Zhuotao Tian, Li Jiang, and Jiaya Jia. Unified language-driven zero-shot domain adaptation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 23407-23415, 2024.
[44] Mert Yuksekgonul, Linjun Zhang, James Y Zou, and Carlos Guestrin. Beyond confidence: Reliable models should also consider atypicality. Advances in Neural Information Processing Systems, 36, 2024.
[45] Sukmin Yun, Jongjin Park, Kimin Lee, and Jinwoo Shin. Regularizing class-wise predictions via self- self-knowledge distillation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 13876-13885, 2020.
[46] Sergey Zagoruyko and Nikos Komodakis. Wide Residual Networks. In Proceedings of the British Machine Vision Conference (BMVC), York, France, January 2016. British Machine Vision Association, arXiv:1605.07 [cs].
[47] Yao Zhang, Cheng-Lin Liu, and Ching Y Suen. Towards robust pattern recognition: A review. Proceedings of the IEEE (P-IEEE), 108(6):894 922, 2020.
[48] Zihan Zhang and Xiang Xiang. Decoupling maxlogit for out-of out-of-distribution detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 3388-3397, 2023.
[49] Fei Zhu, Zhen Cheng, Yao Zhang, and Cheng-Lin Liu. Rethinking confidence calibration for failure prediction. In Proceedings of the Europeon Conference on Computer Vision (ECCV), pages 518-536. Springer, 2022.
13

---

[50] Fei Zhu, Zhen Cheng, Xu-Yao Zhang, and Cheng-Lin Liinng-Linnee Liu. Openmix: Exploring outlier samples for misclassification detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 12074-12083, 2023.
[51] Jianing Zhu, Hengzhuang Li, Jiangchao Yao, Tongliang Liu, Jianliang Xu, and Bo Han. Un- leashing mask: Explore the intrinsic out-of-distribution detection capability. In Proceedings of the International Conference on Machine Learning (ICML), pages 43068-43104, PMLR, 2023.
14

---

Appendix
Overview
This is the supplementary material for our submission titled Typicalness-Aware Learning for Failure Detection. This material supplements the main paper with the following content:
More Experimental Details
A.
Baselines
Evaluation Metries
Training Configuration
A.4
Test Configuration
The Reasons Why OoD method Performs Poor in FD
Additional Results
A More Experimental Details
A.1 Baselines
We compare our proposed TAL method against eleven baseline approaches for failure detection. ! Maximum Softmax Probability (MSP), 2 MaxLogit and 3 Cosine [48 utilize the maximum
softmax probability of the maximum logit (f) value and the cosine similarity between the f and
the corresponding label's one-hot vector as the confidence score, respectively. 4 Energy Score [24]
uses the negative energy of the softmax output as the confidence score. 2 LogitNorm 40 normalizes
the logits to a fixed magnitude to improve confidence score reliability. ④ Entropy
34 employs
the entropy of the softmax distribution as uncertainty measure. ① Mahalanobis
4 computes
confidence scores based on the Mahalanobis distance in the feature space. 8 Gradnorm 17 utilizes
the gradient norm of the loss with respect to the model parameters as a measure of uncertainty. OpenMix [50), SOTA OOD detection method, leverages data augmentation techniques to enhance confidence score separation between in-distribution and out-of-distribution samples. 40 SIRC [41] augments softmax-based confidence scores with feature-agnostic information to better identify OOD samples while maintaining separation between correct and incorrect ID predictions. O Failure Misclassification Feature Propagation (FMFP) [49] SOTA failure detection method, focuses improving model accuracy and confidence score reliability through stochastic weight averaging (SWA) and sharpness-avee minimization (SAM). We also explore combining the proposed TAL with FMFP (I)) to investigate their complementary effects.
A.2 Evaluation Metrics
To comprehensively assess the performance of TAL in failure detection, we adopt nine widely recog- nized evaluation metrics [I8 [9 including Area Under the Risk-Coverage Curve (AURC), Excess Area Under the Risk-Coverage Curve (EAURC), Area Under the Receiver Operating Characteristic Curve (AUROC), False Positive Rate at 95% True Positive Rate (FPR95), True Negative Rate at 95% True Positive Rate (TNR95), Area Under the Rrecision-Reeall curve of Suceess and Error SUURR_Succens and AUPR_EUPeeeeea ! AURC 7 Area Under the Risk Mosk-Coverage Curve, depicting the eттоr rate function of confidence thresholds. 2 EAURC 10:: Excess Area Under the Risk-Coverage Curve, evaluating the ranking ability of confidence scores. 3 AUROC [2]: Area Under the Receiver Operating Characteristic Curve, illustrating the trade-off between true positive rate (TPR) and false positive rate (FPR). 4: False Positive Rate at 95% True Positive Rate. O) and & AUPR AUPR_Success Success rep- curve (AUPR), with AUPR Success sampling thresholds from positive scores while AUPR Error utilizes only scores of observed positive and negative instances as thresholds. 9: Test accuracy, providing a reference for overall model performance.
True Negative Rate at 95% True Positive Rate. O resent two approximations for estimating the Area Under the Precision-
15

---

230
toincamen
siconeec
175
Gesmerre in homnec 3.330
DD coneer in hooren 02533
0 Icoreco
comeet at becreet 6:248
D tzeveo
160
125
12-
0.0
Figure 6: OOD-D methods lead to worse confidence separation between correct and wrong samples.
A.3 Training Configuration
For experiments on the CIFAR (21). we employ an SGD optimizer with initial learning rate of 0.1, a momentum of 0.9, and a weight decay of 0.0005. The models are trained for 200 epochs with a batch size of 256 on a single NVIDIA GeForce RTX 3090 GPU. Furthermore, we adopt CosineAnnealingLLR scheduler adjust the learning rate during training. On ImageNet [5], we use the ResNet-50 architecture as our backbone. The models are trained for 90 epochs with an initial learning rate of 0.1 on single NVIDIA A100. The learning rate is decayed by factor of 0.1 every 30 epochs.
A.4 Evaluation Configuration
To ensure fair and robust evaluation, we conduct experiments with three independent training runs on CIFAR100, each using different random seed. All baseline methods and our proposed TAL are evaluated using identical settings across these three runs. The final performance metrics reported in results averaged across these three sets of weights to account for training variance. For clarity of presentation, we multiply AURC values by 103, while maintaining all other metrics percentage form. This systematic evaluation approach allows to make reliable comparisons between different methods while accounting for the inherent variability in deep learning model training.
For ImageNet experiments, we encountered several implementation challenges with some baseline methods. Specifically, FMFP failed achieve competitive accuracy, while LogitNorm and OpenMix suffered from training instability and collapse. These issues might be attributed to the fact that these methods were not originally validated on ImageNet in their respective papers. While it might be possible to adapt these methods for ImageNet through extensive parameter tuning and modifications, such adaptations would require significant engineering effort and might deviate from the original methods. Therefore, we opted not to report results for these methods on ImageNet to maintain experimental integrity.
B The Reasons Why OoD method Performs Poor in FD
is interesting to note that in the Old Few-Shot Detection (FD) setting, most (OoD) methods, such as Energy Score and LogitNorm, exhibit poor performance compared to baseline methods like Maximum Softmax Probability (MSP) and MaxLogit. This decline in performance can be attributed to the fact that while OoD methods effectively increase the confidence score gap between in-distribution and out-of-distributin hierarchy within the in-distribution samples. As a result, there is a greater overlap in the confidence distributions of correctly and incorrectly predicted in-distribution samples, as illustrated in Fig. 6 This observation highlights the importance of developing dedicated FD methods that can effectively distinguish between correct and incorrect predictions within the in-distribution data.
samples, they inadvertently disrupt the natural confidence score
C
Additional Results
Evaluation on brasse architectures.
Considering the remarkable success of vision transformers as network architectures, is crucial to incorporate transformer-base network in our analysis and evaluate the effectiveness of our proposed
16

---

method. It is worth noting that the substantial disparities between ViT 66 and CNN architectures in terms of their design, feature representation, and learning mechanisms may pose challenges in directly applying methods that have proven effective on CNNs to ViT models.
Implementation Details of ViT. TAL. can also he applied to Transformer architectures. However, it is important to note that certain CNN- methods may not perform well on Transformer- tasks. For example, the accuracy of OpenMix with default settings is significantly lower (approximately 0.20) compared to the baseline. This indicates that OpenMix, originally designed for CNNs, does not effectively detect failures when applied to ViT models without appropriate modifications and adjustments that consider the unique architectural characteristics of ViT.
Specifically, ViT learns the relationships between image patches through the Self-Attention mech- anism, resulting in distinct feature representations and gradient flow patterns compared to CNNs. Moreover, ViT typically employs different normalization techniques, such as Layer Normalization, instead of the commonly used Batch Normalization in CNNs. These differences can impact the effectiveness of certain methods when applied to ViT. To successfully adapt these methods to ViT, appropriate modifications and adjustments may be necessary to account for the unique architectural characteristics of ViT.
To explore the performance of current popular failure detection methods on ViT models, we conducted experiments using the pre-trained Dei'--Smal model "'deit_small Lall___lallllll________ (atch) from the timm library. We employed the SGD optimizer with base learning rate of 0.01, weight decay of 0.0005, and momentum of 0.9. The learning rate scheduler was set to Cosine Asiialnnnn with the T max parameter determined by total training epochs. The experiments were conducted on the CIFARI00 dataset, with a total of 25 training epochs and batch size of 256.
Experimental Results on VIT. The experimental results shown in Tab. and Tab. 5 validate the applicability of TAL to vision transformers. By prioritizing optimization with typical samples and concurrently relaxing the requirements for atypical samples, TAL effectively mitigates overconfidence and enhances failure detection performance.
Table 4: New FD Setting evaluation on CIFARI00 with ViT. Mean and standard deviations of Failure Detection performance on CIFAR benchmarks. The experimental results are reported over five epochs. Best are bolded and second best underlined. AURC and EAURC multiplied by 103, the remaining metrics are percentages except ACC.
Method
AURC 4
EAURC
AUROC 1
IPR95
TNR95
AUPR-Success
AUPR Success
AUPR-
ACC
MSP [5]]
269.42.15.71
60.10.15.80
89.75.1.110
49.65.14.15
50.35 4.15
88.02.1.006
91.48-11.05
0,86.00.00
LogitNorm 40
268 268.20--9440 40
57.8449.90
91.25+0.99
17.78+2.18
62-19-2.17
88.03++2.05
93. 93.38-0611
0,86+00.00
TAL
262.57=7.60
53.3417822
91.6120.65
35.64:1.07
64.34-1005
19.01
93.57=0.28
0.87:0.00
FMFP
255. 29..89-2.99
50.37:3.55
09.09:00.9
45.11=3.19
54.86=3.21
90 90.02:+0:63
92.40=20.73
0.87+0.00
TAL w/ FMFP
246.07
39.96:12.89
03.17:1.76
31.84::2.53
68.15-2.46
9184110.6
94.41:0068
0.87100000
Table 5: Old Setting evaluation on CIFARIO0 with VIT. Mean and standard deviations of Failure Detection performance on CIFAR benchmarks. The experimental results are reported over five epochs. Best are bolded and second best underlined. AURC and EAURC are multiplied by 103. the remaining metrics are percentages except ACC.
Method
Setting
AURC L
EAURC 4
AUROC
FPR95
TNR951
AUPR-Succar Sacc--s
AUPR-Eme
ACC
MSP [5]]
Old FD
27.72-0.61
18.1710.61
88.88_00.3
56.71:12.18
13.26:2.117
97. 97.9610.07
14,361::43
0.86-0.00
LogilNorm 40
Old HD
27 0..
17.54:0.19
8911--0.26
56.991133 3.
42:98:1.355
9.003:40002
54.92::0.93
0.86-0.00
TAL.
27.15-0.33
17.62+0.2 20
89.83
58.65-1.66 1.62
44.28-1.60
98.03-0000
$4.47+0.3
0.87
FMFP 4B
Old FD
22.58.10331
14.30.10.88
90.11-0.41
54.35
45.62:13.27
98.41:10.05
54.67....0
0.87
TAL w/ EMEP
Old FD
21.02:0.332
14.55:10.21
88.79-0.54
53.00:12.43
46,99922.35
98.15:10.07
14.88.11.90
0.87-0000
17

---

NeurIPS Paper Checklist
1. Claims
Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Question:
Answer: [Yes]
The abstract and introduction clearly state the main contributions and scope of the paper, which are consistent with the theoretical and experimental results presented the body of the paper.
Justification:
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification:
The limitations of the proposed method are discussed in the "Limitations"
section.
Guidelines:
The answer NA means that the paper has limitation while the answer No means that paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification. asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested few datasets with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and faireess.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
18

---

Answer: [Yes]
Justification:
The paper provides theoretical analysis and justification for the proposed TAL.
Guidelines:
The answer NA means that the paper does include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should properly referenced.
4. Experimental Result Reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
Justification: The paper provides sufficient details on the experimental setup, datasets, and network architectures to reproduce the main results.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution a novel architecture, describing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of model checkpoint, other means that appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily new algorithm, the paper should make it clear how to reproduce that algorithm. (b) the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g., a large language model), then there should either way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reprodueibility may be tricky in some cases, in which authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
19

---

Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Question:
Answer: [Yes]
Justification:
The code and data will be made available upon publication.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips.. cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-soure benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https
/
for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: The paper specifies the training and test details, including data splits, hyperpa- rameters, and optimization settings.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to a level of detail that is necessary appreciate the results and make sense of them.
The full details can be provided either with the code, appendix, or as supplemental material.
7. Experiment Statistical Significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [Yes]
Justification: The paper reports mean and standard deviations of the experimental results multiple runs.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
20

---

The method for calculating the error bars should be explained (closed form formula, call a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that out of range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures tables in the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification:
The paper provides information the compute resources used for the experi- ments in the Experiments section.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code Of Ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https: http///eeeriip.eee /ttpoh/..rpp..//ooooo..oo/oooo/p
Answer: [Yes]
Justification:
The research conducted in the paper conforms to the NeurIPS Code of Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special cireumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., if there is a special consid- eration due to laws or regulations their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: In the Discussion section, the paper discusses both potential positive and negative societal impacts of the work performed.
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no societal impact why the paper does not address societal impact.
21

---

Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), faireess considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
If there negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: The proposed TAL. is focused on improving failure detection for general classification tasks without requiring the release of potentially unsafe data or models.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Lieenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: The paper properly credits and mentions the licenses for existing assets used.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms use in the package should be provided. For popular datasets, paperssithcode com com/datasets has curated licenses for some datascts. Their licensing guide can help determine the license of dataset.
22

---

For existing datasets that are re-packaged, both the original license and the license of the derived asset (if has changed) should be provided.
If this information is not available online, the authors are encouraged to reach out to the asset's s creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification: The paper introduces codes, which will be documented in the github link. Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember anonymize your assets (if applicable). You either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and sereenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification: The paper does not involve crowdsourcing or research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information tion of the paper involves human subjects, then as much detail as possible should be included the main paper.
supplemental material is fine, but if the main contribu-
According the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country institution) were obtained?
Answer: [NA]
Justification: The paper does not involve research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere
NeurlPS Code of Ethics and the
guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
23


Reviews:
[{"review_id": "cI7AVTiu1J", "reviewer": "Reviewer_RLx5", "summary": "The paper proposed a method for detecting incorrect prediction (Failure Detection). The main hypothesis behind the method is that cross-entropy either increases the logit magnitude or aligns the logit direction to align it with the ground label, which can cause discrepancy on atypical samples at test time. Previous methods, such as LogitNorm, only focussed on the logit magnitude; the proposed method tries to adaptively align training samples based on a metric that measures the typicalness of each sample. The paper also proposes a new setting, 'New FD', for evaluating failure detection methods. Extensive experiments were conducted on CIFAR-10/100, with some preliminary results on ImageNet (not benchmarked properly).", "questions": "1. In Sec 4.2, which model did you use? It is not clear from the manuscript. \n\n2. What are dynamic magnitudes?  Why do we need it?\n\n3. > In this manner, for atypical samples, a higher value \nof $T(\\tau)$ reduces the influence that pulls them towards the label direction\"\n\nCan you explain more how does T influence the direction?\n\n4. Explain more about equation 10. Why atypical samples optimized only by CE loss?", "limitations": "Yes", "rating": 6, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "1. The paper is well-written and easy to follow; the proposed method is intuitive and model-agnostic. \n\n2. Extensive experiments on CIFAR-10/100 show that the proposed method is effective for Failure Detection compared to the existing baselines.\n\n3. The new proposed setting of New-FD is interesting, providing a unified view of both OOD shift and covariate shift. \n\n4. An ablation study was conducted to show how hyper-parameters for the experiments were chosen.", "weaknesses": "While the proposed method looks promising, there are a few weaknesses and areas where the manuscript can be further improved:\n\n1. Most of the experiments were conducted on the CIFAR dataset (Table 1). Results on ImageNet are inconclusive; only MSP was used as a baseline for comparison. It would be helpful to add more baselines for comparison.\n\n\n2. Experiments on ViT were conducted on CIFAR. It is not clear why CIFAR was chosen as a dataset for ViT instead of ImageNet. \n\n3. As SSL-based models or fine-tuning pre-trained foundational models are becoming state-of-the-art models for various tasks, it is unclear if the proposed method can be applied in that setting. Experiments should be conducted on fine-tuning pre-trained foundational models.\n\n4. Some parts of section 3 are not clear:\n\n      a. How are mean and variance calculated for each sample feature? \n      b. Some samples can be incorrectly classified in some parts of the training; how do you handle such samples to update HFQ?\n      c. How is $d_{min}$ and $d_{max}$ are calculated?\n      d. Variables are not explained, e.g., equation 3.\n\n5. It is not clear how equation 10 works. $\\tau$ measures 'typicalness'; a value of 1 indicates a typical sample, and 0 indicates an atypical sample. The CE loss is weighted by $(1-\\tau)$, i.e., the atypical sample ($\\tau=0$) will only be trained by CE loss, which is counterintuitive. \n\n6. Authors only use SVHN for evaluating OOD shifts; systematic evaluation on the WILDS dataset [1] should be added. \n\n7. The need for the New-FD setting (detecting both is not explained, and the authors chose to evaluate the baselines in this setting. The experimental setup (New-FD) is unfair for baselines proposed to detect only OOD shifts (semantic shift). The proposed method should be evaluated for semantic and covariate shifts independently. \n\n[1] Koh et al., WILDS: A Benchmark of in-the-Wild Distribution Shifts", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for the valuable comments and suggestions.\n\n**Q1**: Most experiments were conducted on CIFAR. Results on ImageNet are inconclusive; only MSP was used as a baseline for comparison.\n\nThank you for the reviewers' comments. The question about ImageNet is a common concern among the reviewers. Due to character limits, the response to this important question can be found in the main rebuttal.\n\n**Q2**: Experiments on ViT were conducted on CIFAR. It is not clear why CIFAR was chosen as a dataset for ViT instead of ImageNet.\n\n We would like to humbly clarify that, previous works mainly conducted experiments on CIFAR and they did not report the results on Imagenet. Therefore, we adopted CIFAR for ViT for a fair comparison across all baselines. The experiments on ViT were made to demonstrate the model-agnostic nature of our method, showing its compatibility with both CNN and ViT architectures.\n\n**Q3**：As SSL and fine-tuning pre-trained models become state-of-the-art, it is unclear if TAL method applies. \n\nWe conducted experiments on fine-tuning pre-trained models by freezing the feature extractor and only training the classifier. However, the results were unsatisfactory. We hypothesize this is because our method essentially avoids overfitting to atypical features. The frozen feature extractor limits the effectiveness of our method, as only the classifier layer doesn't have enough model capacity.\n\nIn the discussion period, we will continue exploring fine-tuning all layers of the network to see if our method can still be effective. Going further, we will also explore combining our method with other OOD detection algorithms.\n\n**Q4**：Some parts are not clear:\na. How are mean and variance calculated for features? b. How do you handle incorrect samples to update HFQ? c. How is $\\tau$ and `d' are calculated? d. Variables are not explained, e.g., equation 3.\nThank you for your detailed feedback. We will clarify each point in our revised version:\n\na. We calculate the mean and variance of each sample's feature channels based on insights from CORES (CVPR 2024). This approach stems from the observation that in-distribution samples show larger magnitudes (mean) and variations (variance) in convolutional responses across channels compared to OoD samples, which are a type of atypical sample. As shown in Fig. 5(a), The mean response of OOD samples is smaller than correct ID samples.\n\nb. As stated in line 167 of the main paper, We update the Historical Feature Queue (HFQ) using a first-in-first-out method for correctly predicted samples. We discard the statistics of incorrectly predicted samples.\n\nc. \nWe will incorporate the following clarification to Eq. (7) in the revised version.\nFor each new batch of samples, we calculate a distance `d' for each sample. We then normalize these distances within the batch (dmin and dmax represent the minimum and maximum distances in the batch). This normalization is crucial as typicalness is relative, and we use it to control the strength of optimization direction.\n\nd. We will provide clearer explanations for all variables, including those in Eq. 3, in our revised version.\n\n**Q5**：It is not clear how equation 10 works. The atypical sample $(1-\\tau)$ will only be trained by CE loss, which is counterintuitive.\n\nOur approach tailors the optimization strategy for typical and atypical samples to alleviate overconfidence. \n\nIn particular, for typical samples, we prioritize direction optimization by enhancing the TAL loss optimization with a large $\\tau$. \n\nConversely, for atypical samples, the small $\\tau$ (i.e., large 1 - $\\tau$) emphasizes the optimization of the CE loss that considers \n both direction and magnitude. This may prioritize the magnitude and reduce the impact of atypical samples on the direction, making direction a more reliable confidence indicator.\n\nThank you and we will add the elaboration to the revision.\n\n**Q6**：only use SVHN for OOD shifts;  WILDS dataset [1] should be added.\n\nAs suggested by the reviewer, we have conducted additional experiments using the WILDS dataset. These results will be included in our revised paper. Due to the limited time, the result table will come in the discussion period.\n\n**Q7**: Evaluation for semantic and covariate shifts independently.\n\nThe New-FD setting was introduced in an ICLR 2023 paper, and we incorporated it into our related work section to offer insight into recent advancements in the field. The experiments regarding semantic (OOD) and covariate shifts(Old FD) independently with the evaluation of New FD are shown in Table 8 of rebuttal.pdf. \n\n**Q8**：\n1.In Sec 4.2, which model did you use? \n2.What are dynamic magnitudes? Why ?\n3.how does T influence the direction?\n4.Explain equation 10. \n\n1. Model:\nWe used ResNet50, and the training details were in the supplementary materials.\n\n2. Dynamic Magnitudes:\nDynamic magnitudes regulate the intensity of direction optimization. In contrast to LogitNorm, which employs a fixed logits vector magnitude, our approach adjusts T (magnitude) to modulate the loss function, as depicted in Equation 4. This enables us to dynamically control the optimization strength based on the typicalness of the sample.\n\n3. Influence of T:\nT controls optimization strength to mitigate overconfidence. For instance, in a binary classification scenario with the logit direction [√3/2, 1/2], raising T causes the sigmoid output approaching to 1, thereby decreasing the cross-entropy (CE) loss. Consequently, higher T values lead to a less intense optimization of the logit direction.\n\n4. Explain Equation 10:\nAs mentioned in our earlier response to Q5, emphasizing the CE loss (not only relying on the CE loss, as it is regulated by $\\tau$) for atypical samples enables the optimization of both direction and magnitude. This may help reduce the adverse effects of atypical samples on the direction, enhancing the reliability of direction as a confidence indicator.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "LiX0pb2HCQ", "reviewer": "Reviewer_9uPn", "summary": "This paper proposes a novel training method for improving the failure detection ability of classification models. The authors argue \"overconfidence\" may be in part due to overfitting of a model to the one hot labels of \"atypical\" samples. In order to mitigate this issue, a dynamic modification of the LogitNorm training method is proposed where the logit direction is more focused on for more \"typical\" samples. The proposed TAL is evaluated on image classification failure prediction benchmarks, showing promising performance on CIFAR data.", "questions": "1. As CRL requires training, but SIRC doesn't, and given the length of the rebuttal period, I would be happy with a comparison with just SIRC. \n2. I'd like to see a couple of risk-coverage curves to get a clearer idea of how TAL impacts failure detection (of just TAL vs MSP/CE).\n3. I would **require** ImageNet results with standard accuracies (~76%) which should be achievable using the recipe from the original resnet paper.\n4. Include these references.\n5. I would like to see some more ablations if possible (e.g. using a different measure of typicality, using static mixing of CE/TAL. A toy experiment (e..g 2D classification) showing how OOD samples/samples in lower density areas of the training distribution are assigned lower typicality would help a lot as well. An experiment such as finding atypical samples in the test set (using a model's predictive uncertainty) and then showing how those labels are overfitted when the test set is folded into the training set under CE but not under TAL would also improve the paper in my opinion.\n6. Please clarify this.\n\nDepending on the number of questions/weaknesses addressed I will be happy to raise my score up to 6.\n\nSome additional food for thought for the authors. Knowledge Distillation may also help in softening supervision on atypical samples. Perhaps it could be incorporated into future work. I also think it would be good to clarify that the authors' use of \"typical\" is different to that commonly found in information theory.\n\n\nAs the authors have successfully rebutted my queries, as well as considering the relative scarcity of effective approaches for the problem setting (compared to e.g. OOD detection), I have decided to raise my score to 7.", "limitations": "See above", "rating": 7, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "- I believe the problem setting of failure detection (Fig. 2) is understudied and it is important for more work to focus on it. \n- The results on CIFAR data are promising - TAL is able to perform well compared to a number of existing training-based approaches, both at detecting just misclassifications and mixed OOD + misclassifications. \n- The high-level idea/motivation of TAL, focusing on the difference between \"atypical\" samples, i.e. those on the tail of the training distribution, and optimising them differently is intuitive and appealing.", "weaknesses": "As the reviewing burden has been heavy for this conference (6 papers) please understand that I can only dedicate so much time to this paper. Thus, I may have made mistakes in my understanding of the paper, and I welcome the authors to correct me if this is the case. \n\n1. Missing comparisons. A number of methods that should be compared against are missing. CRL [1] (similar to TAL) is a learning-based approach for better misclassification detection, whilst SIRC [2] is a post-training method explicitly designed for rejecting both ID errors and OOD samples. It would also be better to clearly distinguish training and post-training approaches in the results.\n2. No risk-coverage curves presented. Although scalar scores show overall failure detection performance, plotting out RC curves in my opinion tells a lot more about the performance of an approach. \n3. Odd ImageNet results. The accuracy for ResNet-50 on this benchmark is far below the standard accuracy achievable using cross entropy training (~76%) and standard augmentations. Thus I am yet unconvinced that this approach is able to scale up to more realistic data. CIFAR is a rather special optimisation scenario where training accuracy converges to 100%. Models are typically extremely overparameterised and the data are much smaller scale than real-world CV applications. Thus, CIFAR results may not generalise to real applications. \n4. Missing references/attribution to existing work. There are a number of other published work that targets new failure detection that are not cited [2,3,4]. In particular [2] already discusses the behaviour described in Appendix B at length.\n5. Complexity of method. The actual method of TAL is quite complex, with a number of stages (d calculation, \\tau calculation), hyperparameters (Tmin/max, queue length) and ad hoc design choices (mean/variance calced along feature dimension, wasserstein distance, using \\tau to mix TAL and CE). This complexity limits ease of adoption, and makes it difficult to understand the reasons for TAL's efficacy. A more thorough ablation would greatly improve the understanding of the importance of each choice, aiding potential practitioners. Similarly, some intermediate experiments, (e.g. showing how error-rate changes according to \"typicalness\") would also make the method of TAL more convincing.\n6. Missing details. It is unclear to me what confidence score is used in the end for TAL. Moreover, how is \\tau set during inference? Is it fixed or is it calculated in the same way as during training? What is the OOD dataset for ImageNet?\n\n\nReferences\n\n[1] Moon et al., Confidence-Aware Learning for Deep Neural Networks\n\n[2] Xia and Bouganis, Augmenting Softmax Information for Selective Classification with Out-of-Distribution Data\n\n[3] Narasimhan et al., Plugin estimators for selective classification with out-of-distribution detection\n\n[4] Kim et al., A Unified Benchmark for the Unknown Detection Capability of Deep Neural Networks", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your valuable feedback on our paper.\n\n**Q1**: Missing comparisons. A number of methods that should be compared against are missing. As CRL requires training, but SIRC doesn't, and given the length of the rebuttal period, I would be happy with a comparison with just SIRC.\n\nThe comparison results with SIRC are shown in Tab. 8 (see rebuttal.pdf),  and we will add them to the final version. However, we are deeply sorry that we are unable to finish the results of CRL due to the limited time and resources during the rebuttal period.\n\n**Q2**：No risk-coverage curves presented. Although scalar scores show overall failure detection performance, plotting out RC curves in my opinion tells a lot more about the performance of an approach.\n\nThe risk-coverage curves are shown in Fig. 7(a), and we will them to the main paper.\n\n**Q3**：Odd ImageNet results. The accuracy for ResNet-50 on this benchmark is far below the standard accuracy achievable using cross entropy training (~76\\%) and standard augmentations. Thus I am yet unconvinced that this approach is able to scale up to more realistic data. CIFAR results may not generalise to real applications.\n\nWe apologize for the subpar ImageNet results in our initial submission. Due to time constraints, our focus was primarily on conducting comprehensive experiments on CIFAR, leading to insufficient training time allocated for ImageNet and subsequent underperformance.\n\nTo rectify this issue, we have conducted thorough experiments on ImageNet. The revised results are detailed in Tab. 8, showcasing the efficacy of our approach in scaling effectively to larger and more realistic datasets like ImageNet.\n\n**Q4**：Missing references/attribution to existing work. There are a number of other published work that targets new failure detection that are not cited [2,3,4]. \n\nThank you for your reminder. We have added these references in the revised manuscript.\n\n**Q5**：Complexity of method. The actual method of TAL is quite complex, with a number of stages (d calculation, $\\tau$ calculation), hyperparameters (Tmin/max, queue length) and ad hoc design choices (mean/variance calced along feature dimension, wasserstein distance, using $\\tau$ to mix TAL and CE). This complexity limits ease of adoption, and makes it difficult to understand the reasons for TAL's efficacy. A more thorough ablation would greatly improve the understanding of the importance of each choice, aiding potential practitioners. Similarly, some intermediate experiments, (e.g. showing how error-rate changes according to \"typicalness\") would also make the method of TAL more convincing.\n\nThe following provides additional explanations and experiments to address your concerns:\n\n- 1. Hyperparameters and Intermediate Variables:\nThe hyper-parameters include queue length, Tmin, and Tmax, with ablation studies provided. The intermediate variable d is normalized within each batch to derive $\\tau$, capturing the relative typicality during training.\n\n- 2. Feature Representation:\nWe selected the mean/variance based on insights from CORES (CVPR2024), indicating that ID samples exhibit greater magnitudes and variations in responses compared to OOD samples. Fig.  5 (a) visually represents the disparity in mean responses between ID (typical) and OOD (atypical) samples.\n\n- 3. Ablation of Typicality Measures:\nAs depicted in Fig. 5 (b), we have conduced extra ablation experiments with K-nearest neighbor (KNN) distance and Gaussian Mixture Models (GMM) to assess typicality. These alternative measures did not enhance performance (lower AURC is preferable), thereby reinforcing the validity of our selection of mean/variance criteria.\n\n- 4. Reasons for TAL's effectiveness: Our approach tailors the optimization strategy for typical and atypical samples to alleviate overconfidence.\nIn particular, for typical samples, our method prioritizes direction optimization by enhancing the TAL loss optimization with a large $\\tau$. \nConversely, for atypical samples, the small $\\tau$ (i.e., large 1 - $\\tau$) emphasizes the optimization of the CE loss that considers \n both direction and magnitude. This may prioritize the magnitude and reduce the impact of atypical samples on the direction, making direction a more reliable confidence indicator.\n\n\n- 5. Error Rate vs. typicality:\nWe have included an experiment illustrating the variation in error rates with typically (Fig.  6 (a)). This experiment offers valuable insights into the behavior of TAL.\n\nThank you and we will add the elaboration to the revision.\n\n**Q6**：Missing details. It is unclear to me what confidence score is used in the end for TAL. Moreover, how is $\\tau$ set during inference? Is it fixed or is it calculated in the same way as during training? What is the OOD dataset for ImageNet?\n\n- Confidence Score and Inference:\nIn the inference phase, TAL functions akin to a model trained with conventional cross-entropy. We calculate the cosine value of the predicted direction to derive the confidence score. This aligns with our focus on direction as a more dependable confidence metric, as highlighted in the introduction and the response to Q5-4 (Reasons for TAL's effectiveness).\n\n- Calculation of $\\tau$: $\\tau$ is not used during inference, but only used during training for adjusting the magnitude/direction optimization according to the sample typicalness, as mentioned the response to Q5-4.\n\n- OOD Dataset for ImageNet. We used the Textures dataset as the OOD dataset for ImageNet experiments.\n\nThank you and we will add the above clarifications to the final version.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "SIeMEMvEGU", "reviewer": "Reviewer_hbz4", "summary": "In this work, the authors propose a new approach to failure detection from DNN predictions. They suggest that data samples can be classified as either typical or atypical. The latter includes ambiguous, out of distribution samples or ill annotated data.\nIn order to circumvent this limitation, the authors introduce a new training objective which leverages a simple and effective set of features to determine whether a data sample is typical or not during both training and inference. In particular, they propose to store first and second order moment statistics from training samples and measure the Wasserstein distance from new samples to these distributions. This distance is then leveraged in order to weight the cross entropy logits. As a result, a large distance increases the logits magnitude and thus reduces the requirements from the model. This enables the use of the logits direction as a confidence metric.\nThe authors propose to evaluate the proposed method on Cifar10, Cifar100 and ImageNet for both CNNs ViT, showcasing the added value of the method in these experiments.", "questions": "My main question is: could the authors derive a small set of atypical and typical data (labelled set) from a known ood dataset for Imagenet such as [1] and measure the ability of the proposed method to actually separate typical vs atypical data?\ni believe this work would benefit from such validation\n\n[1] Galil I, Dabbah M, El-Yaniv R. A framework for benchmarking class-out-of-distribution detection and its application to imagenet. ICLR 2023", "limitations": "I believe the authors had a fair description of the current limitations of their work. I would suggest adding a word regarding the specificity of the training loss in the current presentation of the method.", "rating": 6, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "I am not very familiar with the field, but this work seems original to me and manages to achieve a non-negligible improvement over recent methods while remaining fairly simple.\nFurthermore, the proposed method does not appear to be specific to the model architecture but rather to the training loss which is very commonly used: the cross-entropy.\nThe current presentation of the paper is clear.", "weaknesses": "I have two minor concerns regarding this work:\n1. in its current form the empirical quantitative evaluation is not fully convincing as it is bounded to computer vision tasks and models with very few comparison points on large scale datasets.\n2. the authors insist on the intuition behind the added value with an explicit illustration in the motivation but I did not see an explicit evaluation of the typical vs atypical detection.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your valuable comments and kind words to our work.\n\n**Q1**： in its current form the empirical quantitative evaluation is not fully convincing as it is bounded to computer vision tasks and models with very few comparison points on large scale datasets.\n\nThank you and the results on ImageNet are presented in Tab.  8  (see rebuttal.pdf).  \n\nIt's worth noting that recent failure detection works, such as Openmix [CVPR2023] and FMFP [ECCV2022], primarily focused on CIFAR, and we followed the same setting for a fair comparison.\nHowever, the experiments on the large-scale ImageNet further demonstrate the effectiveness of the proposed TAL.\n\n**Q2**： the authors insist on the intuition behind the added value with an explicit illustration in the motivation but I did not see an explicit evaluation of the typical vs atypical detection.\n\nWe would like to clarify that, TAL does not explicitly identify typical or atypical samples but dynamically adjusts the training optimization process for different samples by calculating a value that reflects the degree of typicality, in order to alleviate the overconfidence issue.\nIn other words, TAL tailors the optimization strategy for typical and atypical samples.\n\nSpecifically, for typical samples, our method prioritizes direction optimization by enhancing the TAL loss optimization with a large $\\tau$. \n\n**Q3**：Could the authors derive a small set of atypical and typical data (labeled set) from a known ood dataset for Imagenet such as [1] and measure the ability of the proposed method to actually separate typical vs atypical data? i believe this work would benefit from such validation.\n\nWe have added a visualization of typical and atypical samples in the revised manuscript (as shown in Fig. 7 (b), which demonstrates the distinction between typical and atypical samples. Thank you for this valuable suggestion, and we believe this illustration may facilitate the understanding.\n\nConversely, for atypical samples, the small $\\tau$ (i.e., large 1 - $\\tau$) emphasizes the optimization of the CE loss that considers \n both direction and magnitude. This may prioritize the magnitude and reduce the impact of atypical samples on the direction, making direction a more reliable confidence indicator.\n\nThank you and we will add the above elaboration to the revision.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "s8c3906NFZ", "reviewer": "Reviewer_yqWV", "summary": "This paper identifies overfitting of atypical samples as a potential casue of overconfidence in DNN for Failure Detection.\nThe authors proposes a Typicalness-Aware Learning (TAL) approach that computes a typicalness score of each sample. TAL assigns dynamic logit magnitudes based on typicalness to allow flexible fitting of atypical samples while preserving reliable directions as confidence for atypical and typical samples.", "questions": "1. In Section 3.2 \"We empirically find this simple strately works well in our experiments.\" Please explain why the mean and variance can represent the features of a sample to measure typicality?\n2. The confidence calibration methods aim to achieve more reliable confidence estimates, but fail in failure detection task. Could please explain that?\n3. The OoD data and ID data are presented in equal proportions. How does the ration of them affect the results.", "limitations": "Discussed", "rating": 7, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "1. This paper proposes a novel perspective - overconfidence may be caused by models being forced to conform to labels that fail to accurately describle the image content of atypical samples.\n2. This paper introduces TAL which distinguishes between optimizing typical and atypical samples, thereby improving the reliability of confidence scores.\n3. TAL is model-agnostic and can be readily applied to Transformers and CNNs, demontrating the strong adaptability of TAL.\n4.  TAL is complementary to existing failure dectection methods and can be conbined for further performance gains.", "weaknesses": "1. The definition of atypical samples is vague, making it difficult to accurately determine the typicality of samples, which may affect the method's effectiveness. Please consider providing some mathematical definitions and visual examples.\n2. There is room for improvement in the specific implementation, such as typically calculation and dynamic magnitude generation, by exploring more advanced designs.\n3. The paper only compare failure detection task; the impact of TAL on classification accuracy is not analyzed ( accuracy is noly provided in the table but not discussed).\n4. Minor error: \"pred\" and \"GT\" in Fig.2 should be swapped.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for the valuable comments and suggestions.\n\n**Q1**：The definition of atypical samples is vague, making it difficult to accurately determine the typicality of samples, which may affect the method's effectiveness. Please consider providing some mathematical definitions and visual examples.\n\nThank you for your comments, and we have added the mathematical definition of typicality from the NeurIPS 2023 [ref1] in the revised version of our manuscript.\n\nFurthermore, we present the visual examples in Fig. 7 (b). These examples may aid in clarifying the distinction between typical and atypical samples.\n\n[ref1] Yuksekgonul, \"Beyond Confidence: Reliable Models Should Also Consider Atypicality,\" NeurIPS 2023.\n\n**Q2**：There is room for improvement in the specific implementation, such as typically calculation and dynamic magnitude generation, by exploring more advanced designs.\n\nAs depicted in Fig. 5 (b), we have conduced extra ablation experiments with K-nearest neighbor (KNN) distance and Gaussian Mixture Models (GMM) to assess typicality. These alternative measures did not enhance performance (lower AURC is preferable), thereby reinforcing the validity of adopting the mean and variance.\n\nMoreover, it's important to note that KNN, GMM, and density-based methods may entail higher computational costs compared to our approach, given the high efficiency of utilizing mean and variance in the proposed method. \n\nThank you for your insightful feedback. Since the fundamental mean and variance calculation already yields satisfactory performance in our method, delving into more advanced designs holds promise for further enhancing our method in the future.\n\n**Q3**：The paper only compare failure detection task; the impact of TAL on classification accuracy is not analyzed ( accuracy is noly provided in the table but not discussed).\n\nThank you for highlighting the importance of conducting a more thorough analysis of the impact of our method. In the revised version of our paper, we have incorporated the discussion on classification accuracy:\n- We have analyzed the impact of TAL on classification accuracy across different datasets.\n- We have discussed the relationship between improved failure detection and classification performance.\n- We have compared these results with baseline methods.\n\nThis supplementary analysis will offer a more comprehensive view of the advantages of TAL, demonstrating its value not only for failure detection but also for the overall model performance.\n\n**Q4**：Minor error: \"pred\" and \"GT\" in Fig.2 should be swapped.\n\nWe have corrected this typo in the revised manuscript.\n\n**Q5**：In Section 3.2 \"We empirically find this simple strately works well in our experiments.\" Please explain why the mean and variance can represent the features of a sample to measure typicality?\n\nWe selected the mean/variance based on insights from CORES (CVPR2024), indicating that ID samples exhibit greater magnitudes and variations in responses compared to OOD samples. Fig.  5 (a) visually represents the disparity in mean responses between ID (typical) and OOD (atypical) samples.\n\nMoreover, Fig. 6 (b) depicts the correlation between typicality and density utilizing mean and variance, suggesting that typicality can serve as a substitute for density. Density, calculated by Gaussian KDE (Kernel Density Estimation), represents the likelihood of observing a data point within the distribution.\n\n**Q6**：The confidence calibration methods aim to achieve more reliable confidence estimates, but fail in failure detection task. Could please explain that?\n\nThank you for raising this important question about the difference between confidence calibration and failure detection. This distinction is crucial for understanding the limitations of traditional confidence calibration methods in failure detection tasks.\n\nConfidence calibration methods are designed to ensure that predicted confidence levels align with actual accuracy rates. For instance, in a perfectly calibrated model, out of 10 samples predicted with a confidence level of 0.9, we would expect 9 to be correct and 1 to be incorrect. While this alignment is crucial for calibration purposes, it may not suffice for effective failure detection.\n\nIn failure detection tasks, our objective differs. We aim for high-confidence predictions to be consistently accurate. The occurrence of even one inaccurate prediction among high-confidence samples poses a significant issue. This is due to:\n\n- Failure detection requires the identification of all potential errors, including those within high-confidence predictions.\n\n- In critical applications, it is imperative not to overlook any failures, irrespective of the overall calibration performance.\n\nOur approach, TAL, mitigates this issue by enhancing the reliability of high-confidence predictions, particularly tailored for failure detection tasks. TAL enables us to enhance the detection of potential failures, even in scenarios where conventional calibration metrics indicate the model is well-calibrated.\n\n**Q7**：The OoD data and ID data are presented in equal proportions. How does the ration of them affect the results.\n\nThank you for raising this concern. To tackle this issue, we have conducted experiments in three settings: new failure detection, out-of-distribution (OOD) detection, and old failure detection.\n\nOur results indicate the following:\nTAL outperforms baseline methods in all three settings;\nFor OOD detection, specialized methods perform better than TAL;\nHowever, most OOD-specific methods exhibit reduced effectiveness in general failure detection tasks.\n\nThe robust performance of TAL across all settings underscores its versatility as a solution for various failure detection scenarios. As for the data proportion, we observed that it does not affect our method much. A detailed analysis of these findings will be included in the revised manuscript.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
Thanks to all reviewers and ACs for the valuable comments and suggestions. We appreciate that reviewers described our work as `` well-written, intuitive, effective". We are grateful for the reviewers' positive evaluation of our work as ``well-written, intuitive, effective". Each reviewer's feedback has been carefully addressed individually. The manuscript has been revised in accordance with the suggestions provided. Here is a summary of the main concerns raised by the reviewers:


**Comparison**: Most experiments were conducted on CIFAR. Results on ImageNet have added; 

- As suggested by the Reviewer, we have expanded our evaluation on ImageNet to include additional baselines, as shown in Table 8 of 
 rebuttal.pdf. The evaluation is independently performed on the three settings. The results demonstrate that our approach consistently outperforms existing baselines on ImageNet, aligning with our findings on CIFAR.

- Please note that, because other methods in the community did not report the performance on ImageNet, for a fair comparison, all experiments conducted on ImageNet utilize the same hyper-parameter settings as those used on CIFAR. It is worth mentioning that models with LogitNorm(ICML2022), Openmix(CVPR2023) cannot decently converge on ImageNet, and FMFP(ECCV2022) only achieved 60\% accuracy. Differently, the introduced TAL consistently exhibits enhancements, showcasing its resilience across various data domains.


**Why is TAL effective**: why use mean and variance to represent a sample for measuring typicality?
 We added several visualizations to explain our method's effectiveness (see rebuttal.pdf), including:

- Mean of Features Comparison: ID samples have a higher mean value compared to OOD samples (a type of atypical sample).

- Density and Typicality Relationship: Higher typicality corresponds to lower density, but typicality is faster and more resource-saving.

- Error-over-Typicality Curve: Samples with low typicality show more errors.

- Risk-Coverage Curves.

- Ablation of Other Typicality Measures. 


We sincerely appreciate the valuable suggestions and insightful comments given by the reviewers. The authors look forward to further discussions and are willing to address any of your concerns.

Thanks again.

Best regards,

Authors of paper 10758

Meta-review: There was consensus from the reviewers on the work being well-motivated, well-written and the methodology being intuitive. Much of the reviewer's concerns pre-rebuttal centred on the ImageNet results, the small number of such results, and the poor results. Other concerns included missing references/comparisons with existing work. However, significant work by both the authors and reviewers during the rebuttal period discussion appears to have addressed these concerns for the reviewers, in particular with much improved ImageNet results.
Decision: Accept (poster)