Typicalness-Aware Learning for Failure Detection

Authors: yijun  liu
jiequan  cui
zhuotao  tian
senqiao  yang
qingdong  he
xiaoling  wang
jingyong  su
Venue: NeurIPS_cc_2024_Conference
Published: 2024-05-14 16:36:19

Abstract: Deep neural networks (DNNs) often suffer from the overconfidence issue, where incorrect predictions are made with high confidence scores, hindering the applications in critical systems. In this paper, we propose a novel approach called Typicalness-Aware Learning (TAL) to address this issue and improve failure detection performance. 
We observe that, with the cross-entropy loss, model predictions are optimized to align with the corresponding labels via increasing logit magnitude or refining logit direction. However, regarding atypical samples, the image content and their labels may exhibit disparities. This discrepancy can lead to overfitting on atypical samples, ultimately resulting in the overconfidence issue that we aim to address.
To address this issue, we have devised a metric that quantifies the typicalness of each sample, enabling the dynamic adjustment of the logit magnitude during the training process. By allowing relatively atypical samples to be adequately fitted while preserving reliable logit direction, the problem of overconfidence can be mitigated. TAL has been extensively evaluated on benchmark datasets, and the results demonstrate its superiority over existing failure detection methods. Specifically, TAL achieves a more than 5\% improvement on CIFAR100 in terms of the Area Under the Risk-Coverage Curve (AURC) compared to the state-of-the-art. Code is available at https://github.com/liuyijungoon/TAL.

Main paper: 
 introduction: 
 Failure detection plays a vital role in machine learning applications, particularly in high-risk domains where the reliability and trustworthiness of predictions are crucial. Applications such as medical diagnosis [8] , autonomous driving [18, 41, 42] , and other visual perception tasks [21, 37, 31, 24, 34, 29, 36, 28, 35] require accurate assessments of prediction confidence before making critical decisions. The goal of failure detection is to enhance the reliability and trustworthiness of predictions, ensuring that high-confidence predictions are relied upon while low-confidence predictions are appropriately rejected [25, 46] . This is essential for maintaining the safety and effectiveness of these applications. Indeed, deep neural networks (DNNs) trained using the cross-entropy loss often suffer from the issue of overconfidence. This leads to unreliable confidence scores, which in turn hinder the effectiveness of failure detection methods. It is common for models to make incorrect predictions with high confidence scores, sometimes even close to 1.0. A recent study called LogitNorm [39] has shed light on this problem. It reveals that the softmax cross-entropy loss can cause the magnitude of the logit vector to continue increasing, even when most training examples are correctly classified. This phenomenon contributes to the model's overconfidence.To alleviate the overconfidence problem, LogitNorm [39] proposes assigning a constant magnitude to decouple the influence of the magnitude during optimization. The cross-entropy loss with the Figure 1 : Illustration of the motivation. We observe that directly aligning the predictions of atypical samples to the target label is not appropriate, causing overconfidence (horse with 95% confidence). Instead, the confidence should be aligned with the human perception. During training, the crossentropy loss increases the magnitude ∥f ∥ and adjusts their direction towards the target (represented by the angle α). Consider this example where an image of a human body with a horse head is presented, the loss may optimize towards f 2 in the blue box, which is not the ideal outcome direction. Instead, it would be better to optimize towards f 1 , rather than being biased towards either one, ensuring a more balanced and unbiased representation and allowing for a more accurate estimation of confidence. decoupled logit vector f = f (x; Θ) is defined as what follows:EQUATION)where Θ is the parameters of the DNN model, x is the input image with label y, the logit vector f is decoupled into the magnitude ∥f ∥ and the directions f . Based on the decomposition of the logit vector in Eq. ( 1), we can observe that the overconfidence issue could stem from either increasing ∥f ∥ or decreasing α (the angle between the directions of prediction and label) during training.Key observation & Motivation. Following the exclusion of the logit magnitude's impact by LogitNorm [39] , we posit that the risk of the overconfidence issue still arises from logit directions. Typical samples, which have clear contextual information, help models generalize well. However, optimizing the direction for ambiguous atypical samples can still cause overconfidence. In these cases, the labels do not match the image context well. Aligning the logit direction of atypical samples may still lead to high softmax scores near 1.0 which worsens the overconfidence problem.According to previous work [50, 43] , the definitions of typical and atypical samples are based on their semantic similarity to most samples and the ease with which the model learns them. Specifically:• Typical samples are those that exhibit similarity to a majority of other samples at the semantic level. These samples possess typical features that are easier for deep neural networks to learn and generalize. • Atypical samples, on the other hand, differ significantly from other samples at the semantic level. They pose a challenge for the model to generalize due to their uniqueness. These samples are often located near the decision boundary, causing the model to have higher uncertainty in making predictions for them. In Fig. 1 , we present an atypical example to illustrate the issue at hand. Despite the ground-truth label being a horse, the image depicts a horse with a human body, which could reasonably be predicted as either a human or a horse with a confidence score of around 50%. However, the model incorrectly predicts the image as a horse with an excessively high confidence score of 95%. Upon examining Eq. (1), we observe that the confidence score is determined by two crucial factors: magnitude and direction. This prompts an important question regarding the decoupling of these factors to determine which one is more reliable in accurately approximating real confidence. Addressing this inquiry is essential for effective failure detection.Our approach. Based on the aforementioned observations, we propose a novel approach called Typicalness-Aware Learning (TAL). TAL dynamically adjusts the magnitudes of logits based on the typicalness of the samples, allowing for differentiated treatments of typical and atypical samples. By doing so, TAL aims to mitigate the adverse effects caused by atypical samples and emphasizes that the direction of logits serves as a more reliable indicator of model confidence. In the blue dashed box of Fig. 1 , we provide an example that illustrates the impact of TAL on an atypical sample. The logit vector could be changed from f 2 to f 1 , indicating that the scores obtained with f for both "horse" and "human" become nearly equal. This alignment better aligns with human perception, highlighting the effectiveness of TAL in improving model confidence estimation.The proposed TAL approach is model-agnostic, making it easily applicable to models with various architectures, such as CNN [12] and ViT [6] . Experimental results on benchmark datasets, including CIFAR10, CIFAR100, and ImageNet, demonstrate the superiority of TAL over existing failure detection methods. Specifically, on CIFAR100, our method achieves a significant improvement of more than 5% in terms of the Area Under the Risk-Coverage Curve (AURC) compared to the state-of-the-art method [48] .In summary, the main contributions of this paper are as follows:• We propose a new insight that the overconfidence might stem from the presence of atypical samples, whose labels fail to accurately describe the images. This forces the models to conform to these imperfect labels during training, resulting in unreliable confidence scores. • In order to mitigate the issue of overfitting on atypical samples, we introduce the Typicalness-Aware Learning (TAL), which enables the identification and separate optimization of typical and atypical samples, thereby alleviating the problem of overconfidence. • Extensive experiments demonstrate the effectiveness and robustness of TAL. Besides, TAL has no structural constraints to the target model and is complementary to other existing failure detection methods. 
background and preliminary: 
 Prior to introducing our method, we present the background of Failure Detection (FD). Additionally, we highlight the distinctions between failure detection and two closely related concepts: Confidence Calibration (CC) and Out-of-Distribution detection (OoD-D).Failure Detection. Failure detection (FD) [17, 25, 48, 49] aims to differentiate between correct and incorrect predictions by utilizing the ranking of their confidence levels. In particular, a confidencerate function κ(•) is employed to assess the confidence level of each prediction. High-confidence predictions are accepted, while low-confidence predictions are rejected. By using a predetermined threshold δ ∈ R + , users can make informed decisions based on the following function g:EQUATION)where κ(•) denotes a confidence-rate function, such as the maximum softmax probability.Failure Detection vs. Confidence Calibration. Confidence calibration (CC) [32, 22, 26, 44] primarily emphasizes the alignment of predicted probabilities with the actual likelihood of correctness, rather than explicitly detecting failed predictions as in FD. The goal of CC is to ensure that the predictive confidence is indicative of the true probability of correctness:EQUATIONThis implies that when a model predicts a set of inputs x to belong to class y with a probability p * , we would expect approximately p * of those inputs to truly belong to class y.However, as observed by [48] , models calibrated with CC algorithms do not perform well in FD.Traditional metrics used to evaluate CC, such as the Expected Calibration Error (ECE [27] ), do not accurately reflect performance in FD scenarios. Instead, alternative metrics like the Area Under the Risk-Coverage Curve (AURC [7] ) and the Area Under the Receiver Operating Characteristic Curve (AUROC [2] ) are recommended for assessing FD performance.Failure Detection vs. Out-of-Distribution Detection. While both Out-of-Distribution Detection (OoD-D) and failure detection tasks aim to enhance confidence reliability, they have distinct objectives, as depicted in Fig. 2 . OoD-D focuses on rejecting predictions of semantic shift while accepting in-distribution predictions. However, it does not explicitly address the rejection of cases affected by covariate shifts. Additionally, through empirical observations in Sec. 4 , we find that OoD-D methods are not well-suited for the Failure Detection task.Traditional Failure Detection (Old FD) vs. New Failure Detection (New FD). Traditional failure detection methods [48, 49, 25] primarily focus on assessing the accuracy of predictions for indistribution data. They also evaluate the discriminative performance of distinguishing correct and [20] is evaluated on 6 images under a whole range of relevant distribution shifts: For instance, the 3rd and the 4th images in grayscale depict an airplane and a horse which encounter covariate shifts from that in the original CIFAR10. The 5th and the 6th images depict samples belonging to unseen categories with semantic shifts.incorrect predictions for covariate shift data based on confidence scores. While these approaches address certain aspects of distribution shifts, they overlook the semantic shifts.To address the limitations of Out-of-Distribution Detection (OoD-D) and traditional failure detection (Old FD) methods, [17] proposes a new setting called New FD. The objective of New FD is to accept correct predictions for both in-distribution and covariate shift samples, while rejecting incorrect predictions for all possible failures, including in-distribution, covariate shift, and semantic shift samples. Compared to OoD-D and Old FD, it enables more effective decision-making in real world.Question: Does the paper discuss the limitations of the work performed by the authors? Answer: [Yes] Justification: The limitations of the proposed method are discussed in the "Limitations" section.Guidelines:• The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper. • The authors are encouraged to create a separate "Limitations" section in their paper.• The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be. • The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated. • The authors should reflect on the factors that influence the performance of the approach.For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon. • The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size. • If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness. • While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations. 
method: 
 In this section, we present our proposed strategy called Typicalness-Aware Learning (TAL), as shown in Fig. 3 . First, in Sec. 3.1, we address the shortcomings of existing training objectives and identify overfitting of atypical samples as a potential cause of overconfidence. Next, in Sec. 3.2, we outline the methodology used to calculate the "typicalness" of samples, enabling selective optimization and mitigating the negative impact of atypical samples. Finally, in Sec. 3.3, we introduce the TAL strategy, which incorporates the computed typicalness values for individual samples, resulting in improved performance for the failure detection task.In Eq. ( 1), the optimization of the cross-entropy loss involves either increasing the magnitude of the logits or aligning them better with the labels. However, LogitNorm [39] researchers have observed that as the training progresses and the model becomes more accurate in classifying samples, it tends to generate significantly larger logit magnitudes, leading to overconfidence. To address this issue, The typicalness measure τ is calculated by comparing these statistics between the current batch and the HFQ. This τ influences the overall loss calculation, guiding the model to differentiate between atypical and typical samples. In the inference phase, TAL operates similarly to a model trained with conventional cross-entropy. Confidence is derived from the cosine similarity of the predicted logit direction, emphasizing our approach of using direction as a more reliable confidence metric.The framework distinguishes between typical (high τ ) and atypical (low τ ) samples, influencing the optimization process accordingly.LogitNorm introduces a constant magnitude T in Eq. ( 4) to mitigate the problem:EQUATIONBy keeping the magnitude constant in Eq. ( 4), the model places greater emphasis on producing features that align more closely with the target label in terms of direction, in order to minimize the training loss. However, as shown in Fig. 1 , the presence of atypical samples with ambiguous content and labels may still cause overconfidence. Hence, it is imperative to devise a method that allows for the differentiation and separate optimization of typical and atypical samples.Drawing inspiration from the cognitive process of human decision-making, it is reasonable to distinguish between typical and atypical samples by leveraging the knowledge acquired during the training phase. This approach allows us to effectively mitigate the negative effects of atypical samples, thereby preventing the occurrence of erroneous overconfidence.To differentiate typical samples from atypical ones, we introduce a method for evaluating typicalness and implement typicalness-aware learning (TAL). This approach entails calculating the mean and variance of the feature representations. Specifically, we calculate the mean and variance of each sample's feature channels based on insights from CORES [30] . The insight stems from the observation that in-distribution samples show larger magnitudes (mean) and variations (variance) in convolutional responses across channels compared to OoD samples, which are a type of atypical sample. The mean response of OOD samples is smaller than correct ID samples, as shown in Fig. 5 (a). These statistical characteristics are subsequently compared to a set of historical data, representing typical samples, stored in a structured queue known as the "Historical Feature Queue" (HFQ). By comparing the statistical features of the input samples to those in the HFQ, we can quantify their typicalness.Initialize and update HFQ. We commence the process by initializing the HFQ, denoted as Q, with a predetermined size equivalent to the number of samples in the training dataset. This structured queue is responsible for retaining the mean and variance of feature representations for typical samples identified throughout the training phase.To establish the initial state of the queue, we do not adopt the model trained from scratch. Instead, we employ a model that has been trained for a few epochs, corresponding to a small portion (λ) of the total training epochs. This approach ensures the quality of the queue during the early stages of training. In this study, we set λ = 0.05, which corresponds to 5% of the total training duration.During this initialization phase, for each batch of data, we calculate the mean (µ) and variance (σ 2 ) of the feature vectors for each correctly predicted sample. Each sample in the queue stores its statistical features, denoted as follows, given a prediction ŷ and the ground truth label y:EQUATIONwhere µ i and σ 2 i represent the mean and variance of the feature vectors of the i-th sample, respectively. Once initialized, the statistics (mean and variance) of accurately predicted samples in each batch are directly added to the queue, as shown in Fig. 3 . The queue has a fixed length of 20,000, and the ablation study is provided in Sec 4.3. The queue is updated using a First-In-First-Out (FIFO) approach, guaranteeing that it preserves a representative assortment of typical samples observed throughout the training process, while also adapting to the evolving data distributions. We empirically find this simple strategy works well in our experiments.Typicalness assessment. To evaluate the typicalness τ of a new sample, we first calculate the mean (µ new ) and variance (σ 2 new ) of its features f . Subsequently, we compute the L2 distance d between the feature distribution of the new sample, represented by µ new and σ 2 new , and the distributions of the features stored in the HFQ, denoted as (µ j , σ 2 j ) ∈ Q. Finally, we normalize the resulting distance using min-max normalization to obtain the typicalness τ .EQUATIONEQUATION)Where d min and d max represent the minimum and maximum distances of samples in the batch. Eq. ( 7) normalizes the value of τ within the range of [0, 1], then τ can serve as a indicator of sample typicalness. A high τ value suggests that the sample is highly typical compared to the historical data. Conversely, a low τ value indicates an atypical or anomalous sample.Sec. 3.1 highlights the potential negative impact of atypical samples on the training process. Building upon the insights provided in Sec. 3.2, we now introduce Typicalness-Aware Learning (TAL) in this section. TAL leverages the typicalness τ to distinguish between typical and atypical samples during the optimization process. This approach aims to mitigate the issue of overconfidence that arises from the presence of atypical samples.The training objective of TAL. The training objective of TAL is defined by incorporating an additional loss term L TAL . This is achieved by modifying the LogitNorm equation, denoted as Eq. ( 4), to Eq. ( 8) where the samples x are assigned with dynamic magnitudes T (τ ) based on typicalness τ .EQUATION)Dynamic magnitude T (τ ). Given the upper bound T max and the lower bound T min , the dynamic magnitude T (τ ) can be obtained via:EQUATIONwhere we empirically set T max and T min to 10 and 100, and they perform well on different benchmarks. The ablation study on different values is shown in Sec. 4.3.Specifically, in Eq. ( 9), a smaller magnitude T (τ ) will be assigned to typical samples with large τ , and a larger magnitude T (τ ) will be assigned to less typical samples with smaller τ , enabling different treatments for typical/atypical samples. In this manner, for atypical samples, a higher value of T (τ ) reduces the influence that pulls them towards the label direction. This helps prevent their logit directions from being excessively optimized.In other words, the inverse proportionality between T (τ ) and τ encourages the model to yield directions of f that are well-aligned with the labels for the typical samples with large τ by assigning a small magnitude T (τ ). Conversely, for atypical samples with small τ , the directions are not required to be as precise as the typical ones as the current T (τ ) is large, to mitigate the adverse impacts brought by the ambiguous label. To this end, the direction f can serve as a more reliable indicator of the model confidence.The overall optimization.Fig. 3 illustrates the TAL framework, showing both training and inference processes. During the training process, we utilize both the proposed TAL loss L TAL and cross-entropy loss L CE as it exhibits stronger feature extraction capabilities than LogitNorm [39] . The overall loss is:EQUATIONThe TAL loss, denoted as L TAL , is utilized to optimize the directions of reliable typical samples with large typicalness τ , as well as potentially some atypical samples with small τ .The CE loss (not only relying on the CE loss, as it is regulated by τ ) for atypical samples enables the optimization of both direction and magnitude. This may help reduce the adverse effects of atypical samples on the direction, enhancing the reliability of direction as a confidence indicator. This ensures that the optimization force on the logit directions of atypical samples is weaker compared to that of typical samples. The inference process does not involve the calculation of typicalness, and the only difference from the normal inference process is that our method uses Cosine as the confidence score.To summarize, our proposed approach, as indicated in Eq. ( 10), enables models to selectively and adaptively optimize typical and atypical samples according to their typicalness values. This strategy enhances the reliability of feature directions as indicators of model confidence, ultimately improving the performance on failure detection task.Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?Answer: [Yes] Justification: The code and data will be made available upon publication.Guidelines:• The answer NA means that paper does not include experiments requiring code. • Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted. 
experiments: 
 To evaluate the effectiveness of the proposed Typicalness-Aware Learning (TAL) strategy, we conduct extensive experiments on various datasets, network architectures, and failure detection (FD) settings. More details such as the training configuration can be found in Appendix A.Datasets and models. We first evaluate on the small-scale CIFAR-100 [20] dataset with SVHN [11] as its out-of-distribution (OOD) test set. To demonstrate scalability, we further conduct experiments on large-scale ImageNet [5] using ResNet-50, with Textures [3] and WILDS [19] serving as OOD data.For CIFAR-100, we verify TAL's effectiveness across diverse architectures including ResNet [13] , WRNet [45] , DenseNet [15] , and the transformer-based DeiT-Small [38] . Detailed experimental results are provided in Appendix C.Three settings. We evaluate TAL under three different settings: Old FD setting, OOD detection setting, and New FD setting (detailed in Section 2). While Old FD distinguishes between correct and incorrect in-distribution predictions, and OOD detection identifies out-of-distribution samples, our New FD setting aims to separate correctly predicted in-distribution samples from both misclassified and out-of-distribution samples. We maintain a 1:1 ratio between in-distribution and out-of-distribution samples in testing, and also report results for Old FD and OOD detection settings for completeness.Baselines. We compare our proposed TAL method against classical Maximum Softmax Probability (MSP), MaxLogit [14] , Cosine [47] , Energy [23] , Entropy [33] , Mahalanobis [4] , Gradnorm [16] , SIRC [40] and recent LogitNorm [39] , OpenMix [49] and (FMFP) [48] . It is worth noting that FMFP focuses on improving accuracy for failure detection.Evaluation metrics. To comprehensively assess the performance of TAL in failure detection, we adopt three widely recognized evaluation metrics [17, 48, 9] To showcase the scalability of our approach, we present the results on ImageNet in Table 2 . It is obvious that our TAL strategy consistently enhances the failure detection performance of the baseline method, significantly improving the reliability of confidence. Notably, TAL reduces the AURC by 3.7 and 11.6 points, indicating a better overall performance in distinguishing between correct and incorrect predictions. It is worth noting that TAL achieves these impressive improvements while maintaining a comparable accuracy to the MSP baseline.Additionally, we present the Risk-Coverage (RC) curves (Fig. 5(c )) for both old and new FD task settings on ImageNet. The comparison between TAL and baseline RC curves demonstrates the effectiveness of our method. Fig. 5 (d) further visualizes typical and atypical data examples. For the fish category in ImageNet, typical data includes common fish images, while atypical data comprises both rare fish images from ImageNet and out-of-distribution samples.The ablation study of key components. We conduct experimental ablations on the components of our TAL loss with CIFAR100. The results are summarized in Table 3 . With the dynamic magnitude T (τ ) strategy, we achieve substantial enhancements in Failure Detection performance, which manifests the effectiveness of integrating typicalness-aware strategies into training approaches. The impacts of T min and T max . We perform an ablation study using ResNet110 on the CIFAR10 and CIFAR100 datasets to examine the impact of T min and T max on failure detection performance. Fig. 4 (a) and Fig. 4 (b) present the experimental results of the failure detection metric EAURC for CIFAR10 and CIFAR100, respectively. Darker regions in the figures correspond to lower values of the metric, indicating superior failure detection performance. The findings suggest that while T min should not be set too small, a moderate increase in T max can enhance failure detection capabilities.The effects of the length of Historical Feature Queue. We conduct an ablation study on the CIFAR100 dataset to examine the impact of queue length on failure detection performance. The original CIFAR100 dataset consists of 50,000 training images, with 5,000 images reserved for validation and the remaining 45,000 images used for training. The results, depicted in Figure 4 (c), demonstrate that queue lengths ranging from 10,000 to 50,000 yield similar failure detection performance. However, when the queue length exceeds 50,000, there is a noticeable decline in failure detection performance.  
concluding remarks: 
 Summary. This paper introduces Typicalness-Aware Learning (TAL), a novel approach for mitigating overconfidence in DNNs and improving failure detection performance. The effectiveness of TAL can be attributed to a crucial insight: overconfidence in deep neural networks (DNNs) may arise when models are compelled to conform to labels that inadequately describe the image content of atypical samples. To address this issue, TAL leverages the concept of typicalness to differentiate the optimization of typical and atypical samples, thereby enhancing the reliability of confidence scores.Extensive experiments have been conducted to validate the effectiveness and robustness of TAL. We hope TAL can inspire new ideas for further enhancing the trustworthiness of deep learning models.Limitations. The main contribution of TAL lies in recognizing the issue of overfitting atypical samples as a cause of overconfidence and proposing a comprehensive framework to tackle this problem. Given that the methods adopted in this work are simple yet effective, there is still potential for further improvement by incorporating more advanced designs, such as the methods for typicalness calculation and the dynamic magnitude generation. These are left as future work to be explored.Broader impacts. As deep learning models become increasingly integrated into critical systems, from autonomous vehicles to medical diagnostics, the need for accurate and reliable confidence scores is paramount. TAL's ability to improve failure detection performance directly addresses this need, potentially leading to safer and more dependable AI systems. For experiments on the CIFAR [20] , we employ an SGD optimizer with an initial learning rate of 0.1, a momentum of 0.9, and a weight decay of 0.0005. The models are trained for 200 epochs with a batch size of 256 on a single NVIDIA GeForce RTX 3090 GPU. Furthermore, we adopt a CosineAnnealingLR scheduler to adjust the learning rate during training. On ImageNet [5] , we use the ResNet-50 architecture as our backbone. The models are trained for 90 epochs with an initial learning rate of 0.1 on a single NVIDIA A100. The learning rate is decayed by a factor of 0.1 every 30 epochs.To ensure a fair and robust evaluation, we conduct experiments with three independent training runs on CIFAR100, each using a different random seed. All baseline methods and our proposed TAL are evaluated using identical settings across these three runs. The final performance metrics reported in our results are averaged across these three sets of weights to account for training variance. For clarity of presentation, we multiply AURC values by 10 3 , while maintaining all other metrics in percentage form. This systematic evaluation approach allows us to make reliable comparisons between different methods while accounting for the inherent variability in deep learning model training.For ImageNet experiments, we encountered several implementation challenges with some baseline methods. Specifically, FMFP failed to achieve competitive accuracy, while LogitNorm and OpenMix suffered from training instability and collapse. These issues might be attributed to the fact that these methods were not originally validated on ImageNet in their respective papers. While it might be possible to adapt these methods for ImageNet through extensive parameter tuning and modifications, such adaptations would require significant engineering effort and might deviate from the original methods. Therefore, we opted not to report results for these methods on ImageNet to maintain experimental integrity.It is interesting to note that in the Old Few-Shot Detection (FD) setting, most Out-of-Distribution (OoD) methods, such as Energy Score and LogitNorm, exhibit poor performance compared to baseline methods like Maximum Softmax Probability (MSP) and MaxLogit. This decline in performance can be attributed to the fact that while OoD methods effectively increase the confidence score gap between in-distribution and out-of-distribution samples, they inadvertently disrupt the natural confidence score hierarchy within the in-distribution samples. As a result, there is a greater overlap in the confidence distributions of correctly and incorrectly predicted in-distribution samples, as illustrated in Fig. 6 . This observation highlights the importance of developing dedicated FD methods that can effectively distinguish between correct and incorrect predictions within the in-distribution data.Evaluation on Transformer-based architectures.Considering the remarkable success of vision transformers as network architectures, it is crucial to incorporate a transformer-based network in our analysis and evaluate the effectiveness of our proposed method. It is worth noting that the substantial disparities between ViT [6] and CNN architectures in terms of their design, feature representation, and learning mechanisms may pose challenges in directly applying methods that have proven effective on CNNs to ViT models.Implementation Details of ViT. TAL can also be applied to Transformer architectures. However, it is important to note that certain CNN-based methods may not perform well on Transformer-based tasks. For example, the accuracy of OpenMix with default settings is significantly lower (approximately 0.20) compared to the baseline. This indicates that OpenMix, originally designed for CNNs, does not effectively detect failures when applied to ViT models without appropriate modifications and adjustments that consider the unique architectural characteristics of ViT.Specifically, ViT learns the relationships between image patches through the Self-Attention mechanism, resulting in distinct feature representations and gradient flow patterns compared to CNNs. Moreover, ViT typically employs different normalization techniques, such as Layer Normalization, instead of the commonly used Batch Normalization in CNNs. These differences can impact the effectiveness of certain methods when applied to ViT. To successfully adapt these methods to ViT, appropriate modifications and adjustments may be necessary to account for the unique architectural characteristics of ViT.To explore the performance of current popular failure detection methods on ViT models, we conducted experiments using the pre-trained DeiT-Small model ("deit_small_patch16_224") from the timm library. We employed the SGD optimizer with a base learning rate of 0.01, a weight decay of 0.0005, and a momentum of 0.9. The learning rate scheduler was set to CosineAnnealingLR, with the T_max parameter determined by total training epochs. The experiments were conducted on the CIFAR100 dataset, with a total of 25 training epochs and a batch size of 256.Experimental Results on ViT. The experimental results shown in Tab. 4 and Tab. 5 validate the applicability of TAL to vision transformers. By prioritizing optimization with typical samples and concurrently relaxing the requirements for atypical samples, TAL effectively mitigates overconfidence and enhances failure detection performance. Justification: The abstract and introduction clearly state the main contributions and scope of the paper, which are consistent with the theoretical and experimental results presented in the body of the paper. Guidelines:• The answer NA means that the abstract and introduction do not include the claims made in the paper. • The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers. • The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings. • It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper. 
experimental setting/details: 
 Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?Answer: [Yes] Justification: The paper specifies the training and test details, including data splits, hyperparameters, and optimization settings.Guidelines:• The answer NA means that the paper does not include experiments.• The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them. • The full details can be provided either with the code, in appendix, or as supplemental material. 
experiment statistical significance: 
 Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?Answer: [Yes]Justification: The paper reports mean and standard deviations of the experimental results over multiple runs.Guidelines:• The answer NA means that the paper does not include experiments.• The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper. • The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).• The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.) • The assumptions made should be given (e.g., Normally distributed errors). • It should be clear whether the error bar is the standard deviation or the standard error of the mean. • It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified. • For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates). • If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text. 
experiments compute resources: 
 Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?Answer: [Yes]Justification: The paper provides information on the compute resources used for the experiments in the Experiments section.Guidelines:• The answer NA means that the paper does not include experiments.• The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage. • The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute. • The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper). 
code of ethics: 
 Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines?Justification: The research conducted in the paper conforms to the NeurIPS Code of Ethics.Guidelines:• The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.• If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics. • The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction). 
broader impacts: 
 Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?Answer: [Yes]Justification: In the Discussion section, the paper discusses both potential positive and negative societal impacts of the work performed.Guidelines:• The answer NA means that there is no societal impact of the work performed.• If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.• Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations. • The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster. • The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology. • If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML). 
safeguards: 
 Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)? Answer: [NA] Justification: The proposed TAL is focused on improving failure detection for general classification tasks without requiring the release of potentially unsafe data or models. Guidelines:• The answer NA means that the paper poses no such risks.• Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters. • Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images. • We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort. 12. Licenses for existing assets Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected? Answer: [Yes] Justification: The paper properly credits and mentions the licenses for existing assets used. Guidelines:• The answer NA means that the paper does not use existing assets.• The authors should cite the original paper that produced the code package or dataset.• The authors should state which version of the asset is used and, if possible, include a URL. • The name of the license (e.g., CC-BY 4.0) should be included for each asset.• For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided. • If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset.• For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided. • If this information is not available online, the authors are encouraged to reach out to the asset's creators. 13. New Assets Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets? Answer: [Yes] Justification: The paper introduces new codes, which will be documented in the github link. Guidelines:• The answer NA means that the paper does not release new assets.• Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc. • The paper should discuss whether and how consent was obtained from people whose asset is used. • We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution. • For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review. 
 
 Appendix:  


Decision: There was consensus from the reviewers on the work being well-motivated, well-written and the methodology being intuitive. Much of the reviewer's concerns pre-rebuttal centred on the ImageNet results, the small number of such results, and the poor results. Other concerns included missing references/comparisons with existing work. However, significant work by both the authors and reviewers during the rebuttal period discussion appears to have addressed these concerns for the reviewers, in particular with much improved ImageNet results.

Reviews: review: paper_summary: paper_summary: The paper proposed a method for detecting incorrect prediction (Failure Detection). The main hypothesis behind the method is that cross-entropy either increases the logit magnitude or aligns the logit direction to align it with the ground label, which can cause discrepancy on atypical samples at test time. Previous methods, such as LogitNorm, only focussed on the logit magnitude; the proposed method tries to adaptively align training samples based on a metric that measures the typicalness of each sample. The paper also proposes a new setting, 'New FD', for evaluating failure detection methods. Extensive experiments were conducted on CIFAR-10/100, with some preliminary results on ImageNet (not benchmarked properly).
strength_weakness: strength_weakness: While the proposed method looks promising, there are a few weaknesses and areas where the manuscript can be further improved:

1. Most of the experiments were conducted on the CIFAR dataset (Table 1). Results on ImageNet are inconclusive; only MSP was used as a baseline for comparison. It would be helpful to add more baselines for comparison.


2. Experiments on ViT were conducted on CIFAR. It is not clear why CIFAR was chosen as a dataset for ViT instead of ImageNet. 

3. As SSL-based models or fine-tuning pre-trained foundational models are becoming state-of-the-art models for various tasks, it is unclear if the proposed method can be applied in that setting. Experiments should be conducted on fine-tuning pre-trained foundational models.

4. Some parts of section 3 are not clear:

      a. How are mean and variance calculated for each sample feature? 
      b. Some samples can be incorrectly classified in some parts of the training; how do you handle such samples to update HFQ?
      c. How is $d_{min}$ and $d_{max}$ are calculated?
      d. Variables are not explained, e.g., equation 3.

5. It is not clear how equation 10 works. $\tau$ measures 'typicalness'; a value of 1 indicates a typical sample, and 0 indicates an atypical sample. The CE loss is weighted by $(1-\tau)$, i.e., the atypical sample ($\tau=0$) will only be trained by CE loss, which is counterintuitive. 

6. Authors only use SVHN for evaluating OOD shifts; systematic evaluation on the WILDS dataset [1] should be added. 

7. The need for the New-FD setting (detecting both is not explained, and the authors chose to evaluate the baselines in this setting. The experimental setup (New-FD) is unfair for baselines proposed to detect only OOD shifts (semantic shift). The proposed method should be evaluated for semantic and covariate shifts independently. 

[1] Koh et al., WILDS: A Benchmark of in-the-Wild Distribution Shifts
questions: questions: 1. In Sec 4.2, which model did you use? It is not clear from the manuscript. 

2. What are dynamic magnitudes?  Why do we need it?

3. > In this manner, for atypical samples, a higher value 
of $T(\tau)$ reduces the influence that pulls them towards the label direction"

Can you explain more how does T influence the direction?

4. Explain more about equation 10. Why atypical samples optimized only by CE loss?
limitations: limitations: Yes
paper_summary: paper_summary: This paper proposes a novel training method for improving the failure detection ability of classification models. The authors argue "overconfidence" may be in part due to overfitting of a model to the one hot labels of "atypical" samples. In order to mitigate this issue, a dynamic modification of the LogitNorm training method is proposed where the logit direction is more focused on for more "typical" samples. The proposed TAL is evaluated on image classification failure prediction benchmarks, showing promising performance on CIFAR data.
strength_weakness: strength_weakness: As the reviewing burden has been heavy for this conference (6 papers) please understand that I can only dedicate so much time to this paper. Thus, I may have made mistakes in my understanding of the paper, and I welcome the authors to correct me if this is the case. 

1. Missing comparisons. A number of methods that should be compared against are missing. CRL [1] (similar to TAL) is a learning-based approach for better misclassification detection, whilst SIRC [2] is a post-training method explicitly designed for rejecting both ID errors and OOD samples. It would also be better to clearly distinguish training and post-training approaches in the results.
2. No risk-coverage curves presented. Although scalar scores show overall failure detection performance, plotting out RC curves in my opinion tells a lot more about the performance of an approach. 
3. Odd ImageNet results. The accuracy for ResNet-50 on this benchmark is far below the standard accuracy achievable using cross entropy training (~76%) and standard augmentations. Thus I am yet unconvinced that this approach is able to scale up to more realistic data. CIFAR is a rather special optimisation scenario where training accuracy converges to 100%. Models are typically extremely overparameterised and the data are much smaller scale than real-world CV applications. Thus, CIFAR results may not generalise to real applications. 
4. Missing references/attribution to existing work. There are a number of other published work that targets new failure detection that are not cited [2,3,4]. In particular [2] already discusses the behaviour described in Appendix B at length.
5. Complexity of method. The actual method of TAL is quite complex, with a number of stages (d calculation, \tau calculation), hyperparameters (Tmin/max, queue length) and ad hoc design choices (mean/variance calced along feature dimension, wasserstein distance, using \tau to mix TAL and CE). This complexity limits ease of adoption, and makes it difficult to understand the reasons for TAL's efficacy. A more thorough ablation would greatly improve the understanding of the importance of each choice, aiding potential practitioners. Similarly, some intermediate experiments, (e.g. showing how error-rate changes according to "typicalness") would also make the method of TAL more convincing.
6. Missing details. It is unclear to me what confidence score is used in the end for TAL. Moreover, how is \tau set during inference? Is it fixed or is it calculated in the same way as during training? What is the OOD dataset for ImageNet?


References

[1] Moon et al., Confidence-Aware Learning for Deep Neural Networks

[2] Xia and Bouganis, Augmenting Softmax Information for Selective Classification with Out-of-Distribution Data

[3] Narasimhan et al., Plugin estimators for selective classification with out-of-distribution detection

[4] Kim et al., A Unified Benchmark for the Unknown Detection Capability of Deep Neural Networks
questions: questions: 1. As CRL requires training, but SIRC doesn't, and given the length of the rebuttal period, I would be happy with a comparison with just SIRC. 
2. I'd like to see a couple of risk-coverage curves to get a clearer idea of how TAL impacts failure detection (of just TAL vs MSP/CE).
3. I would **require** ImageNet results with standard accuracies (~76%) which should be achievable using the recipe from the original resnet paper.
4. Include these references.
5. I would like to see some more ablations if possible (e.g. using a different measure of typicality, using static mixing of CE/TAL. A toy experiment (e..g 2D classification) showing how OOD samples/samples in lower density areas of the training distribution are assigned lower typicality would help a lot as well. An experiment such as finding atypical samples in the test set (using a model's predictive uncertainty) and then showing how those labels are overfitted when the test set is folded into the training set under CE but not under TAL would also improve the paper in my opinion.
6. Please clarify this.

Depending on the number of questions/weaknesses addressed I will be happy to raise my score up to 6.

Some additional food for thought for the authors. Knowledge Distillation may also help in softening supervision on atypical samples. Perhaps it could be incorporated into future work. I also think it would be good to clarify that the authors' use of "typical" is different to that commonly found in information theory.


As the authors have successfully rebutted my queries, as well as considering the relative scarcity of effective approaches for the problem setting (compared to e.g. OOD detection), I have decided to raise my score to 7.
limitations: limitations: See above
paper_summary: paper_summary: In this work, the authors propose a new approach to failure detection from DNN predictions. They suggest that data samples can be classified as either typical or atypical. The latter includes ambiguous, out of distribution samples or ill annotated data.
In order to circumvent this limitation, the authors introduce a new training objective which leverages a simple and effective set of features to determine whether a data sample is typical or not during both training and inference. In particular, they propose to store first and second order moment statistics from training samples and measure the Wasserstein distance from new samples to these distributions. This distance is then leveraged in order to weight the cross entropy logits. As a result, a large distance increases the logits magnitude and thus reduces the requirements from the model. This enables the use of the logits direction as a confidence metric.
The authors propose to evaluate the proposed method on Cifar10, Cifar100 and ImageNet for both CNNs ViT, showcasing the added value of the method in these experiments.
strength_weakness: strength_weakness: I have two minor concerns regarding this work:
1. in its current form the empirical quantitative evaluation is not fully convincing as it is bounded to computer vision tasks and models with very few comparison points on large scale datasets.
2. the authors insist on the intuition behind the added value with an explicit illustration in the motivation but I did not see an explicit evaluation of the typical vs atypical detection.
questions: questions: My main question is: could the authors derive a small set of atypical and typical data (labelled set) from a known ood dataset for Imagenet such as [1] and measure the ability of the proposed method to actually separate typical vs atypical data?
i believe this work would benefit from such validation

[1] Galil I, Dabbah M, El-Yaniv R. A framework for benchmarking class-out-of-distribution detection and its application to imagenet. ICLR 2023
limitations: limitations: I believe the authors had a fair description of the current limitations of their work. I would suggest adding a word regarding the specificity of the training loss in the current presentation of the method.
paper_summary: paper_summary: This paper identifies overfitting of atypical samples as a potential casue of overconfidence in DNN for Failure Detection.
The authors proposes a Typicalness-Aware Learning (TAL) approach that computes a typicalness score of each sample. TAL assigns dynamic logit magnitudes based on typicalness to allow flexible fitting of atypical samples while preserving reliable directions as confidence for atypical and typical samples.
strength_weakness: strength_weakness: 1. The definition of atypical samples is vague, making it difficult to accurately determine the typicality of samples, which may affect the method's effectiveness. Please consider providing some mathematical definitions and visual examples.
2. There is room for improvement in the specific implementation, such as typically calculation and dynamic magnitude generation, by exploring more advanced designs.
3. The paper only compare failure detection task; the impact of TAL on classification accuracy is not analyzed ( accuracy is noly provided in the table but not discussed).
4. Minor error: "pred" and "GT" in Fig.2 should be swapped.
questions: questions: 1. In Section 3.2 "We empirically find this simple strately works well in our experiments." Please explain why the mean and variance can represent the features of a sample to measure typicality?
2. The confidence calibration methods aim to achieve more reliable confidence estimates, but fail in failure detection task. Could please explain that?
3. The OoD data and ID data are presented in equal proportions. How does the ration of them affect the results.
limitations: limitations: Discussed

Comments: title: Thanks to reviewer
Thanks to reviewer
Responses to the question and Thanks to reviewer
Reply to the authors
Responses to the question
Responses to the question
Satisfied with the rebuttal and another question.
Responses to the question
Responses to the questions
Responses to the question
Experiments on WILDS dataset
further question
comment: We sincerely appreciate the time and effort you've invested in our work. Your insightful questions have highlighted areas in our paper that require clearer articulation. 
With your professional suggestions, the quality of our paper is significantly enhanced.
Thank the reviewer for the careful and thorough reading of our manuscript.
We are deeply grateful for your thorough review and positive recognition of our work. The insightful comments demonstrate a deep understanding of our research, which we greatly value. Your initial evaluation has been a significant source of encouragement for our team, motivating us to further refine and improve our paper. We truly appreciate your professionalism and constructive approach throughout the review process.
**Q1**: It's interesting that SIRC leads to more degradation on old FD with the cosine score. Maybe this is because it is not a softmax score and so the values are generally farther away from 1.0 and closer to (see Fig. 4 of the SIRC paper). I would be good to see some more analysis if the authors have time.

Thank the reviewer for the suggestion. More results are presented in the table below.

| Method | Old setting FD | | | OOD Detection | | | New setting FD | | | ID-ACC |
|--------|----------------|-------|-------|---------------|-------|-------|----------------|-------|-------|--------|
| | AURC↓ | FPR95↓ | AUROC↑ | AURC↓ | FPR95↓ | AUROC↑ | AURC↓ | FPR95↓ | AUROC↑ | |
|MSP_res(SIRC) | 72.70 | 63.40 | 86.18 | 276.45 |  30.85 | 90.73 | 330.73 | 37.27 | 89.79 | 76.13 |
| TAL | 64.66 | 64.93 | 87.11 | 290.5 | 47.66 | 87.51 | 338.45 | 50.11 | 88.29 | 76.43 |
cos_z(TAL) | 64.55 | 63.66 | 87.15 | 288.23 | 46.91 | 87.88 | 336.56 | 49.68 | 88.35 | 76.43 |
| cos_res(TAL) | 65.74 | 66.62 | 86.69 | 283.77 | 45.42 | 88.50 | 333.16 | 49.20 | 88.47 | 76.43 |
| (cube_root cos)_res(TAL) | 65.62 | 65.91 | 86.73 | 284.09 | 44.94 | 88.44 | 333.38 | 48.51 | 88.43 | 76.43 |
| (square_root cos)_res(TAL) | 65.58 | 66.05 | 86.75 | 284.19 | 45.10 | 88.42 | 333.45 | 48.68 | 88.42 | 76.43 |

For completeness of our work, we will include non-TAL SIRC Res results across all ImageNet cols in the final manuscript.

Additionally, we implemented the reviewer's suggestion to modify the range of cosine similarity. However, the square or cube root of the cosine score did not significantly alter the experimental results. We will add an explicit discussion to explore the problem in the new revision.

We are deeply grateful for the reviewer's exceptional feedback. The reviewer's dedication contributes significantly to the quality improvements of our paper.
The insightful comments and suggestions demonstrate the reviewer's profound expertise. We sincerely appreciate the time and effort invested in providing such comprehensive and constructive comments.
Thank you for providing detailed answers to the questions raised during the review period. The authors have resolved most of the concerns raised during the review period. New experiments on ImageNet and WILDS are really helpful. Since most of my concerns have been resolved, I have increased the score.
I'm a little confused by the evaluation using WILDS in this case. My understanding (although I'm not intimately familiar with the dataset) is that it contains various training data and covariate shifted test data. Using it as OOD data to ImageNet doesn't particularly make sense as there's no guarantee with regards to class overlap.

However, I would encourage reviewer RLx5 to overlook this as the authors have improved their evaluation with the inclusion of ImageNet-1k results
Thanks for the results. I will be also expecting non-TAL SIRC Res results across all ImageNet cols in the final manuscript.
Accordingly I will raise my score to 7. 

It's interesting that SIRC leads to more degradation on old FD with the cosine score. Maybe this is because it is not a softmax score and so the values are generally farther away from 1.0 and closer to  (see Fig. 4 of the SIRC paper). I would be good to see some more analysis *if* the authors have time. I have a hunch that taking the square or cube root of the cosine score before SIRC would reduce degradation in old FD.  

Thank you for a fruitful rebuttal period.
Thank you for your efforts during the rebuttal period. I think my concerns have been addressed adequately. Considering the questions and suggestions from the other reviewers, and trusting that the authors will include the corresponding experiments, necessary explanations and other recommended components, I decide to raise my score to 7.
**Q1**:  So to be more precise, I would like to see the results, on ImageNet+Textures, of SIRC with the residual score.

We greatly appreciate the reviewer's insightful suggestion. Following your recommendation, we combined our TAL method with different versions of SIRC and evaluated their performance on ImageNet+Textures. The results are presented in the table below.

| Method | Old setting FD | | | OOD Detection | | | New setting FD | | | ID-ACC |
|--------|----------------|-------|-------|---------------|-------|-------|----------------|-------|-------|--------|
| | AURC↓ | FPR95↓ | AUROC↑ | AURC↓ | FPR95↓ | AUROC↑ | AURC↓ | FPR95↓ | AUROC↑ | |
| TAL | 64.66 | 64.93 | 87.11 | 290.5 | 47.66 | 87.51 | 338.45 | 50.11 | 88.29 | 76.43 |
| TAL+SIRC_cos_z | 64.55 | 63.66 | 87.15 | 288.23 | 46.91 | 87.88 | 336.56 | 49.68 | 88.35 | 76.43 |
| TAL+SIRC_cos_res | 65.74 | 66.62 | 86.69 | 283.77 | 45.42 | 88.50 | 333.16 | 49.20 | 88.47 | 76.43 |

The results reveal that TAL combined with SIRC and residual scores demonstrates superior performance in OOD-D and New FD tasks. 
On the task of Old FD task without OOD data during inference, TAL combined with SIRC and features achieved the best results.
**Q1**: What is the underlying reason for the observed difference in mean values of the features between in-distribution (ID) and out-of-distribution (OoD) data? Please provide further elaboration on this point.


Thank the reviewer for the question.


The underlying reason for the observed difference in mean values of the features between in-distribution (ID) and out-of-distribution (OOD) data, according to CORES [ref 1], is that the convolutional kernels in a trained deep neural network are inherently tuned to extract fundamental attributes of input samples. These kernels exhibit strong responses to patterns they recognize - inputs that are consistent with the training data distribution (ID). In contrast, their response diminishes for patterns they do not recognize, characteristic of out-of-distribution (OOD) inputs.

- [ref1] Tang, Keke, et al. CORES: Convolutional Response-based Score for Out-of-distribution Detection. CVPR2024.




If you have any further questions or need additional clarification, please don't hesitate to let us know. We are more than happy to provide more details.
Thanks for your detailed rebuttal. I appreciate your efforts to explain my questions. Please update the corresponding explanation in the revised paper for better quality. 

Besides, I have another question: What is the __underlying reason__ for the observed difference in mean values of the features between in-distribution (ID) and out-of-distribution (OoD) data?  Please provide further elaboration on this point.
Thanks for the response and apologies for the delayed reply.

In my earlier comment, I was referring to the performance of SIRC on failure detection that includes OOD data. SIRC is not meant to improve detection performance when there isn't semantically shifted data. So to be more precise, I would like to see the results, on ImageNet+Textures, of SIRC with the residual score. 

I'd like to clarify that SIRC is a post-hoc method and as such doesn't directly compete with TAL (and as you have demonstrated can complement TAL). As such I am interested in if the best version of SIRC is able to further boost TAL's performance. 

I would also suggest that the authors revise the tables in their paper to more clearly delineate training-free and training-based approaches. 

Finally, I have been pleasantly surprised with this rebuttal period and feel like the quality of the manuscript will be significantly improved as a result. If the authors can address my final requests I will increase my score to 7.
Table 1: The performance differences in Old FD task among various SIRC variants.
| Method | AUROC$\uparrow$ | FPR95$\downarrow$ | AURC$\downarrow$ |
|--------|-------|-------|------|
| SIRC_H_z | 85.41 | 67.62 | 74.80 |
| SIRC_MSP_z | 86.11 | 63.67 | 72.91 |
| SIRC_doctor_z | 86.12 | 64.76 | 72.88 |
| SIRC_H_res | 85.54 | 66.67 | 74.44 |
| SIRC_MSP_res | 86.18 | 63.40 | 72.70 |
| SIRC_doctor_res | 86.16 | 65.71 | 72.73 |
|TAL | 87.11 | 64.93|64.66|
|SIRC_cos_z(TAL)|87.15|63.66|64.55|

The Table shows the performance for all of the variants of the SIRC method on ImageNet.
We observe that SIRC_MSP_z and SIRC_MSP_res achieve similar performance. Our TAL algorithm can significantly surpass the baselines. Thank the reviewer for the question.
Thanks for the further clarification. 
 
One final thing, I’d still like to see the results for SIRC with the residual score. Although the residual score is somewhat unreliable depending on the distribution shift, it seems to do well in OOD detection here, so I would expect it to still help a fair amount. I am surprised in this case that you state the version with the feature norm is better.
We thank the reviewer for valuable suggestions and insights, which have significantly improved our manuscript.

**Q1**: Which version of SIRC did you use? SIRC is just a method to combine scores and it's unclear which scores are combined. In the original paper, the best-performing combination was Entropy+Residual for example. SIRC is also typically used with softmax scores, so when applied to TAL, is it used with the cosine measure or a softmax score?

In our experiments, we utilized the most effective variant of SIRC for the FD task, i.e. SIRC_MSP_z. This configuration combines Maximum Softmax Probability (MSP) with the feature. For the TAL+SIRC combination, we opted for a cosine measure paired with the feature.

**Q2**: Could you include the training recipe for the updated ImageNet results? Also, how are the OOD/ID datasets balanced i.e. what is the ratio between the number of incorrect ID predictions and OOD samples?
The use of cosine distance as a confidence score is important to the story, and helps me understand the approach a lot better. I would suggest making this clear to the reader early on (e.g. in Figure 1). I would generally suggest editing the manuscript to provide a clearer story, i.e. incorporating some of the new plots into the method section. Hopefully the authors see the value in this.

- Training protocol for the updated ImageNet experiments: We employed a ResNet50 architecture as the base model.
    For data preprocessing, we implemented standard augmentation techniques. These include random cropping, horizontal flipping, and normalization.
Our training configuration includes a batch size of 256 and an initial learning rate of 0.1. The learning rate was managed by a StepLR scheduler, which reduced it by a factor of 0.1 every 30 epochs. 
We chose SGD as our optimizer, with a momentum of 0.9 and weight decay of 0.0001. The total training duration was set to 90 epochs.
To accelerate training, we utilized distributed training across four 3090 GPUs. 
For the TAL-specific parameters, we kept Tmax and Tmin unchanged. The queue length and its initial epochs were adjusted proportionally, following the same ratio as in CIFAR experiments. This approach ensures consistency across different datasets while accounting for the larger scale of ImageNet.

- Ratio of OOD/ID data: ID (correct+incorrect) : OOD = 1:1 . For baseline，incorrect ID: OOD = 451:1880 ;  For TAL, incorrect ID: OOD = 445:1880.

-  Thanks for the suggestion. In the new revision, we will revise Figure 1 for a more clear explanation of our method. We will explicitly illustrate the training and testing processes respectively. We believe this addition will effectively address the reviewer's question and improve the clarity of our paper.

**Q3**： How are the error/density plots calculated? On the training set at a particular snapshot? On the test set using the final queue?

Density, calculated using Gaussian KDE (Kernel Density Estimation), represents the likelihood of observing a data point within the distribution. The density is computed using the ImageNet test set. Typicality, denoted by $\tau$ in the original paper, is determined by calculating the distance of each test sample to the training samples, using a historical queue constructed from the mean and variance of the training set (recomputed by the last snapshot) to obtain typically.

**Q4**：
FMFP appears to fail on ImageNet. I am not familiar with the method but it would be good to offer insight here, as well as caution that the method may suffer from poor scalability.

FMFP employs Stochastic Weight Averaging (SWA) and Sharpness-Aware Minimization (SAM). SWA and SAM can suffer from sensitive hyperparameters for different datasets. Since FMFP did not report experimental results on ImageNet, we directly transferred the hyperparameters from CIFAR to ImageNet. We speculate that FMFP might need to carefully adjust the hyperparameters on ImageNet to achieve sufficient accuracy. On the other hand, for our TAL method, we also directly transfer the hyperparameters from CIFAR to ImageNet and achieve significant improvements over baselines (shown in Table 8), which demonstrates the stability of our algorithm.
**Q1**： My question would be, then why can't we derive a typical vs atypical detection method from this value? If we can, it would be interesting to do so in order to empirically validate that the intuition behind the proposed method actually corresponds to what occurs during training.

Yes. Our $\tau$ can indeed be used to distinguish between typical and atypical samples, similar to other methods for assessing typicality (such as density). Density is calculated using Gaussian kernel density estimation (Gaussian KDE) and represents the likelihood of observing a particular data point within a distribution. Fig. 6(b) in our rebuttal.pdf provides a scatter plot showing the relationship between density and our calculated typicality with $\tau$, which demonstrates a positive correlation. The advantage of our method is that it consumes fewer resources and computes quickly, making it suitable for use during the training process. Thank the reviewer for the suggestion.
Thank you for taking the time and effort to address my questions (which I think have strengthened the paper). Although I am broadly happy with the authors' response, I still have a few things I want to clarify.
- Which version of SIRC did you use? SIRC is just a method to combine scores and it's unclear which scores are combined. In the original paper the best-performing combination was Entropy+Residual for example. SIRC is also typically used with softmax scores, so when applied to TAL, is it used with the cosine measure or a softmax score?
- Could you include the training recipe for the updated ImageNet results? Also, how are the OOD/ID datasets balanced i.e. what is the ratio between the number of incorrect ID predictions and OOD samples?
- The use of cosine distance as a confidence score is important to the story, and helps me understand the approach a lot better. I would suggest making this clear to the reader early on (e.g. in Figure 1). I would generally suggest editing the manuscript to provide a clearer story, i.e. incorporating some of the new plots into the method section. Hopefully the authors see the value in this.
- How are the error/density plots calculated? On the training set at a particular snapshot? On the test set using the final queue?
- FMFP appears to fail on ImageNet. I am not familiar with the method but it would be good to offer insight here, as well as caution that the method may suffer from poor scalability.
**Q6**：only use SVHN for OOD shifts;  WILDS dataset [1] should be added.

Here are the failure detection evaluation results of TAL on the WILDS dataset. Thank you and we will supplement these results to the final version.

| **Architecture** | **Method** | **AURC↓ (Old FD)** | **FPR95↓ (Old FD)** | **AUROC↑ (Old FD)** | **AURC↓ (OOD Detection)** | **FPR95↓ (OOD Detection)** | **AUROC↑ (OOD Detection)** | **AURC↓ (New FD)** | **FPR95↓ (New FD)** | **AUROC↑ (New FD)** | **ID-ACC** |
|------------------|------------|--------------------|---------------------|---------------------|---------------------------|----------------------------|----------------------------|--------------------|---------------------|---------------------|------------|
|                  |            | Imagenet vs WILDS                                                                                                                   |
| **ResNet50**     | MSP        | 72.73              | 63.95               | 86.18               | 272.93                    | 59.27                      | 87.72                      | 326.85             | 60.19               | 87.42               | 76.13      |
|                  | Cosine     | 102.98             | 69.93               | 79.49               | 255.91                    | 68.67                      | 89.85                      | 326.97             | 68.92               | 87.81               | 76.13      |
|                  | Energy     | 118.66             | 76.33               | 75.81               | 235.88                    | 37.67                      | 94.22                      | 318.89             | 45.27               | 90.60               | 76.13      |
|                  | MaxLogit   | 113.35             | 72.11               | 77.29               | 237.28                    | 38.93                      | 93.97                      | 317.46             | 45.46               | 90.69               | 76.13      |
|                  | Entropy    | 74.61              | 67.07               | 85.48               | 259.01                    | 51.20                      | 90.44                      | 316.26             | 54.32               | 89.47               | 76.13      |
|                  | mahalanobis| 208.22             | 96.19               | 54.23               | 264.11                    | 77.17                      | 88.20                      | 382.46             | 80.91               | 81.51               | 76.13      |
|                  | Residual   | 238.18             | 97.01               | 49.00               | 282.46                    | 81.30                      | 85.09                      | 409.89             | 84.39               | 77.99               | 76.13      |
|                  | gradnorm   | 206.99             | 89.66               | 57.88               | 237.31                    | 25.37                      | 94.84                      | 363.03             | 38.02               | 87.56               | 76.13      |
|                  | SIRC       | 72.91              | 63.67               | 86.11               | 267.33                    | 52.17                      | 89.03                      | 322.41             | 54.43               | 88.46               | 76.13      |
|                  | LogitNorm  | -                  | -                   | -                   | -                         | -                          | -                          | -                  | -                   | -                   | -      |
|                  | openmix    | -                  | -                   | -                   | -                         | -                          | -                          | -                  | -                   | -                   | -      |
|                  | FMFP       | -                  | -                   | -                   | -                         | -                          | -                          | -                  | -                   | -                   | 60.11      |
|                  | TAL (ours)        | 64.66              | 64.93               | 87.11               | 232.11                    | 40.97                      | 94.28                      | 288.67             | 45.55               | 92.91               | 76.43      |
I would like to thank the authors for their response. I am still confused about the typical vs atypical detection.
In the response, you state that the proposed method "dynamically adjusts the training optimization process for different samples by calculating a value that reflects the degree of typicality". My question would be, then why can't we derive a typical vs atypical detection method from this value? If we can, it would be interesting to do so in order to empirically validate that the intuition behind the proposed method actually corresponds to what occurs during training.