Paper: Out-of-Distribution Detection & Applications With Ablated Learned Temperature Energy (ICLR-2024)

Reviewer comment (Theory):
This work trains an objective similar to generalized ODIN and uses an energy score with trained temperature for OOD detection. A theoretical analysis between ODIN seems to be missing. For example, previous energy score research pointed out it was better than the maximum confidence score because of the contradiction of maximum logit and energy score. Even if such analysis exists, the contribution will still be limited.

Author response:
Thanks for the comments! In general it sounds like you would like to see a more clear and/or rigorous theoretical interpretation as to why Abet works and potentially compared to ODIN. Is the explanation in section 3 not sufficient in providing intuition? If so, the authors would be happy to formalize this intuition into a theorem.

Reviewer comment (Theory):
Sec 5 only addresses the empirical understanding rather than theoretical.

Author response:
Thanks for the comments! In general it sounds like you would like to see a more clear and/or rigorous theoretical interpretation as to why Abet works and potentially compared to ODIN. Is the explanation in section 3 not sufficient in providing intuition? If so, the authors would be happy to formalize this intuition into a theorem.

Reviewer comment (Theory):
(For weakness 1) Why using energy score with trainable temperature for energy-score is better than the OOD score in generalized ODIN? Maybe following this idea? What determines the performance in energy score is mainly the logsumexp of the logits, rather than temperature. Thus, its analysis was mainly in the denominator of the softmax function and temperature does not play a major role there. Given this fact/approximation, the function $\hat f_{y_i}(x_i; \theta)$ (just below equ(1)) could be analyzed similarly. Taking the log of $\hat f_{y_i}(x_i; \theta)$, we will have two terms, one of which will be the AbeT score.

Author response:
Thanks for the comments! In general it sounds like you would like to see a more clear and/or rigorous theoretical interpretation as to why Abet works and potentially compared to ODIN. Is the explanation in section 3 not sufficient in providing intuition? If so, the authors would be happy to formalize this intuition into a theorem.

Reviewer comment (Theory):
Given the similarity between Generalized ODIN and energy score, it seems the really interesting result is that it does not need OOD samples during training. (2.1) Can you confirm this? (2.2) If so, can you explain why the trained temperature only could change the OOD performance theoretically? If this explanation is convincing, it may be more important than the empirical perspectives of the methods.

Author response:
Thanks for the comments! In general it sounds like you would like to see a more clear and/or rigorous theoretical interpretation as to why Abet works and potentially compared to ODIN. Is the explanation in section 3 not sufficient in providing intuition? If so, the authors would be happy to formalize this intuition into a theorem.

Reviewer comment (Novelty):
The idea of learned temperature has been fully studied in previous works like GODIN (Liu et al., 2020). This paper seems to be an incremental work.

Author response:
It is true that the idea of a learned temperature has been studied in GODIN. However, this is the first work where it is combined with an energy score. We additionally contribute the dropping of the forefront temperature constant. And we adapt our score the semantic segmentation and object detection settings.

Reviewer comment (Experiments):
In figure 1, as the Learned Temperature contradicts with energy score, how about using the negative temperature to boost the energy, rather than simply ablating the temperature?

Author response:
we attempted using various inversion techniques on the temperature, but found that performing the ablation performs approximately the same as all of our tested inversion techniques. This is because our AbeT score already is inversely proportional to the learned temperature, as a result of the exponential divisor term.

Reviewer comment (Evaluation):
The experimental results should be further discussed. For example, in Table 1, as vanilla AbeT achieves 40% of FPR95 in ImageNet-1k, why a simple combination with ASH leads to such a surprising 3.7% of FPR95?

Author response:
In the original ASH paper [1], ASH-S 95.12 is already at AUROC using ImageNet-1k as their ID dataset. As can be seen in Table 1 of our manuscript, we found a similar result in combining Energy and ASH, with a slight improvement to 96.59 AUROC. Given that AbeT is a significant improvement upon Energy (as can be seen by our performance metrics), we find that this jump from 96.59 of Energy + ASH to 99.00 AUROC of AbeT + ASH to be in alignment with other experiments and literature. 

[1] Djurisic, Andrija, et al. "Extremely simple activation shaping for out-of-distribution detection." arXiv preprint arXiv:2209.09858 (2022).

Reviewer comment (Novelty):
Lack of novelty: While being able to achieve good performance empirically, the proposed method is a simple adaptation of two existing methods.

Author response:
* Reviewer comment: While being able to achieve good performance empirically, the proposed method is a simple adaptation of two existing methods.	
* Response: We do acknowledge that our method is simple. However, we view simplicity in the presence of efficacy as far superior to complexity in the presence of efficacy.

Reviewer comment (Theory):
Lack of theoretical justifications: While empirically proven, the proposed decision on dropping Forefront Temperature Constant, a crucial part of the proposed method, lacks theoretical justifications.

Author response:
* Reviewer comment: Lack of theoretical justifications: While empirically proven, the proposed decision on dropping Forefront Temperature Constant, a crucial part of the proposed method, lacks theoretical justifications.
* Response: we ask of the reviewer if the explanation in section 3 is not sufficient in providing intuition. If so, the authors would be happy to formalize this intuition into a theorem.

Reviewer comment (Presentation):
Formatting: the paper is not in the ICLR conference formatting.

Author response:
* Reviewer comment: Formatting: the paper is not in the ICLR conference formatting.
* Response:If accepted, we will change the formatting to be according to the ICLR template.

Reviewer comment (Experiments):
Based on my understanding of table 1, no ablation study was done to compare the effect of dropping Forefront Temperature Constant. How does the performance of Energy + learned temperature compare to the proposed AbeT?

Author response:
* Reviewer comment: Based on my understanding of table 1, no ablation study was done to compare the effect of dropping Forefront Temperature Constant. How does the performance of Energy + learned temperature compare to the proposed AbeT?
* Response: please see Appendix Table 10 for results of energy + learned temperature with and without ablation, as requested