Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: The authors tackle the issue of out-of-distribution detection for deep learning classifiers by proposing two regularization terms that can be added to a loss-function to fine-tune a pretrained network in order to ensure a calibrated probability output of P(X) that can be used to detect OOD samples. 

Existing approaches rely on either using a function of the logits P(Y|X) or directly estimating P(X) based on high-level features of the deep learning classifier. However, the logits of a deep network can be overconfident (mis-calibrated) [1] and are often not linearly correlated with P(X) for OOD samples [2]. Furthermore, density estimation on a biased low-dimensional projection of X might not produce an unbiased estimate of P(X). 

Instead, the authors propose to compute P(X) based on the softmax logits (P(Y|X)) after first (re)-calibrating the joint distribution P(X,Y) (g(x,y) in Eq. 3). Towards this goal, they derive a density-consistency regularization (DCR) term  based on the asymptotic testing of the consistency of P(Y) (based on logits) and its empirical density function, in a batch-wise fashion. 
This regularization term effectively (although asymptotically) re-calibrates v(Y) and thus, according to the author’s derivations, re-calibrates the joint distribution h(X,Y). Since a discriminative model already optimizes for the accuracy of P(Y|X), calibrating P(X,Y) = P(Y|X) P(X) should ensure a calibrated P(X) which can be used as a reliable OOD score.

As a second contribution, this paper introduces a contrastive distribution regularization (CDR) term that incentivizes a high likelihood ratio between augmented samples (assumed to be distribution-deviating) and in-distribution samples. 


[1] Matthias Hein, Maksym Andriushchenko, and Julian Bitterwolf. Why relu networks yield high-confidence predictions far away from the training data and how to mitigate the problem. In 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 41–50, 2019

[2] Weitang Liu, Xiaoyun Wang, John Owens, and Yixuan Li. Energy-based out-of-distribution detection. In Advances in Neural Information Processing Systems, pages 21464–21475, 2020.

strengths_and_weaknesses: Strengths:
* The proposed approach is simple and the derivation is straightforward. 
* The idea of converting the asymptotic consistency test into a regularization term for fine-tuning seems novel to me. The idea of calibrating P(Y) to ensure the calibration of P(X) also seems new.
* The authors present an extensive empirical study in which their approach outperforms most state of the art benchmarks. 

Weaknesses:
* The writing could be clearer. For example, the authors claim to propose a new approach for estimating P(X) when they are actually re-calibrating the logits-based estimation of P(X). “Calibration” was not once mentioned in the paper even though it is equivalent to the assumption of Eq. 3 (that the learned model is faithful to the joint distribution).
* The authors emphasize on the novelty of the density-consistency regularization term. However, from the empirical study as well as previous literature, it is highly likely that the contrastive-loss term is the main driver of the improved performance. Ablation studies do not disentangle the contribution of this term from that of the density-consistency one (Table 2 is lacking a row for enabling CDR alone).
* It is difficult to assess the significance of these contributions due to the lack of a detailed (rather than aggregated) ablation study and the use of smaller networks than commonly deployed for the ImageNet results (which I believe are more interesting than those of CIFAR-10/100). 

[3] Minderer, Matthias, Josip Djolonga, Rob Romijnders, Frances Hubis, Xiaohua Zhai, Neil Houlsby, Dustin Tran, and Mario Lucic. "Revisiting the calibration of modern neural networks." Advances in Neural Information Processing Systems 34 (2021): 15682-15694.

questions: 1. It is unclear to me why the OOD performance degrades with a larger batch size (Table 3) as it should ensure a tighter CLT bound thus a more accurate analytical approximation in the density-consistency regularization (DCR) term. Do the authors have any intuition regarding this?
2. To establish the utility of density-consistency regularization, could the authors compare and report the CDFs of the joint (X,Y) or marginals between the networks fine-tuned by the baselines and that using the DCR loss?
3. The aggregation of the results across different metrics for the ablation study somewhat defies the purpose of disentangling which features matter for which datasets, especially when there is a high variance of difficulty across the datasets (20% FPR vs. ~0). Would it be possible to de-aggregate these results in the appendix?
4. This approach does indeed seem simple and, based on the reported results, effective. However, could the authors provide some information on the inference time/memory requirements of this approach (as compared to some memory-intensive approaches such as Gram, for example) ?
5. Could you report the performance of adding CDR alone in Table 2? While the paper is anchored on the novelty of the density consistency regularization term, it seems that the most empirical gain might be from the contrastive distribution regularization term.
6. What do you mean by "q is a detached tensor dynamically determined to lienarly map the exponent..." shouldn't this just be a constant for numerical stability rather than being an adaptive tensor?
limitations: 1. The performance on ImageNet is not as impressive as that on CIFAR-10/100 which brings up the question of whether this approaches scales with the number of classes or size of dataset. Furthermore, larger networks that are becoming more commonplace for ImageNet-derived tasks might be better calibrated [3]. It would be interesting to verify that this approach can be as effective on these modern architectures. 
2. While the authors claim robustness to hyper parameter choices, the FPR metric does seem to change significantly (on the average benchmark) with different choices of “r” and batch size. It would be interesting to see if that's on a subset of the dataset or a common theme.
review_title: null

final_score: rating: 5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.
review_content: summary: A method is presented to fine-tune pretrained networks by introducing regularization terms such that the marginal density $p(y)$ estimated from logits matches the empirical density $n_y/N$. The claim is that this method allows the sum of logit-exponentials $\log(\sum_y \exp(h_y(x)))$ to be used as a score for OOD detection. Effectiveness on various datasets is examined with good results.
strengths_and_weaknesses: 

The exposition is heavy on mathematical proofs at the cost of conceptual clarity and readability. A lot of emphasis is placed on various proofs, but the intuition and concept do not come across as clearly. For instance, in a paper on OOD detection, what is used as the OOD score should be clearly emphasized. However, here the fact that $\log(\sum_y \exp(h_y(x)))$ is used as the eventual OOD score is mentioned only fleetingly, and is easy to miss on the first reading. 
Some recommendations for improvement:
1) I feel that the training algorithm should be included in the main manuscript instead of the appendix since it clarifies how the various regularization terms are calculated and gives a clearer picture of the overall flow. Some of the details of the lemmas and proofs can be moved to the Appendix.
2) Mathematical notation should be simplified and made consistent to improve readability. For instance, $y$ is used both as the dependent variable in Eq. (2) and also as the index of summation in the denominator. The term $\Upsilon^y_n$ in Eq. (9) seems to be a scalar. However, in Eqs. (12) and (13), the matrix transposition operator is applied to it. Can the authors please clarify how to interpret $\Upsilon^y_n$? Also, notation should be explained where it is introduced. For instance $\mathbb{S}$ is introduced in Eq. (16), but is explained almost a page later on line 274. 

The paper is well motivated, though, and the results seem good, but there is a question about benchmarks (explained in the next question before). 


questions: 1) In lines 243-245, the OOD score is specified as $\log(\sum_y \exp(h_y(x)))$. Since log is a monotonic function, we can focus instead on the term inside the logarithm i.e. $\sum_y \exp(h_y(x))$, which is basically the sum (or average) of logit exponentials (which is the same as the denominator of the softmax function). Can the authors motivate this intuitively? How and why is it better than using the softmax score, itself? Specifically, how does introducing the regularization terms during training impact the characteristics of the logit such that this term $\log(\sum_y \exp(h_y(x)))$ can serve as an effective OOD score?

2) A related question: it is stated that "existing detectors relying on discriminative probability suffer from the overconfident posterior estimate for OOD data." This is true, particularly for Softmax based OOD scores. This is closely related to the problem of uncertainty calibration and plenty of research has been done on softmax calibration. Can the authors comment if their OOD score, which is also closely derived from logit-exponentials, is better calibrated? 

3) Can the authors please clarify how the numbers reported in Table 1 were obtained? Assuming FPR95% means FPR when TPR=95%, then the benchmarks would seem to be performing remarkably poorly, getting false positive rates of well over 70-80% ? If so, these do not seem to agree with what is widely reported in literature for these methods. For example, TNR rates of well over 80% are reported in reference [9] (Mahalanobis), which would imply FPRs of under 20%. The authors should clarify any differences in settings that were used to obtain their results on benchmarks.

4) The distribution deviating operator $\mathcal{S}$ comprises fairly standard transformations such as rotations, crops, etc. Typically these are used to generate positive or similar samples in contrastive learning, but here they are being used to synthetically generate out-of-distribution samples. Wouldn't this result in such inputs (from the in-distribution dataset that have been rotated, cropped, etc.) being classified as OOD? 

5) It is claimed that the computational algorithm that is developed, "can be used for any pretrained classification network." Hence, the proposed approach is akin to fine-tuning it's parameters with additional regularization terms. Can the authors comment if they tried training a network from scratch (ie with randomly initialized weights), with their combined loss function?
limitations: No concerns with potential negative societal impact.
review_title: null

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: This paper highlights the drawbacks of current OOD detection methods that most of them try to model ID p(x) using the features that are intially trained to fit p(y|x).
To solve the distribution misalignment, a novel density-consistency regularizaiton is proposed.
A contrastive criterion is also used to flatten p(x) so that OOD samples can locate in low-density area to be identified easily.
The proposed method is evaluated upon both OE and non-OE settings, and the performance exceeds the previous methods in a large margin.
strengths_and_weaknesses: Pro:
- The paper is well-motivated, and the methods can address the problem intuitively.
- The method rely on proven assumptions with theoretical proof.
- The paper obtains significant great results across various experiments with fixed hyperparameters.


Con:
- The results in Table 1 are super charming. However, for CIFAR-100 experiments, I am more interested in the results when CIFAR-10 is OOD, as CIFAR-10 and CIFAR-100 are generally looks alike, but they are from different label distribution. This result usually clearly show how the method capture clues to distinguish ID/OOD. 
In CV, OOD samples are usually considered as semantic anomalies to ID datasets. However, a common shortcoming of current OOD detection evaluation protocols use different datasets as OOD, so that some methods can use low-level superficial shortcut to make good results (e.g., a simple low/high-resolution image classifier might do extra well on some OOD datasets). But actually, we hope the model uses semantic difference to distinguish ID/OOD. I hope the proposed method gets the better result not by superficial shortcut. The easiest way to proof is to show (ID: CIFAR-10, OOD: CIFAR-100) and (ID: CIFAR-100, OOD: CIFAR-10) cases.
- The paper points out the semantically overlapping objects between ImageNet and iNaturalist. In some other works such as `ViM: Out-Of-Distribution with Virtual-logit Matching`, `https://github.com/hendrycks/natural-adv-examples`, `Scaling Out-of-Distribution Detection for Real-World Settings`, some cleaner OOD datasets for ImageNet are provided. Authors are suggested to do experiments on these clean ood data. Or to support the reason that the proposed method does not do well on Texture / iNaturalist due to the label noise, the author shall show the appropriated sampled semantically-overlapping objects get high confidence.
- It would be great if the authors provide more visualization or deeper analysis on what DCR and CDR impact (maybe on the feature space) to help readers better understand the method.
questions: - How is Mah implemented? Only use the last layer feature to compute mah dist or use features from multiple layers and use validation set to learn optimal weight combination of different feature layers, as the original paper suggest?
limitations: The authors do not include the discussion on the limitation of the method.
We hope the authors can provide more analysis on its weakness in discusssion.
review_title: null

final_score: rating: 4: Borderline reject: Technically solid paper where reasons to reject, e.g., limited evaluation, outweigh reasons to accept, e.g., good evaluation. Please use sparingly.
review_content: summary: The paper proposes to improve OOD detection with energy score [1] by incorporating a density-based regularizer during training that promotes better estimate of P(Y). In particular, the regularizer is derived based on the rejection criteria of a hypothesis test to ensure the consistency of Monte Carlo mean of P(y) (estimated per batch) and the empirical mean in the training set (n_y/N). 




[1] Liu et al., Energy-based out-of-distribution detection, NIPS 2020. 
strengths_and_weaknesses: Strengths
- The motivation that existing OOD detection scores suffer from poor density estimation is clear and the task is important. 
- The proposed method based on law of large numbers and central limit theorem is simple. 

Weaknesses
- The rationale behind the proposed method is unclear to me. Currently the works promotes Monte Carlo estimate of P(Y=y) to be closer to n_y/N in the training set. However, during test time, this assumption may not hold: OOD detection concerns with label shift and it is unclear why one wants to overfit the density of label distribution in the training set.
- While both Lemma 1 and Lemma 2 hold when N goes to infinity,  the batch size used in practice is 256, where Monte Carlo estimate can result in large variance and unreliable estimation. Therefore, there exists significant gaps between the proposed theoretical insights and practical implementation.
- While y is in low dimensional space, obtaining P(Y) requires integrating over x, which are high dimensional vectors and therefore the effects of dimensionality on the accuracy of estimation is non-negligible. 

questions: It would be great if the authors can provide further clarifications on the above Weaknesses points and the following two points:

- It is unclear why ImageNet(R)  and ImageNet(C) can be used as OOD sets for CIFAR (ID). If OOD is defined based on the label, ImageNet contains categories with overlapping semantics with CIFAR and therefore may not serve as valid OOD test sets.

- The reported results for energy score seem much worse than the numbers in the original paper [1] under the same architecture (WideResNet). Are there any potential reasons for the discrepancy? 


[1] Liu et al., Energy-based out-of-distribution detection, NIPS 2020. 
limitations: The paper included discussions on limitations.
review_title: null

Meta-review: metareview: This paper develops a method for improving out-of-distribution detection in deep learning based on a novel regularization term.  There was significant variance in review scores with two championing the paper for acceptance and two borderline scores (7, 7, 5, 4) resulting in an aggregated score just above borderline accept.  The reviewers arguing for acceptance found the method novel, the simplicity of the algorithm compelling and the experiments extensive and convincing.  One reviewer was concerned that baseline comparisons provided in the paper seem less strong than reported in other work.  Two reviewers questioned the mathematical derivations and some of the underlying assumptions of the paper.  

That two reviewers are arguing for acceptance is a signal that the paper could be a useful contribution and interesting to the community.  Since the experiments seem extensive and seem to demonstrate that the method consistently works well, and given that it is simple to implement, that seems to validate the underlying assumptions and it could provide a useful baseline.  Therefore the recommendation is to accept the paper.  Please make sure to address the remaining reviewer concerns in the final manuscript.  
confidence: Less certain
Decision: Accept