Venue: NeurIPS 2021 Conference

Reviews:

final_score: rating: 7: Good paper, accept
review_content: summary: This proposes ReAct to reduce model overconfidence on OOD data.
This paper observes the mean activation of OOD data in the penultimate layer has larger variations across units and is biased towards having sharp positive values, which produces overconfident predictions on OOD data.
ReAct rectifies the activations at an upper limit for OOD data and preserves the activation for ID data.
The results show that ReAct makes output distributions between ID and OOD data more separable.
main_review: The overall methods is simple and effective to detect OOD data.
Extensive expermental and theoretical analysis are given.
limitations_and_societal_impact: -- ReAct performs well on the classification networks. Can ReAct help OOD detection in other computer vision tasks, such as detection and segmentation networks?

-- As  OOD data can trigger unit activation patterns that are significantly different from ID data. Why not detection OOD data from such separable unit activation patterns directly?
review_title: null

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: The authors propose a simple method for OOD detection, called ReAct, which truncates high activations at test time. Experiments show that ReAct performs better than state-of-the-art OOD detection methods. The authors also theoretically verify the effectiveness of ReAct.
main_review: The proposed ReAct is simple, novel, and shows high performance compared to conventional OOD detection methods. In addition, the entire paper is very readable with clear arguments. However, there are some inadequacies in the verification and other aspects as follows.

- I think that the performance of ReAct is greatly affected by the layer at which it is applied, but this is not fully verified in this paper. The Appendix shows the differences in activation for each layer, suggesting that the deeper the layer, the easier it is to distinguish between ID and OOD. However, the authors do not indicate the extent to which these differences actually affect performance. In addition, the Appendix examines the activity values of ResNet-50, but they should verify and show whether the trend is similar for other architectures (such as ResNet-18 used in Sec. 4.2).
- In the proof, the authors model the distribution of OOD as the epsilon skew-normal distribution, since it is positively skewed. In the proof, the distribution of OOD is positively skewed, so it is modeled by the epsilon skew-normal distribution. Under this assumption, they prove that ReAct is effective. However, it does not explain why the distribution of OOD becomes positively skewed. In addition, it is not clear whether there is a case where the distribution of OOD is not positively skewed (which is directly related to a limitation of the proposed method). The authors should clarify them.
- According to the setup, the authors use a validation set of Gaussian noise to determine the value of p. However, in order for this Gaussian noise tuned p to be valid for testing, the validation set of Gaussian noise and the test set, i.e., the OOD, should have the same distributional trend. The authors should verify this.
- Other minor comments
  - This method is simple and should be easy to implement, but I think it would help the reader to understand it better if the code is included.
  - There should be a clear formulation of what S is around Equation 3.  I found out later that this is a score function, meaning any existing OOD scoring function, but it should not be introduced into Equation 3 without explaining.
limitations_and_societal_impact: The authors claim that the proposed method shows good performance on a variety of data and settings and can be combined with various existing OOD methods, but they are not very explicit about the limitations of the proposed method. For example, this study proves that the proposed method is effective under the hypothesis that the distribution of OOD is positively skewed, but if this does not hold, there should be no guarantee that the proposed method is effective. The authors should include such discussions.
review_title: null

final_score: rating: 4: Ok but not good enough - rejection
review_content: summary: The paper proposes a very simple way of improving out-of-distribution detection performance in neural network classifiers by rectifying the pre-logit layer's activations.

main_review: Strengths:
The approach is very simple to apply. 
It is good that the authors demonstrate results on ImageNet which is not standard in the field of OOD detection.

Weaknesses:
The authors claim to compare to state-of-the-art methods, but the highest performing methods in the literature are variations on outlier exposure [16]. The authors correctly point out that this method uses additional data at train time. I think the authors should include separate evaluations, with their method being applied to outlier-aware approaches (like [16]) in addition to the outlier-unaware approaches that they study.

In the related work section the authors bring up [14] as an example of using an "auxiliary OOD dataset" during training. This seems like a stretch since that paper uses a noise distribution generated from the in-distribution as training out-distribution. By the same logic one could claim that this paper uses an "auxiliary OOD dataset" because they use a noise dataset as validation set for hyperparameter selection.

The technical contribution by the paper is trivial. Also, as the authors mention, the idea of clipping activations has been around for over ten years (clippedReluLayer even having an official implementation in matlab...), so the novelty of the approach is extremely limited.

The paper should definitely include their OOD performance measures for each test out-distribution (as opposed to just the means) in the appendix.

The authors show that their method leads to a degradation in test accuracy (which they correctly point out can be mitigated by an additional forward pass). However, it would be interesting to know if this drop in performance still occurs when training or even just fine-tuning with the ReAct activation. In general, I don't see why the paper doesn't investigate if doing this changes their performance at all. 


Clarity:
The paper is mostly simple to follow. However, there are a few issues.
In line 68 f is defined as a function from X to Y. However, later (e.g. in Eq. 2) it is clearly assumed that f represents logits and that f is a function onto R^|Y|.

It is not entirely obvious how the percentile is chosen. Line 120 states "We use a validation set of Gaussian noise" so presumably this is the OOD dataset that is used to choose the percentile. However, first of all no details of this Gaussian noise are given (mean, variance, clip to image space or not?) and the authors do not even state which metric is being optimized for (AUROC, FPR95, AUPR?).


Additional feedback:
In line 243 the authors state that it is surprising that using OOD batch norm statistics normalizes the mean and variance of the activations. I don't see how this is surprising at all. That is the point of batch norm after all.

The writing of the paper needs much improvement. The style is occasionally too informal (e.g. line 22 "[...] modern neural networks can, contrary to popular belief, produce overconfident prediction [...]", who holds this "popular belief"?) and there are many typos (e.g. line 84 an->a, line 115 class->classes, line 140 given->give etc...). 

The formatting of paragraphs is somewhat inconsistent. Some end with periods ".", others do not.
limitations_and_societal_impact: The paper omits the mandatory discussion of their paper's weaknesses and limitations. No reason for this is given in the checklist.
review_title: null

final_score: rating: 7: Good paper, accept
review_content: summary: This paper identifies that out-of-distribution (OOD) data tend to cause overconfident activations much more often than in-distribution data. Based on this observation, a simple approach called ReAct (Rectified Activations) is proposed, which essentially applies a threshold to the activations of the penultimate layer of a CNN classifier. Extensive experiments demonstrate that ReAct works with different network architectures and OOD detection algorithms, and the improvements over previous SOTA methods are significant.
main_review: **Strength**

- ReAct is well motivated. Though the design is simple, it works very well with different network architectures and is compatible with several existing OOD detection methods.
- A theoretical analysis is provided to justify ReAct's effect on reducing both the activations and output for OOD data.
- The experiments are extensive and the results are strong.
- The analysis part is insightful, especially the discussions that connect the overconfidence issue to BatchNorm. I think the findings and results could be useful to research beyond OOD detection, such as OOD generalization and domain adaptation.
- The paper is well written and easy to follow.

**Weakness**

I do not find major issues in this paper. The following comments are mainly related to the presentation.

- By just reading the abstract and the introduction, it is unclear how and why ReAct helps OOD detection. It would be clearer to mention that the OOD detection part is based on existing methods such as softmax scores or the energy-based scores. It would also be better to highlight that the idea of ReAct generalizes to different network architectures and works with different OOD detection methods.
- Though the motivation for trimming the activations is intuitive, the reason why ReAct improves the OOD scoring functions is a bit vague. Taking the energy function (Liu et al., 2020) as example, which essentially associates the sum of logits to the marginal distribution (the likelihood). It is less intuitive why lowering the confidence on OOD samples helps make the OOD scores more distinguishable.
- Line199: Fig.2 shows that the left tail has a higher frequency than the right tail. Why the right tail has a higher density? Is this a typo or is there something wrong with my understanding? Please clarify on this.

---
Post-rebuttal update:

I have read the authors' responses and other reviewers' comments.

Overall, I find the proposed approach simple and well-motivated, and the experiments solid and extensive enough for supporting the claim. Hence, I think the paper is worth sharing with the community and keep my rating unchanged.
limitations_and_societal_impact: N/A
review_title: null

Meta-review: This paper observes that OOD samples tend to have overconfident activations and it proposes a simple activation clipping mechanism to reduce model overconfidence on OOD data. Although the proposed method is simple and straightforward, the reviewers found the theoretical analysis and insightful discussion valuable to the research field. Given the strong results and clear presentation of the ideas, I recommend acceptance. 
Decision: Accept (Poster)