Venue: NeurIPS 2021 Conference

Reviews:

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: The manuscript proposes using the gradient norm of a trained classifier for out-of-distribution detection. Concretely, the norm of the gradient of the KL divergence between the actual prediction and the uniform prediction with regard to the network parameters is used to distinguish in-distribution and out-of-distribution data. Results on several benchmarks show this method has better OOD detection performance than some other methods.

main_review: ** Update**: Increase score to 6 due to added experiment SVHN vs CIFAR10

I found the manuscript fairly easy to read and understand, the method seems sensible and the results are interesting.

I see some weaknesses in the discussion of related work and in the evaluation.

I would suggest to more clearly and extensively discuss in one place exactly the relation of this work to [28]. And refer to that more extensive discussion in the introduction already, as it seems very crucial in order to determine what is novel about this work. Like it seems to me that that [28] use may be the same gradients as in this manuscript? From their “To generate gradients, we utilize a confounding label where all classes are positive (i.e. a vector of all 1’s).” Or is there a difference still between the two? Somehow it seems the motivation in this manuscript and theirs is opposite, judging from ““If the model is unfamiliar with the given input, however, the amount of updates required to 1) learn new features to properly represent the new input, and 2) associate the new features with the confounding label, would be larger” [28] and “Our operating hypothesis is that using the KL divergence for backpropagation, gradient norm is higher for ID data than that for OOD data“ (this manuscript). Do you have any comments on that? Or maybe I have some misunderstanding?

Regarding empirical evaluation, where do the numbers for [28] come from? Comparing Table 4 in this manuscript to Table 1 in [28] the numbers seem completely different. Also, shouldn’t your method perform worse since their method even trains an extra classifier that sees OOD data? It would be good to also clearly show what is the difference in method that causes your results to be better, if they are indeed so.

Also I think it would be good if you could clearly delineate what methods you are including and excluding in your comparison. There are so many variants for OOD detection methods, generative, discriminative, with outliers, without outlines etc. and it would be good to know what you consider fair comparisons and what not.

It would also be important to report values not only for CIFAR10 in-dist vs SVHN OOD but also reverse case, SVHN in-dist vs CIFAR10 OOD as some methods degrade the performance in the latter “easier” setting.

How do the results in 4.2 table 3 relate to the results in table 2? Seems even the best results from table 3 do not reach the results from table 2?

You should include https://arxiv.org/abs/1911.07421 and https://arxiv.org/abs/2006.10848 in related work generative models and your supplementary comparison for generative models. https://arxiv.org/abs/2007.05566 could be another interesting related work.

Analysis and Ablation studies seem interesting to me.

What was the motivation for training your own networks and not using pretrained networks on CIFAR10 etc.?


limitations_and_societal_impact: I only saw a very minimal discussion of this at the beginning of the introduction.
review_title: null

final_score: rating: 7: Good paper, accept
review_content: summary: The work presents GradNorm which uses the norm of gradient, back-propagated from the KL divergence between the softmax output and a uniform probability distribution, as the uncertainty score for OOD detection. The experimental results show that the proposed method outperforms the baselines on ImageNet1k vs OOD and CIFAR vs OOD benchmarks. Ablation studies are presented to help better understand the proposed method. The paper is written well and the experiments are comprehensive .

main_review: - Originality: The proposed method GradNorm is novel, simple and efficient, as demonstrated by the experimental results. Most existing work rely on the output (MSP, ODIN, SNGP) or feature space (Mahalanobis) for deriving OOD scores. This work suggests making use of the information from the gradient space for OOD detection. 

- Quality: The results are presented clearly. Comprehensive ablation studies were conducted and that did help me better understand the method.  It would be great to include the benchmark dataset CIFAR10 vs CIFAR100 as one of the near-OOD tasks.

- Clarity: The work is well written and the content is well organized. One comment I have is regarding one of the baseline methods, called Lee and AlRegib [28]. It is not clear how the experiment for this method was conducted exactly. As mentioned in line 151-152, Lee and AlRegib requires “a binary classifier trained on the OOD datasets, which can unfairly overfit to the test data and does not suit OOD-agnostic settings in the real world”. But it is also stated in line 139-140 that “for a fair comparison, all the methods use the same pre-trained backbone, without regularizing with auxiliary outlier data”.

- Significance: the results are important. This work inspires the future work to use the information from the gradient space for OOD detection. Combining the information from gradient space with that from the output space and feature space, would potentially result in a better OOD score.


Update: Thank the authors for responding to my questions. I keep my score. I would suggest the authors to incorporate the reviewers' comments to the final version. 
limitations_and_societal_impact: The limitations were not addressed. It is unknown if the method works well on discrete data like text. That can be left as a future work.

Societal Impact was not discussed.
review_title: null

final_score: rating: 8: Top 50% of accepted NeurIPS papers, clear accept
review_content: summary: The work proposes a novel method for OOD detection. The method itself uses as scoring function the norm of the gradient of the KL divergence of the softmax vector to a uniform distribution. Importantly, this method does not require ground-truth OOD data and does not require hyperparameter tuning. The main contribution of the work is empirical demonstration that the method achieves state-of-the-art performance on a variety of datasets, as well as ablations to further understanding into the effectiveness of the proposed method. 

main_review: I think this paper is overall a solid contribution and would recommend acceptance for the following reasons. 
- The method is relatively simple and practical. The only operation is taking the norm of the gradient of the KL divergence, and does not require ground-truth label information. This latter fact is especially important as it might obviate the need for excessive human intervention in practice. Methods that use ground-truth label information almost seem to be “cheating”, and seem to have a scaling problem by requiring samples from the OOD dataset to begin with. 
- The writing and organization were exceptionally clear throughout. I do not actively do work in OOD detection, yet I found the work easy to follow. 
- The experimental methodology is high-quality and demonstrates a significant improvement over baselines. 
    - The baselines were tuned, which is especially important because the paper has many experiments on imagenet, but some of the baselines were originally only tested on the cifar datasets. 
    - There is a large variety of datasets considered as well, which helps to give a balanced sense of the performance of the proposed method. Indeed, the method performs worse than some baselines on some datasets, yet performs the best on average across all datasets. 
    - The ablations (to KL divergence, different L_p norms, different temperatures, gradnorm of different layers, one-hot vs. all labels, feature and output space components) were extensive and interesting. I especially found comparison to using just the norm of the KL divergence interesting; based on the motivation on line 35, I would have expected KL divergence to do better than the proposed method (more on this point below). 
- I liked the mathematical analysis in section 5 to decompose the gradnorm into a feature space quantity and an output space quantity, and especially appreciated how it led to another ablation experiment. Given that the main contribution of the paper is empirical, I saw this section as a nice bonus to further understanding of the proposed method, and in future work would look forward to seeing more extensive mathematical analysis. 

There did remain limitations of the paper, but these were not enough to outweigh its advantages above. I think addressing these limitations would make the paper stronger.
- I found the intuition for the gradnorm to be lacking. The main intuition is on lines 35 and 94, where I get the impression the ID data should have a higher KL divergence. However, it’s not clear to me that the gradient norms for ID data should be higher. The experiments certainly demonstrate that the gradient norm is a good discriminator of ID and OOD data, but I would have liked better intuition for this result. 
- There was no discussion of limitations or negative potential impact, which is a problem considering the explicit NeurIPS requirement this year. As a suggestion to the authors, I think the derivation of the gradnorm decomposition in section 5 could be left to the appendix so that more room is left in the main body for a discussion of limitations and negative potential impact of the work. 

limitations_and_societal_impact: copied from above: 

There was no discussion of limitations or negative potential impact, which is a problem considering the explicit NeurIPS requirement this year. As a suggestion to the authors, I think the derivation of the gradnorm decomposition in section 5 could be left to the appendix so that more room is left in the main body for a discussion of limitations and negative potential impact of the work.
review_title: null

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: This work presents a new method for detecting out-of-distribution inputs that relies on the gradient norm of the KL divergence between the model’s softmax output and a uniform probability distribution.  The method is easy to implement (using the norm of the gradient of a standard cross entropy loss with a uniform label vector), can be used at inference time without access to true labels, and does not require training an additional model.


The authors present empirical evidence that their method outperforms prior work by a significant margin, reducing the false positive rate (FPR95) for out-of-distribution detection by 10.89% and 14.47% on ImageNet and CIFAR-10 respectively. Additionally, the authors perform an ablation study that explores how their method performs as they vary layer parameters, type of target distribution used (uniform versus real), temperature, type of gradient norm, and using only individual terms that form a decomposition of the gradient norm.

main_review: **Major Comments**

1. *Evaluation on covariate shifts.* The authors only evaluate on datasets where the class hierarchy is distinct from that of the in-distribution dataset.  It is not clear that GradNorm would still perform well on covariate shifts. The authors claim that the GradNorm technique is effective since the in-distribution predictions tend to concentrate on the ground truth class and are thus distributed less uniformly.  It’s possible that covariate-shifted data would similarly have a higher GradNorm because the predictions would concentrate on the ground truth class. For ImageNet, the authors could have evaluated on ImageNetV2, ObjectNet, ImageNet-R, ImageNet-C, ImageNet-Vid-Robust, or YouTube-Bounding-Boxes. For CIFAR, the authors could have evaluated on CINIC-10, CIFAR-10.1, or CIFAR-10C. 

2. *Hyperparameter to set the gradient norm threshold.*  The authors claim that their work is hyperparameter free if one sets the temperature to T=1.  However, isn’t the threshold on the gradient norm for separating in-distribution versus out-of-distribution data ($\gamma$) an important hyperparameter?  Clarifying how this parameter is set for different datasets would improve the paper.

**Minor comments**

3. *Calibration*. From Figure 7, it looks like the authors only tried one temperature value that was smaller than T=1. The authors also do not give an explanation for why temperatures smaller than T=1 would work. It would also be interesting to see if the optimal temperature for calibration would fare any better than T=1.


4. *Explanation of prior work.* It would be helpful if the authors gave a brief description of prior work they compare against, e.g. MSP, ODIN, Generalized ODIN, Energy or Mahalanobis. 

5. *Effect of label noise.* Mislabeled images might force the in-distribution predictions to be more uniformly distributed. Have the authors looked at whether GradNorm can be used to detect label noise? Would the technique be able to distinguish between out of distribution and label noise?

6. *Organization.* The section “Effect of temperature scaling” is an ablation study and should be included in Section 4.


**Originality**: The method is novel and the authors describe how it is distinct from prior work.

**Quality**: See comments above.

**Clarity**: Paper is clearly written, but could be improved by clearer descriptions of related work.

**Significance**: Problem is important, and proposed algorithm is easy to implement and deploy.

If major comments are addressed, I would consider raising my score to a 6.
limitations_and_societal_impact: The author should discuss potential negative societal impacts of their work in the main paper.
review_title: null

Meta-review: This paper proposes using the KL divergence between the softmax output and the uniform distribution to detect OOD examples (OOD examples are expected to have lower KL divergence). Authors further show extensive empirical results in support of their method.

All reviewers agree that this work is novel, well-written and is a significant contribution to the community. Reviewers have provided detailed comments to improve this work and increase its impact and I highly recommend authors to take all these suggestions into account. In particular, I suggest further explanation and investigation about the effectiveness of this technique when the class label sets are distinct as well as when there is covariate shift.

Given reviewers' consensus, I recommend accepting the paper.
Decision: Accept (Poster)