Venue: ICLR 2022 Conference

Reviews:

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: The paper proposes to use a score based on the Fisher-Rao information metric (the Riemannian metric in the space of probability distributions) for the detection of out-of-distribution samples input to a trained DNN. While the output SoftMax probabilities are used in the black-box and grey-box scenarios, the learnt features in the intervening DNN layers are additionally used in the white-box scenario. The approach models each sample as providing posterior probabilities – (a) the SoftMax probability in the label space, and (b) class-conditional PDFs over the corresponding feature spaces for each DNN layer. These latter are modeled as multivariate Gaussian distributions with diagonal covariance matrices. Extensive experiments on existing benchmarks are conducted for comparative results against the state of the art demonstrating promising results. 
main_review: ## Strengths & Weaknesses 

**Clarity**: The paper is mostly clear and well written. The writing can be improved in some places (e.g. paragraph 2 in the Introduction)   

**Novelty**: While the Fisher-Rao metric has been applied in the context of deep learning (natural gradient, regularization of training, etc. -- need references here), I've unaware of its use for anomaly/ out-of-distribution (OOD) detection. It's a reasonable extension to evaluate the utilization of Mahalanobis distance in the prior art (Hendycks and Gimpel, ICLR'17). The novelty meets the bar for a publication without rising to the level of being significant.  

**Technically correctness**: The material is technical correct in the large. I went through the approach including the math in the paper and skimmed through the Appendix. I'm mostly familiar with the topic and the material seems correct though I didn’t check the appendix comprehensively. 

*(TC 1) Gaussianity*: There are two main parts for the proposed score – one based on the SoftMax output – this seems ok. My main concern is with the other part -- using the multivariate Gaussian model with a diagonal covariance for the feature spaces without first validating the premise with a Gaussianity test. While testing for high-dimensional multivariate Gaussians may pose difficulties, the diagonality assumption should make this feasible. I recommend that evaluation and analysis of this assumption is added to the paper. 

*(TC 2) PMFs and PDFs*: while the SoftMax formulation provides a posterior distribution in the label space, having this interpretation for the feature layers for a single input sample without further grounding in theory/ past literature makes the approach ad hoc. I'd like to see authors clarify this. 

*(TC3) Max and Average*: The authors comment that (for the SoftMax output) they obtain slightly better results using (6) -- average() rather than (5) -- min() over the classes.  

(a) Would using the average be tantamount to computing the probability with respect to a mixture distribution and admit a model more applicable for the scenario (single sample case) than a PMF/ PDF estimate interpretation?  

(b) Was this also tried with the features corresponding to equations (12) and (13)?  

**Experimental Evaluation**: A good number of experiments have been conducted and presented in the main paper for the black-box (Table 1) and the white-box (Table 2) scenarios. These are further supported by additional experiments in the Appendix E (Tables 5-11) as well as ablation studies for various (hyper)parameters important to the proposed approach. This is very good.  

On the flip side, some parts of the experimental validation can be improved to support better the central claims – that OOD scores based on the Fisher-Rao information metrics outperform the previous state of the art. 

*(EE1) Difficulty of directly comparing with published results*: Since the experimental settings seem different from other papers (DNN models used, finetuning dataset, etc.), it is really hard to directly compare tables in the paper to those published in the compared SOTA. I tried to do this both for the Tables in the main paper as well as in the Appendix. Since this beats the due diligence review process, the authors should explain why this is ok. 

(a) Black-Box settings: Entries in Table 1 (and Table 8) can't be compared with Table 2 in Hendryks and Gimpel (ICLR 2017) referred to as Baseline, Table 1 or Table 2 in Liu et al (NeurIPS 2020) or with ODIN - Liang et al (ICLR 2018). 

(b) White-Box settings:  I compared some numbers across Tables 10 and 11 with Table 2 in Lee et al (NIPS 2018) -- discrepancies in results exist, sometimes very significant,  but mostly (not all) the comparative accuracy improvements hold. 

I gave up on a more comprehensive cross-checking of the results but since there are differences in the setups.  

*(EE2) SOTA used for comparison*: Apart from Liu et al (NeurIPS 2020), all the compared approaches seem 3-4 years old. The authors should respond on whether those results are SOTA. 

*(EE3) Inconclusive evidence for improved performance*: Performance improvements vis-a-vis the reported SOTA is mixed – ref. Tables 8 – 11 though I consider the results to be promising. The authors should discuss this in their response. 

**Reproducibility**: I expect the results to be reproducible since enough details are shared in the paper and the code has been made publicly available at https://github.com/igeood/Igeood.   

 
summary_of_the_review: I recommend to accept the paper.  

It is a good addition to the set of methods on an important topic - OOD. The approach is novel, reasonably principled and the code has been made publicly available at [https://github.com/igeood/Igeood](https://github.com/igeood/Igeood) which is good for the community to able to put the above methods to test.  The validation is comprehensive though the results are somewhat inconclusive though promising. There are some concerns regarding the validation of assumptions and the experimental evaluation. My preliminary assessment is that the submission passes the criteria for acceptance to this venue.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This paper presents IGEOOD, a new method for detecting OOD samples by using geodesic (Fisher-Rao) distance in confidence scoring. It further combines confidence scores from the logit outputs and the layer-wise features of a deep neural network. The method is validated under various testing environments such as the availability of OOD data or the accessibility of latent features of a deep network. The idea of using Fisher-Rao distance for OOD detection seems novel and interesting. 
main_review: The theoretical basis of the proposed methodology is well described and the experiments appear to be well-designed overall. Empirically the proposed method outperforms other methods, especially in the white-box setting where OOD validation samples are available. 

According to Table S8 (Table 8 in the Appendix), the performance difference of the different methods appears to be marginal. Can you provide statistical significance for the performance differences? In addition, according to Table S9 (grey-box), ODIN performs better than the proposed method in most cases. Can you add some discussion on why? Moreover, the current description of the results given in the main text is a bit misleading because these observations are not properly explained.  

In the white-box setups, IGEOOD seems to perform best when OOD samples are available for validation (Table S10), but it does not when only the adversarially generated samples are used for validation(Table 2, Table S11). This result does not match the purpose of this study presented in Abstract and in Introduction (e.g., "IGEOOD applies to any pre-trained neural network, does not require OOD samples or assumptions on the OOD data").

Recently, many SSL (self-supervised learning)-based OOD detection methods have been developed. Examples are SSD[1] and CSI[2] that use the same confidence score as in Lee et al.[3] while utilizing self-supervision (simclr). I wonder 1) if the proposed method can be compared to these self-supervision-based models, and 2) whether the proposed geodesic distance can be applied to the SSL-based approaches.

[1] Sehwag, Vikash, Mung Chiang, and Prateek Mittal. "SSD: A Unified Framework for Self-Supervised Outlier Detection." International Conference on Learning Representations. 2020.
[2] Jihoon Tack, Sangwoo Mo, Jongheon Jeong, and Jinwoo Shin. Csi: Novelty detection via contrastive learning on distributionally shifted instances. Advances in Neural Information Processing Systems, 33, 2020.
[3] Lee, Kimin, et al. "A simple unified framework for detecting out-of-distribution samples and adversarial attacks." Advances in neural information processing systems 31 (2018).

The proposed model is based on the assumption that the layer output follows a Gaussian distribution. It is possible to provide validation or discussion of this assumption? 

Regarding Eqs. (5) and (6), the authors note that taking the sum (5) instead of the minimum distance to the class conditional centroid produces better results. This looks interesting. Can you provide empirical results or more analysis on this?

It would be interesting to see the score distributions as shown in Figure 1(c) for the real datasets used in this study. 
summary_of_the_review: This paper presents a novel idea of using Fisher-Rao distance for confidence scoring in OOD detection. The main idea is interesting, the paper is well structured in terms of methodological description and the problem/experimental setups, but the empirical results do not appear to be sufficient to validate the intended purpose of this study.
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: The paper proposes a new group of methods for supervised OOD detection. In particular, the authors propose to use the Fisher-Rao distance between output distributions on the in-distribution data and test samples to detect OOD. The authors additionally propose to use Fisher-Rao distance in the hidden layer feature space, when possible (white-box setting). The method achieves strong empirical performance, improving upon standard baselines (such as Odin, Mahalanobis), especially in the white-box setting.
main_review: ## Methodology

The authors propose two related methods: (1) measuring distance in the output space, and (2) measuring distance in the space of the hidden layer activations. (1) is quite intuitive, and as far as I know novel: to the best of my knowledge prior work typically considers just the confidence of the classifier, and not the full predictive distribution. (2) is quite similar to Mahalanobis [1], with the main difference being the distance metric used. However, the authors show significant improvements in performance compared to [1] in Section 4.3, justifying the proposed method.

## Results

The empirical results constitute the main strength of this paper: the method outperforms the considered baselines across the board. However, I have two concerns:

- The improvements in the Black-box setting appear very minor. In many cases, the method does not outperform the baselines, our improves the results by $O$(0.1%) (see e.g. the AUROC results for ResNet, in Table 1). It would potentially be helpful to add error bars to Table 1.
- The paper claims to set the new *state-of-the-art* on visual out-of-distribution detection, but the considered baselines are somewhat limited. OOD detection is a very active field, with many papers claiming to improve on the considered baselines [see e.g. 2-6]. From a quick comparison it seems like the results reported by the authors are competitive with the best methods I could find, but I think the authors need to do a better job comparing to prior work, in order to claim SOTA.

On the other hand, I want to highlight that the authors perform a fairly exhaustive experimental evaluation in terms of the out-of-distribution datasets considered for each in-distribution dataset, including both near- and far-OOD.

## Writing

The writing is generally clear, but there are several typos and minor inaccuracies.

## Comparison to other distance metrics?

As far as I understand, it is possible to replace the Fisher-Rao metric with any other distance metric in the method proposed by the authors? I think it would be interesting to see a comparison of the results with a few other standard metrics in the Black-Box set-up, e.g. KL-divergence, total-variation distance, etc. If the Fisher-Rao metric provides significantly better results, this experiment would strengthen the paper.


## References

[1] A Simple Unified Framework for Detecting Out-of-Distribution Samples and Adversarial Attacks
Kimin Lee, Kibok Lee, Honglak Lee, Jinwoo Shin

[2] Generalized ODIN: Detecting Out-of-distribution Image without Learning from Out-of-distribution Data
Yen-Chang Hsu, Yilin Shen, Hongxia Jin, Zsolt Kira

[3] Detecting Out-of-Distribution Examples with In-distribution Examples and Gram Matrices
Chandramouli Shama Sastry, Sageev Oore

[4] Hybrid Models for Open Set Recognition
Hongjie Zhang, Ang Li, Jie Guo, Yanwen Guo

[5] Deep Residual Flow for Out of Distribution Detection
Ev Zisselman, Aviv Tamar

[6] A Simple Fix to Mahalanobis Distance for Improving Near-OOD Detection
Jie Ren, Stanislav Fort, Jeremiah Liu, Abhijit Guha Roy, Shreyas Padhy, Balaji Lakshminarayanan
summary_of_the_review: In summary, I recommend a weak accept for this paper. The empirical results are good, and the method is generally novel. I am open to increasing my score, if the authors address the concerns I raised in my review.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: UPDATE:

I acknowledge that I've read the author responses as well as the other reviews. 

While the authors added further analysis on the proposed method, I am skeptical of the performance of the method since the proposed method requires validation OOD data to achieve SOTA performance and the runtime is much larger than MSP and energy baseline. However, I think the rebuttal clarified much of my concerns and therefore raise the score to 5 weak reject.
 
=================================================================

The paper proposes an out-of-distribution detection (OOD) metric based on Fisher-Rao distance which can be applied for pre-trained classifiers. First, the authors derive a Fisher-Rao distance applied to the distribution of softmax. Also, they motivate a toy example where the FIsher-Rao distance outperforms the conventional OOD metric, Mahalanobis distance. Furthermore, they formulate the Fisher-Rao distance-based framework, IGEOOD on Black(Grey)-Box, where we can only get access on the logit of the network output, and White-Box, where we can get access to intermediate feature layers. Finally, the authors compare IGEOOD against conventional OOD metrics on various out-of-distribution data and in-distribution data.
main_review: Strengths of the paper
1) The concept of applying the Fisher-Rao distance to out-of-distribution seems novel and important to me. Furthermore, examples on the Gaussian strengthen the motivation of the FIsher-Rao distance's theoretical benefit.
2) Proposed Fisher-Rao distance can be combined with pre-existing techniques, input-preprocessing, temperature scaling, and feature ensembling, to boost the OOD performance.
3) The proposed method shows competitive results on the black-box OOD detection setting.

Weakness of the paper
1) One major issue of the proposed algorithm is that we have to tune the parameter of centroid parameter of each class. While the authors have noted that they optimized for 100 epochs, I wonder how the extra computation time scales compared to the baselines. (e.g., Mahalanobis distance, energy-based distance), especially in the CIFAR-100 dataset, where we have to evaluate the mean of the centroid 100 times.
2) Furthermore, given the computation time of 1), experiment results are weak to champion IGEOOD as the OOD method suitable for pre-trained classifiers. In the grey-box setting, ODIN outperforms IGEOOD. Furthermore, in the white-box setting, the method is only compared against the Mahalanobis distance. 

Comments


1. As mentioned in the weakness section, the wall clock time of the IGEOOD on the various datasets can help to solve the time efficiency issue of the method.


2. Furthermore, in the white-box setup, I suggest comparing the method not only against the Mahalanobis distance but also on the recently proposed methods (e.g., [1])


3. In figure 1(c), while it's convincing that the Fisher-Rao distance improves detection against type-2 OOD data, the two histograms are not normalized enough to make comparisons. I suggest matching the area of in-distribution data frequencies between two histograms. 


4. ." Empirically, we show in the appendix (see Section C) that this confidence score does not degrade and
sometimes improves the in-distribution test classification accuracy".<<<< I found the results and the claim are rather redundant. If there is no stark improvement, why should we take a look into it?


5. For Figure 2, I suggest adding the distributions of the baselines for direct comparisons.


6. How the choice of validation dataset impacts performance <<< I am skeptical about the absolute, or relative robustness of IGEGOOD against the other algorithms. First, 8% variation in TNR does not seem so robust since the paper's gain against baselines is not bigger than 8%. Furthermore, I cannot find a major difference against baselines given the results in E.3.

==================================================================================================
References
[1] Detecting Out-of-Distribution Examples with Gram Matrices. C. S. Satry and S. OOre, 

summary_of_the_review: The paper proposed a new OOD detection framework, IGEGOOD. My biggest concern is that the experiment results are fairly weak given the complex procedure of obtaining the detection scores. Therefore, I am leaning towards the rejection of the paper.
review_title: null

Meta-review: This paper introduces a novel approach for out of distribution detection that generates scores from a trained DNN model by using the Fisher-Rao distance between the feature distributions of a given input sample at the logit layer and the lower layers of the model and the  corresponding mean feature distributions over the training data.

The use of Fisher-Rao distance is novel in the context of OOD, and the empirical evaluations are extensive.  The main concerns of the reviewers were the limitations of the Gaussianity assumption used in computing the Fisher-Rao distance and the use of the sum of the Fisher-Rao distances to the class-conditional distributions of the target classes rather than the minimum distance. These concerns were addressed satisfactorily in a revision. In terms of technical novelty, experimental evaluation and novelty, the paper is above the bar of acceptance.
Decision: Accept (Poster)