Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: This work proposes TRUSTED: an out-of-distribution detection approach that uses integrated rank weighted (IRW) depth to assess whether an example falls within a set of in-distribution samples in the embedding space of a transformer model. The average embeddings across all transformer layers are used for optimal performance. TRUSTED is compared against 1) integrated rank weighted depth using final embedding layer, 2) Mahalanobis distance instead of IRW and 3) maximum softmax probability on a text OOD benchmark and shows superior performance.
strengths_and_weaknesses: Strengths:
1. Approach and experiment settings are clearly explained.
2. Code is provided for reproducibility.
3. The detector can be applied to new transformer models without any training, as opposed to data-driven methods.

Weaknesses:
1. Experiments didn't show comparison against (any) existing works. 
- [36]
- [82]
- Arora, Udit, William Huang, and He He. "Types of Out-of-Distribution Texts and How to Detect Them." EMNLP 2021.

2. Approach is fairly simplistic. Novelty is limited. More suitable as a "strong baseline" type of approach.
- There's even very limited contributions to benchmarking of text OOD detection given that approach is simple. At the current state, a sound & large OOD benchmark with diverse text, different types of OOD, with text from multiple domains may be the easiest way to boost the contributions of this work.

3. Questions:
- Table 2 uncertainties are large even after averaging over 1440 configurations. Why is that?

=========================

With comparisons to existing works, the authors have adequately addressed my strong concerns on the experiments. I have updated the score.

questions: See Weaknesses.
limitations: Limitations are sufficiently addressed
review_title: null

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: The authors propose a novel out-of-distribution (OOD) detector for text classification based on similarity scores from the training distribution. The goal is to identify OOD samples with an unsupervised and fast method by leveraging information from the hidden layers of the representation model.  The main contributions are: i) use of hidden representations to compute a distance score for OOd detection, ii) comprehensive comparison of models and hyper-parameters, and iii) open source for replicability. The study shows that the proposed method has competitive results compared to related work.
strengths_and_weaknesses: Strengths


- Clear description of background knowledge and related work needed to understand the proposed approach.  

- Clear description of the proposed approach.

- The authors perform a comprehensive comparison with related work.



Weaknesses


- It is not clear how the parameter initialization  and selection of hyper-parameters could affect the method performance.
questions: Questions to the Authors

Please address the following questions during the rebuttal:

- Could you elaborate on the hyper-parameter selection and define the classifier architecture. 
- During training, are the pre-trained models freeze?
- As an extra contribution, the authors can compute calibration and sharpness of the predicted probability outputs. Please speculate on the relation between well calibrated output probabilities and your method to identify OOD samples. 

limitations: The authors have addressed limitations of the proposed approach.
review_title: null

final_score: rating: 4: Borderline reject: Technically solid paper where reasons to reject, e.g., limited evaluation, outweigh reasons to accept, e.g., good evaluation. Please use sparingly.
review_content: summary: This paper proposes an OOD detector for Transformer models. Their motivation is that all hidden layers of transformer-based models contribute to OOD detection. They verify their method with extensive experiments and show the performance jump on AUROC.
strengths_and_weaknesses: - Their motivation that going deeper inside the network can improve OOD detection is somehow limited. Also, intermediate representation for detection tasks is not new, at least in the vision domain. This paper works on NLP domain but is still somehow incremental.
- Their observation that using all hidden layers is straightforward and reasonable, but simply taking an average of those latent representations is simply.
questions: - Their method is based on the Mahalanobis score, which depends on the gaussian assumption. However, this is questionable; such an assumption still fits a real-world setting. Instead of the Mahalanobis score, I am wondering if the authors consider other feature-space scores for their method. Also, the computation burden or Mahalanobis also needs to be discussed.
- They use Integrated Rank-Weighted depth in their method, which has been used in some previous related works. This downgrade the novelty of the contribution in this paper.
- Woking on textual OOD problem is reasonable; however, since their methods rely on the transformer model, the natural question is, could it be applied to other domains with Transformer-based models, such as the vision domain?
- Their selected datasets have different domain characteristics. For example, IMDB is a movie review, while QA is another domain. I wonder if they consider some harder OOD settings during their experiments?
- It is interesting their observation that training longer the classifier hurts detection. Do the authors consider training those models from scratch without pretraining, for example, train 6-layer transformer vs. 12-layer transformer from scratch and show the power of their metric without pretrained weights.
limitations: The detector proposed in this paper is simple and effective. However, this method is still incremental and still has some room to be improved. The detector proposed in this paper is simple and effective. However, this method is still incremental and still has some room to be improved. For example, if this paper uses intermedia hidden layers, then if the pretrained model affects the use of hidden states needs to be discussed. Also, when it comes to the vision domain, could this method still work?
review_title: null

final_score: rating: 4: Borderline reject: Technically solid paper where reasons to reject, e.g., limited evaluation, outweigh reasons to accept, e.g., good evaluation. Please use sparingly.
review_content: summary: This paper develops a new tool for out-of-distribution (OOD) detection in the NLP domain, which aims to complement the need of deformation techniques as that in computer vision. The key to detecting OOD relies on the computation of the OOD score that is realized by an integrated rank-weighted (IRW) depth.  Extensive experiments are conducted to show the superiority of the proposed method.
strengths_and_weaknesses: Strengths:
1. This paper aims to solve an interesting topic.
2. Extensive experiments are conducted and show competitive performance for the proposed method.
3. This paper has delivered a thorough analysis of the related work.

Weakness:
1. The paper suffers from limited technical novelty.  For example, 

1) The proposed IRW depth is borrowed for existing work. 

2) The paper has claimed that one of the contributions is to aggregate all hidden layers of a neural network.  However, this kind of trick has been proposed in the literature.  The authors should cite the related work and highlight their main differences.

3) The reviewer doubts whether there is any theoretical guarantee on the proposed method.  In particular, the reason for choosing "the halfspace depth" in line 174 is unknown.  

2. Some main technical implementations are not clear.  For example, 

1) In line 149, the notation of \delta is not defined.  It makes it hard for the reviewer to know how to compute the similarity score and justify its correctness.  Moreover, \delta has appeared twice in the paper.  One is in lines 132-133, and the other is in line 149.  This makes the reviewer a little confused about understanding the meaning.  

2) In line 147, the authors define F_{PM}(S) as the distribution of the mean-aggregation of the training distribution samples with the same predicted target as x.  However, it applies different aggregation strategies in Sec. 4.3, lines 246-255.

3) In line 159, “a pre-score aggregation function”: the definition is not shown.

4) In lines 182-183, u_k and n_{proj} are not defined. 
  

3. Critical issues in the experiments:
a) Though the authors claim that the motivation for introducing IRW depth is to reduce the computational complexity, there are no such experiments to verify this issue.  The paper should address it through experimental support.
b)	In Table 2, it contains some contradictory conclusions and lacks sufficient experiment comparison.

i.	The paper claims the information from aggregating all layers in the neural network is more useful. However, for the D_M score in Table 2, the result of the last layer F_L is much better than of all layers F_{cat} and F_{PM}, which shows a contradiction result.  The authors have tried to explain the results in lines 269-270, i.e., the multivariate Gaussian assumption may not hold for the dataset.  It makes the reviewer confuse about the experimental comparison and makes unconvincing the experimental results because the aggregation of F_L is also under the Gaussian assumption and it beats F_{PM}. 

ii.	In D_{IRW}, it only compares F_{cat} and F_{PM} with F_L and F_{L+1} and shows that  F_{cat} and F_{PM} beat F_L and F_{L+1}.  The comparison is not sufficient.  For example, the authors can consider other combinations, such as concatenating F_L and F_{L+1} or averaging F_L and F_{L+1}.

questions: 1. As there are many different tools to solve the OOD problem, what is the exact motivation of borrowing the IRW depth to solve this problem? 
2. The reviewer expects to see the connection of the proposed IRW depth and the deformation technique in CV.  Can the authors provide some theoretical results?

limitations: Yes
review_title: null

Meta-review: metareview: The paper proposes a out-of-distribution detection approach using integrated rank weighted (IRW). Its main novel feature is leveraging the information from all layers of the model for this task. The detector can be applied to new transformer models without any training, as opposed to data-driven methods. The method is assessed in a comprehensive evaluation and code is provided for reproducibility. One of the limitations, however, is the difficulty to compare the proposed method with related work.  Presentation, especially of the technical content, should be improved in the final version.

The AC disagrees with the authors' complaint about the biasedness of some reviews. Indeed two reviewers had critical remarks on several aspects of the paper, yet this criticism appears to be fair and driven by the scientific discourse. The answers provided in the rebuttal have clarified most of the reviewers' concerns. 
confidence: Less certain
Decision: Accept