Paper: Out-of-Distribution Detection by Leveraging Between-Layer Transformation Smoothness (ICLR-2024)

Reviewer comment (Evaluation):
While Figure 1 qualitatively illustrates the smoothness difference, the author did not offer an explanation for the observed difference between in-distribution (ID) and out-of-distribution (OOD) data. I wonder whether there is an analysis or understanding of this phenomenon, specifically why the learned progressions on ID data do not align with OOD data and how to define this misalignment.

Author response:
**3) Explanation of difference in smoothness** - On the operational level, BLOOD works because the learning algorithm learns the smooth transitions for the data it was trained on, while the rest of the representation space is either left largely unchanged (if the task was simple enough) or sharpened as the byproduct of trying to fit the more complex data, as we showed in the paper.  
We introduced the intuition and motivation for our method in subsection 3.2. Our intuition is based on the phenomenon known from the literature that different Transformer layers model features at different levels of abstraction. Since the model learns on the ID training data, it should learn the natural progression between the layers of abstractions inherent to the task, i.e., ID data. However, the unseen OOD data will likely have different features in a given level of abstraction than the ID data since the nature of the data is different, so the model is not familiar with them or their progression. Because the model has not learned how to handle the mismatch in the transition between the levels of abstraction, it exhibits sharper between-layer transformations of those representations, i.e., there is a more considerable difference between how OOD representations get mapped onto the next layer and how its infinitesimally small neighborhood gets mapped.

Reviewer comment (Evaluation):
In Table 2, the calculated effect size is greater in the “Mean” approach than in the “Last” approach, which appears to contradict the results in Table 1.

Author response:
**2) Table 2 contradicts Tabe 1** - No, that is as expected. A lower CLES implies a higher AUROC because the learning algorithm has sharpened the OOD region of representation space along with the smoothening of the ID region instead of just smoothening the ID region. That emphasizes the difference in smoothness between ID and OOD data. And since the learning algorithm had to “invest more work” in the last layers (as seen in Table 1), it sharpened the OOD region of representation space as a byproduct, leading to lower CLES for the last layer and higher AUROC when using the last layer.
The reasoning behind the CLES of lower vs. higher layers is parallel to the reasoning behind the CLES of simpler vs. more complex datasets. If a dataset is more complex, the learning algorithm changes the model more during training, which, instead of just focusing on the ID region of the representation space, also makes changes to the OOD region of the representation space as a byproduct. Similarly, in higher layers, which are more influenced by the learning algorithm, there are more changes in the OOD region of space.

Because of that, the only instances where CLES in the last layer is larger than the CLES_mean for RoBERTa is for theMG and NG datasets, which are exactly the datasets on which BLOOD_{mean} outperforms BLOOD_{L-1} on RoBERTa.

Reviewer comment (Evaluation):
In Section 4.3, there are certain inconsistencies in the presented results (e.g., RoBERTa model on the MG and NG datasets), which seem to challenge the hypotheses by the authors. Given the variations in performance, can you provide additional evidence or discussion, such as whether certain architectures or settings are preferred by the proposed method?

Author response:
**2) Table 2 contradicts Tabe 1** - No, that is as expected. A lower CLES implies a higher AUROC because the learning algorithm has sharpened the OOD region of representation space along with the smoothening of the ID region instead of just smoothening the ID region. That emphasizes the difference in smoothness between ID and OOD data. And since the learning algorithm had to “invest more work” in the last layers (as seen in Table 1), it sharpened the OOD region of representation space as a byproduct, leading to lower CLES for the last layer and higher AUROC when using the last layer.
The reasoning behind the CLES of lower vs. higher layers is parallel to the reasoning behind the CLES of simpler vs. more complex datasets. If a dataset is more complex, the learning algorithm changes the model more during training, which, instead of just focusing on the ID region of the representation space, also makes changes to the OOD region of the representation space as a byproduct. Similarly, in higher layers, which are more influenced by the learning algorithm, there are more changes in the OOD region of space.

Because of that, the only instances where CLES in the last layer is larger than the CLES_mean for RoBERTa is for theMG and NG datasets, which are exactly the datasets on which BLOOD_{mean} outperforms BLOOD_{L-1} on RoBERTa.

Reviewer comment (Experiments):
I’m curious if BLOOD can be extended to open-box settings.

Author response:
**4) BLOOD in an open-box setting** - While we focused on the white-box scenarios because of the wider useability of our method, we are thankful for this great suggestion. 
We experimented with extending BLOOD by using a small sample of OOD examples to learn the optimal parameters for OOD detection (as is done by many popular OOD methods), e.g., learn the optimal weights for the weighted average of scores in different layers. We used a validation set of 5% of original training sets and fitted a logistic regression to learn the optima weights for a weighted sum of BLOOD values in different layers. This approach generally outperforms the open-box versions of our method. We provide results (AUROC) of this experiment below. We will be adding the results in the Appendix of our paper:


| Model   | SST   | SUBJ  | AGN   | TREC  | BP    | AR    | MG    | NG    |
|---------|-------|-------|-------|-------|-------|-------|-------|-------|
| RoBERTa | 73.94 | 82.98 | 81.39 | 91.73 | 93.47 | 96.25 | 92.46 | 89.97 |
| ELECTRA | 77.67 | 77.30 | 82.76 | 98.73 | 96.54 | 91.97 | 90.92 | 85.19 |

Reviewer comment (Evaluation):
Could you provide more detailed explanation of how the CLES measurements in Tables 2 and 3 were computed?

Author response:
**6) Detailed explanation of CLES** - We take the intermediate representations of the ID and OOD data in each of the layers $l$ denoted by $r^l_P$ from the pre-trained model (i.e., before fine-tuning on training data). Then, after we trained the model on training data, we again took intermediate representations of ID and OOD data from each layer denoted by $r^l_F$. We then calculate the Euclidean distance between representations from the model before fine-tuning and the representations from the model after fine-tuning $c^l = || r^l_P-r^l_F|| $. We then use this measure $c^{l}$, which represents how much a particular data point’s intermediate representation has changed after training in the layer $l$, and calculate two different quantities: (1) MEAN = $1/L \sum_l c^l$ and (2) LAST = $c^L$, where $L$ is the last layer.

We then order all ID and OOD points based on their value of either $c_{MEAN}$ or $c_{L}$. To calculate the CLES, we create a Cartesian product between ID and OOD data points and then calculate the percentage of times ID data points have larger $c$ than OOD data points (it is mathematically equivalent to taking the AUROC of ID and OOD data based on $c$). 
The interpretation of CLES is that (1) if CLES is larger than 50% ID representations changed more than OOD representations, (2) if CLES is close to 50% ID and OOD representations changed roughly the same, and (3) if CLES is less than 50% ID representations changed less than OOD representations.

Reviewer comment (Experiments):
The experimental results are not very strong or convincing. I would have expected some experiments on vision datasets as there has been a significant amount of OOD detection work focussing on vision datasets. Also, this approach can be easily applied on those tasks and will give better understanding how well this approach works.

Author response:
**1) CV experiments** - BLOOD is competitive and all of the conclusions of our paper hold when applied to vision datasets paired with Vision Transformer. However, when paired with CNNs, BLOOD behaves differently, likely due to differences in how the two architectures process data.

Reviewer comment (Evaluation):
Why does BLOOD_mean performance significantly worse than the baselines in most of the cases? Whereas in some case it performs better than BLOOD_{L-1}. Can the authors explain that?

Author response:
**3) Relationship between BLOOD_mean and BLOOD_{L-1}** - BLOOD_mean introduces noise to the method because it includes BLOOD scores from the lower layer that did not change much during training and thus retain their BLOOD scores from the pre-trained models where there is no difference between ID and OOD data. And since BLOOD scores in the low layers have higher magnitudes, they take over the average. However, for datasets MG and NG on RoBERTa, BLOOD_mean works better than BLOOD_{L-1}. The reasoning behind that is the same (BLOOD scores are kept the same as before training in low layers), but in these scenarios, there exists a priori difference in BLOOD score between ID and OOD data, as seen from Figures 9 and 10.