An Empirical Study on Robustness of Language Models via Decoupling Representation and Classifier
Authors: Hao Yuan, Xin Wang, Yingji Li, Ying Wang


Reviewer paper summaries: This paper fine-tunes multiple Pretrained Language Models (PLMs) in MNLI and analyzes their representations and behavior of classifiers in MNLI and HANS.

Analysis of PLM representations of MNLI and HANS data shows that representations from the two datasets are both linearly separable, but the classification score for HANS drops since representations of HANS data have large cosine similarity regardless of their labels.

Further analysis of debiasing methods shows that the main reason behind their benefits is their optimization of classifiers, not the model representations. This shows the importance of classifiers in training robust models.
Summary of strengths: 1. The decoupling of representation and classifiers provide an profound insight on the role of classifiers in preventing shortcut learning, which was overlooked in previous works. 

2. The findings of the paper are empirically supported by multiple experiments with various PLMs and learning rates.

3. The descriptions for the debiasing methods and probing methods are mainly about how to interpret the probing and debiasing experiment results. Such writing style reduces unnecessary information and makes the paper easy to comprehend. The figures and tables also effectively illustrate the results.
Summary of weaknesses: 1. As mentioned in the Limitations, the authors analyze PLMs on a single task. As the paper aims to investigate model behaviors on OOD and provides purely empirical findings, the findings might not be consistent in other datasets. To confirm that the findings are generally applicable, they should be verified on more tasks.

2. The comparison between [CLS] and [MEAN] embeddings can be interesting, but it is irrelevant to the other contents of this paper.

3. The t-SNE and DIRECTPROBE clusters for each task are calculated independently. This setting does not correctly reflect the research question, as it ignores the relationship between HANS and MNLI clusters. For example, if the HANS-Entailment cluster mainly overlaps with the MNLI-Not-Entailment cluster, HANS can be still linearly separable but the model would not be robust. The authors should analyze HANS clusters based on MNLI clusters to verify the impact of representations on the generalization setting.
Comments, suggestions and typos: 1. The different task difficulties of MNLI and HANS could have contributed to the performance drops on HANS. Reporting the performances of PLMs fine-tuned on HANS will address such concerns.

2. Table 10 shows that the fine-tuned models achieve higher scores in “overlap”, compared to other non-entailment categories in HANS. Is this clear distinction relevant to the findings from the analysis?
Overall assessment scores: 3