Pathologies of Pre-trained Language Models in Few-shot Fine-tuning (2022)

Abstract: Although adapting pre-trained language models with few examples has shown promising performance on text classification, there is a lack of understanding of where the performance gain comes from. In this work, we propose to answer this question by interpreting the adaptation behavior using post-hoc explanations from model predictions. By modeling feature statistics of explanations, we discover that (1) without fine-tuning, pre-trained models (e.g. BERT and RoBERTa) show strong prediction bias across labels; (2) although few-shot fine-tuning can mitigate the prediction bias and demonstrate promising prediction performance, our analysis shows models gain performance improvement by capturing non-task-related features (e.g. stop words) or shallow data patterns (e.g. lexical overlaps). These observations alert that pursuing model performance with fewer examples may incur pathological prediction behavior, which requires further sanity check on model predictions and careful design in model evaluations in few-shot fine-tuning.

## introduction pre-trained language models @xcite @xcite have shown impressive adaptation ability to dowstream tasks, achieving considerable performance even with scarce task-specific training data, i.e., few-shot adaptation @xcite @xcite . existing fewshot adaptation techniques broadly fall in finetuning and few-shot learning @xcite @xcite . specifically, fine-tuning includes directly tuning pre-trained language models with few task-specific examples or utilizing a natural-language prompt to transform downstream tasks to masked language modeling task for better mining knowledge from pre-trained models @xcite @xcite . few-shot learning lever-ages unlabeled data or auxiliary tasks to provide additional information for facilitating model training @xcite @xcite . although much success has been made in adapting pre-trained language models to dowstream tasks with few-shot examples, some issues have been reported. @xcite found that models obtained from few-shot prompt-based finetuning utilize inference heuristics to make predictions on sentence pair classification tasks. @xcite discovered the instability of model performance towards different prompts in few-shot learning. these works mainly look at prompt-based fine-tuning and discover some problems. this paper looks into direct fine-tuning and provides a different perspective on understanding model adaptation behavior via post-hoc explanations @xcite . specifically, post-hoc explanations identify the important features (tokens) contribute to the model prediction per example. we model the statistics of important features over prediction labels via local mutual information (lmi) @xcite . we track the change of feature statistics with the model adapting from pre-trained to fine-tuned and compare it with the statistics of few-shot training examples. this provides insights on understanding model adaptation behavior and the effect of training data in few-shot settings. we evaluate two pre-trained language models, bert @xcite and roberta @xcite , on three tasks, including sentiment classification, natural language inference, and paraphrase identification. for each task, we test on both in-domain and out-of-domain datasets to evaluate the generalization of model adaptation performance. we discover some interesting observations, some of which may have been overlooked in prior work: (1) without fine-tuning, pre-trained mod-els show strong prediction bias across labels; (2) fine-tuning with a few examples can mitigate the prediction bias, but the model prediction behavior may be pathological by focusing on non-taskrelated features (e.g. stop words); (3) models adjust their prediction behaviors on different labels asynchronously; (4) models can capture the shallow patterns of training data to make predictions. the insight drawn from the above observations is that pursuing model performance with fewer examples is dangerous and may cause pathologies in model prediction behavior. we argue that future research on few-shot fine-tuning or learning should do sanity check on model prediction behavior and ensure the performance gain is based on right reasons. ## setup tasks. we consider three tasks: sentiment classification, natural language inference, and paraphrase identification. each task contains an indomain/out-of-domain dataset pair: imdb @xcite /yelp @xcite for sentiment classification, snli @xcite /mnli @xcite for natural language inference, and qqp @xcite /twitterppdb (tppdb) @xcite for paraphrase identification. the data statistics are in ## experiments we report the prediction results (averaged across 5 runs) of bert and roberta trained with different ratio (r% : 0 ∼ 1%) of in-domain training examples on both in-domain and out-of-domain test sets in we look into the predictions of models from pretrained to fine-tuned and analyze model prediction behavior change during adaptation via post-hoc explanations. in subsection 3.1, we observe that pretrained models without fine-tuning show strong prediction bias across labels. the models fine-tuned with a few examples can quickly mitigate the prediction bias by capturing non-task-related features, leading to a plausible performance gain. in subsection 3.2, we further quantify the prediction behavior change by comparing the feature statistics of model explanations and training data. we discover that the models adjust their prediction behavior on minority labels first rather than learning information from all classes synchronously and can capture the shallow patterns of training data, which may result in pathologies in predictions. ## conclusion in this work, we take a closer look into the adaptation behavior of pre-trained language models in few-shot fine-tuning via post-hoc explanations. we discover many pathologies in model prediction behavior. the insight drawn from our observations is that promising model performance gain in fewshot learning could be misleading. future research on few-shot fine-tuning or learning requires sanity check on model prediction behavior and some careful design in model evaluation and analysis. timo schick and hinrich schütze. 2021a. exploiting cloze-questions for few-shot text classification and natural language inference. in proceedings of the 16th conference of the european chapter of the association for computational linguistics: main volume, pages 255-269, online. association for computational linguistics. timo schick and hinrich schütze. 2021b. it's not just size that matters: small language models are also fewshot learners. in proceedings of the 2021 conference of the north american chapter of the association for computational linguistics: human language technologies, pages 2339-2352, online. association for computational linguistics. tal schuster, darsh shah, yun jie serene yeo, daniel roberto filizzola ortiz, enrico santus, and regina barzilay. 2019. towards debiasing fact verification models. in proceedings of the 2019 conference on empirical methods in natural language processing and the 9th international joint conference on natural language processing (emnlp-ijcnlp), pages 3419-3425, hong kong, china. association for computational linguistics.