On Robustness of Finetuned Transformer-based NLP Models (2023)

Abstract: Transformer-based pretrained models like BERT, GPT-2 and T5 have been finetuned for a large number of natural language processing (NLP) tasks, and have been shown to be very effective. However, while finetuning, what changes across layers in these models with respect to pretrained checkpoints is under-studied. Further, how robust are these models to perturbations in input text? Does the robustness vary depending on the NLP task for which the models have been finetuned? While there exists some work on studying robustness of BERT finetuned for a few NLP tasks, there is no rigorous study which compares this robustness across encoder only, decoder only and encoder-decoder models. In this paper, we characterize changes between pretrained and finetuned language model representations across layers using two metrics: CKA and STIR. Further, we study the robustness of three language models (BERT, GPT-2 and T5) with eight different text perturbations on classification tasks from General Language Understanding Evaluation (GLUE) benchmark, and generation tasks like summarization, free-form generation and question generation. GPT-2 representations are more robust than BERT and T5 across multiple types of input perturbation. Although models exhibit good robustness broadly, dropping nouns, verbs or changing characters are the most impactful. Overall, this study provides valuable insights into perturbation-specific weaknesses of popular Transformer-based models which should be kept in mind when passing inputs.

## introduction pretrained transformer-based language models @xcite have revolutionized the field of natural language processing (nlp). specifically, pretrained transformer models such as bert @xcite (bidirectional encoder representations from transformers), gpt-2 @xcite (generative pre-trained transformer) and t5 @xcite ) (text-to-text transfer transformer), have set new benchmarks for various downstream nlp tasks. to adapt these models for downstream tasks, researchers finetune them on task-specific data. finetuning modifies the representations generated by each layer. how do these modifications compare with respect to corresponding pretrained models? further, if we perturb the inputs supplied to these language models, does that lead to changes in layer representations and also prediction accuracy? how does this robustness vary across different nlp tasks for which these models have been finetuned? it is important to understand answers to these questions to ensure that we account for perturbations for which these models are not very robust. recent investigations on adversarial text perturbations @xcite @xcite @xcite have revealed that even strong language models are susceptible to adversarial examples, which can increase the risk of misclassification of input data, leading to incorrect results. however, existing studies on robustness have used only a few adversarial text perturbations and experimented with bert on a few downstream nlp tasks. hence, it is important to study the representations generated by pre-trained and finetuned transformer models and evaluate their robustness when exposed to text perturbations. specifically, we aim to answer the following questions: (i) is the effect of finetuning consistent across all models for various nlp tasks? (ii) to what extent are these models effective in handling input text perturbations? and (iii) do these models exhibit varying levels of robustness to input text perturbations when finetuned for different nlp tasks? earlier studies use representation similarity analysis (rsa) as the metric to quantify the similarity between the representations generated by different models @xcite . recently, @xcite introduced a new similarity metric, centered kernel alignment (cka), to better measure the representations. in another recent study, @xcite overcame the limitations of cka by introducing @xcite to analyze the shared invariances of models to meaningless perturbations. hence, we use both cka and stir to examine the representation similarity across various layers of pretrained versus finetuned models. we measure robustness as a function of relative change in performance when perturbations are applied to inputs. we make the code and models publicly available 1 . our key contributions are as follows: (1) our analysis of finetuned models versus pretrained models shows that the last layers of the models are more affected than the initial layers when finetuning. (2) gpt-2 exhibits more robust representations than bert and t5 across multiple types of input perturbation. (3) although transformers models exhibit good robustness, the models are seen to be most affected by dropping nouns, verbs or changing characters. (4) we also observed that while there is some variation in the affected layers between models and tasks due to input perturbations, certain layers are consistently impacted across different models, indicating the importance of specific linguistic features and contextual information. ## related work the nlp community has shown increasing concern for the robustness of pre-trained language models, when exposed to adversarial examples. to assess this, various studies have been conducted to examine the models' susceptibility to modifications in the input text. some works investigated modifications including typos or word replacements, while @xcite ; @xcite ; @xcite evaluated the models' capacity to adapt to different data distributions and linguistic phenomena, such as coreference or sarcasm. to address the concern for robustness, @xcite introduced a multi-task benchmark for the evaluation of language models. more broadly, @xcite perform robustness analysis of videolanguage models. studies on model probing, such as @xcite @xcite , have analyzed the degree to which syntactic and semantic features are captured in the different layers of bert-like models. additionally, @xcite ; @xcite performed a comprehensive analysis of how finetuning affects the representations in the bert model using a combination of probing and analytical techniques. in terms of similarity metrics, @xcite used a form of canonical correlation analysis (pw-cca; @xcite ) to examine the layerwise evolution of representations in deep neural networks. on the other hand, @xcite utilized representation similarity analysis (rsa; @xcite ) to assess the models' representations. in recent work, @xcite applied centered kernel alignment (cka; @xcite ) to pre-trained transformers like bert and gpt-2, focusing mainly on cross-model comparisons. our study, however, specifically delves into comparing the representations generated by pre-trained and finetuned transformer models, and analyzing their layer-wise similarity and shared invariances. additionally, our work seeks to contribute to the ongoing efforts to better understand the strengths and limitations of transformer models in handling input text perturbations. ## conclusion our research has provided a comprehensive analysis of the layer-wise similarity and shared invariance between pre-trained and finetuned transformer models, and the impact of text perturbations on their performance. a key to our study is that we leverage stir @xcite , a recent approach that estimates how much of the invariances to specific perturbations learned by one source model are shared with a second target model @xcite . our findings suggest that model performance is task-sensitive, and the type of data it was trained on influences its performance on a particular task. layer-wise analysis showed that some layers have a more significant impact on performance than others. also, the robustness scores of bert, gpt-2, and t5 under different perturbations demonstrate that these models exhibit varying degrees of robustness depending on the task and type of perturbation. gpt-2 is more resilient to perturbations than t5 and bert. overall, our study provides insights into the strengths and limitations of bert, gpt-2, and t5, serving as a foundation for future research to develop more resilient nlp models.