How Emotionally Stable is ALBERT ? Testing Robustness with Stochastic Weight Averaging on a Sentiment Analysis Task (2021)

Abstract: Despite their success, modern language models are fragile. Even small changes in their training pipeline can lead to unexpected results. We study this phenomenon by examining the robustness of ALBERT (Lan et al., 2020) in combination with Stochastic Weight Averaging (SWA)—a cheap way of ensembling—on a sentiment analysis task (SST-2). In particular, we analyze SWA’s stability via CheckList criteria (Ribeiro et al., 2020), examining the agreement on errors made by models differing only in their random seed. We hypothesize that SWA is more stable because it ensembles model snapshots taken along the gradient descent trajectory. We quantify stability by comparing the models’ mistakes with Fleiss’ Kappa (Fleiss, 1971) and overlap ratio scores. We find that SWA reduces error rates in general; yet the models still suffer from their own distinct biases (according to CheckList).

## introduction current language models perform well on data that resemble the distribution they are trained on, but even a slight variation in the model training setup can lead to results that diverge from what is originally reported @xcite . furthermore, when the model relies on spurious correlations for decision making, it then contains biases that are not represented by real world data. ideally a model should be robust to data that has (slightly) different characteristics from the data it was trained on. accuracy and related metrics, despite their popularity, are usually not sufficient to identify these frailties. this is known as underspecification (d' @xcite : different predictors can achieve similar results on a specific task, but exhibit diverging performance on other tasks due to different induced biases. stress tests are an increasingly popular method for exposing biases of a model. to test the linguis-tic capabilities and robustness of models, @xcite introduce checklist, an evaluation methodology that is comparable to the aforementioned stress tests for robustness. checklist can be used to investigate which linguistic phenomena are fully captured by a model and for which the model is thus expected to be robust across datasets. robustness and generalization can be improved by ensembling multiple models. training different models, however, is expensive. stochastic weight averaging (swa) @xcite is a way of ensembling without the need to train different models. during training, the weights of the model at specific timepoints are averaged, avoiding the need to keep track of several models. the idea is that swa explores different solutions close to a high performing minimum. in this paper, we study the effect of swa on the robustness to both a standard sentiment analysis dataset and different checklist capabilities. we investigate if models varying only in their random seeds still have different behavior on the same data when trained using swa. specifically, we train albert-large @xcite on sst-2 @xcite , a sentiment analysis dataset, with 10 random seeds. we perform one run with swa turned off (termed vanilla models) and repeat the procedure with swa turned on (termed swa models). we explore the robustness of the trained models using the checklist methodology by looking at the stability of mistakes. we quantify this stability to measure the agreement in mistakes between the different models and compare the resulting values between the vanilla and swa models. our main hypothesis is that using swa leads to more stable models. we therefore expect more overlap across random seeds in the results on the sst-2 evaluation data. we also expect swa to lead to more overlap in mistakes for checklist items that are captured by part of the vanilla models. we also anticipate (minor) improvements of general performance in both cases. for checklist phenomena that are already largely captured or not at all, on the other hand, we do not expect to see major differences between vanilla models and swa in terms of general performance or overlap. we make the following contributions: we find that swa improves on error rates in general, but results on increased stability are mixed: models with different random seeds still hold onto their own distinct induced biases on linguistic information captured by part of the models in our checklist evaluation. there is minor improvement in stability on the fleiss' kappa score on the development set of sst-2, but results are not conclusive. finally, we observe a large error rate for one of the random seeds on both sst-2 and checklist, which also leads to a less strong result on increasing agreement between models. ## related work to the best of our knowledge, we are the first to combine swa with checklist and apply it to a bert-based model to understand its effect on robustness with different random seeds. the work closest to ours has used variations of swa for investigating the differences in interpretability on cnns and lstms among different random seeds @xcite . a similar method to stochastic weight averaging was employed by @xcite with a different objective: improving the fine-tuning process of bert. they propose to average the bert model at each time-step and two types of knowledge distillation to improve fine-tuning of the model. the averaging receives slightly better results and their variant of knowledge distillation works the best. however, it is unclear what the effect of this is on different random seeds. instead of looking at a form of ensembling, @xcite investigate the effect of injecting noise in bert as a regularizer on the stability (sensitivity to input perturbation) of the models and show that fine-tuning performance improves. they point out that this improves generalizability as well, by looking at the difference in accuracy on the training and test set. however, training and test set might contain the same biases and hence might not reveal generalization issues @xcite . varying performance most work until now has focused on behavioral changes of models on train and test data when changing an arbitrary choice of the pipeline, such as the random seed @xcite . investigating the behavior of language models with different pre-training and fine-tuning random seeds on an instance-level, @xcite find that the fine-tuning random seed is influential for the variation in performance on an instance-level. this contrast in performance is also highlighted by @xcite ; they release multiple bert checkpoints with a different weight initialization and show diverging performance between similarly trained models. such behavior has also been observed for out-ofdistribution samples @xcite @xcite , where different induced biases are found when the random seed is modified and checkpoints behave differently on unseen data, even when evaluation performance is similar. @xcite . @xcite show that outputs from explainability methods also vary when changing hyperparameters, e.g. the random seed. model evaluation evaluating models on a development set might not expose certain biases or weaknesses a model has acquired due to the possibility of the same biases occurring in the training set. hence, scalable diagnostic methodologies are useful to investigate a model's capabilities @xcite @xcite . even though these methodologies all focus on evaluation, the approach can vary between the methods. @xcite tackle evaluation from a counterfactual point of view. @xcite not only examine counterfactuals but also grouping queries to ensure that error analysis is scaled to all instances. likewise, @xcite exploit such subpopulation grouping, in addition to adversarial attacks, perturbations, and evaluation sets. it is possible to be unaware of certain subpopulations for which the model is weak, and therefore d' @xcite introduce a method that looks for such weak groups. ribeiro et al. (2020) provide a methodology to analyze robustness toward basic capabilities and operationalize this with different test types (e.g. invariance to specific perturbations, basic capabilities). there are also more task specific efforts for evaluation, such as perturbations for robustness in task-oriented dialog @xcite and evaluation of bias in a sentiment analysis setting @xcite . ## method to examine stochastic weight averaging's effect on model stability due to underspecification, we finetune a pretrained albert-large version 2 on the sst-2 dataset. we train two types of models 10 times: foot_0 ## results this section presents the outcome of our experiments. we first provide results on the original dataset and then the results on checklist items. lastly we examine how stable vanilla and swa models are by looking at the label agreement between models trained from different seeds. ## discussion this research illustrates the potential impact of random seeds. first, our original sample of 5 seeds contained an outlier that performed far worse than the other seeds (as well as the original study). second, while initial results on the sst-2 development set were promising when looking at the 4 random seeds that showed normal behavior, these results did not hold when adding 5 additional random seeds. this highlights the necessity for proper analysis and the fragility of deep language models. possibly, the initial random seeds were closer to each other in the weight space and hence swa appeared to increase the agreement significantly. the additional random seeds could lie farther away, thus subsiding the increased agreement. in the future, more comprehensive research on the proximity and behavior of different random seeds could therefore be useful. even though checklist provides an easy way to investigate the capabilities of a model, automatizing some tests can be hard. there can be situations in which labels indicated for a specific capability might not hold for a certain test case. for instance, negating a negative sentence might not always lead to a positive sentence, it can also be neutral. similarly, we applied negations on some instances from the test set but the label is not required to flip, depending on the placement of the negation. therefore, we leave out the results in our conclusions as the labels did not always make sense upon investigation. in some instances, it is also unclear what the resulting label should be. we have added the results for these specific capabilities in appendix b for completeness. for further experiments, we would like to manually generate some checklist capabilities to ensure validity of the labels. this will also enable us to focus on the creation of more subjective tests, cases that are less black-and-white than the tests conducted in this research. we can then gain more insights into the fragility of models when it comes to border cases. ## conclusion we combine swa with the checklist methodology to explore the effects of swa on the robustness of a bert-based model (albert-large) on different random seeds and apply it to a sentiment analysis task. to understand how swa affects the stability amongst different random seeds, we analyze in-depth the results and mistakes made on the development set and checklist test items and provide error rates, overlap ratios, and fleiss' kappa agreement values. while swa is able to reduce the error rate in general amongst most of the random seeds, on the checklist tests, there are still some capabilities that models make their own distinct mistakes on with swa incorporated. the stability on the development set also improves only slightly. in the future, we would like to create more hand-crafted checklist capabilities for further rigorous study. furthermore, it could be useful to thoroughly investigate the impact of adjacency of random seeds on their error agreement. matthew watson, bashar awwad shiekh hasan, and noura al moubayed. 2021. agree to disagree: when deep learning models with identical architectures produce distinct explanations. arxiv preprint arxiv:2105.06791. thomas wolf, lysandre debut, victor sanh, julien chaumond, clement delangue, anthony moi, pierric cistac, tim rault, rémi louf, morgan funtowicz, et al. 2019. huggingface's transformers: state-of-the-art natural language processing. arxiv preprint arxiv:1910.03771. tongshuang wu, marco tulio ribeiro, jeffrey heer, and daniel weld. 2019. errudite: scalable, reproducible, and testable error analysis. in proceedings of the 57th annual meeting of the association for computational linguistics, pages 747-763, florence, italy. association for computational linguistics. tongshuang wu, marco tulio ribeiro, jeffrey heer, and daniel s weld. 2021. polyjuice: generating counterfactuals for explaining, evaluating, and improving models. in proceedings of the 59th annual meeting of the association for computational linguistics. yige xu, xipeng qiu, ligao zhou, and xuanjing huang. 2020. improving bert fine-tuning via self-ensemble and self-distillation. arxiv preprint arxiv:2002.10345. ruiqi zhong, dhruba ghosh, dan klein, and jacob steinhardt. 2021. are larger pretrained language models uniformly better? comparing performance at the instance level. in findings of the association for computational linguistics: acl-ijcnlp 2021, pages 3813-3827, online. association for computational linguistics.