Measuring the Instability of Fine-Tuning (2023)

Abstract: Fine-tuning pre-trained language models on downstream tasks with varying random seeds has been shown to be unstable, especially on small datasets. Many previous studies have investigated this instability and proposed methods to mitigate it. However, most of these studies only used the standard deviation of performance scores (SD) as their measure, which is a narrow characterization of instability. In this paper, we analyze SD and six other measures quantifying instability of different granularity levels. Moreover, we propose a systematic evaluation framework of these measures’ validity. Finally, we analyze the consistency and difference between different measures by reassessing existing instability mitigation methods. We hope our results will inform better measurements of the fine-tuning instability.

## introduction since the introduction of bert @xcite , the pre-train-then-fine-tune paradigm has achieved state-of-the-art performance across many nlp benchmarks @xcite @xcite . however, despite its wide success, the fine-tuning process, especially when fine-tuning large models on small datasets, is shown to be unstable: fine-tuning a given model with varying random seeds can lead to different performance results @xcite @xcite . this instability makes the investigation of better architectures and instability mitigation methods (imms) challenging @xcite . many previous studies have investigated finetuning instability @xcite @xcite . in these studies, the most prevalent instability measure is the standard deviation of performance (sd, e.g. the standard deviation of f1-scores). however, as we discuss in §3 and §6, sd can only offer very limited assessments. for example, classifiers can obtain the same accuracy score (i.e. zero sd) even when they neither make the same predictions on each example (prediction instability) nor have the same hidden representations (representation instability). therefore, it is important to also use other measures that can address the weaknesses of sd. however, it is difficult to decide which measures to use: because instability is an abstract concept, it is hard to examine to which extent a measure indeed quantifies what it intends to measure. this property is called validity in measurement theory @xcite . for example, using the average accuracy of models as an instability measure would have low validity, because how accurate these models make predictions does not reflect their stability. to better assess the instability of fine-tuning pretrained language models (plms), we study more measures concerning instability at different granularity levels @xcite @xcite @xcite and develop a framework to assess their validity. we focus on bert and roberta for their popularity, but our framework can also be applied to other plms. concretely, • first, we discuss six other instability measures at different granularity levels in addition to sd, and categorize them into prediction measures and representation measures based on the type of instability they focus on ( §3). ## instability measures despite its wide usage, sd only provides a narrow view of the instability of models. for example, consider fine-tuning two pre-trained models on the same classification task. if one of them makes correct predictions only on half of the test data, while the other model makes correct predictions only on the other half, these two models will both have a 0.5 accuracy score and therefore no instability would be measured using sd. however, they actually make different predictions on each data point (i.e. prediction instability). moreover, even if these two models achieve the same accuracy by making identical predictions, due to the overparameterization of plms @xcite , they can have different sets of hidden representations (i.e. representation instability). to better assess these two types of instability, we study six other instability measures at different granularity levels in addition to sd. furthermore, according to the instability types that these measures intend to quantify, we categorize these measures into two types: prediction measures ( §3.1) and representation measures ( §3.2). all these instability measures have a continuous output range 0-1, with higher values indicating lower stability. it is worth noting that similar categorizations have been used before. for example, @xcite categorized measures as functional and representational. however, they used functional similarity to refer to the function compositions that different components of the models realize. also, summers and dinneen (2021) categorized measures as performance variability and representation diversity. however, they used performance variability to specifically refer to sd and used representation diversity to refer to all other measures at different granularity levels that we study here. notation formally, suppose we have a dataset consisting of n data points. we fine-tune m bert models {m 1 , m 2 , . . . , m m }, with the same settings except for m different random seeds. we use p k i and ŷk i to denote the class probability and the prediction of m i on the k-th test sample. assume the l-th layer of m i consists of e neurons, we use m l @xmath0, w.r.t. all n data points (all representation measures discussed below require us to center the representations). representation measures involve computing the distances between the representations derived from the same layer of two different models. we use d l i,j to represent the distance between m l i and m l j . ## experimental setup we study the instability of fine-tuning bert @xcite and roberta @xcite empirically. following @xcite and @xcite , we perform our experiments on three small datasets of the glue benchmark @xcite : rte, mrpc @xcite , and cola @xcite , because models trained on small datasets are observed to be less stable @xcite . 4 unless specified, we fine-tune bert-large and roberta-large models from huggingface transformers @xcite , with a 16 batch size, a 0.1 dropout rate, and a 2 × 10 -5 learning rate, using de-biased adam, as well as a linear learning rate warm-up during the first 10% steps followed by a linear decay, following @xcite . consistent with @xcite , we train the models for five epochs with 20 random seeds. consistent with @xcite , we divide the validation data into two equally sized parts, respectively as new validation and test data, because we have no access to the glue test datasets. moreover, we keep the checkpoint with the highest validation performance and obtain all our results on the test set. more details are provided in appendix a. ## assessing the validity of instability measures it is not trivial to assess the validity of instability measures, because there is no clear ground truth. nevertheless, we can still perform validity assessments by building on approaches from measurement theory. here, we propose a framework to assess two important types of validity (william m. k., 2023), by computing their correlations with for figure 1b and 1c, x-axis and y-axis are the layer depth and instability scores respectively. figure 1b shows the differences of representation measures between successful ( * s ) and failed runs ( * f ) for bert on rte. figure 1c shows the consistency of different measures for bert on mrpc across different sub-samples (different lines). ## conclusion in this paper, we study measures that quantify the instability of fine-tuning plms. in addition to the most commonly used measure, sd, we study six other measures at different granularity levels and propose a framework to evaluate their validity. using this framework, we show that all these measures except svcca have good validity. moreover, by reassessing existing imms, we show that different instability measures are not always consistent, and that they are more consistent when the models are less stable. finally, based on our observations, we offer two suggestions for selecting instability measures in future studies.