Two-Stage Fine-Tuning for Improved Bias and Variance for Large Pretrained Language Models (2023)

Abstract: The bias-variance tradeoff is the idea that learning methods need to balance model complexity with data size to minimize both under-fitting and over-fitting. Recent empirical work and theoretical analysis with over-parameterized neural networks challenges the classic bias-variance trade-off notion suggesting that no such trade-off holds: as the width of the network grows, bias monotonically decreases while variance initially increases followed by a decrease. In this work, we first provide a variance decomposition-based justification criteria to examine whether large pretrained neural models in a fine-tuning setting are generalizable enough to have low bias and variance. We then perform theoretical and empirical analysis using ensemble methods explicitly designed to decrease variance due to optimization. This results in essentially a two-stage fine-tuning algorithm that first ratchets down bias and variance iteratively, and then uses a selected fixed-bias model to further reduce variance due to optimization by ensembling. We also analyze the nature of variance change with the ensemble size in low- and high-resource classes. Empirical results show that this two-stage method obtains strong results on SuperGLUE tasks and clinical information extraction tasks. Code and settings are available: https://github.com/christa60/bias-var-fine-tuning-plms.git

## introduction transformer-based neural language models, such as bidirectional encoder representations from transformers (bert) @xcite , have achieved state-of-the-art (sota) performance for a variety of natural language processing (nlp) tasks through the process of fine-tuning. given an nlp task, the process often involves searching for optimal pretrained models and hyperparameters * co-first authors while continuing to train the pretrained model on a domain-specific dataset, with the aim of building generalizable and robust fine-tuned models. based on the classic notion of the bias-variance tradeoff @xcite , where increasing model capacity decreases bias but increases variance (leading to a u-shaped test error curve), large pretrained models (lpms) should have large variance and overfit domain-specific data which is often sparsely labeled and extremely imbalanced for classification. however, recent empirical work and theoretical analysis of neural networks challenge this classic bias-variance trade-off notion @xcite . it has been suggested that no such trade-off holds: as the width/depth of the network grows, bias monotonically decreases while variance initially increases followed by a decrease. this is why transformerbased lpms often achieve better performance compared to less complex models like long short-term memory (lstm)-based models or feature-rich methods (e.g., support vector machines). in the context of the new bias-variance paradigm, these lpms are seemingly complex enough to have low bias and variance, however, so far there has been no method to justify whether those sota models are generalizable and robust in solving a variety of downstream tasks. in this paper, we (1) show that many sota models are very sensitive to data and training randomness, and (2) provide a variance decomposition-based justification method. we also aim to improve model performance, reducing the generalization error of lpms by reducing their bias and variance. recent findings in @xcite show that the generalization error mainly comes from bias. bias can be reduced by modifying the model architecture, e.g., making the neural networks wider and deeper as in transformer-based lpms. however, pretraining new or larger language models can be challenging due to the technical and computational resource requirements afforded by only a few institutions -a topic outside the scope of this paper. we focus on the problem of reducing variance of neural models to further boost model performance, given a fixed bias (i.e., a fixed pretrained model). ensemble methods have been successful in boosting predictive performance of single learners @xcite presents a comprehensive review) and thus are a promising venue to explore. we propose a two-stage fine-tuning framework that first justifies the generalization status of a selected pretrained model through a concrete metric, and then uses the fixed-bias model to further reduce variance due to optimization through ensembling techniques. to the best of our knowledge, we are the first to provide such a metric and perform theoretical and empirical analysis using ensembles for improved bias and variance for lpms. we conduct experiments on the superglue tasks as well as on information extraction from clinical text as it is a domain of high significance and presents data challenges due to the limitations of sharing patientrelated data. we believe our proposal is of interest to any unstructured domain where neural models are used. specifically we make the following contributions: ## preliminaries in this section we present the bias-variance decomposition for squared loss in the new paradigm studied in @xcite and @xcite . we also present a further decomposition of variance. we denote f as a supervised learning task such that f : @xmath0, based on a training dataset s of m i.i.d. samples drawn from a joint distribution d of (x , y). the learning target is to minimize the mean squared error e(f ) = e (x,y) ∥ yf (x) ∥ 2 , where (x, y) ∼ d. we consider the predictor f θ as a random variable depending on the random variable s for training dataset and the random variable o for optimization randomness, where θ = a(s, o) ∈ r p represents the weights of neural networks produced by the learning algorithm a. p denotes the dimension of θ. the notations and their descriptions are shown in ## related works in a fine-tuning setting, searching for an optimal setting of pretrained models and hyperparameters is challenging due to the high dimensionality of the search space, as well as the infinite values for each dimension. in previous works of fine-tuning tasks @xcite @xcite , the sota models are single learners carefully selected and fine-tuned based on evaluation results, such as p, r, and f1 scores, using grid-search or random-search. to improve the stability of the pre-trained transformerbased language models, @xcite suggests using small learning rates with bias correction and increasing the number of iterations to avoid vanishing gradients. prior efforts also highlight the comparable effect of weight initialization and training data order to the variance of model performance @xcite . ensemble methods have been successful in boosting the predictive performance of single learners @xcite present a comprehensive review; also see @xcite @xcite as well as in estimating predictive uncertainty @xcite @xcite . among these studies, @xcite and @xcite theoretically prove that ensembles can perform better than the average performance of component learners for prediction accuracy and consistency of learning models. @xcite empirically evaluates the application of ensemble methods to fine-tuned transformer-based models for clinical nlp tasks. the findings demonstrate that ensemble methods improve model performance, particularly when employing dynamic snapshot ensembling. although it is common knowledge that ensembles can reduce variance thus reducing the generalization error, no previous work has discussed or measured this in the context of variance decomposition. furthermore, no previous work has investigated the impact of ensembles on imbalanced datasets. ## conclusion different from the prevailing fine-tuning settings, we propose a two-stage fine-tuning method to improve the generalization ability of a fine-tuned lpm. we provide a variance decomposition-based justification method to empirically measure the generalization ability of the lpm w.r.t. the upper limit of its learning ability. in stage-i, the rsota setting is built by ratcheting down bias and variance in an iterative way. in stage-ii, given the rsota setting, the fine-tuned lpm is guaranteed to be further generalized through ensembling techniques by reducing the variance due to optimization. the proposed justification method provides a concrete metric to track this process. we provide empirical evidence by conducting experiments on the superglue tasks and two clinical datasets. furthermore, we perform theoretical analysis on how ensembles improve variance due to optimization. we investigate the nature of variance change for the ensemble size in lowand high-resource classes in classification tasks. different from previous theoretical analyses using only model complexity and data size which depends on least squared regression, our variance decomposition-based justification method in stage-i does not rely on specific factors thus leading to a more generalizable measurement. the ens further boosts performance without risk of computational cost and overfitting. our analysis on imbalanced data reveals another advantage of ensemble algo-rithms in improving model performance on lowresource classes. as future work, we are interested in (1) rigorously proving variance decomposition-based justification criteria, (2) quantifying low-and highresource classes with specific features that interplay with ensemble size. if properly used, we believe the theoretical and empirical findings discussed in this paper can guide practitioners to fine-tune more generalizable models.