Bi-level Finetuning with Task-dependent Similarity Structure for Low-resource Training (2023)

Abstract: Training a large language model in low-resource settings is challenging since they are susceptible to overfitting with limited generalization abilities. Previous work addresses this issue by approaches such as tunable parameters reduction or data augmentation. However, they either limit the trained models’ expressiveness or rely on task-independent knowledge. In this paper, we propose the Bi-level Finetuning with Task-dependent Similarity Structure framework where all parameters, including the embeddings for unseen tokens, are finetuned with task-dependent information from the training data only. In this framework, a task-dependent similarity structure is learned in a data-driven fashion, which in turn is used to compose soft embeddings from conventional embeddings to be used in training to update all parameters. In order to learn the similarity structure and model parameters, we propose a bi-level optimization algorithm with two stages—search and finetune—to ensure successful learning. Results of experiments on several classification datasets in low-resource scenarios demonstrate that models trained with our method outperform strong baselines. Ablation experiments further support the effectiveness of different components in our framework. Code is available at https://github.com/Sai-Ashish/BFTSS .

## introduction finetuning pretrained large models in low-resource scenarios 1 face many challenges @xcite @xcite . one of the challenges is the overfitting of the model when finetuned on the small training data. different approaches have been proposed to tackle this problem and achieved great results. some approaches restrict the number of parameters to be updated in the finetuning process to avoid overfitting to small amounts of data @xcite . however, parameter restriction while model finetuning may impact the model's expressiveness. other methods such as data augmentation @xcite aim at increasing the training data size through synthesizing new training data examples to boost generalization ability. these methods rely on either external lexical resources and heuristics, which are limited in domain and language; or pretrained language models, the semantic similarity space of which is not taskdependent. for example, apple may be replaced by microsoft in synonym replacement based on some lexical resource, but the "replace-ability" really depends on whether the task is "separate tech companies from gas companies" or "find companies with their headquarters in washington state", the information of which pretrained language models or static lexical resources are not able to provide. our motivation is to combine strengths of both approaches. for generalization ability to unseen data, all parameters, especially embeddings of words not in the training data, should participate in the finetuning. at the same time, no external knowledge source should be required for such finetuning to ensure the method being scalable to different tasks. the ideal training framework for these goals should allow training signals to flow from tokens in training data to unseen tokens in a task-dependent way. such a framework will ensure that the generalization ability of the trained model is strengthened through finetuning without the risk of overfitting quickly to a small amount of training data. our approach proposed in this paper, bi-level finetuning with task-dependent similarity structure (bftss), aims to meet these goals. first, we propose a low-resource finetuning method where all parameters of the model, including the embeddings of unseen tokens, can be tuned directly through soft embeddings. the soft embeddings are constructed through the use of a similarity matrix with pairwise similarity scores between words, termed a similarity structure foot_0 in this paper. second, we propose a bi-level optimization algorithm to learn a task-dependent similarity structure with lowresource task data, where no extra data, knowledge source or task-dependent prior knowledge is required. since the similarity structure is usually very large, two different methods are proposed to reduce its size and make the learning tractable. finally, we conduct extensive experiments on different datasets and with different model sizes. comparison to baseline models and ablated models shows the effectiveness of our approach, where the performance of the models trained in the proposed method surpasses all baselines by large margins. ## related work low-resource training has been a challenging but important task in natural language processing @xcite . approaches have been proposed to tackle the issues encountered in lowresource training. robust finetuning methods are applied in such training scenarios, such as approaches restricting tunable parameters @xcite @xcite and noise-robust training methods @xcite @xcite . they alleviate the overfitting problem but introduce no new information beyond the training data into the model. data augmentation on the token level @xcite @xcite as well as on the sentence level using syntax @xcite , back-translation @xcite or generation models @xcite @xcite @xcite are approaches which aims at introducing extra information into model training. other similar methods rely on pseudo-labeling extra data @xcite @xcite using task insights and heuristics. most of them are designed for a specific task, or require external knowledge sources. bi-level optimization (blo) has wide applications in machine learning. the neural architecture search proposed by @xcite uses blo. it is also used in data selection @xcite @xcite and meta-learning @xcite . @xcite proposed a blo-based optimization framework for hyperparameter tuning. @xcite proposed blo-based learning rate adaptation. noisy label correction using blo is proposed by @xcite . among the papers mentioned above, the lower parameters are the model weights, and the upper parameters are the meta-variables, such as hyperparameters, architecture, training example weights, etc., to be optimized using blo. ## bi-level finetuning with task-dependent similarity structure the bi-level finetuning with task-dependent similarity structure framework, as shown in figure 1 , centers around how to learn and utilize a taskdependent similarity structure s. the structure is first initialized and learned along with model parameters with bi-level optimization on task data, where soft embeddings of words are derived from the structure to propagate training signals into unseen words. after this first phase of training, named search phase, the finetune phase follows in which only model parameters are updated with the similarity structure fixed. ## conclusion to mitigate the impact of overfitting leading to low generalization ability of large language models, especially in low-resource scenarios, we propose bilevel finetuning with task-dependent similarity structure-bftss, a bi-level optimization framework to learn a task-specfic similarity structure and further enable the generalization ability of updating 'unseen' or 'similar' words of language models on a given task. we also introduce two variants, bftss top-k and bftss u-v, to reduce the dimensionality and make computation efficient. extensive experimental results on various datasets show the effectiveness of our approaches in low-resource scenarios when comparing to different baseline methods.