On Enhancing Fine-Tuning for Pre-trained Language Models (2023)

Abstract: The remarkable capabilities of Natural Language Models to grasp language subtleties has paved the way for their widespread adoption in diverse fields. However, adapting them for specific tasks requires the time-consuming process of fine-tuning, which consumes significant computational power and energy. Therefore, optimizing the fine-tuning time is advantageous. In this study, we propose an alternate approach that limits parameter manipulation to select layers. Our exploration led to identifying layers that offer the best trade-off between time optimization and performance preservation. We further validated this approach on multiple downstream tasks, and the results demonstrated its potential to reduce fine-tuning time by up to 50% while maintaining performance within a negligible deviation of less than 5%. This research showcases a promising technique for significantly improving fine-tuning efficiency without compromising task- or domain-specific learning capabilities.

## introduction neural based language models are functions or algorithms that are trained to predict the likelihood of a sequence of words @xcite . these models were trained using large volumes of textual content and are able to provide an accurate approximation for language features and structure. these models provide an important tool for analyzing and understanding the nuance of language, as well as for building applications that rely on natural language understanding @xcite . fine-tuning neural language models refers to the process of further training a pre-trained language model on a specific task or domain with a smaller dataset. the pre-trained language model, such as bert or gpt, has already learned a significant amount of knowledge about natural languages from a large corpus of text. however, it may not have been trained specifically for the task at hand or on the specific domain of interest. fine-tuning involves updating the pre-trained model's parameters to optimize its performance on the given target so it can learn more task-specific or domain-specific information. fine-tuning large language models (llms) proved to be very effective and efficient to achieve higher accuracy and state of the art numbers in many downstream tasks @xcite . various techniques were suggested to ensure that the resulting models achieve optimal accuracy. one of the challenges faced during the fine-tuning of language models is overfitting. overfitting occurs when the model performs well on the training or fine-tuning data but poorly on new, unseen data. this happens because the model has learned to fit the noise in the training data rather than capturing the underlying patterns. to address overfitting, several regularization techniques were proposed in the literature, such as weight decay and dropout. these methods help prevent the model from memorizing the training data and promote better generalization to unseen data. additionally, achieving optimal results with fine-tuning involves hyperparameter tuning, where efforts are made to select the best set of hyperparameters for the model. hyperparameters, such as the learning rate and number of layers, can significantly influence the model's performance and generalization capabilities. properly tuning these hyperparameters is essential for obtaining the best possible results during fine-tuning @xcite . in this research, we pursue a different direction for finetuning language models by exploring a methodology that involves limiting backpropagation to a specific number of layers. this approach offers several benefits, including effectively addressing the issue of over-fitting and significantly reducing the fine-tuning time. our primary objective is to identify the most impactful layers that contribute to achieving the best performance, and then extend this investigation to various pre-trained mod-els. the key contributions of this research are as follows: the remainder of the article is structured as follows: in the next section, we provide background information on the evolution of language models and natural language processing. subsequently, in the third section, we present our methodology, introducing the language models and tasks we will be experimenting with. following that, we present the results and engage in a discussion in the fourth section. finally, in the fifth section, we present our conclusions and outline the prospects for our ongoing work. ## methodology to explore the extent of the proposed method, we limit the scope of our investigation to the following pre-trained models and tasks, more models would be worth of investigating in the future work. camelbert @xcite created and distributed a pre-trained language model that combined modern standard arabic (msa), dialectal arabic (da), and classical arabic (ca). the collection included over 167gb of text ( 17.3b tokens). qarib @xcite trained a model on a collection of 420 million tweets and 180 million sentences of text. the tweets contains both msa and da, while the text content is mostly msa. marbert abdul-mageed et al. (2021) created and distributed large-scale pre-trained masked language model focused on both dialectal arabic (da) and modern standard arabic (msa). it was trained on a dataset of 1 billion arabic tweets from an in-house dataset of about 6 billion tweets. ## conclusion and future work our results suggest that freezing limited numbers of layers from the bottom in combination with top layers provide an optimal performance. it successfully addressed the challenge of time-consuming fine-tuning for large language models. this indicate that the perturbation from the fine-tuning can be controlled best using this approach; further, the approach might generalized better for out of domain data, as it keeps all the knowledge learnt during the pre-training. by introducing the layers freezing, we were able to achieve impressive time savings that reached up to 50% of time required for fine-tuning compared to conventional methods. this achievement in computation time optimization adds to the major advancement in the field of nlp and deep learning in general. it not only empowers researchers to conduct experiments and iterate more swiftly but also enhances the practicality of implementing large language models in real-world applications. for future work, we plan to expand this research to cover more tasks to ground these findings. more models with different architecture will be needed as well as applications in other languages. in other direction, we plan to explore the impact of the approach on generalization to out of domain and unseen data. such explorations will validate the approach and demonstrate its merits.