L 3 M asking: Multi-task Fine-tuning for Language Models by Leveraging Lessons Learned from Vanilla Models (2024)

Abstract: When distributional differences exist between pre-training and fine-tuning data, language models (LMs) may perform poorly on downstream tasks.Recent studies have reported that multi-task learning of downstream task and masked language modeling (MLM) task during the fine-tuning phase improves the performance of the downstream task.Typical MLM tasks (e.g., random token masking (RTM)) tend not to care tokens corresponding to the knowledge already acquired during the pre-training phase, therefore LMs may not notice the important clue or not effective to acquire linguistic knowledge of the task or domain.To overcome this limitation, we propose a new masking strategy for MLM task, called L3Masking, that leverages lessons (specifically, token-wise likelihood in a context) learned from the vanilla language model to be fine-tuned.L3Masking actively masks tokens with low likelihood on the vanilla model.Experimental evaluations on text classification tasks in different domains confirms a multi-task text classification method with L3Masking performed task adaptation more effectively than that with RTM.These results suggest the usefulness of assigning a preference to the tokens to be learned as the task or domain adaptation.

## introduction language models (lm) pre-trained on generic documents such as bert @xcite or gpts (e.g., @xcite ) may perform poorly on downstream tasks when the vocabulary or context used in the documents in each pre-training and downstream task differs @xcite . to bridge the domain gap between pre-training and fine-tuning, continual pre-training is used. continual pre-training re-trains a model by applying the 1 the code is available at https://github.com/usk-kim ura/l3masking m lm af t er pr e-t r ai n i n g to overcome this, we propose a new masking strategy for mlm called l3masking (leveraging lessons learned from vanilla model) as an effective task or domain adaptation. figure 1 highlights the difference between l3masking and a popular and simple masking strategy random token masking. l3masking identifies tokens with low likelihoods as task-or domain-specific tokens that appear less frequently in the similar contexts in the generic documents, and it actively masks them so that lm learns these tokens during fine-tuning. unlike causal language modeling which computes the likelihood of a token in a sentence only from the preceding tokens, mlm can compute the likelihood of the token conditional on both preceding and subsequent tokens. this difference has led to variations in the idea of a sentence's likelihood and has been noted in what is called the pseudo-loglikelihood (pll) @xcite . based on the pll, this study defines a token-wise pseudolikelihood in the downstream task sentence and actively mask tokens with low pseudo-likelihood. in consequence, the contributions of this paper can be summarized as follows: ## related studies this section describes the task or domain adaptation methods that have been studied in contexts of continual pre-training and fine-tuning. ## l3masking: the proposed method this paper propose a new masking strategy of mlm task for task or domain adaptation, called l3masking. the basic idea is to improve task and domain adaptability by actively masking tokens in sentences that are not well trained during pre-training. figure 2 depicts the overview of l3masking. l3masking captures the tokens that are most likely to represent task-or domain-specific linguistic knowledge based on a token-wise likelihood. since the likelihood cannot be calculated simply in bidirectional lm, l3masking calculates the token-wise pseudo-likelihood and then masks more tokens with lower the pseudo-likelihood. ## evaluation experiment to evaluate l3masking, we conducted experiments to answer the following four questions: ## conclusion this paper introduced l3masking as a novel masking strategy for fine-tuning of masked language models to text classification. our method leverages likelihood scores from the vanilla models to actively mask task-or domain-specific tokens. for calculating mask probability on the bidirectional mlms, token-by-token pseudo-likelihood scores are used. our method focuses more on tokens that are underrepresented in the pre-training corpus but are crucial for downstream tasks. through the experimental evaluation of three text classification tasks from different domains, we demonstrated that l3masking outperforms traditional random token masking, particularly in domain-specific language models such as scibert and clinicalbert. future work will focus on refining the token selection algorithm to handle diverse datasets better and exploring l3masking's potential in other nlp tasks beyond text classification. additionally, applying l3masking to the continual pre-training of large language models (llms) represents a significant future direction. by leveraging l3masking in llms, we aim to achieve more accurate domain adaptation, task-specific learning, and effective utilization of large-scale datasets, ultimately enhancing llms' overall performance and applicability in various specialized and general domains.