Know Where You’re Going: Meta-Learning for Parameter-Efficient Fine-Tuning (2023)

Abstract: A recent family of techniques, dubbed lightweight fine-tuning methods, facilitates parameter-efficient transfer by updating only a small set of additional parameters while keeping the parameters of the original model frozen. While proven to be an effective approach, there are no existing studies on if and how such knowledge of the downstream fine-tuning approach calls for complementary measures after pre-training and before fine-tuning. In this work, we show that taking the ultimate choice of fine-tuning into consideration boosts the performance of parameter-efficient fine-tuning. By relying on optimization-based meta-learning using MAML with certain modifications for our distinct purpose, we prime the pre-trained model specifically for parameter-efficient fine-tuning, resulting in gains of up to 4.96 points on cross-lingual NER fine-tuning. Our ablation settings and analyses further reveal that the specific approach we take to meta-learning is crucial for the attained gains.

## introduction the pre-@xmath0, owing to state-of-the-art performance on a wide variety of tasks @xcite . the impressive effectiveness of this approach does not come at a low price. it requires iterative adjustment of anywhere between millions @xcite to staggering billions of parameters @xcite . with this many parameters, fine-tuning all parameters, as is common, becomes exceedingly computationally expensive: where many models need to be fine-tuned, serving a separate copy of all a model's parameters for each instance is costly in terms of storage. recent works on parameter-efficient (pe) foot_1 fine-tuning address this issue by introducing methods that alternatively rely on only changing a tiny set of extra parameters @xcite @xcite @xcite or a small fraction of the existing model's parameters @xcite . these methods have been shown to be competitive with full fine-tuning despite modifying only as little as 0.01% of all the parameters @xcite . with this shift towards lightweight fine-tuning, we ask if the pre-training needs to be complemented in any way as well. ought we further modify the pre-trained model, knowing that we are going to opt for pe fine-tuning? specifically, can we extend pre-training in a way that leads to parameter initializations that better suit pe finetuning than the initializations coming outright from the pre-trained language model (plm) and used by full fine-tuning? in this work, we show that, in fact, we can use optimization-based meta-learning to further modify the parameters from a plm so that they are more beneficial for pe fine-tuning and result in improved performance on the target task after transfer. we term this step, which sits between conventional pre-training and fine-tuning, "priming." specifically, as we describe in §3.2, we tweak the popular meta-learning approach maml @xcite for priming and crucially simulate the actual pe fine-tuning procedure in the inner loop of the algorithm. this means that instead of including all the parameters in the inner loop gradient update, we only consider those that will be updated by the pe fine-tuning method. thus, during the meta-gradient update in the outer loop of the algorithm, this information about the ultimate fine-tuning approach will be incorporated into the pre-trained values. we choose cross-lingual transfer for named entity recognition (ner) as the testbed to show the effectiveness of priming stage. we show that prim-ing a plm boosts the performance of cross-lingual pe fine-tuning for ner by up to 4.96 f1 points. we provide the details of our lightweight fine-tuning setup in §4. our ablation study in §5.1 reveals that simulating the fine-tuning procedure is indispensable to the observed improvements: it is not meta-learning in general, but how we formulate the meta-learning setup that leads to observed gains. our contributions are: 1) we propose a metalearning based mechanism termed "priming" to further update the parameters of a plm in a way that improves the final pe transfer performance; 2) we show the effectiveness of priming for crosslingual transfer for ner as an exhibit; 3) we justify and shed more light on the importance of the design elements in the priming algorithm through an ablation analysis. ## meta-learning background the meta-learning problem can be viewed as acquiring meta-parameters θ using meta-training data d meta-train such that θ, when used for adaptation, improves performance on a new task with training data d train @xcite . optimization-based metalearning algorithms formulate adaptation as an optimization procedure during which task parameters ϕ are obtained by fine-tuning meta-parameters θ: ## experimental setup while our proposed priming algorithm is modelagnostic, we need a concrete pe fine-tuning and meta-training setup for empirical evaluation. for lightweight fine-tuning, we choose adapters @xcite . in our experiments, we add a single adapter after the last layer of the pretrained transformer. our model then computes the logits for input as: h(g(f (x; θ p ); θ a ); θ h ), where f is the pre-trained model, g is the single adapter layer at the top, and h is the task-specific head. as a testbed, we experiment with cross-lingual ner. for this case, we can design the priming (meta-learning) and fine-tuning stages as such: meta-learning: using one or more source lan-guages, we construct the meta dataset and run priming. per our problem formulation, θ p and θ a are shared among languages, but each source language l has a separate head, parameterized by θ h l . fine-tuning: for each desired target language, we use the pre-trained and adapter parameter initializations acquired during meta-learning along with randomly initialized new head parameters as the model's starting point. we then fine-tune only the adapter parameters and the head parameters. in our single adapter layer setup, this means only updating fewer than 0.4% of all the parameters. ## results and analysis per ## conclusion we propose to add "priming" between the conventional pre-training and parameter-efficient finetuning to incorporate awareness of the transfer procedure in the plm. we model this as optimizationbased meta-learning, which integrates such knowledge by updating pre-trained parameters under pe fine-tuning simulation. we show the effectiveness of priming in improving baseline pe fine-tuning on cross-lingual transfer for ner. further analysis reveals that our decisions to 1) model priming with meta-learning instead of simple fine-tuning and 2) simulate the actual pe fine-tuning in the metalearning instead of using it unadjusted contribute to the effectiveness of priming.