Multitask Pre-training of Modular Prompt for C hinese Few-Shot Learning (2023)

Abstract: Prompt tuning is a parameter-efficient approach to adapting pre-trained language models to downstream tasks. Although prompt tuning has been shown to match the performance of full model tuning when training data is sufficient, it tends to struggle in few-shot learning settings. In this paper, we present Multi-task Pre-trained Modular Prompt (MP2) to boost prompt tuning for few-shot learning. MP2 is a set of combinable prompts pre-trained on 38 Chinese tasks. On downstream tasks, the pre-trained prompts are selectively activated and combined, leading to strong compositional generalization to unseen tasks. To bridge the gap between pre-training and fine-tuning, we formulate upstream and downstream tasks into a unified machine reading comprehension task. Extensive experiments under two learning paradigms, i.e., gradient descent and black-box tuning, show that MP2 significantly outperforms prompt tuning, full model tuning, and prior prompt pre-training methods in few-shot settings. In addition, we demonstrate that MP2 can achieve surprisingly fast and strong adaptation to downstream tasks by merely learning 8 parameters to combine the pre-trained modular prompts.

## introduction pre-trained models (ptms) @xcite @xcite with prompt-based learning have achieved remarkable progress in few-shot learning. a major reason behind their success is the closed gap between upstream pre-training and downstream fine-tuning @xcite . since the downstream tasks are reformulated into a unified (masked) language modeling ((m)lm for short) task, one can reuse the pre-trained (m)lm head instead of training a randomly initialized classification head to solve tasks with limited data. however, prompt-based learning (e.g., pet @xcite and lm-bff @xcite ) usually fine-tunes all the parameters of the ptm for each downstream task, which can be computationally expensive and deployment-inefficient, especially for large ptms such as @xcite . recently, much effort has been devoted to parameter-efficient prompt tuning @xcite @xcite , which only learns a small number of soft prompt parameters while keeping the main body of the ptm untouched. in contrast to full model tuning, prompt tuning can get specialized models for specific tasks by simply attaching taskspecific prompts, and therefore is highly efficient for serving different tasks. though it has been demonstrated that prompt tuning can match the performance of full model tuning when training data is sufficient @xcite , the soft prompt cannot be well trained from scratch in few-shot learning settings @xcite because the randomly initialized soft prompt introduces a new gap between pre-training and fine-tuning. to bridge the gap between pre-training and finetuning for prompt tuning, we present multi-task pre-trained modular prompt (mp 2 ). as illustrated in figure 1 , we insert a second pre-training procedure before downstream fine-tuning, in which we pre-train a set of modular prompts with multitask learning. the modular prompts are selectively activated and combined by a trainable router for specific tasks. by this, we can achieve fast adaptation to downstream tasks by learning to combine and reuse the pre-trained modular prompts. drawing inspiration from the success of deep prompt tuning @xcite , we inject soft prompt into every layer of the ptm. further, considering that a variety of tasks cannot be reformulated into a (m)lm task, we instead recast upstream and downstream tasks into a unified machine reading comprehension (mrc) task, which has shown great potential to unify various nlp tasks @xcite . we pre-train mp 2 on 38 chinese nlp tasks and evaluate on 14 downstream tasks including sentiment analysis, topic classification, natural language inference, question answering, multiple choice classification, and keyword extraction. experimental results in few-shot learning settings demonstrate that mp 2 outperforms prompt tuning, full model tuning, and previous prompt pre-training methods @xcite by a large margin. we also evaluate the compatibility of mp 2 with black-box tuning (bbt) @xcite and bbtv2 @xcite , which are gradient-free prompt tuning methods. as a result, mp 2 achieves significant improvement over bbt and bbtv2. besides, we demonstrate that mp 2 can achieve surprisingly fast adaptation to target tasks by merely tuning the router (only 8 parameters) while freezing the ptm and all the prompts. foot_0 ## related work this work lies in the line of parameter-efficient tuning (pet) @xcite , which trains a small portion of parameters to adapt ptms to downstream tasks. the small tunable parameters can be lightweight neural adapters between ptm layers @xcite , or soft prompt attached to the input examples @xcite or hidden states @xcite , or bias terms in the ptm parameters @xcite , or lowrank matrices to be added to attention weights @xcite . especially, this work is closely related to two prior works on prompt tuning, namely ppt @xcite and spot @xcite . ## methods we first introduce the mrc format used to unify different tasks in §3.1, and then describe the deep modular prompt in §3.2, and finally we detail the procedure of multi-task pre-training and downstream fine-tuning in §3.3 and §3.4, respectively. ## conclusion this work aims to bridge the gap between pretraining and fine-tuning of soft prompt tuning for few-shot learning. to achieve this, we extend the soft prompt in two dimensions, depth and width. the extended prompt, named deep modular prompt, is pre-trained on a mixture of 38 public chinese nlp tasks, which are reformulated into the mrc format. for adaptation to downstream tasks, we propose the two-stage tuning, where we first learn to combine and reuse pre-trained prompts and then tune the selected prompts with gradient descent or black-box optimization. extensive experiments on 14 downstream tasks demonstrate that, the multitask pre-trained modular prompt (mp 2 ) significantly outperforms prompt tuning, full model tuning, and previous prompt pre-training methods, namely ppt and spot. surprisingly, we demonstrate that mp 2 can achieve extremely fast adaptation to downstream tasks by only learning to combine pre-trained prompts.