Self-supervised Meta-Prompt Learning with Meta-Gradient Regularization for Few-shot Generalization (2023)

Abstract: Prompt tuning is a parameter-efficient method, which learns soft prompts and conditions frozen language models to perform specific downstream tasks. Though effective, prompt tuning under few-shot settings on the one hand heavily relies on a good initialization of soft prompts. On the other hand, it can easily overfit to few-shot training samples, thereby undermining generalizability. Existing works leverage pre-training or supervised meta-learning to initialize soft prompts but they fail to data-efficiently generalize to unseen downstream tasks. To address the above problems, this paper proposes a novel Self-sUpervised meta-Prompt learning framework with MEta-gradient Regularization for few-shot generalization (SUPMER). SUPMER leverages self-supervised meta-learning with a diverse set of well-designed meta-tasks to learn a universal prompt initialization for efficient adaptation using only unlabeled data. Additionally, it jointly meta-learns a gradient regularization function to transform raw gradients into a domain-generalizable direction, thus alleviating the problem of overfitting. Extensive experiments show that SUPMER achieves better performance for different few-shot downstream tasks, and also exhibits a stronger domain generalization ability. The code for SUPMER will be available at https://github.com/beepkh/SUPMER.

## introduction recent nlp accomplishments witnessed the rapid development of pre-trained language models (plms) (e.g., @xcite @xcite . finetuning, which tunes the entire plm parameters, has achieved outstanding performances in various nlp tasks. however, as the pre-trained model scale increases, tuning the entire set of parameters * work done when interning at alibaba damo academy. though prompt tuning is an efficient and effective paradigm, @xcite shows it performs much worse than fine-tuning under few-shot settings. we argue that the performance is not satisfactory mainly due to two limitations: 1) the performance of pt is highly sensitive to the soft prompt initialization, especially for few-shot tasks. as shown in figure 1 (a), different soft prompt initialization leads to significant performance variations. 2) few-shot pt risks overfitting to some spurious correlations as soft prompts are tuned on limited training samples, thus undermining the generalizability of plms. as shown in figure 1 (b) , the performance of few-shot vanilla pt degrades significantly in the final training steps. recent research mainly focused on the first limi-tation, leveraging pre-training or supervised metalearning for soft prompt initialization. a pretrained prompt tuning method (ppt) @xcite is proposed from the beginning, which utilizes self-supervised tasks to pre-train soft prompts and then applies them in the few-shot scenario. however, without explicitly optimizing the fast adaptation ability of the model, ppt suffers from a train-test mismatch between the pre-training data and the downstream data. so it limits generalization to unseen few-shot tasks, especially when there is a significant disparity in task domains or formats. metaprompting @xcite , as another effort, seeks assistance from model-agnostic meta-learning ( @xcite for fast adaptation in few-shot settings. however, in each task, metaprompting requires plenty of labeled data within certain classes to perform supervised meta-learning for prompt initialization, which is often inaccessible in practical few-shot scenarios. and the learned initialization can only generalize to the remaining classes of the same task in a fewshot manner, exhibiting weak task transferability. furthermore, all these existing works ignore the second limitation, i.e., the propensity for few-shot prompt tuning to lead to overfitting. to address the shortcomings of existing works, we propose supmer, a self-supervised meta-prompt learning framework with meta-gradient regularization. it leverages self-supervised metalearning to universally learn an efficient soft prompt initialization, also with a meta-gradient regularization function to mitigate overfitting. this comprehensive process only requires a one-time execution and enables seamless adaptation to different downstream few-shot tasks, while also facilitating faster convergence for downstream prompt tuning. specifically, to address the first limitation, we design a novel self-supervised meta-learning method for prompt initialization, which automatically generates a diverse set of meta-training tasks from large-scale unlabeled corpora and explicitly learns to fast adapt across these tasks. to ensure task diversity, we initially design a collection of anchor self-supervised meta-training tasks with different formats. and then a curriculum-based task augmentation method is further proposed to enrich the task distribution dynamically in terms of the current model capability. for the second issue, we integrate a metagradient regularization function into meta-prompt learning. as we simulate distribution shift through task augmentation, the meta-gradient regularization parameters are jointly optimized to align gradient directions across different distributions within our proposed meta-prompt learning paradigm. consequently, in downstream tasks, these optimized parameters can be directly utilized to transform raw gradients over few-shot samples into a domaingeneralizable direction, preventing prompt tuning overfitting to some domain-specific correlations. overall, our contributions are mainly three-fold: to efficiently train task-adaptive soft prompts in few-shot scenarios, some studies @xcite @xcite employ task adaptation techniques, obtaining source prompts from source tasks in a supervised way and interpolating them into the target prompts. other works focus on training improved prompt initializations. ppt @xcite pre-trains the soft prompts with some self-supervised tasks on unlabeled corpora, but it doesn't explicitly optimize the fast adaptation ability of the model. metaprompting @xcite utilizes supervised meta-learning for soft prompt initialization, splitting each dataset into two sets with disjoint data classes. one split is used to initialize soft prompts while the other serves as the downstream task. in comparison, supmer differs from metaprompting in the following ways: 1) for each downstream task metaprompting focuses on a fixed supervised dataset to reinitialize soft prompts, whereas supmer can universally generalize to different unseen tasks with large-scale unlabeled corpora for initialization; 2) metaprompting doesn't freeze plm parameters, while sup-mer only tunes the soft prompts as the general soft prompt tuning methods do. ## method in this section, we describe the whole framework of supmer (shown in figure 2 ). with pre-defined preliminaries, we first introduce the way to construct anchor self-supervised meta tasks and the foundation of task augmentation to densify task distributions. then we elaborate on the supmer model, including the meta-gradient regularization function. finally, we upgrade the original task augmentation method into a curriculum-based one. besides, we formalize all tasks in a text-to-text format following the t5 fashion @xcite . ## conclusion in this paper, we present supmer, a selfsupervised meta-prompt learning framework with meta-gradient regularization. with a diverse set of well-designed self-supervised meta-training tasks, supmer jointly meta-learns a universal prompt initialization and an effective gradient regularization function for efficient few-shot generalization. extensive experiments on few-shot learning and domain generalization show that supmer outperforms other prompt methods and full-model tuning, achieving state-of-the-art performance.