Prompt Tuning for Discriminative Pre-trained Language Models (2022)

Abstract: Recent works have shown promising results of prompt tuning in stimulating pre-trained language models (PLMs) for natural language processing (NLP) tasks. However, to the best of our knowledge, existing works focus on prompt-tuning generative PLMs that are pre-trained to generate target tokens, such as BERT. It is still unknown whether and how discriminative PLMs, e.g., ELECTRA, can be effectively prompt-tuned. In this work, we present DPT, the first prompt tuning framework for discriminative PLMs, which reformulates NLP tasks into a discriminative language modeling problem. Comprehensive experiments on text classification and question answering show that, compared with vanilla fine-tuning, DPT achieves significantly higher performance, and also prevents the unstable problem in tuning large PLMs in both full-set and low-resource settings.

## introduction recent years have witnessed the great success of the pre-training-then-fine-tuning paradigm in natural language processing (nlp) @xcite @xcite @xcite . typically, language models are first pre-trained on large-scale corpora via self-supervised generative or discriminative tasks to learn universal text representations, and then fine-tuned to adapt to downstream tasks @xcite . however, the significant gap between the objective forms of model pre-training and fine-tuning hinders taking full advantage of plms in downstream tasks @xcite . prompt tuning has recently shown its effectiveness in stimulating the capability of plms by transforming downstream tasks into the same form as pre-training @xcite @xcite @xcite . however, to the best of our knowledge, existing works focus on prompt-tuning generative plms (i.e., plms pre-trained by generating target textual tokens from the context, such as bert @xcite and @xcite ). it is still unknown whether and how discriminative plms can be effectively prompt-tuned (i.e., plms pre-trained by discriminating replaced tokens, such as electra @xcite and wklm @xcite ). since discriminative plms typically enjoy competitive performance and superior computational efficiency compared with their generative counterparts @xcite , it can be especially appealing to prompt-tuning discriminative plms. in this work, we present dpt, the first prompt tuning framework for discriminative plms. dpt reformulates downstream tasks into a discriminative language modeling problem, maximally mitigating the gap between model pre-training and tuning. specifically, as shown in figure 1 , models are asked to discriminate correct answer tokens (e.g., correct labels for text classification, or answer spans for question answering) from the input tokens based on the reused discriminative language modeling head, where the objective form is identical to pre-training. you may to make up a book on the movie subject [cls] … … dlm head original replaced dlm head original replaced … … … … … … [sep] (b) fine-tuning a graceful movie [cls] [sep] cls head positive neutral a graceful movie [cls] [sep] swering in both full-set and low-resource settings. experimental results show that despite its simplicity, dpt significantly outperforms vanilla finetuning (e.g., 4.1% accuracy improvement in the low-resource sst-5 evaluation). moreover, previous works have shown that fine-tuning large plms can be highly unstable and even produce divergent results @xcite , which undermines the practicality of large plms. we show that dpt also addresses the unstable problem in tuning large discriminative plms. the contributions of our work are summarized as follows: (1) we present the first prompt tuning framework for discriminative plms. (2) comprehensive experimental results on text classification and question answering demonstrate the effectiveness of the proposed prompt tuning framework. ## preliminary in this work, without loss of generality, we take electra @xcite as a representative example of discriminative plms, while applying dpt to other discriminative plms is also applicable. here we introduce the main procedure of pre-training and fine-tuning, and we refer readers to the paper @xcite for more details. pre-training. during pre-training, a generator first corrupts the text via token replacement. then the discriminator is asked to detect the replaced tokens, by classifying each token into binary categories, i.e., {original, replaced}, as shown in figure 1 . finally, the generator is discarded and the discriminator is fine-tuned on downstream tasks. vanilla fine-tuning. (1) during fine-tuning, to perform text classification, a new classification head is typically introduced to classify the hidden representation of the [cls] token in the last layer @xcite . (2) for general multispan question answering, the answer could be multiple spans from the input text @xcite . state-of-the-art fine-tuning approaches formulate the task as a sequence-labeling problem, and classify each input token into binary labels based on a new classification head, indicating whether the token belongs to the answer or not @xcite . note that the classification head typically introduces new parameters, and learning the parameters from scratch usually requires a large amount of labeled data. moreover, previous works have shown that fine-tuning large plms can be highly unstable, and even produce divergent results @xcite . as a result, multiple fine-tuning trials are usually needed to find a good random seed that leads to a stably fine-tuned plm, which undermines the practicality of large plms. ## methodology in this section, we introduce the framework of dpt for prompt-tuning discriminative plms. we first introduce dpt using text classification as the running example, and then illustrate its application in question answering. dlm-based reformulation. dpt reformulates nlp tasks into a dscriminative language modeling problem, maximally mitigating the gap between pre-training and tuning. specifically, as shown in figure 1 (c), for a text classification task with class set @xmath0, c 2 , . . . , c n }, dpt defines a template that contains all answer candidates t (•; c). given an input text x (e.g., "a graceful movie."), dpt fills the input text into the template as follows: intuitively, t (x; c) can be understood as creating a virtual context that assumes all candidate classes are correct for the input text x. it is then straightforward for discriminative plms to decide whether each class candidate token is proper in the context, by classifying the tokens into original (i.e., correct), or replaced (i.e., incorrect) based on the reused dlm head. in our experiments, we find that the order of classes in template has minimal influence on the performance, and a random order can produce good prompt-tuning results. dpt training. after template filling, t (x; c) is fed into plms to obtain the hidden representations {h @xcite , h 1 , h 2 , . . . , h m , h [sep] }. plms are then prompted to discriminate whether each class is correct. specifically, we compute the score of class c i based on the representation of the corresponding token t i as: foot_0 dpt for question answering. besides text classification, dpt can also be applied for the question answering task. given a question and a paragraph, directly concatenating them without additional templates can already create a good prompting context. then similar to text classification, we ask plms to discriminate whether each token in the paragraph is part of the answer (i.e., original), or not (i.e., replaced) based on the reused dlm head. during inference, we threshold the token scores to obtain multiple answer spans. ## experiments in this section, we empirically evaluate dpt on the task of text classification and question answering. datasets. we evaluate dpt on four widely used text classification datasets, including sst-2, sst-5, trec and agnews. for question answering, we adopt the challenging quoref dataset, where for each question, there may exist multiple answer spans in the paragraph. we refer readers to section b for more dataset details. ## conclusion and future work in this work, we present a simple and effective prompt tuning approach for discriminative plms. we note directly performing large-scale classification (e.g., for hundreds of classes) with dpt may be computationally inefficient. in future, we plan to address the problem by classifying text following class hierarchies, where each hierarchical layer typically consists of a moderate number of classes.