Decoder Tuning: Efficient Language Understanding as Decoding (2023)

Abstract: With the evergrowing sizes of pre-trained models (PTMs), it has been an emerging practice to only provide the inference APIs for users, namely model-as-a-service (MaaS) setting. To adapt PTMs with model parameters frozen, most current approaches focus on the input side, seeking powerful prompts to stimulate models for correct answers. However, we argue that input-side adaptation could be arduous due to the lack of gradient signals and they usually require thousands of API queries, resulting in high computation and time costs. Specifically, DecT first extracts prompt-stimulated output scores for initial predictions. On top of that, we train an additional decoder network on the output representations to incorporate posterior data knowledge. By gradient-based optimization, DecT can be trained within several seconds and requires only one PTM query per sample. Empirically, we conduct extensive natural language understanding experiments and show that DecT significantly outperforms state-of-the-art algorithms with a 200x speed-up. Our code is available at https://github.com/thunlp/DecT .

## introduction recent advances in pre-trained models (ptms) demonstrate the power of the "pre-trainingfine-tuning" paradigm, which empowers broad downstream nlp tasks with a single backbone model @xcite @xcite . given the million even billionscale models, model-as-a-service (maas) has become an emerging practice in deploying massive ptms, where users can only get access to model inference apis @xcite . under such a scenario, ptms' parameters are frozen, and users cannot fine-tune the model on downstream tasks for adaptation. to find an alternative way, researchers have studied maas ptm adaptation methods extensively. most existing approaches in this line are based on prompts, which modify inputs with specific patterns. by wrapping inputs into cloze-style questions or prepending inputs with a few demonstrative examples, ptms could produce the right outputs directly and show strong "in-context" learning abilities @xcite without any parameter update. besides heuristic prompt design, some recent works try to optimize the input prompts without gradients. among them, black-box tuning (bbt) @xcite and bbtv2 @xcite apply evolutionary algorithm @xcite on continuous prompt tokens, while rlprompt @xcite adopts reinforcement learning to find discrete prompt tokens. nevertheless, gradientfree optimization is rather difficult and these input-side methods need to query the ptms thousands of times for optimization, which leads to huge inference costs in terms of time and computation resources. moreover, their final performance is not satisfying as well. given the flaws of input-side adaptation, we turn to output-side adaptation, which builds tunable decoder networks on model outputs. comparatively, output-side adaptation enjoys two major advantages: (1) we can directly tune decoder networks on top of model outputs with back-propagation rather than arduous alternatives. (2) we can reduce thousands of model queries to only once per sample. however, designing decoder networks is not straightforward. past studies have shown that merely tuning an mlp or lstm @xcite over output features cannot provide satisfying results @xcite , leaving this path underexplored. in this work, we aim to solve the performance issue for output-side adaptation, and we argue that there are two critical reasons behind it: (1) simply utilizing ptms as feature extractors ignores the infilling ability of ptms, which is a strong prior for adaptation. (2) mlp and lstm are not proper networks especially when training data is not sufficient. based on these findings, we present decoder tuning (dect), an enhanced output-side adaptation method. specifically, dect has two crucial design choices to address the above issues. first, dect queries the ptm with prompts and adopts model output scores as the initial predictions, which takes advantage of internal model knowledge. second, on top of the output representations, we select a prototypical network (protonet) @xcite as the decoder network and train it to fit the training data, which is more suitable for few-shot learning. in this way, dect modifies the initial model scores with subsequent training data, thus achieving better performance. through few-shot learning experiments on ten language understanding datasets, we highlight three advantages of dect (see figure 1 ). (1) dect achieves over 3% absolute accuracy improvement on average, greatly outperforming previous works. ## preliminaries given a set of training data d @xmath0, y i )} n @xmath1, we need to predict the label @xmath2, . . . , k} for sample x, where k is the number of classes. we assume that each class has the same amount of n training samples. in the maas setting, m is a black-box inference api with fixed parameters. therefore, we can only query the model with input x and get corresponding outputs. to better utilize the ptms, it has been a common practice to wrap input samples into prompts. specifically, we enclose each input x into a template t with a [mask] token (here we assume using a masked language model). then, we query m with t (x) and get the final layer hidden states h at the [mask] position and scores @xmath3. take sentiment analysis as an example, we can use as the template with @xmath4, great} as label words for negative and positive sentiment respectively. the output scores on these label words further correspond to the classes. ## methodology in this section, we elaborate on our proposed decoder tuning (dect) method for the classification task. we start with reviewing current input-side adaptation methods, then give an overview of dect and finally detail it step-by-step. ## analysis in addition to main experiments, we further provide more analytical experiments for understanding dect. we conduct ablation study on several components in section 5.1. then we evaluate the scaling effect (section 5.2), the impact of hyperparameter λ (section 5.3) and templates (section 5.4) respectively. we further conduct transferability experiments in appendix b. ## conclusion in this paper, we present dect, which performs both data and parameter-efficient adaptation with off-shelf ptms. by fusing prior model knowledge and posterior data knowledge, dect achieves su-perior performance on ten language understanding tasks. meanwhile, dect exceeds existing baselines by three orders of magnitude in terms of training time and the number of queries, highlighting its advantages in real-world deployment. in future works, we are eager to explore how to combine input and output-side adaptation methods for better ptm adaptation, and how to extend this line of research to more challenging scenarios.