Context-Tuning: Learning Contextualized Prompts for Natural Language Generation (2022)

Abstract: Recently, pretrained language models (PLMs) have had exceptional success in language generation. To leverage the rich knowledge encoded by PLMs, a simple yet powerful paradigm is to use prompts in the form of either discrete tokens or continuous embeddings. In existing studies, these prompting methods are typically independent of the inputs, lacking sufficient consideration of input semantics. To address this issue, we propose a novel continuous prompting approach, called context-tuning , to fine-tuning PLMs for natural language generation. Firstly, the prompts are derived based on the input text to elicit useful knowledge from PLMs for generation. We refer to such prompts as contextualized prompts . Secondly, we use continuous inverse prompting to improve the process of natural language generation by modeling an inverse generation process from output to input, making the generated text more relevant to the inputs. Furthermore, we utilize a lightweight context-tuning method that fine-tunes only 0.12% of the parameters while maintaining good performance. Our code is publicly available at https://github.com/RUCAIBox/Context-Tuning .

## introduction natural language generation (a.k.a. text generation) aims to produce plausible and readable text in human language from input data @xcite . recently, large-scale pretrained language models (plms) such as bart @xcite have had exceptional success in language generation. to leverage the encoded knowledge from plms, prompting methods have been proposed @xcite , where the original input to plms has been extended by prepending discrete tokens or continuous embeddings (called prompts). following this paradigm, this work aims to study how to develop story i think that live-action works can't be considered art. they feel more like ::::::::::: documentaries or ::::: theater :::: pieces with ::: cgi combined. the superiority of animated works is that they are more abstract and imaginative and characters show more emotion and variety of designs. early methods focused on human-written (discrete) prompts by manually constructing taskspecific prompt templates @xcite , such as "tl;dr:" for the summarization task. recent work has further proposed utilizing continuous prompts @xcite ) for text generation. continuous prompts consist of trainable parameters that do not correspond to real tokens and can be easily optimized during fine-tuning. however, existing prompting approaches typically adopt static prompts for generation, i.e., the prompts contain task-related information but remain the same for different input texts. in this work, we mainly focus on challenging open-ended generation, such as story generation @xcite and review generation @xcite . under this setting, the input text usually contains very limited information, while the task goal is to generate an output sequence containing informative contents based on the given limited input. the example in to address the above issues, we propose context-tuning, a novel continuous prompting approach to fine-tuning plms for natural language generation. our approach has three major technical contributions. firstly, the prompts are derived based on input text to enrich the input by eliciting related knowledge from plms. specifically, by concatenating limited input and a sequence of "[mask]" tokens into bert @xcite , we leverage its excellent mask-filling ability to predict these tokens, and the last hidden state of tokens can be used as prompt vectors. since the prompts are highly related to the input context, we refer to them as contextualized prompts. secondly, to further enhance the relevance between the generated text and the input text, we extend inverse prompting @xcite by incorporating continuous prompts. we refer to them as continuous inverse prompting. by maximizing the likelihood of predicting inputs conditioned on the generated text and continuous prompts, context-tuning can generate texts highly relevant to the input text. moreover, to ease the training burden, we propose to use a lightweight context-tuning method @xcite that only fine-tunes the bias term of all model parameters. in this way, we can achieve a comparable performance (98.0% of the full-tuned performance) by only tuning 0.12% of the parameters, compared to full-tuned context-tuning. to our knowledge, we are the first to encode input-related information into continuous prompts for text generation. our context-tuning approach can elicit relevant knowledge according to specific input text and enhance the relevance between the generated text and the input text. we compare our method with several baseline models for the evaluation of four natural language generation tasks. extensive experiments demonstrate the effectiveness of our proposed context-tuning approach. prompting learning. prompting methods prepend task instructions to the input and generate the output from plms. most typical methods utilize manually designed task-specific prompts to adapt to different generation tasks @xcite ). however, it is time-consuming and laborious to construct human-written prompts for various generation tasks. as a result, recent research has concentrated on automating the search for discrete prompts @xcite . nonetheless, searching for prompts over discrete space is challenging to optimize due to the non-differentiable issues and continuous nature of neural networks. to handle these problems, many studies propose optimizing continuous prompts @xcite , which are more expressive and flexible for any task. among these works, prefix-tuning @xcite and prompt tuning @xcite are two representatives focused on text generation and natural language understanding (nlu) tasks, respectively. compared with these continuous approaches, our context-tuning encodes the context information of inputs into the contextualized prompts and adopts continuous inverse prompting to enhance relevance further. most existing prompting methods @xcite @xcite focus on nlu tasks, which are choice questions that can easily be converted into filling ## the proposed approach in this section, we present the proposed contexttuning to fine-tune plms for natural language generation. we first introduce the contextualized prompts based on the input text for generating informative text. to further enhance the relevance of the generated text to the input, we utilize continuous inverse prompting to enforce the prediction of inputs given the generated text and continuous prompts. figure 1 presents an overall illustration of the proposed context-tuning approach. for natural language generation, we consider a general task setting, where the model generates the output sequence y conditioned on the input sequence @xmath0, . . . , x l . the output text is usually composed of multiple sentences: @xmath1: y j,1 , . . . , y j,t , . . . , y j,n j } m @xmath2. in contexttuning, we introduce contextualized prompts p @xmath3, . . . , p c k into the input side. thus, the promptbased generation task can be formulated as: pr(y|x , p c ) = pr(y1, . . . , ym|x1, . . . , x l , p c ). (1) ## experiment in this section, we first set up the experiments and then report the results and analysis. ## conclusion this paper presents a novel continuous prompting approach, i.e., context-tuning, to fine-tuning plms for natural language generation. the core idea is to inject input-related context information into continuous prompts, called contextualized prompts, to enhance the informativeness of generation. the contextualized prompts can elicit input-relevant knowledge from plms to enrich the input text. furthermore, to enhance the relevance of the generated text to the inputs, we adopt a continuous inverse prompting method to refine the forward generation process by modeling an inverse generation process from output to input. we also propose a lightweight method for efficient training. extensive experiments on four generation tasks have demonstrated the effectiveness of our model in fine-tuning plms for text generation tasks. in future work, we will consider integrating more types of context information (e.g., sentiment) to derive more expressive prompts and investigate how our model could be applied to other tasks. colin raffel, noam shazeer, adam roberts, katherine lee, sharan narang, michael matena, yanqi zhou, wei li, and peter j. liu. 2020. exploring the limits of transfer learning with a unified text-totext transformer. journal of machine learning research, 21(140):1-67. leonardo f. r. ribeiro, martin schmitt, hinrich schütze, and iryna gurevych. 2021. investigating pretrained language models for graph-to-text generation. in proceedings of the 3rd workshop on natural language processing for conversational ai, pages 211-227, online. association for computational linguistics. timo schick and hinrich schütze. 2021a. exploiting cloze-questions for few-shot text classification and natural language inference. in proceedings of the 16th conference of the european chapter of the association for computational linguistics: main volume, pages 255-269, online. association for computational linguistics. timo schick and hinrich schütze. 2021b. few-shot text generation with natural language instructions. in proceedings of the 2021 conference on empirical methods in natural language processing, pages 390-402, online and punta cana, dominican republic. association for computational linguistics. taylor shin, yasaman razeghi, robert l. logan iv, eric wallace, and sameer singh. 2020. autoprompt: eliciting knowledge from language models with automatically generated prompts. in proceedings of the 2020 conference on empirical methods in natural language processing (emnlp), pages 4222-4235, online. association for computational linguistics. ruize wang, duyu tang, nan duan, zhongyu wei, xuanjing huang, jianshu ji, guihong cao, daxin jiang, and ming zhou. 2021. k-adapter: infusing knowledge into pre-trained models with adapters. in findings of the association for computational linguistics: acl-ijcnlp 2021, pages 1405-1418, online. association for computational linguistics. thomas wolf, lysandre debut, victor sanh, julien chaumond, clement delangue, anthony moi, pierric cistac, tim rault, remi louf, morgan funtowicz, joe davison, sam shleifer, patrick von platen, clara ma, yacine jernite, julien plu, canwen xu, teven le scao, sylvain gugger, mariama drame, quentin lhoest, and alexander rush. 2020. transformers: state-of-the-art natural language processing. in proceedings of the 2020 conference on empirical methods in natural language processing: system demonstrations, pages 38-45, online. association for computational linguistics. yizhe zhang, siqi sun, michel galley, yen-chun chen, chris brockett, xiang gao, jianfeng gao, jingjing liu, and bill dolan. 2020. dialogpt : largescale generative pre-training for conversational response generation. in proceedings of the 58th annual meeting of the association for computational linguistics: system demonstrations, pages 270-278, online. association for computational linguistics. xu zou, da yin, qingyang zhong, hongxia yang, zhilin yang, and jie tang. 2021. controllable generation from pre-trained language models via inverse prompting. in proceedings of the 27th acm sigkdd conference on knowledge discovery & data mining, kdd '21, page 2450-2460. association for computing machinery. thank you for taking time out of your busy schedule to participate in our scientific research evaluation! our research work is to let the machine generate corresponding story, for a given title, and hope that it is as close as possible to what humans write. hence, we need to evaluate whether it meets the standards that people think. in this task, you will see a title, idea or introduction such as: you have a very shitty type of precognition . then you'll see a corresponding story or comment, either written by a human or a machine, but you don't know which one was written, and you'll have to rate it based on the following standard: for turing test, just judge the text by instinct. for the other four factors, we adopt a 5-point likert scale as the scoring mechanism, in which 5-point means "very satisfying", and 1-point means "very terrible".