Attribute Controlled Dialogue Prompting (2023)

Abstract: Prompt-tuning has become an increasingly popular parameter-efficient method for adapting large pretrained language models to downstream tasks. However, both discrete prompting and continuous prompting assume fixed prompts for all data samples within a task, neglecting the fact that inputs vary greatly in some tasks such as open-domain dialogue generation. In this paper, we present a novel, instance-specific prompt-tuning algorithm for dialogue generation. Specifically, we generate prompts based on instance-level control code, rather than the conversation history, to explore their impact on controlled dialogue generation. Experiments on popular open-domain dialogue datasets, evaluated on both automated metrics and human evaluation, demonstrate that our method is superior to prompting baselines and comparable to fine-tuning with only 5%-6% of total parameters.

## introduction fine-tuning has been frequently used when deploying generative pretrained language models (plms) to downstream tasks since the advent of gpt @xcite and bert @xcite . however, this requires storing a full copy of parameter states for every downstream task, which is memoryconsuming and expensive to serve when working with large-scale models with billions of parameters like gpt-3 @xcite . in this work, we design a lightweight prompting module for adapting pretrained language models for attribute controlled dialogue generation. more precisely, for each attribute such as persona, intention, emotion etc. we only save an additional prompt module. since the prompting module is a fraction of the size of the pretrained dialogue model, this allows many controlled dialogue systems to be stored on a device without too much ing. idpg @xcite consumes inputs in a two-layer perceptron module to generate instancedependent prompts in classification tasks rather than generation tasks. in addition, @xcite proposes dialogprompt which performs instancespecific prompting for dialogue generation by conditioning the prompt on the entire dialogue history. however, their prompting module consists of gpt-2, which is a full-fledged language model, and the approach is as costly as storing an entire fine-tuned base model. recent works contrastive prefixes @xcite and tailor @xcite both propose attribute-based prompts, instead of instance-specific, to include either single-attribute or multi-attribute prompts into controlled text generation tasks, which reveal the powerful potential of controllability of continuous prompts. in contrast to previous work, we propose controlled dialogprompt for applying prompt-tuning in controlled dialogue generation, which optimizes prompts based on provided control codes rather than the previous conversation history and we further explore the controllability of prompts at the instance level. the size of the prompt encoder is strictly limited and we freeze the pretrained transformer during training in order to preserve memory efficiency. in addition, we would like to highlight that our work focuses more on open-ended text generation rather than natural language understanding, such as entailment, paraphrase detection, extractive qa, as seen in other parameter-efficient fine-tuning methods @xcite @xcite . we posit that generating high-quality text is a more challenging task that requires a more nuanced approach to prompt tuning. ## controlled dialogprompt in this section, we present controlled dialog-prompt (controlled dp) for dialogue generation, which is expected to provide attribute information such as the dialogue intention or the user's persona within the prompt and steer the pretrained model efficiently. soft prompt-tuning @xcite learns soft tokens for different tasks and then prepends them to the conversation context as well as control attributes. this approach yields a static shallow prompt since the soft tokens are static (i.e., fixed for a task) and shallow (only added as an input to the language model). in contrast, prefix-tuning proposes a more effective technique that adds soft tokens in the form of key-value pairs at every attention block of the transformer @xcite . this allows the soft tokens to influence each stage of the language model and therefore it is referred to as a static deep prompt. figure 1 (bottom right) shows our proposed controlled dialogue prompt (deep version). instead of training static soft tokens for the dialogue task, we train a lightweight prompt module that takes as input a control attribute, either an intention label or persona sentences, and outputs key-value pairs that are prepended to each layer of the language model. since the soft token embeddings change depending on the control attribute, this corresponds to an instance-specific prompt. for the shallow prompt (figure 1 bottom left), we follow soft prompt-tuning which adds an additional trainable embedding layer to encode the attribute. for the deep prompt module, we consider two architectures: i) a simple multilayer perceptron (two fully connected layers of size 512 with tanh activation) applied to each token of the control attribute, and ii) a two-layer transformer decoder with embedding size of 256. the embedding size of each architecture was chosen to yield roughly the same number of parameters. this number of parameters is about 5%-6% of the number of parameters of the language model. for a given domain, training the prompt module is done as follows. an intention label or persona sentences are fed to the prompting module, which outputs key-value pairs added at each layer of the frozen pretrained dialogue system. gradients to maximize the likelihood of response tokens are back-propagated through the dialogue system and prompting module, but only the weights of the prompting module are updated. ## conclusion and future work in summary, we presented a novel prompting technique, conditioned on a dialogue attribute (persona or intent), for controlled dialogue generation. the prompting module requires only 5%-6% of the total number of parameters, which allows the storage of several fined-tuned prompting modules for different dialogue generation tasks at a fraction of the cost of a full dialogue model. however, controlled dialogprompt currently studies conditioning on simple control attribute sentences like the user's persona and the work can be extended to more extensive and complex sentences such as background knowledge documents to further evaluate the controlled prompt's encoding capabilities. additionally, combining multiple controlled dialogprompts on several control attributes and automatically triggering various dialogue skills is an interesting and unexplored direction.