Control Prefixes for Parameter-Efficient Text Generation (2022)

Abstract: Prefix-tuning is a parameter-efficient and powerful technique for adapting a pre-trained language model to a downstream application. However, it uses the same dataset-level tuned set of parameters for all examples in the dataset. We extend the framework with a dynamic method, Control Prefixes, which allows for the effective inclusion of input-dependent information, thereby demonstrating how prefix-tuning can be used for controlled text generation tasks. The method incorporates attribute-level learnable representations into different layers of a pre-trained Transformer, enabling the generated text to be guided in a particular direction. We provide a systematic evaluation of the technique and apply it to five datasets from the GEM benchmark for natural language generation (NLG). Using only 0.1–2% additional trainable parameters, we show Control Prefixes can even outperform full fine-tuning methods, and present state-of-the-art results on several data-to-text datasets, including WebNLG. We also examine the common case where input-dependent information is unavailable at test time and show Control Prefixes can excel in this setting also.

## introduction approaches in text generation have been dominated by adapting pre-trained language models (plm) to various downstream tasks. as the scale of plms continue to climb, the cost of updating all the plm parameters per task, and resultant overhead of entirely new parameter-sets per task becomes impractical. furthermore, full fine-tuning has been shown to result in catastrophic forgetting where knowledge learnt from the pre-training task is lost and natural language understanding overwritten @xcite . recent work has demonstrated that it is possible to train these models by optimizing a negligible fraction (0.01-2%) of additional parameters while leaving the base plm parameters unchanged @xcite . such parameter-efficient transfer learning (petl) can achieve performance comparable to fine-tuning. prefix-tuning @xcite , which trains a prefix of additional key-value pairs at each layer, and adapters @xcite are the two current most popular petl methods. another alternative is in-context learning (icl) @xcite , which supplies hand-written prompts and requires no gradientbased training. icl has however shown to result in poor performance as the number of fine-tuning examples increases beyond a handful @xcite . we therefore believe that petl methods provide a more promising direction for study. a weakness of most petl methods is that the same additional parameters are used for all examples within a single task. as yet, there has been little research exploring petl methods that incorporate input-dependent parameters @xcite for finer-grained control. our work closes this gap by introducing a novel framework which extends prefix-tuning and demonstrates the utility of controlling parameter-efficient learning for data-to-text tasks. the method uses multiple modular control prefixes, trained simultaneously, which can change alongside the input according to the guidance signal. these dynamic prefixes operate together with the static prefix parameters and allow for finer-grained control over the frozen plm. the chosen attributes can provide additional context about the input, for example, the sub-domain of a data-to-text triple, or specify some aspect of the desired output, such as the target length for text simplification. controlled text generation aims to guide generation towards the desired attributes, by incorporating various types of guidance (e.g. highlighted phrases @xcite ). previous work has focused on directly updating all the existing model's parameters @xcite or using a discriminator to guide generation @xcite . other methods aim to generate text with specific target qualities, independent of overall task performance @xcite . in contrast, our proposed method is designed for maximizing downstream task performance through controlled text generation, while also doing it in a way that is parameterefficient and compatible with petl. the resulting parameter-efficient architecture outperforms previous approaches, many of them based on full fine-tuning, when evaluated on the webnlg @xcite , webnlg+ 2020 @xcite , @xcite and e2e clean @xcite data-totext datasets using the official evaluation scripts. we also show that although these modular prefixes are formed from shared reparameterizations and operate at every layer, they provide a level of interpretability, as similar control prefix representations are learned by the model for semantically similar attribute labels. this fact allows us to employ a zero-shot technique to deal with the more common case in controlled generation, where attribute-level information is absent at inference time. in addition, we show the superiority of the architecture to an alternative architecture of introducing identical guidance signal into prefix-tuning. in total, we evaluate control prefixes on five popular datasets from the gem benchmark @xcite for natural language generation and demonstrate the technique is easily extendable to new tasks. 1 ## data-to-text results we indicate the guidance signal(s) used by each control prefixes model with a 1 for the webnlg subdomain category and a 2 for the dart sub-dataset source. results in the sota for webnlg is a t5-large model fine-tuned on webnlg and the human annotated portion of @xcite . compared to this model, control prefixes achieves a 0.83 higher bleu overall, and 1.33 on the seen categories. notably, control prefixes (a 1 ) outperforms control prefixes (a 1 ,a 2 ) on the seen component of the dataset, but does not generalize as well to the unseen categories, indicating the benefit of using both controllable attributes. the prefix-tuning model with additional dart data, like the sota, is trained on only the human annotated portion and yields a minor performance increase of 0.05 bleu compared to prefix-tuning solely trained on webnlg. we believe this indicates that for fine-tuning, training on a complementary type of additional data allows the plm to maintain more nlu by not over-fitting a narrow distribution, leading to better lm generalization. in contrast, for prefix-tuning, much of this gain has already been realized by retaining the original frozen parameters. the sota @xcite for e2e clean consists of a fine-tuned gpt-2 with a semantic fidelity classifier trained on additional generated data. control prefixes (a 2 ), which can leverage the heterogeneous dart datasets, outperforms this model in terms of the bleu score. we also report results on the less popular webnlg+ 2020 @xcite ) dataset (gem), the second official webnlg competition, in appendix d. ## token-level control for comparison, we also investigated a simpler architecture: prefix-tuning combined with control tokens @xcite . in this setting, the model receives the same guidance signals as con-trol prefixes, but instead uses trainable control tokens for representing the attribute values. the main model is kept frozen, while the general prefix is optimized along with embeddings for the control tokens, allowing us to benchmark against a different parameter-efficient architecture. note, we chose to compare against a prefix-tuning based architecture as the fully fine-tuned models lag behind prefix-tuning in the results for this experiment are included in appendix g. we found that control prefixes consistently outperformed control tokens on all three data-to-text datasets. this indicates that con-trol prefixes is a superior parameter-efficient framework for leveraging additional information, whilst maintaining the fixed-lm property. control tokens lack the shared re-parameterization of static and dynamic parameters. they are only able to inject information at the embedding level, making them less expressive than the control prefixes method. control prefixes fundamentally depends on the strength of the guidance signal. this is reflected in the constraint of attribute information being available with the dataset. however, we show that control prefixes is a powerful general method which can utilize this signal to achieve a consistent improvement across an array of tasks. ## applicability to other tasks finally, we investigate the application of control prefixes to generation tasks beyond the data-totext setting. for these experiments, we integrate the method with a sequence-to-sequence model trained for text simplification on the wikilarge (zhang and lapata, 2017) dataset. following @xcite , the model uses four simplification-specific attributes as control prefixes: the length compression ratio, replace-only levenshtein similarity, aggregated word frequency ratio and dependency tree depth ratio. 12 in ## related work controlled generation aims to incorporate various types of guidance beyond the input text into the generation model @xcite . @xcite trained a translation model with control tokens to encode each language, and @xcite pre-trained a 1.63b parameter model, alongside conditional control tokens demonstrating these learnt to govern stylistic aspects. in addition to having the benefit of updating all model parameters, these methods only act at the embedding level. alternatives exist, such as using a plug-and-play mechanism to perturb the lm hidden states towards a target attribute @xcite . strategies such as these are computationally intensive, resulting in a slow generation speed and the shift in conditional probability has been shown to increase text degeneration @xcite . gsum @xcite is an example of work that has explored using learned guidance prediction models at test time. however, both the prompt and lm parameters are tuned. there has been little work using controlled generation in the data-to-text domain. @xcite were able control both the intra-text sentence and inter-sentence structure of generated output. this architecture exhibits inferior performance to our method on the mutual evaluation dataset webnlg. additionally, control prefixes uses fewer additional parameters and can incorporate multiattribute control with prefixes of varying sizes. several successive works @xcite @xcite employ prompt tuning, where unlike the discrete text prompts in icl, trainable soft embeddings are prepended to the input. again the technique acts only at the embedding level, thus limiting any control that can be exerted from data-point guidance. this shortcoming also exists with icl and multitask prompting @xcite . prefix-tuning is more expressive and, along with @xcite , serves as inspiration for this work. however, prefix-tuning trains each prefix separately and no relationship between prefixes is modelled. the parameters are static with no mechanism to incorporate guidance. there have been few works exploring inputdependent parameters trained alongside static prompt parameters @xcite . perhaps most similar to our work is @xcite , who use an attribute alignment function to encode to-kens of attributes. unlike our work, there are no dedicated task parameters and the method aims to generate text with specific target attributes, independent of task performance. with control prefixes, the intention is to also maximize taskspecific performance, which is why we maintain a large static component to specify the task itself, which is directly learnt simultaneously with the dynamic parameters in a shared framework. ## conclusion we have proposed control prefixes, a general framework for integrating attribute-level information into pre-trained language models. in addition to the general prefix for the overall task, special prefixes are optimized for each attribute value and incorporated into different levels of the transformer. this allows for finer-grained control over generated text, either by providing additional context about each input example or by allowing the user to specify some aspect of the desired output. the main language model parameters are kept frozen while only the multiple prefixes are optimized for a particular task, providing a very parameter-efficient method. our experiments show that control prefixes outperforms all existing methods for several datato-text tasks including webnlg and dart. this is in spite of learning less than 2% of the base lm's parameters and using signal from attribute level information that is available for the tasks. control prefixes also achieves higher results when compared to an alternative prefix-tuning architecture that makes use of the same attribute-level information, showing that the proposed framework is better able to integrate the additional signals with the rest of the model. we also saw that the method can still be applied when suitable prefixes do not exist for a particular attribute value, by constructing the required prefix based on semantic similarity. experiments on text simplification also verified that control pre-fixes can be applied on other tasks and datasets beyond the data-to-text setting. in future work, additional guiding attributes can be investigated for text generation, such as the desired formality and sentiment. in addition, this method can be integrated with a wider range of model architectures, beyond text generation applications, that require parameter-efficient methods of control.