Tailor: A Soft-Prompt-Based Approach to Attribute-Based Controlled Text Generation (2023)

Abstract: Attribute-based Controlled Text Generation (CTG) refers to generating sentences that satisfy desirable attributes (e.g., emotions and topics). Existing work usually utilize fine-tuning or resort to extra attribute classifiers, yet suffer from increases in storage and inference time. To address these concerns, we explore attribute-based CTG in a parameter-efficient manner. In short, the proposed Tailor represents each attribute as a pre-trained continuous vector i.e., single-attribute prompt), which guides the generation of a fixed pre-trained language model (PLM) to satisfy a pre-specified attribute. These prompts can be simply concatenated as a whole for multi-attribute CTG without any re-training. Nevertheless, this may raise problems of fluency downgrading and position sensitivity. To solve this, Tailor provides two solutions to enhance the combination. The former contains a multi-attribute prompt mask and a re-indexing position sequence to bridge the gap between the training (one single-attribute prompt for each task) and the testing stage (concatenating two prompts). The latter introduces a trainable prompt connector to further enhance the combinations. Experiments demonstrate that, only requiring 0.08% extra training parameters of the GPT-2, Tailor can achieve effective and general improvements on eleven attribute-specific generation tasks.

## introduction attribute-based ctg @xcite focuses on generating sentences satisfying pre-specified attributes such as topic and sentiment, which remains extremely challenging in recent progress @xcite . specifically, single-attribute ctg typically resorts to attribute-specific data, guiding the ctg model learning with supervised objectives @xcite @xcite . nevertheless, multi-attribute ctg is generally zero-shot since no example of a sentence with specified attribute combination is accessible during training @xcite . for both single and multi-attribute ctg, existing efforts can be roughly divided into two types: 1) fine-tuning a pre-trained language model (plm) on the attribute-specific data @xcite and 2) utilizing extra attribute classifiers. the former usually introduces control codes to generate various styles of sentences with one plm, such as keywords @xcite and numerical sequence @xcite . the latter applies extra attribute classifiers to guide a plm, such as backpropagating gradients of these classifiers @xcite or weighting output logits @xcite . however, this two types suffer from expensively re-training whole plm @xcite ) and higher latency during inference @xcite , respectively. to overcome the aforementioned limitations, we propose tailor -text-attribute general controller, a soft-prompt-based approach to jointly include both single-attribute ctg and multi-attribute ctg in a unified manner. 1 the key idea is to represent each attribute as a trainable continuous vector (i.e., the single-attribute prompt). these single-attribute prompts could be separately used or concatenated as a whole to control a fixed gpt-2 @xcite for single and multi-attribute ctg, respectively. 2 as simply concatenating always suffers from poor performances (see appendix f), tailor provides two effectively concatenating strategies without or with training after single-attribute ctg, namely non-training and training methods. first of all, we argue that the undesirable results of simply concatenating is due to the gap between the training and the testing stage. specifically, the single-attribute prompt only attends to itself while being individually trained by the attribute-specific data. while testing, the second prompt also attends to the first one in the concatenation, with the simultaneous change of the position embeddings. to fill this gap, the non-training method introduces a multi-attribute prompt mask (map mask) and a re-indexing position sequence (rp sequence) for the fixed gpt-2. map mask prevents distinct single-attribute prompts from cross-attention, and rp sequence ensures stable position information for the plm after swapping, by individually numbering each prompt. such a non-training method could be easily implemented and gets promising performances, but still has much space for improvement -there is no multi-attribute specific training stage for these prompts to adapt to work together. therefore, the training method contains a trainable prompt to connect two single-attribute prompts as a whole to multi-attribute ctg. inspired by the role of 'and' in connecting parallel phrases for natural sentences @xcite , as shown in figure 1 , the proposed multi-attribute prompt connector (map connector) can be concatenated with any two singe-attribute prompts and hints a gpt-2 to multi-attribute ctg. meanwhile, a pseudo-prompt based strategy is also provided for training the connector in unsupervised settings. with map connector, the combinations show strong performances on multi-attribute ctg on the popular benchmark yelp dataset @xcite . furthermore, map connector can get encouraging improvements for the unseen combinations in the training stage (see appendix f). the main contributions are: attribute-based ctg focuses on generating sentences containing pre-specified attributes, such as sentiment and topic. as a vital demand for intelligent writing @xcite , existing efforts include fine-tuning plms and utilizing extra attribute classifiers. the first type usually fine-tunes separately and stores a full copy of plm for each desirable attribute @xcite . to alleviate the storage problem, ctrl @xcite provides 55 kinds of control codes (i.e., special keywords) to fine-tune one plm for generating sentences of various styles. styleptb @xcite also proposes several style transfer tokens (i.e., a sequence of numbers) to guide a gpt-2 @xcite to multiple styles transfer. gsum @xcite introduces four guidance signals (e.g., keywords and relations) to enhance the controllability of plms in text summarization. although they make successful attempts in attribute-based ctg, re-training whole plms could be expensive @xcite . to improve the flexibility and extensibility of the ctg model, the second type makes efforts in the inference stage. in short, utilizing extra attribute classifiers to guide plms in each generating step. pplm @xcite iteratively modifies latent representations of a gpt-2 referring to the gradient of attribute classifiers, yet notably increasing the inference time. to solve this problem, fudge @xcite uses an attribute predictor to adjust the output probabilities of a plm. similarly, gedi @xcite uses smaller plms as generative discriminators to hint a larger plm generating sentences that satisfy desirable attributes. despite their progress, the fluency of generating sentences tends to decrease compared with the original plm (see § 4.2) and extra inference time costs still existed. in comparison, utilizing tailor, plms can benefit from the manner of controllability on single-attribute prompt combinations, with a negligible decrease on text quality. prompt learning is a new paradigm in nlp summarised as "pre-train, prompt and predict" @xcite . in short, it guides a single plm to solve various downstream tasks by reformulating these tasks into a text-to-text manner. recently, the continuous prompt has attracted attention @xcite @xcite , which usually forms as a set of continuous task-specific vectors to the input. despite their encouraging progress, the prompt composition is rarely explored but undoubtedly important in prompt learning. in that case, a composable task could be accomplished by composing various subtasks with multiple sub-prompts @xcite . to achieve it, ptr @xcite introduces manual sub-prompts for entity recognition and relation classification, respectively. then, these two kinds of prompts are composed by logic rules as a complete prompt for the relation extraction task. unfortunately, the composition of continuous prompts is rarely explored yet has demonstrated great potential @xcite . the main difference between contrastive prefix @xcite and tailor is that the former needs attribute data to be occurred contrastively (e.g, positive and negative attribute data must be available at the same time), which might be limited for the single attribute. for multi-attribute, contrastive prefix trains a new prompt (twice the size of their single prompt) for each combination. instead of it, tailor only trains an extra prompt connector to enhance the combinations of single prompts. it can act as an efficient plug-and-play manner with extremely low training parameters to attribute-based ctg. ## conclusions in this paper, we explore attribute-based ctg in a soft-prompt-based manner-tailor, which represents each attribute as a continuous prompt and effectively combines them as a multi-attribute prompt. for enhancing these combinations, tailor provides two solutions, namely non-training (map mask + rp sequence) and training methods (map connector). as our first attempt to multiattribute ctg, combining more than two attributes still needs to be discussed. in the future, we will investigate extending tailor to connect wider ranges of attributes, and expand it to other text-to-text generation tasks.