An Extensible Plug-and-Play Method for Multi-Aspect Controllable Text Generation (2023)

Abstract: Recently, multi-aspect controllable text generation that controls the generated text in multiple aspects (e.g., sentiment, topic, and keywords) has attracted increasing attention. Although methods based on parameter efficient tuning like prefix-tuning could achieve multi-aspect controlling in a plug-and-play way, the mutual interference of multiple prefixes leads to significant degeneration of constraints and limits their extensibility to training-time unseen aspect combinations. In this work, we provide a theoretical lower bound for the interference and empirically found that the interference grows with the number of layers where prefixes are inserted. Based on these analyses, we propose using trainable gates to normalize the intervention of prefixes to restrain the growing interference. As a result, controlling training-time unseen combinations of aspects can be realized by simply concatenating corresponding plugins such that new constraints can be extended at a lower cost. In addition, we propose a unified way to process both categorical and free-form constraints. Experiments on text generation and machine translation demonstrate the superiority of our approach over baselines on constraint accuracy, text quality, and extensibility.

## introduction multi-aspect controllable text generation (mctg), which aims at generating fluent text while satisfying multiple aspects of constraints simultaneously, has attracted increasing attention in recent years @xcite @xcite . to effectively control diverse aspects such as sentiment, topic, and detoxification, extensive efforts have been devoted to the task, including methods based on conditional generative model (keskar † indicates equal contribution. * corresponding authors: m.sun (sms@tsinghua.edu .cn) and y.liu (liuyang2011@tsinghua.edu.cn) 1 the source code is available at https://github.com/ thunlp-mt/promptgating4mctg et al., 2019), decoding-time regulation @xcite , and parameter efficient tuning @xcite . despite their effectiveness, existing methods still suffer from low extensibility. ideally, suppose a multi-aspect controllable text generation system has learned how to control sentiment, topic and keywords separately, it should be extensible to any combinations of the three aspects, e.g., generating a sports-themed sentence with negative sentiment containing keywords "new york" (see figure 1 ). moreover, an extensible system should also be easily extended to control new aspects in a plug-andplay way. however, it is non-trivial for existing methods to achieve this goal. specifically, the dedicated conditional generative models @xcite mostly need to be trained from scratch or finetuned when facing new aspect combinations. the decoding-time regulation based methods @xcite intervene in the probabilities of sentences by light-weight attribute classifiers or language models during inference, which significantly impairs text fluency when multiple distributions are interpolated. the parameter efficient tuning based methods @xcite control aspects by inserting trainable prompts or prefixes into the model, referred to as plugins. by leveraging one plugin for each aspect, these methods can naturally work in a plug-and-play way, showing better potential to achieve high extensibility. however, existing studies show that directly combining multiple plugins results in significantly lower controllability of the corresponding aspects than before combining (i.e., attribute degeneration) @xcite . @xcite argue that mutual interference of the plugins is the major reason for attribute degeneration, which is further justified by our theoretical and empirical analyses. previous works alleviate the problem by introducing connectors to connect mulfirst, plugins are trained on single-aspect labeled data separately (left). then, arbitrary plugins can be combined by simply concatenation and plugged into the pretrained model to satisfy corresponding combinations of constraints (right). due to the separate training of different plugins, the cost of extending a new constraint is relatively low. besides, our approach restrains the accumulation of mutual interference, alleviating the degeneration of constraints. in this paper, we propose an extensible plug-andplay method, prompt gating, for multi-aspect controllable text generation. we derive a theoretical lower bound for the interference of plugins and reveal that it accumulates with the increasing number of layers where prefixes are inserted. based on these findings, we propose attaching trainable gates to the plugins, which normalize the interventions of plugins. as a result, the mutual interference has been significantly reduced such that the control of arbitrary combinations of aspects can be realized by simply concatenating the corresponding plugins. thus, our method is both extensible and plug-andplay. moreover, we represent the constraints of the aspects in textual form, which makes our method applicable not only to categorical aspects (e.g., sentiment) but also to free-form aspects (e.g., lexical constraint). our contributions are three-fold: ## background in this section, we illustrate the widely-used prefixtuning-based method @xcite @xcite for multi-aspect controllable text generation. generally, prefix-tuning @xcite prepends light-weight continuous vectors to the multi-head attention sublayer of each transformer layer @xcite : during inference, for a combination of multiple aspects, corresponding prefixes are selected and synthesized by either concatenating @xcite or finding their intersection @xcite , and then the generation is conditioned on the synthesis. without loss of generality, we take two aspects as an example. the conditioned probability can be represented as although existing methods alleviate the mutual interference of prefixes by joint training @xcite @xcite , they are based on pre-defined closed-set constraints, which increases the overhead of extending a new constraint and thus limits extensibility. thus, to maintain high extensibility, reducing mutual interference without joint training still remains a challenge. ## analyses on mutual interference to alleviate mutual interference while maintaining extensibility, we conduct theoretical and empirical analyses. first, we provide a definition of mutual interference as follows. definition. mutual interference (mi) is the interference between multiple plugins which are trained separately during training but are combined to guide the pretrained model simultaneously during inference (i.e., in the zero-shot setting). however, the exact interference is hard to analyze because of the complexity of deep neural networks. intuitively, suppose multiple plugins are optimized simultaneously during training, which requires multi-aspect labeled data, their interference will be minimized because they have learned to work cooperatively under supervision (i.e., in the supervised setting). therefore, we use the differences between the hidden states of the supervised and zero-shot settings 2 note that x is the source sentence or context for tasks like machine translation @xcite or summarization @xcite and can be eliminated when it is not present. empirical analysis. then, as mutual interference has been defined as the norm of gap between hidden states in the zero-shot and supervised settings, we can empirically estimate it on the authentic dataset. by calculating the averaged norm on the yelp dataset (lample et al., 2019), we plot the variations of mutual interference with the number of transformer layers for prompt-tuning @xcite and prefix-tuning @xcite in figure 2 . we can find that the interference accumulates with insertions of trainable parameters. moreover, the magnitude of mutual interference at the last transformer layer (i.e., the 12-th layer in figure 2 ) is consistent with the performance gap, which is the difference between the fulfillment of constraints in single-and multi-aspect settings (see ## prompt gating we propose a novel approach that attaches trainable gates to the plugins, which alleviates the mutual interference of multiple plugins and makes the model highly extensible. figure 3 shows the architecture of our approach. we first provide intuition in §4.1, then define our approach formally in §4.2. ## experiments we conducted experiments on two representative tasks in natural language generation, which are text generation and machine translation. ## related work multi-aspect controllable text generation (mctg) @xcite @xcite that simultaneously satisfies multiple constraints is a challenging task for which highly extensible methods make more practical sense. approaches to it can be roughly divided into the following three categories. ## conclusion in summary, we propose an extensible plug-andplay method for multi-aspect controllable text generation. by replacing trainable prefixes with trainable prompts and gates, our approach alleviates the mutual interference of multiple plugins while providing sufficient interventions. experiments on text generation and machine translation show its superiorities over baselines on the cost of extending to new combinations of aspects, the fulfillment of constraints, and text fluency.