Tailor: A Soft-Prompt-Based Approach to Attribute-Based Controlled Text Generation
Year: 2023

Abstract:
Attribute-based Controlled Text Generation (CTG) refers to generating sentences that satisfy desirable attributes (e.g., emotions and topics). Existing work usually utilize fine-tuning or resort to extra attribute classifiers, yet suffer from increases in storage and inference time. To address these concerns, we explore attribute-based CTG in a parameter-efficient manner. In short, the proposed Tailor represents each attribute as a pre-trained continuous vector (i.e., single-attribute prompt), which guides the generation of a fixed pre-trained language model (PLM) to satisfy a pre-specified attribute. These prompts can be simply concatenated as a whole for multi-attribute CTG without any re-training. Nevertheless, this may raise problems of fluency downgrading and position sensitivity. To solve this, Tailor provides two solutions to enhance the combination. The former contains a multi-attribute prompt mask and a re-indexing position sequence to bridge the gap between the training (one singleattribute prompt for each task) and the testing stage (concatenating two prompts). The latter introduces a trainable prompt connector to further enhance the combinations. Experiments demonstrate that, only requiring 0.08% extra training parameters of the GPT-2, Tailor can achieve effective and general improvements on eleven attribute-specific generation tasks.

heading: 1 Introduction
text: Attribute-based CTG (Zhang et al., 2022) focuses on generating sentences satisfying pre-specified attributes such as topic and sentiment, which remains extremely challenging in recent progress (Dathathri et al., 2020). Specifically, single-attribute CTG typically resorts to attribute-specific data, guiding the CTG model learning with supervised objectives (Keskar et al., 2019; Lyu et al., 2021; Ziegler et al., 2019). Nevertheless, multi-attribute CTG is
∗ Work is done during internship at DAMO Academy † Corresponding author.
generally zero-shot since no example of a sentence with specified attribute combination is accessible during training (Lample et al., 2019).
For both single and multi-attribute CTG, existing efforts can be roughly divided into two types: 1) fine-tuning a pre-trained language model (PLM) on the attribute-specific data (Ziegler et al., 2019) and 2) utilizing extra attribute classifiers. The former usually introduces control codes to generate various styles of sentences with one PLM, such as keywords (Keskar et al., 2019) and numerical sequence (Lyu et al., 2021). The latter applies extra attribute classifiers to guide a PLM, such as backpropagating gradients of these classifiers (Dathathri et al., 2020) or weighting output logits (Krause et al., 2021; Yang and Klein, 2021). However, this two types suffer from expensively re-training whole PLM (Yang and Klein, 2021) and higher latency during inference (Qian et al., 2022), respectively.
To overcome the aforementioned limitations, we propose Tailor – Text-attribute general controller, a soft-prompt-based approach to jointly include both single-attribute CTG and multi-attribute CTG in a unified manner.1 The key idea is to represent each attribute as a trainable continuous vector (i.e., the single-attribute prompt). These single-attribute prompts could be separately used or concatenated as a whole to control a fixed GPT-2 (Radford et al., 2019) for single and multi-attribute CTG, respectively.2 As simply concatenating always suffers from poor performances (see Appendix F), Tailor provides two effectively concatenating strategies without or with training after single-attribute CTG, namely non-training and training methods. First of all, we argue that the undesirable results of simply concatenating is due to the gap between the training and the testing stage. Specifically, the
1Our code and corpus will be released at https://github. com/yangkexin/Tailor.
2Following Lample et al. (2019); Qian et al. (2022), we focus on the multi-attribute task that contains two attributes.
410
single-attribute prompt only attends to itself while being individually trained by the attribute-specific data. While testing, the second prompt also attends to the first one in the concatenation, with the simultaneous change of the position embeddings. To fill this gap, the non-training method introduces a Multi-Attribute Prompt mask (MAP mask) and a Re-indexing Position sequence (RP sequence) for the fixed GPT-2. MAP mask prevents distinct single-attribute prompts from cross-attention, and RP sequence ensures stable position information for the PLM after swapping, by individually numbering each prompt.
Such a non-training method could be easily implemented and gets promising performances, but still has much space for improvement – there is no multi-attribute specific training stage for these prompts to adapt to work together. Therefore, the training method contains a trainable prompt to connect two single-attribute prompts as a whole to multi-attribute CTG. Inspired by the role of ‘and’ in connecting parallel phrases for natural sentences (Rudolph, 1989), as shown in Figure 1, the proposed Multi-Attribute Prompt connector (MAP connector) can be concatenated with any two singe-attribute prompts and hints a GPT-2 to multi-attribute CTG. Meanwhile, a pseudo-prompt based strategy is also provided for training the connector in unsupervised settings. With MAP connector, the combinations show strong performances on multi-attribute CTG on the popular benchmark YELP dataset (Lample et al., 2019). Furthermore, MAP connector can get encouraging improvements for the unseen combinations in the training stage (see Appendix F). The main contributions are:
• We propose Tailor, a soft-prompt-based approach to attribute-based CTG. To jointly in-
clude both single-attribute and multi-attribute CTG in a unified paradigm, Tailor employs a set of pre-trained prefixes to guide a fixed PLM to generate sentences with pre-specified attributes, and effectively concatenate them to generate multi-attribute sentences.
• We experimentally reveal the combining ability of continuous prompts. To enhance this combination, we explore two effective strategies without training (MAP mask + RP sequence) or with training (MAP connector) after single-attribute CTG. Especially, the MAP connector achieves strong performances on six multi-attribute generation tasks, and even works on the unseen ones.

heading: 2 Related Work
text: Attribute-Based CTG focuses on generating sentences containing pre-specified attributes, such as sentiment and topic. As a vital demand for intelligent writing (Zhang et al., 2022), existing efforts include fine-tuning PLMs and utilizing extra attribute classifiers. The first type usually fine-tunes separately and stores a full copy of PLM for each desirable attribute (Ziegler et al., 2019). To alleviate the storage problem, CTRL (Keskar et al., 2019) provides 55 kinds of control codes (i.e., special keywords) to fine-tune one PLM for generating sentences of various styles. StylePTB (Lyu et al., 2021) also proposes several style transfer tokens (i.e., a sequence of numbers) to guide a GPT-2 (Radford et al., 2019) to multiple styles transfer. GSum (Dou et al., 2021) introduces four guidance signals (e.g., keywords and relations) to enhance the controllability of PLMs in text summarization. Although they make successful attempts in attribute-based CTG, re-training whole PLMs could be expensive (Yang and Klein, 2021). To improve the flexibility and extensibility of the CTG model, the second type makes efforts in the inference stage. In short, utilizing extra attribute classifiers to guide PLMs in each generating step. PPLM (Dathathri et al., 2020) iteratively modifies latent representations of a GPT-2 referring to the gradient of attribute classifiers, yet notably increasing the inference time. To solve this problem, Fudge (Yang and Klein, 2021) uses an attribute predictor to adjust the output probabilities of a PLM. Similarly, GeDi (Krause et al., 2021) uses smaller PLMs as generative discriminators to hint a larger PLM generating sentences that satisfy desirable
attributes. Despite their progress, the fluency of generating sentences tends to decrease compared with the original PLM (see § 4.2) and extra inference time costs still existed. In comparison, utilizing Tailor, PLMs can benefit from the manner of controllability on single-attribute prompt combinations, with a negligible decrease on text quality. Prompt Learning is a new paradigm in NLP summarised as “Pre-train, Prompt and Predict” (Liu et al., 2021a). In short, it guides a single PLM to solve various downstream tasks by reformulating these tasks into a text-to-text manner. Recently, the continuous prompt has attracted attention (Gu et al., 2021; Liu et al., 2021b, 2022), which usually forms as a set of continuous task-specific vectors to the input. Despite their encouraging progress, the prompt composition is rarely explored but undoubtedly important in prompt learning. In that case, a composable task could be accomplished by composing various subtasks with multiple sub-prompts (Liu et al., 2021a). To achieve it, PTR (Han et al., 2021) introduces manual sub-prompts for entity recognition and relation classification, respectively. Then, these two kinds of prompts are composed by logic rules as a complete prompt for the relation extraction task. Unfortunately, the composition of continuous prompts is rarely explored yet has demonstrated great potential (Qian et al., 2022). The main difference between contrastive prefix Qian et al. (2022) and Tailor is that the former needs attribute data to be occurred contrastively (e.g, positive and negative attribute data must be available at the same time), which might be limited for the single attribute. For multi-attribute, contrastive prefix trains a new prompt (twice the size of their single prompt) for each combination. Instead of it, Tailor only trains an extra prompt connector to enhance the combinations of single prompts. It can act as an efficient plug-and-play manner with extremely low training parameters to attribute-based CTG.

heading: 3 Methodology

heading: 3.1 Tailor for Single-Attribute CTG
text: Different from fine-tuning a full copy of PLMs for each attribute, our basic idea is to guide the generation of a PLM with a set of pre-trained continuous vectors, namely single-attribute prompts. Meanwhile, each prompt represents a desirable attribute. As shown in Figure 2 (top), we fix the parameters of a GPT-2 and train each prompt on the attributespecific data. After training, these prompts can
act as plug-ins for desirable single-attribute CTG. For the prefix “Once upon a time”, the GPT-2 can continue with “I had to order my tacos ...” with a prompt representing the Mexican food topic or “ the food was good” with a prompt representing the positive sentiment. In this way, our method can be easily expanded: if a new attribute emerges, we only need to train an attribute prompt and then control a PLM to generate attribute-specific sentences. To be exact, we use language modeling learning object to train such a set of single-attribute prompts. In detail, k-th single-attribute prompt Sk with length lk is first initialized randomly, where Sk ∈ Rlk×demb . demb is the word embedding dimension of the GPT-2. Meanwhile, given an attribute-specific sentence x = {x1, x2, ..., xn} with length n, we get a word sequence matrix Xemb ∈ Rn×demb after being embedded by GPT2. Then, Sk is concatenated with Xemb to form a input matrix as [Sk;Xemb] ∈ R(lk+n)×demb , and this matrix is fed into a fixed GPT-2. Finally, the language-modeling based learning object is:
Lsingle = n∑
t=1
logPθg ;θSk (xt|Sk, x<t) , (1)
where θg and θSk denote the parameters of GPT-2 and the single-attribute prompt, respectively. Only θSk are updated during the training stage.

heading: 3.2 Tailor for Multi-Attribute CTG
text: Inspired by the composition of discrete prompts (Han et al., 2021) to accomplish a complex task, our intuitive idea is to combine single-attribute prompts as a multi-attribute prompt to hint a PLM for multi-attribute CTG. To enjoy the benefit of our paradigm in single-attribute CTG, we first consider simply concatenating several single-attribute prompts as a whole multi-attribute prompt. Surprisingly, such a multi-attribute prompt can guide a GPT-2 to generate sentences containing multi attributes and get encouraging performances in unsupervised settings without any training (see § 4.2). Despite the progress, this straightforward method suffers from fluency decrease compared with single-attribute CTG. Meanwhile, it is position sensitive, i.e., the PLM tends to focus more on the single-attribute prompt that is closer to the input prefix (see Appendix F).
To polish such a paradigm while keeping plugand-play and storage-friendly advantages, as shown
in Figure 2 (bottom), Tailor introduces a nontraining method to quickly and effectively alleviate the above problems of simply concatenation. Afterward, a training method is further provided to greatly enhance the combinations. We elaborate the two methods separately as follows.

heading: 3.2.1 Non-Training Method
text: To make better use of single-attribute prompts, reducing disparities between the training (a singleattribute prompt for each task) and the testing stage (concatenating more than one single-attribute prompt) is undoubtedly important. Specifically, the single-attribute prompt only attends to itself in the attention matrix while training, as each prompt is individually trained by the attribute-specific data. However, while in the testing stage for multiattribute CTG, the second prompt also focuses on the first one in the concatenation, with the simultaneous change of the position embedding. To fill this gap, MAP mask and RP sequence are introduced to the fixed PLM while generating. MAP mask avoids cross-attention between representations of single-attribute prompts to approximate the condition in the single-attribute CTG training
stage. Meanwhile, the RP sequence keeps a stable prompt position for swapping, preventing such concatenating paradigm from position sensitivity. MAP Mask For the ease of implementation, we introduce MAP mask matrix Mp to the softmax logits of GPT-2. Given a vanilla attention module:
A = Softmax( QK⊤√
d ) ∈ Rn×n, (2)
where n is the length of input sentence x and Q,K denote representations of query and key, respectively.3 For MAP Mask, given two single-attribute prompts Su, Sv with length being lu, lv, respectively, the attention module is then modified as:
A = Softmax( QK⊤√
d +Mp) ∈ R(lp+n)×(lp+n),
M ijp = { −∞ i ∈ [lu, lv] and j ∈ [0, lu], 0 otherwise,
(3)
where lp = lu + lv. 3The multi-head mechanism is omitted for illustration.
RP Sequence Simple concatenation of singleattribute prompts always suffers from position sensitivity. To address this issue, we propose a simple but effective method to ensure position consistency while swapping. In short, we modify the position sequence of the PLM while concatenating.4 Given the original position sequence:
id = { 1, ..., lu,︸ ︷︷ ︸ Length of Su lu + 1, ..., lp,︸ ︷︷ ︸ Length of Sv lp + 1, ..., lp + n︸ ︷︷ ︸ Length of input prefix }, (4) the RP sequence can be defined as:
idRP = { 1, ..., lu,︸ ︷︷ ︸ Length of Su

heading: 1, ..., lv,︸ ︷︷ ︸
text: Length of Sv lv + 1, ..., lv + n︸ ︷︷ ︸ Length of input prefix },
(5) note that, lv = lu. In that case, swapping does not bring any changes, since the position of prompts is fixed by the RP sequence while avoiding crossattention by the MAP mask.

heading: 3.2.2 Training Method
text: While the non-training method partly addresses the issues of combination, there is no multi-attribute specific training stage for these prompts to adapt to work together. Therefore, we provide a training method – MAP connector, which is also a continuous prompt trained for combining two singleattribute prompts to multi-attribute CTG. To utilize only single-attribute sentences for multi-attribute CTG, we propose a pseudo-attribute prompt based training strategy for MAP connector. The details of the pseudo-attribute prompt building method and the workflow of the MAP connector are as follows.
Building Pseudo Single-Attribute Prompt Our key idea is to build another pseudo-attribute prompt for each single-attribute sentence, thus MAP connector could be trained in a multi-attribute circumstance. An overview of our building method is
4In this case, position sequence denotes position indexes of input tokens in the position embeddings for GPT-2.
demonstrated in Figure 3, where a sentence with the topic of Mexican food is used as a showcase.5 To be exact, we first train an attribute classifier on the same single-attribute CTG training set. Thus, such a classifier with nclass classes corresponds to the pre-trained single-attribute prompt set S = {S1, S2, ..., Snclass}. Given an attribute-specific sentence x of other attribute category, we first get the class probabilities set p = {p1, p2, ..., pnclass}. Then, the pseudo single-attribute prompt can be obtained by two methods:
Sa = SIndex(argmax(p)),
Sw =
nclass∑
z=1
pzSz , (6)
where argmax-pseudo prompt method obtains the pseudo prompt Sa by using a single-attribute prompt corresponding to the predicted sentiment, Index(·) means getting the corresponding index. In contrast, weighted-pseudo prompt method utilizes the predicted probability distribution to multiply corresponding single-attribute prompts, respectively. Then these weighted prompts form a whole prompt Sw by element-wise addition. The MAP Connector Workflow Figure 2 bottom illustrates the workflow of the MAP connector. In the training stage, we unify sentences containing different single attributes to train the MAP connector, each of which is added an extra pseudo singleattribute prompt (boxes with the slash pattern) by employing the aforementioned method. Specifically, for each training sample, we first concatenate two single-attribute prompts (real and pseudo), the MAP connector and the input sentence into a sequence, and then feed it into a fixed GPT-2. It is worth noting that only the parameters of the MAP connector are updated in the training stage. Therefore, given two single-attribute prompt Su and Sv, MAP connector C with the length lC , C ∈ RlC×demb , we concatenate Su, Sv, C and the input sentence matrix Xemb to form a input matrix as [Su;Sv;C;Xemb]. The learning object is:
Lmulti = n∑
t=1
logPθ (xt|Su, Sv, C, x<t) , (7)
where θ = [θg; θSu ; θSv ; θC ]. θg, θSu , θSv , and θC denote the parameters of GPT-2, two single-
5In the implementation for multi-attribute CTG, we use YELP data of two sentiments attributes (Positive / Negative) and three kinds of food type (Mexican / American / Asian)
attribute prompts and MAP connector, respectively. Only θC are updated during the training stage. In the inference stage, we just decompose each multiattribute generation task as several single-attribute generation tasks and find corresponding singleattribute prompts. Then, these prompts are concatenated with MAP connector to generate sentences that satisfy multi attributes.

heading: 4 Experiments

heading: 4.1 Experimental Setup
text: Datasets We conduct experiments on the widelyused benchmark dataset YELP (Lample et al., 2019). It contains multiple single-attribute data that can verify Tailor’s performance on both singleattribute and multi-attribute CTG, while ensuring that the combination of these attributes is reasonable. Following previous works that conduct experiments on attributes of emotions and topics for multi-attribute CTG, we choose Yelp restaurants reviews of sentiment attributes (positive (PO) and negative (NE)) and topics of food type (Mexican (ME), American (AM) and Asian (AS) foods) to evaluate models. Specifically, each attribute contains 30,000 / 3,000 sentences for training / validation. For evaluation, to keep in line with previous works (Yang and Klein, 2021; Dathathri et al., 2020), we use 15 attribute-unrelated prefixes6 and ask the model to continue writing with them (for each of the 15 prefixes, 100 completions are generated, total: 1500 for each attribute) while satisfying pre-specified attribute as the final results.7 Automatic Evaluation Following Yang and Klein (2021); Dathathri et al. (2020), we automatically evaluate generation results from three aspects: (1) Correctness. We used RoBERTaLarge (Liu et al., 2019) based attribute classifiers to compute the fraction of final sentences that contain a pre-specified attribute, details in Appendix C. (2) Text Quality. Grammar (GRAM) (Warstadt et al., 2019) indicates the averaged grammaticality probabilities of all final sentences, evaluated by a RoBERTa-based CoLA grammaticality model (Yang and Klein, 2021). Perplexity (PPL), we average the scores from GPT-2Base, GPT-2Medium and GPT-2Large version of GPT-2 (Radford et al., 2019) as the final result. (3) Diversity. Following Li et al. (2015), we report the distinctness of the final results. Specifically, we count the number of unigrams, bigrams
6https://github.com/uber-research/PPLM 7More details can be found in Appendix B
and trigrams and then normalize them by the total number of words (i.e., Dist-1 / Dist-2 / Dist-3). Human Evaluation Following Qian et al. (2022), we also conduct the human evaluation. For each model, three crowdsource evaluators are shown 15 randomly selected samples (one per each attributeunrelated prefixes) for each generation task (Total: 75 samples for single-attribute CTG and 90 samples for multi-attribute CTG), respectively. Then, they are asked to rate model results in two categories: the text quality of generation sentences and whether they contain the target attribute. Scores are ranged from 1 to 5, the higher the better.8 Tailor Settings TailorSingle denotes the singleattribute prompts. For multi-attribute, ConcatSimple means simply concatenating two single-attribute prompts and TailorConcat is our non-training method. TailorArgmax and TailorWeight represent using argmax-pseudo and weighted-pseudo prompts when training the MAP connector, respectively. Baselines We compare Tailor with mainstream competitive models as follows. (1) Finetune, finetuning the original GPT-2Base on attribute-specific data. As multi-attribute CTG is unsupervised, following Lyu et al. (2021), we sequentially apply the GPT-2 trained for corresponding singleattribute data multiple times. (2) Adapter, following Li and Liang (2021), we use the adapter for GPT-2 as same as Lin et al. (2020). Note that for multi-attribute CTG, we first use the same training method as mentioned in Finetune for Adapter. Besides, we use the same argmax-pseudo labeled sentences (see § 3.2.2) to train the Adapter (marked with ‘Pseudo’). (3) GeDi (Krause et al., 2021), using small PLMs to hint large ones. (4) PPLM (Dathathri et al., 2020), back-propagating gradients of extra attribute classifiers to a PLM9.

heading: 4.2 Main Results
text: Single-Attribute CTG As shown in Table 1, TailorSingle outperforms PPLM and GeDi to a great extent on both correctness and text quality. Meanwhile, compared with other parameter-efficient learning model Adapter, TailorSingle also gets improvements on both correctness (e.g, + 9.19% of Food) and diversity (e.g, + 0.02% / + 0.12% / + 0.25% of Food) with a similar scale of training parameters. However, with 0.08% training parameters of the GPT-2, TailorSingle still has a
8Details can be found in Appendix D. 9The implementation details of baselines and Tailor can be
found in Appendix A
performance gap with Finetune, e.g., - 4.14% correctness on Food. Fortunately, as the length of TailorSingle increases (see Appendix F), this gap appears to narrow (- 0.33%, TailorSingle with the prompt length of 256). Then, we illustrate human evaluations in Table 3. Different from automatic evaluations, TailorSingle obtains the same
score with Finetune, even outperforms all baselines on attribute-relevance score. This experimental discovery demonstrate the limitations of only resorting to automatic evaluation, as also be mentioned in Welbl et al. (2021); Qian et al. (2022). Multi-Attribute CTG As shown in Table 2, we compare three instantiations of Tailor and strong baselines in the single-attribute CTG experiment. First, TailorConcat shows encouraging performances without any training, especially on correctness, outperforms fine-tuning (+ 13.51% Sentiment / + 4.53% Food) and Adapter (+ 13.44% Sentiment / + 6.00% Food). Besides, our training methods TailorWeight and TailorArgmax show improvements on all scores compared with TailorConcat, e.g., + 4.58% / + 11.22% correctness on the topic of food type attribute. Meanwhile, Tailor also outperforms Adapter with the same pseudo label strategy on both correctness and diversity, with a notable
scale discrepancy of training parameters (0.08% vs 0.60%, i.e., 1:7.5). Meanwhile, Tailor seems to suffer from lower text diversity compared to PPLM and GeDi. This is because these methods have poor fluency (with many unreasonable words), while Dist-1/2/3 measure different words without considering whether they are reasonable. We supplement it by human evaluation. As shown in Table 3, the diversity of words considered by the index attribute, which shows the superiority of our method.

heading: 4.3 Further Discussions
text: Few-Shot Learning We conduct a few-shot learning setting to further analyze the effectiveness of Tailor. In detail, following Li and Liang (2021), we randomly sample from full dataset and obtain the few-shot dataset (training / validation / testing: 150 / 20 / 20 samples). Specifically, we sample three different few-shot datasets and average the scores of each method on three datasets as the final results. As shown in Table 4, three types of Tailor outperform other baselines on correctness, with 0.00% / 0.08% extra training parameters of Finetune. Cross Domain Dataset Evaluating We further evaluate the performances of Tailor on combining attribute from different domain.10 Specifically, we choose SST-2 (Socher et al., 2013) and AG News (Zhang et al., 2015) for data sources of sentiment and topic attribute, respectively. As shown in
10The details can be found in Appendix E
Appendix Table 10, Tailor still outperforms baselines in both correctness and diversity. Meanwhile, the text quality of Tailor has been improved by the Map Connector training (GRAM 0.59 to 0.68). Inference Speed We also compare Tailor with extra classifier based CTG method on inference speed. As shown in Table 5, TailorSingle outperforms baselines to a great extend on inference speed, which indicates computational efficacy of Tailor.
Ablations of TailorConcat Whether TailorConcat enjoys the benefits from the MAP mask and the RP sequence is of concern. As shown in Table 6, both the MAP mask and the RP sequence are important to TailorConcat. More importantly, using these two strategies simultaneously can improve the performance while avoiding the position sensitivity.

heading: 5 Conclusions
text: In this paper, we explore attribute-based CTG in a soft-prompt-based manner—Tailor, which represents each attribute as a continuous prompt and effectively combines them as a multi-attribute prompt. For enhancing these combinations, Tailor provides two solutions, namely non-training (MAP mask + RP sequence) and training methods (MAP connector). As our first attempt to multiattribute CTG, combining more than two attributes still needs to be discussed. In the future, we will investigate extending Tailor to connect wider ranges
of attributes, and expand it to other text-to-text generation tasks.
Limitations
As we tentatively give a successful implementation of leveraging soft-prompt-based manner to benefit both single and multi-attribute CTG, such a paradigm deserves a closer and more detailed exploration. First, we explore multi-attribute CTG in the scenario of two-attribute composition, yet combining more attributes when generating a completion is more challenging and thrilling, and still in its fledgeless stage. Besides, while extensive experiments demonstrate that Tailor consistently improves attribute-based CTG, applying our approach on a wider variety of PLMs will evaluate the effectiveness of Tailor in a more generally way.
Ethics Statement
All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards. This article does not contain any studies with animals performed by any of the authors. Informed consent was obtained from all individual participants included in the study.

heading: B Yelp Dataset
text: In this section, we elaborate the workflow of filtering, pre-processing and sub-sampling to get the attribute-specific dataset for training all models and the classifiers For correctness evaluation. First of all, we get the YELP dataset from Lample et al. (2019). In detail, each sample of the YELP dataset contains a review and the corresponding attributes.15 Then, we select the restaurant reviews sub-set as our original dataset. For dataset filtering, we use the dataset setup scripts offered by Lample et al. (2019), which contains a fastText(Joulin et al., 2017) classifier to filter sentences that are not written in English. After that, we filter the sentences with rated 3 stars, since they could be neutral in sentiment (Shen et al., 2017). Finally, we get the pre-processed dataset as illustrated in Table 8. For the classifiers that are used in correctness evaluation, we use the full dataset and details in Appendix C. Aside from it, for training Tailor and baselines, we randomly sample 30,000 / 3,000 sentences as training/validation data set for each attribute.

heading: C Classifiers For Correctness Evaluation
text: We use the RoBERTaLarge based model to train two classifiers for both sentiment and topic of food type attributes. To obtain a balanced dataset, we randomly over-sampling the raw dataset. Finally, we
14https://github.com/uber-research/PPLM/blob/ master/paper_code/pplm.py
15The format can be found via https://github.com/ shrimai/Style-Transfer-Through-Back-Translation
get 1500k / 15k / 15k topic-specific sentences and 1380k / 1k / 1k sentiment-specific sentences for training/validation/testing, respectively. For training two classifiers, the learning rate is set to 5e-5 and the warm-up step is set to 200. The performances on the testing set can be found in Table 7.

heading: D Human Evaluation Details
text: For human evaluation, we first set a guideline for evaluating, which includes the task background, key points, detailed descriptions, and examples of evaluation scores from 1 to 5. Then, we set an entry barrier for annotators. In detail, we organize a training program and a preliminary annotating examination (90 examples for each model) to select appropriate annotators with an approval rate higher than 95%. Score Definition We define two categories in the human evaluation as follows:
1. Quality means whether the sentence corresponding to the option is fluent.
2. Attribute means whether the sentence corresponding to the option aligns with the target single attribute or multi attributes.
The scores are ranged from 1 to 5, and the higher score is better. The details are specified in Table 9. Following Qian et al. (2022), to obtain separate scores for both text quality and attribute correlation, the annotators are required to not attend to attribute correlation when evaluating the text quality (and vice versa). Aside from it, when the annotators feel that the sentences generated by different models perform similarly in terms of text quality, they are asked to give higher quality scores for sentences with longer lengths, which have more scope and diversity for expression yet have been ignored by automatic text quality evaluation metrics. Inter-annotator agreement We use Fleiss’ kappa (Fleiss, 1971) to measure three annotator’s
reliability.16 The results are: 0.24 for score the quality (fair agreement), 0.55 for the attribute score (substantial agreement).

heading: E Experiments on Cross Domain Dataset
text: We also evaluate Tailor on the cross domain dataset SST-2 (Socher et al., 2013) and AGNews (Zhang et al., 2015). For the classifiers that are used in correctness evaluation, we also use the RoBERTaLarge based model to train two classifiers for both sentiment and topic of news attributes and reuse the parameters setting as in the experiment for YELP datasets. The F1 scores of two classifiers are 89.80 (sentiment) and 94.95 (news), respectively. For baselines and Tailor, we use the same experimenting setup as described in Appendix A. The experimental results of cross domain dataset are shown in Table 10.

heading: F Ablations
text: Length of Tailor As shown in Figure 4, we explore the length of both TailorSingle and TailorArgmax.
16https://www.nltk.org/_modules/nltk/metrics/ agreement.html
For singe-attribute prompt TailorSingle, the performances increase alongside the length. But for TailorArgmax, it obtains the best performances with a length of 128, and the performances have a slight drop when we continue to increase the length.
Position Sensitivity We investigate the position sensitivity problem when concatenating two singleattribute prompts. As shown in Table 11, for simply concatenation, the GPT-2 tends to focus more on the prompt that is closer to the input prefix (i.e., the attribute behind the dash in Table 11). For instance, NE attribute gets a 3.14% improvement if we put the corresponding prompt close to the input prefix.
However, it also brings a 3.4% decrease for AM attribute as being away from the input prefix at the same time. In contrast, TailorConcat keeps the same performance after swapping.
Unseen Combination In this part, we analyze the combining ability of Tailor on the unseen combination, which does not appear in Tailor’s training stage. In the implementation, we randomly select one combination, remove the corresponding data from the training set for the MAP connector, and then test the performance of the MAP connector on this multi-attribute generation task. As shown in Table 12, TailorArgmax still works to the unseen combination PO+ME and outperforms the non-training method TailorConcat with 2.35% improvements.

heading: G Case Study
text: To intuitively display the effects of various attributes, we show some generation results of singleattribute CTG in Table 13 and multi-attribute CTG in Table 14, respectively.
Multi Attributes Method Generation Results
Negative + Mexican
Finetune Once upon a time I ordered from there, it was just ok . the service was ok, the
food is not worth the price ...
Adapter Once upon a time I was in phoenix and the place was not clean , I went back to try to find one more spot. no problem...
Adapter (P) Once upon a time I came here, the food was ok. however, they had an overpriced
chicken burrito on tap and their guacamole is not as good or fresh ...
TailorConcat Once upon a time they had Mexican cuisine . The service is terrible , it’s not clean and we were left hungry ! (I am so sorry for any inconvenience)...
TailorArgmax Once upon a time, we would be served the burritos that were cooked with no flavor . They didn’t do it right and I will not return !...
TailorWeight Once upon a time I ’d had some of this Mexican food and it was pretty bland .
now the burritos are not good ...
Positive + Mexican
Finetune Once upon a time I had the chicken tacos and my fiancé ordered the carne asada torta . Both were outstanding . very clean , well prepared ...
Adapter Once upon a time I found it. The food and service was excellent as well. our server, kate, had an outstanding experience with...
Adapter (P) Once upon a time I went, we were in town for some reason and ordered the tacos that day. everything was amazing ! food is fresh ...
TailorConcat Once upon a time we had some amazing lunch, which included two tortillas
and one taco . the service was great ! - no complaints there are plenty of...
TailorArgmax Once upon a time, I had the red bell chile and it was great ! our waitress came
to get us as soon . We ordered some tacos with chicken nachos that...
TailorWeight Once upon a time I had the carne asada burrito and they were so good that it
was one of my favorites . I will go back again for sure! ...
Negative + American
Finetune Once upon a time, I tried it and had the worst hangover. after finishing my meal that day ( which was a great one ) all of sudden there is a cockroach...
Adapter Once upon a time I had the lobster rolls , they were cold and not appetizing .
I also received one of these with chicken wings ...
Adapter (P) Once upon a time, I ordered the chicken sandwich . It was good but not quite
as juicy or flavorful with any flavor at all ...
TailorConcat Once upon a time, I would have ordered the shrimp and fish salad . it was
very dry with no flavor ! I ate this place on Sunday night so...
TailorArgmax Once upon a time, this was the place to be. I ordered my chicken burger and
then there is no more fries or burgers at all! ( if you don ’t like that one )?
TailorWeight Once upon a time they would serve you the burger , but it was not cooked .
No sauce in there! ( I’re sorry )? so that could be considered food poisoning ...
Positive + American
Finetune Once upon a time they brought us our food. the staff was very kind and friendly .
I ordered shrimp cocktail , it’s one of those restaurants...
Adapter Once upon a time, the food came fast and fresh . the staff was attentive and we got everything ordered promptly . I recommend this...
Adapter (P) Once upon a time I had the lobster bisque , it was absolutely delicious . Service
is very attentive and friendly ...
TailorConcat Once upon a time, I had the lobster sandwich that was good. it is one of my favourite dishes on this menu! ( no other place in vegas has been more awesome .)...
ACL 2023 Responsible NLP Checklist

heading: A For every submission:
text: A1. Did you describe the limitations of your work?
Left blank.
A2. Did you discuss any potential risks of your work? Left blank.
A3. Do the abstract and introduction summarize the paper’s main claims? Left blank.
A4. Have you used AI writing assistants when working on this paper? Left blank.

heading: B Did you use or create scientific artifacts?
text: Left blank.
B1. Did you cite the creators of artifacts you used? Left blank.
B2. Did you discuss the license or terms for use and / or distribution of any artifacts? Left blank.
B3. Did you discuss if your use of existing artifact(s) was consistent with their intended use, provided that it was specified? For the artifacts you create, do you specify intended use and whether that is compatible with the original access conditions (in particular, derivatives of data accessed for research purposes should not be used outside of research contexts)? Left blank.
B4. Did you discuss the steps taken to check whether the data that was collected / used contains any information that names or uniquely identifies individual people or offensive content, and the steps taken to protect / anonymize it? Left blank.
B5. Did you provide documentation of the artifacts, e.g., coverage of domains, languages, and linguistic phenomena, demographic groups represented, etc.? Left blank.
B6. Did you report relevant statistics like the number of examples, details of train / test / dev splits, etc. for the data that you used / created? Even for commonly-used benchmark datasets, include the number of examples in train / validation / test splits, as these provide necessary context for a reader to understand experimental results. For example, small differences in accuracy on large test sets may be significant, while on small test sets they may not be. Left blank.

heading: C Did you run computational experiments?
text: Left blank.
C1. Did you report the number of parameters in the models used, the total computational budget (e.g., GPU hours), and computing infrastructure used? Left blank.
The Responsible NLP Checklist used at ACL 2023 is adopted from NAACL 2022, with the addition of a question on AI writing assistance.
C2. Did you discuss the experimental setup, including hyperparameter search and best-found hyperparameter values? Left blank.
C3. Did you report descriptive statistics about your results (e.g., error bars around results, summary statistics from sets of experiments), and is it transparent whether you are reporting the max, mean, etc. or just a single run? Left blank.
C4. If you used existing packages (e.g., for preprocessing, for normalization, or for evaluation), did you report the implementation, model, and parameter settings used (e.g., NLTK, Spacy, ROUGE, etc.)? Left blank.

heading: D Did you use human annotators (e.g., crowdworkers) or research with human participants?
text: Left blank.
D1. Did you report the full text of instructions given to participants, including e.g., screenshots, disclaimers of any risks to participants or annotators, etc.? Left blank.
D2. Did you report information about how you recruited (e.g., crowdsourcing platform, students) and paid participants, and discuss if such payment is adequate given the participants’ demographic (e.g., country of residence)? Left blank.
D3. Did you discuss whether and how consent was obtained from people whose data you’re using/curating? For example, if you collected data via crowdsourcing, did your instructions to crowdworkers explain how the data would be used? Left blank.
D4. Was the data collection protocol approved (or determined exempt) by an ethics review board? Left blank.
D5. Did you report the basic demographic and geographic characteristics of the annotator population that is the source of the data? Left blank.

heading: Future Work
text: In the future, we will investigate extending Tailor to connect wider ranges
of attributes, and expand it to other text-to-text generation tasks. Limitations
As we tentatively give a successful implementation of leveraging soft-prompt-based manner to benefit both single and multi-attribute CTG, such a paradigm deserves a closer and more detailed exploration. First, we explore multi-attribute CTG in the scenario of two-attribute composition, yet combining more attributes when generating a completion is more challenging and thrilling, and still in its fledgeless stage. Besides, while extensive experiments demonstrate that Tailor consistently improves attribute-based CTG, applying our approach on a wider variety of PLMs will evaluate the effectiveness of Tailor in a more generally way