Seen to Unseen: Exploring Compositional Generalization of Multi-Attribute Controllable Dialogue Generation
Year: 2023

Abstract:
Existing controllable dialogue generation work focuses on the single-attribute control and lacks generalization capability to out-of-distribution multiple attribute combinations. In this paper, we explore the compositional generalization for multi-attribute controllable dialogue generation where a model can learn from seen attribute values and generalize to unseen combinations. We propose a prompt-based disentangled controllable dialogue generation model, DCG. It learns attribute concept composition by generating attribute-oriented prompt vectors and uses a disentanglement loss to disentangle different attributes for better generalization. Besides, we design a unified reference-free evaluation framework for multiple attributes with different levels of granularities. Experiment results on two benchmarks prove the effectiveness of our method and the evaluation metric.

heading: 1 Introduction
text: Recently, large pre-trained language models (PLMs) like DialoGPT (Zhang et al., 2020), BlenderBot (Roller et al., 2020) and Meena (Adiwardana et al., 2020) can produce fluent and relevant responses for dialogue contexts. However, the generated responses are often uninformative and factual inconsistent. Hence, controllable dialogue generation (CDG) is proposed to guide dialogue generation towards the desired attributes such as emotions (Zhou et al., 2018), acts (Li et al., 2017), and personas (Zhang et al., 2018). Previous work focused on directly fine-tuning the large-scale PLMs (Keskar et al., 2019) or using an extra attribute discriminator (Krause et al., 2021; Dathathri et al., 2019) to guide generation. The former is expensive and requires extensive annotated attribute labels. The decoding of the latter is computationally intensive, reducing the response fluency and generation speed.
∗The first two authors contribute equally. Weiran Xu is the corresponding author.
Although these methods have made some progress in CDG, most of them focus on singleattribute generation where there is only one attribute label like happiness in emotion and pay less attention to the multi-attribute generation, which is a more practical setting. Therefore, we are committed to filling this gap in CDG. Noted that different from single-attribute, the control signal of the multiattribute generation is a combination of multiple values from different attributes, which faces the challenge of lacking sufficient annotated attributespecific data. We also find state-of-the-art methods for multi-attribute controllable text generation (Yang et al., 2022; Qian et al., 2022), which combine controllers learned from single-attribute, only suitable for discrete attributes with specific labels (Li et al., 2017) but not for continuous attributes (Zhang et al., 2018). More importantly, we further show directly applying all existing models achieves superior attribute accuracy on seen attribute combinations but drops significantly on unseen combinations, as shown in Figure 1. It proves that previous work lacks compositional generalization capability from seen attribute values to unseen combinations. Besides, the evaluation of controllability in CDG is severely limited by attribute types and annotated attribute data (Du and Ji, 2021), which is not ap-
14179
plicable to all cases. Therefore, it is valuable to explore a unified and efficient evaluation metric.
In this paper, we try to explore the compositional generalization for multi-attribute controllable dialogue generation where a model could learn from seen attribute values and generalize to unseen combinations. Figure 2 shows two granularities of multi-attribute compositional generalization, where the token-level attribute labels are regarded as coarse-grained discrete attributes and the sentence-level attribute descriptions are regarded as fine-grained continuous attributes. Specifically, we propose a Disentangled Controllable Generation model (DCG), for compositional generalization in multi-attribute controllable dialogue generation. Inspired by prompt learning (Lester et al., 2021), we adopt the attribute values in a combination as attribute-oriented prompts to elicit knowledge from PLMs where the prompts for all instances learn a shared transformation layer, instead of learning an independent prompt representation for each attribute value (Clive et al., 2022; Qian et al., 2022; Yang et al., 2022). Our method helps transfer attribute concepts from seen values to unseen combinations by learning different prompt embeddings and is easily applied to attribute combination with a huge number of discrete or continuous attribute values. To further disentangle different attribute values, we construct a set of pseudo combinations and design a novel objective of controllable attribute combinations for prompt-tuning, which separates desired attribute combination from others.
Furthermore, to unify the evaluation of different granularity attributes, we design a novel and general reference-free evaluation framework, i.e. Multiple Attribute Evaluation (MAE), to measure the consistency between desired seen/unseen
attribute combinations and generated responses. Specifically, the evaluation of each attribute is converted to a text-to-text generation task based on T5 (Raffel et al., 2020) with handcrafted templates, and the generated probability of "yes" is regarded as the controllability score. To mitigate the potential bias of different handcrafted modalities (Zhao et al., 2019; Ke et al., 2022), we add a trainable continuous prompt to improve stability and robustness. Through human evaluation, we show that our proposed evaluation metric can handle both coarse-grained discrete attributes and fine-grained continuous attributes well.
Our contributions are as follows: (1) To the best of our knowledge, we are the first to explore the compositional generalization for multi-attribute controllable dialogue generation and find existing models lack generalization capability to outof-distribution multi-attribute combinations. (2) We propose a disentangled controllable generation, DCG, which learns attribute concepts from seen values to unseen combinations via a shared mapping of attribute-oriented prompts and uses a disentanglement loss to disentangle different attribute combinations. (3) We introduce a unified reference-free evaluation framework, MAE, for different granularities of attributes. Two benchmarks are established and sufficient experiment results prove the effectiveness of our method and evaluation metric.

heading: 2 Related Work
text: Controllable Dialogue Generation Currently, there have existed many studies on CDG (Zhou et al., 2018; Li et al., 2017; Zhang et al., 2018). CTRL (Keskar et al., 2019) used 55 kinds of attribute control codes to finetune an LM which is expensive and requires extensive annotated attribute labels. Krause et al. (2021); Dathathri et al. (2019); Yang and Klein (2021); Lin and Riedl (2021) addressed these limitations by employing an attribute discriminator to update the hidden activations or re-weight the next token distributions, resulting in a slow inference speed. Despite the progress, these models all focus on the single-attribute CDG where the attribute only contains coarse-grained discrete values, such as happiness in emotion-controlled generation. It is also vital to explore multi-attribute CDG with multi-granularity attributes. Recently, some works (Yang et al., 2022; Qian et al., 2022) extend to multi-attribute controllable text genera-
tion by simply concatenating the prefixes trained for single attribute. However, they are only suitable for discrete attributes but not for fine-grained continuous attributes like personas (Zhang et al., 2018). Besides, we find all these methods have a large performance drop from seen attribute values to unseen combinations. Therefore, in this paper, we are the first to explore the compositional generalization for multi-attribute CDG where a model could learn from seen attributes and generalize to out-of-distribution (OOD) combinations. Compositional Generalization in NLP Compositional generalization has gradually attracted the interest of NLP researchers. The main application is in semantic parsing, involving grammar-based approaches (Herzig and Berant, 2021), data augmentation strategies (Oren et al., 2020), disentangled representations (Zheng and Lapata, 2022), etc. Recently, a large-scale benchmark, STYLEPTB, is constructed to advance the development of compositional style transfer (Lyu et al., 2021), and a template-based input representation is also performed on the data-to-text task (Mehta et al., 2022). Overall, the application of compositional generalization in NLP tasks is not widespread and there is no related work on CDG at all. Prompt Learning Prompt-based methods have achieved significant success in many NLP fields (Lester et al., 2021; Schick and Schütze, 2021). Li and Liang (2021) proposed the task-specific continuous prompts to finetune a NLG model. For controllable generation, Clive et al. (2022); Qian et al. (2022); Yang et al. (2022) applied the prompt learning to represent each attribute value as an independent prefix. However, those methods are impractical for fine-grained attributes with a large value set. In contrast, we use the control codes to generate attribute-oriented prompts to guide the generation via a shared MLP layer.

heading: 3 Problem Formulation
text: Given a predefined set of attributes X = {A,B,C, ...}, each attribute contains various values A = {a1, ..., ak} and k is the number of values of attribute A. Multi-attribute controlled dialogue response generation aims to generate responses r that satisfy multiple desirable attributes c = (a1, b2, ...) conditioned on the dialogue history d, where a1 and b2 are one value of the attribute A and B, and c ∈ Cv is a combination of attribute values. It can be symbolized as p(r|d, a1, b2, ...),
(a1 ∈ A, b2 ∈ B, ...). In this paper, we further focus on the multiattribute compositional generalization, where the combinations of multiple attribute values for the training set and the test set are disjoint, i.e., Cv,train ∩ Cv,test = ∅.

heading: 4 Methodology
text: As shown in Figure 3, our model is on the basis of the framework of DialoGPT (Zhang et al., 2020) with the compositional prompt module.

heading: 4.1 Compositional Prompt

heading: 4.1.1 Prompt Design
text: To better use the control signals, we design two types of prompts to elicit the attribute-related information from the PLM: Attribute-oriented Prompt We use the combination of controlled attribute values corresponding to each instance as prompts to guide the model to focus on the controlled information in the dialogue. Here, the controlled attribute values are discrete attribute labels in DailyDialog or continuous attribute descriptions in ConvAI2. The multiple attribute values ai,· in the corresponding combination c are simply concatenated as an attribute-oriented prompt sequence, i.e., patt = [a1, b2, ...]. We encode the prompt tokens using the word embedding layer of a pre-trained DialogGPT and then employ a shared MLPθ1 to generate the embeddings Eatt of the attribute-oriented prompts. Note that we don’t require independent parameters for each attribute value like Clive et al. (2022); Qian et al. (2022); Yang et al. (2022), but only a shared transformation MLP layer. Task-oriented Prompt Although attribute-oriented prompts capture the instance-specific control signals, the dialogue response generation task also is guided by the instance-independent global features. Following Lester et al. (2021), we adopt a series of randomly initialized tokens as the task-oriented
prompt, i.e., ptask = [p1, ..., pm], where m is the length of the task-oriented prompt sequence. We look up this prompt sequence in the randomly initialized embedding table Mθ2 and get the prompt embeddings Etask.
Finally, we concatenate the two prompt embeddings as the whole prompt embeddings, i.e., Ep = [Eatt;Etask].

heading: 4.1.2 Disentanglement Learning
text: Given an instance (d, c), d is the dialogue history and c is the combination of controllable attribute values. To force the model to distinguish different combinations of multiple attribute values, we design some pseudo combinations to enhance the diversity of the prompts, which improves the generalization ability of our model. A disentanglement loss LD is further introduced to disentangle the combination representations and train multiple compositional prompts simultaneously:
LD = −log P (r|d, c)
P (r|d, c) +∑c′∈Cpse P (r|d, c ′)
(1) where Cpse is the set of pseudo combinations and at least one value in the combination c ′ is different from the corresponding value in the golden combination.1 Here, we maximize the generated likelihood of the desirable positive combination P (r|d, c) against the generated likelihood of pseudo combinations P (r|d, c′) to generate more controllable responses relevant to given attributes.

heading: 4.2 Training Strategy
text: We use DialoGPT (Zhang et al., 2020) as the backbone of our model. Given the dialogue history d, the embedding Ed is obtained by DialoGPT. Then, the embeddings of the prompt sequence Ep are prepended to the Ed as a whole input embedding matrix. Overall, the PLM loss is calculated as:
LPLM = − T∑
t=1
log pθ1,θ2,φ(yt|y<t, d, patt, ptask)
(2) where T is the length of generated sequence, i.e., the dialogue history and response. φ is the parameter of the PLM and is fixed. The parameters of two prompts, θ1 and θ2, are the only updated parameters. Therefore, the training loss L is the
1We find constructing pseudo combinations with at least one different attribute value is slightly better than with all different attributes in the experiments.
L = αLD + (1− α)LPLM (3) When the training is completed, we save all parameters of the prompt module. During the inference, the data from the test set is mapped to the representations of prompts only via the embedding matrices, where the features of the attributes seen in the training set can be transferred to the unseen combinations.

heading: 5 Method of MAE
text: To fill the gap in metrics for multi-attribute controllable dialogue generation, we propose a unified and efficient evaluation framework without additional large-scale labeled data, as shown in Figure 4, which converts the evaluation of each attribute to a unified text-to-text generation task, just like Gu et al. (2022). T5 (Raffel et al., 2020) is used as the base model for our work. A template is designed as discrete prompts, i.e., "The emotion/act/persona controls the response [MASK]". To alleviate the potential bias of different handcrafted patterns (Ke et al., 2022), we further add a trainable continuous task-oriented prompt to improve stability and robustness.
Specifically, the continuous prompt sequence is prepended to the response as a prefix, which makes up the input of the encoder. Another continuous prompt sequence, the attribute values, and the template are concatenated and fed to the decoder. We take the probability of generating "yes" corresponding to [MASK] token as the controllability score. In training process, only embeddings of continuous prompts are updated and the parameters of T5 are fixed. Note that our model-based evaluation ap-
proach gets rid of the reliance on golden response when tested and can be uniformly applied to various granularities of attributes.

heading: 6 Experiments

heading: 6.1 Datasets
text: We construct two datasets based on DailyDialog (Li et al., 2017) and ConvAI2 (Dinan et al., 2020) for compositional generalization in multi-attribute controllable dialogue response generation. DailyDialog-CG DailyDialog is an open-domain dialogue dataset with two controllable attributes: emotion and act. Here, we treat the labels of the two attributes as an attribute combination, e.g., (surprise, inform). For dialogues, each utterance with two attribute labels is regarded as the response and all preceding texts of this utterance are considered as the corresponding dialogue history. In this way, we get 14,879 examples. We count the attribute combinations labeled in all examples, 18 of which are selected as Cv,train and the other 6 are Cv,test. Then, the examples are divided into the training set and test set according to the combination set. We also extract 10% samples from the training set as the validation set. ConvAI2-CG ConvAI2 is a persona-based dialogue dataset in which the persona profile of each dialogue is consisting of 4 or 5 personalized sentences. We treat each sentence as an attribute value and the sentences in the same position belong to the same attribute. The persona profile is regarded as an attribute combination, e.g., ("My mom is my best friend.", "I’ve four sisters.", "I believe that mermaids are real.", "I love iced tea."). For each dialogue, we choose the first 4 utterances as the dialogue history and the 5th utterance as the response. Consistent with the processing method of DailyDialog-CG, we select 11,566 combinations as Cv,train 2 and the other 873 combinations as Cv,test.
2The 1,883 combinations of the validation set are included in the 11,566 combinations of the training set.
After that, we obtain the corresponding training set, validation set, and test set.
The statistics about the two datasets are shown in Table 1.

heading: 6.2 Baselines
text: We compare our methods with several competitive baselines. The common dialogue generation models are included: (1) DialoGPT-Ori (Zhang et al., 2020); (2) FUDGE (Yang and Klein, 2021); (3) PPLM (Dathathri et al., 2019); (4) Cocon (Chan et al., 2020); (5) Fine-tuning; (6) CTRL (Keskar et al., 2019). We also implement some promptbased methods for comparison: (1) Prompt-tuning (Lester et al., 2021); (2) CatPrompt (Yang et al., 2022). More details can be seen in Appendix A3.

heading: 6.3 Evaluation Metrics
text: In this work, we focus on evaluating the attribute controllability and text quality for different controllable generation methods. Attribute Controllability It aims to evaluate whether the method can generate responses constrained by multiple attributes successfully.
1. For the control of coarse-grained discrete attributes in DailyDialog-CG, we use the classification accuracy, i.e., E-ACC and A-ACC, for each attribute computed by an independently trained Roberta classifier (Liu et al., 2019), respectively.
2. For the control of fine-grained continuous attributes in ConvAI2-CG, we calculate the cosine similarity between the representations of attribute sentences and the generated response, i.e., PSIM(Du and Ji, 2021). We also evaluate the model by measuring the consistency of attribute sentences with the generated response via a Roberta-based Natural Language Inference (NLI) model, i.e., PNLI(Madotto et al., 2019).
3. We propose a unified model-based evaluation metric, i.e., MAE, for various granularities of
3Our code, models and other related resources are publicly available at https://github.com/Zeng-WH/Seen-to-Unseen.
attributes, the details can be seen in Section 5. Text Quality We use the BLEUs (Papineni et al., 2002) and METEOR (Banerjee and Lavie, 2005) to measure the match scores between generated responses and ground-truth references.

heading: 6.4 Main Results
text: Results on DailyDialog-CG Table 2 presents the results of controllable dialogue generation about unseen attribute combinations for DailyDialogCG. 4 We conduct experiments based on some strong controllable dialogue generation models and novel prompt-based methods. In general, our DCG outperforms all other baselines in terms of attribute controllability and text quality. Compared to CTRL, our model improves by 1.6%, 2.7%, 4.1% in BLEU-1, BLEU-2, METEOR for text quality, and 3.3%, 3.1%, 5.5%, 5.5% in E-ACC, E-MAE, A-ACC, A-MAE for attribute controllability. We also find the FUDGE and PPLM, two methods based on the decoding strategy, perform poorly
4Our DCG improves text quality and controllability. The BLEUs seem low because we adopt the same calculation as ParlAI (Miller et al., 2017), which is lower than results in (Li et al., 2017) for different smooth functions.
here, especially in text quality, which illustrates the incompatibility of these decoding strategies for combinatorial generalization. Besides, as observed, Catprompt is a relatively well-performing prompt-based baseline, but it is still far worse than our method. This is because it directly concatenates all trained single-attribute prompts as the multi-attribute prompt for test. This inconsistency between training and testing stages decreases the performance. Different from these methods, our method optimizes the language modeling loss only based on discrete prompts for attribute combination and continuous task-oriented prompt, which can focus on the features of multiple attributes at the same time also during the training and achieve a better transfer via a learnable mapping.
Besides, we also concern whether DCG benefits from attribute-oriented prompt, task-oriented prompt, and disentanglement learning. We find that DCG w/o AOP is the same with Prompt-tuning and it performs poorly in attribute controllability, which shows attribute-oriented prompt plays an important role in guiding the model to focus on the controlled information. After removing the taskoriented prompt, the DCG w/o TOP decreases to
19.18%, 6.74%, and 15.63% on text quality, but still maintains high controllability. It proves taskoriented prompt helps improve text quality. We also conduct experiments to prove that TOP can improve text quality when combined with other methods. (See Appendix H). Besides, after removing disentanglement learning, the DCG w/o DL drops significantly, which shows disentanglement learning effectively disentangles attribute combinations and improves the ability of compositional generalization. Results on ConvAI2-CG Table 3 presents the results of generalization on unseen attribute combinations for ConvAI2-CG. Due to the diversity of attribute values and attribute combinations, it is very difficult to implement CatPrompt in ConvAI2CG. Therefore, we remove this baseline. We also remove FUDGE and Cocon for their poor generation quality and slow decoding speed, which is shown in Table 2 and Table 5. We can observe that the trend of overall performance is consistent with that of DailyDialog-CG. Compared to CTRL, our model achieves a great improvement in attribute controllability and text quality, which proves the generality of our methods on the coarse-grained discrete attribute control and fine-grained continuous attribute control. It also shows the effectiveness of our method when more attributes are combined. However, all BLEU scores are low, which is because the ConvAI2-CG has more diverse and complex attribute combinations and leads to the instability of models facing new attribute combinations. Generally, the results show that the compositional generalization for multi-attribute controllable dialogue generation is necessary and meaningful. Noted that we also conduct experiments on the setting with changed number of attributes from training to inference (See in Appendix G).

heading: 7 Qualitative Analysis

heading: 7.1 Comparison between Seen and Unseen
text: Attribute Values
Figure 5 displays the comparison of the performance on seen and unseen attribute combinations for DailyDialog-CG. We report the controllability metrics, E-ACC (emotion) and A-ACC (act), and the BLEUs of the Fine-tuning, CTRL, and our DCG. The top of each box denotes the result of seen attribute combinations and the bottom represents unseen attribute combinations. We find all methods achieve significantly superior performance on
seen attribute combinations than on unseen combinations. For example, CTRL achieves 71.27% E-ACC and 43.15% A-ACC on seen attribute combinations but drops to 67.34%(-3.93) and 33.50%(- 9.65) on unseen combinations. It strongly proves previous methods suffer from the difficulty of compositional generalization for the multi-attribute controllable dialogue generation. However, we find our proposed DCG can greatly alleviate this gap. The DCG has a smaller drop of 0.41% and 0.11% for E-ACC and A-ACC, and it also outperforms CTRL on both controllability and text equality of unseen attribute combinations. The results confirm the effectiveness of our method for transferring seen attributes to unseen combinations. We find CTRL achieves a higher A-ACC on seen combinations but a lower score on unseen combinations than Finetuning, which demonstrates directly adding control codes may cause overfitting to seen attribute combinations.

heading: 7.2 Correlation Results on Metrics
text: Following Guan and Huang (2020), we adopt Pearson (r), Spearman (ρ), and Kendall (τ ) correlation coefficients between our proposed automatic metric, MAE, and human judgments (details can be seen in Appendix D) to measure the quality of different metrics. Table 4 shows the overall results on the controllability of coarse-grained discrete attributes, emotion and act, and the fine-grained continuous attributes, persona description. We can observe that our MAE outperforms classic metrics, E-ACC, A-ACC, P-SIM, and P-NLI, by a large margin, indicating the effectiveness of our unified metric on different granularities. We also conducted experiments on some variants of MAE. After the removal of continuous prompts, the correlation scores decrease. It is because the task-oriented prompts are the only parameters can be fine-tuned, which is important for MAE. We also implement MAE on another PLM, BART, to demonstrate generality for our model. Robustness Analysis To verify the effect of the bias of the handcrafted template, we design another two templates. The Template 1 is "The response is related to the emotion/act/persona [MASK]" and Template 2 is "The response is about the emotion/act/persona [MASK]". As shown in Table 4, MAE (T1) and MAE (T2) achieve similar correlation results (within 0.50%) while the results of MAE w/o Prompt (T1) and MAE w/o Prompt (T2)
are quite different. It suggests the trainable continuous task-oriented prompt can alleviate the potential bias of different handcrafted templates and further improve the robustness of MAE.

heading: 7.3 Prompt Visualization
text: To show the effect of prompts for compositional generalization, we display a visualization of the concatenated prompt embeddings of two attributes via PCA (Jolliffe and Cadima, 2016) on DailyDialog-CG in Figure 6. For CatPrompt in Figure 6(a), all the multi-attribute combinations (6(emotion)× 4(act) = 24) almost collapse into four dots where each dot is of the same act attribute value but of different emotion values. We find directly concatenating two single-attribute prompts makes the model only focus on the latter attribute (act), i.e., position sensitive, so that the CatPrompt cannot distinguish different combinations with the other attribute (emotion). Therefore, it’s hard for CatPrompt to learn multi-attribute compositional generalization. In Figure 6(b), We find that DCG w/o DL can distinguish different multi-attribute combinations to some extent. However, the combinations of different attribute values are tightly entangled, such as (a0, b2) and (a4, b1). Figure 6(c) shows that our DCG has a close distribution
with prompts of the same attribute value, i.e., (a0, b0), (a0, b1), (a0, b2), and a sparse distribution with prompts of different attribute values, e.g., (a0, b2) and (a4, b1). It proves our DCG can disentangle attribute combinations and learn relations between different attributes. Furthermore, DCG learns generalization capability from seen attributes to unseen combinations. For example, (a2, b1) -> (a0, b1) (unseen path) is equal to (a2, b0) -> (a0, b0) (seen path). The results confirm that our proposed attribute-oriented prompt outperforms the models that learn an independent prompt for each attribute value. The shared embedding mapping helps learn attribute concepts from seen values to unseen combinations.

heading: 7.4 Few-shot Learning
text: To study the effect of few-shot learning, we randomly select a ratio of original training data from DailyDialog-CG to train CTRL or DCG in lowresource settings and evaluate the model performance on the original test set. "Full" denotes the same setting as the main results. 5000, 1000, and 500 denote the number of examples chosen from the original training data respectively. The results are shown in Figure 7. Note that we keep the original test set fixed for a fair comparison. As the
size of training data decreases, the performance of both CTRL and DCG presents a dropping trend and our DCG model is consistently better than CTRL, which confirms our model has a strong capability for multi-attribute controllable dialogue generation.

heading: 8 Case Study
text: Figure 9 (See in Appendix) shows two examples from Dailydialog-CG and ConvAI2-CG, respectively. For example one in the DailyDialog-CG, the CTRL generates the word "great", showing that the generated response is emotionally controllable. However, both sentences in the response are declarative sentences, which does not control the act question. As observed, the response generated by our
DCG contains the word "Wow", which strongly expresses the emotion of happiness. Besides, a question sentence is also generated. Example two in ConvAI2-CG needs to control 5 attributes, of which the golden response contains 2 attributes. The CTRL only controls "like to skate", while our DCG controls "like to write poetry and skate", which is highly consistent with the golden response. Compared with previous models, our model addresses many difficult issues in compositional generalization for multi-attribute controllable dialogue generation. With an attribute-oriented prompt and a task-oriented prompt, our method learns attribute concepts from seen attribute values to unseen attribute combinations. Through a disentanglement learning, some artificial-constructed unseen pseudo combinations are injected into the training process, which greatly improves the generalization ability of our model.

heading: 9 Conclusion
text: In this paper, we study the compositional generalization for multi-attribute controllable dialogue generation. We propose a prompt-based disentangled controllable dialogue generation model which generates attribute-specific prompt vectors from control codes and uses a disentanglement loss to disentangle different attributes. Further, we develop a unified reference-free evaluation framework, MAE, for multi-attribute generation with different levels of granularities. Experiments and analysis show our method achieves better text quality and controllability scores. Moreover, our proposed MAE has a higher correlation with human judgments for evaluation on CDG.

heading: Acknowledgements
text: We thank all anonymous reviewers for their helpful comments and suggestions. We are also grateful to the track organizers for their valuable work. This work was partially supported by National Key R&D Program of China No. 2019YFF0303300 and Subject II No. 2019YFF0303302, DOCOMO Beijing Communications Laboratories Co., Ltd, MoE-CMCC "Artifical Intelligence" Project No. MCM20190701. Jingang Wang is funded by Beijing Nova Program(Grant NO. 20220484098)
Limitations
Although DCG achieves significant improvements compared with existing baselines, there are still avenues to be explored in future research. (1) DCG in this paper focuses on the compositional generalization for multi-attribute on controllable dialogue generation. We hope to extend the method to other generative tasks, including but not limited to dialogue summarization and story generation. (2) In this paper, we explored the control of coarsegrained discrete attributes and the control of finegrained ones separately, and we intend to study the combination of these two attributes in future research.
Ethics Statement
Controllable dialogue generation(CDG) is an essential task in Natural Language Processing (NLP) and has been widely studied for decades, which aims to guide dialogue generation toward the desired attributes such as emotions, acts, and personas. In the open-domain dialogue scenario, CDG can generate emotional and diverse responses to enhance the user’s sense of participation. In the task-oriented dialogue scenario, CDG can generate responses that meet the user’s needs according to the user’s intent. However, most previous works focus on singleattribute generation where there is only one attribute label like happiness in emotion and pay less attention to the multi-attribute generation, which is a more practical setting. Different from singleattribute, the control signal of the multi-attribute generation is a combination of multiple values from different attributes, which faces the challenge of lacking sufficient annotated attribute-specific data. Therefore, we explore the compositional generalization for multi-attribute controllable dialogue generation where a model could learn from seen
attribute values and generalize to unseen combinations. We also design a novel and general referencefree evaluation framework to unify the evaluation of different granularity attributes. The experimental results prove the effectiveness of our model and evaluation framework. Besides, there is no huge biased content in the datasets and the models. If the knowledge base is further used, the biased content will be brought into the generated responses, just like biased content posted by content creators on the Web which is promoted by a search engine. To prevent the technology from being abused for disinformation, we look forward to more research effort being paid to fake/biased/offensive content detection and encourage developers to carefully choose the proper dataset and content to build the knowledge base.

heading: A Baselines
text: DialoGPT-Ori: Proposed by (Zhang et al., 2020), this model is a dialogue generative pre-trained transformer. Here, we use the original DialoGPT for open-domain dialogue generation. DialoGPT is the backbone for all other baselines except CoCon. Fine-tuning: We use dialogue history in datasets to fine-tune the DialoGPT for dialogue generation. CTRL: Proposed by (Keskar et al., 2019), this method provides attribute control codes for a language model trained from scratch. We concatenate
multi-attribute control codes with dialogue history to fine-tune the DialoGPT. CoCon: Proposed by (Chan et al., 2020), this method uses a content input to control an GPT’s output text at a fine-grained level. PPLM: Proposed by (Dathathri et al., 2019), this method is a gradient-based baseline that uses a plug-and-play language model(PPLM) to guide the language model. We train a joint classifier of emotion and dialogue act which takes a single response as input and predicts the attribute combination of the emotion and dialogue act on DailyDialog-CG. Noted that the attribute classifiers of PPLM can not directly generalize to unknown attribute combinations, so we use both training data and test data to train the attribute classifiers. We use the bagof-words attribute model which encodes persona profile to control the DialoGPT on ConvAI2-CG. FUDGE: Proposed by (Yang and Klein, 2021), this method is a weighted decoding baseline which uses a future discriminator for generation(FUDGE) to guide the DialoGPT. We train a joint discriminator that takes the dialogue history and the current response as input and predicts the attribute combination of emotion and dialogue act on DailyDialogCG. Prompt-tuning: Proposed by (Lester et al., 2021), this method uses continue prompts to fine-tune language models. We apply this method to the DialoGPT for dialogue generation. CatPrompt: Inspired by Yang et al. (2022); Qian et al. (2022), we initialize an unique prompt for each single attribute value and concatenate singleattribute prompts as the multi-attribute prompts. We fine-tune multi-attribute prompts for dialogue generation. Note that CatPrompt is only applied to coarse-grained discrete attributes like emotion and act instead of persona. Because persona has a large value set, resulting in numerous parameters (see Table 6).
B Implementation Details
Our implementation is based on the Hugging Face Transformer models5. DialoGPTSmall is used as a backbone and the input sequence length is truncated to 512 tokens. Following the HuggingFace default setup, we use an AdamW optimizer (Loshchilov and Hutter, 2017) and a linear learning rate scheduler with an initial rate of 7.5 · 10−5, and the batch size is set to 8. The prompt lengths
5https://github.com/huggingface/transformers
are set to 50 and 150, the attribute-oriented prompt lengths are set to 6 and 100, the disentanglement loss weight is set to 0.1 and 0.03, and the number of Pseudo Combinations is set to 8 and 6 for DailyDialog-CG and ConvAI2-CG, respectively. Our model is trained on Tesla V100 machines, taking 24 minutes per epoch on DailyDialog-CG and 36 minutes per epoch on ConvAI2-CG. For all experiments, we set the number of training epochs to 30. At the decoding phase, we use a greedy search and max generated tokens of 150.
C Inference Efficiency
We compare the average inference efficiency of our methods with the baselines. As we can observe from Table 5, the inference speed of PPLM, FUDGE, and CoCon is far slower than the original GPT-2 model. Prompt-based methods are much faster than that decoding strategy based methods. The inference speed of our method is close to the original DialoGPT methods. As shown in Table 6, with the growth of attribute combinations, the trainable parameters of CatPrompt increase rapidly, from 0.84M to 224M, which even exceeds the 117M trainable parameters of full DialoGPT. While our method achieves better results with a lower number of trainable parameters on DialyDialogCG and ConvAI2-CG.

heading: D Human Evaluation
text: To validate the good performance of DCG, we further deploy a set of human evaluations to compare the controllability and text quality between several methods. We randomly sample 100 examples from two datasets and collect the corresponding generated responses of CTRL, DCG, and DCG w/o DL. For the controllability, 5 human annotators are invited to evaluate on a scale of 1-3, where score 1
means that the generated response is completely inconsistent with the expected attribute label, score 2 denotes that the generated response has the same meaning as the expected attribute label, but no explicit attribute-related words, and score 3 means that the generated response contains some clear attribute words. For the text quality, we ask the annotators to evaluate the fluency and context relevancy of the generated responses on a scale of 1-5, where a higher score indicates better quality. The inter-annotator agreement on the controllability and text quality is 0.63 and 0.61 for DailyDialog-GC, and 0.58 and 0.60 for ConvAI2-CG. For all metrics, the average score of the 5 annotators is treated as the final score.
As shown in Table 7, the text quality scores of all models are high, which is because the models finetuned on contextualized language backbones can generate fluent sentences with relevant information. For controllability, our DCG achieves better performance than CTRL both on the coarse-grained discrete attributes and fine-grained continuous attributes, which suggests that our shared prompt mapping can learn the attribute concepts from seen attribute values to unseen attribute combinations and is useful for diverse attributes. Besides, when removing the disentanglement learning, the scores of our DCG w/o DL drop significantly, which further shows the effectiveness of the combination disentanglement to improve the generation ability.

heading: E Effect of Model Parameters
text: Prompt Length Figure 8 (a) displays the effect of overall prompt lengths of Ep. Since the length of attribute-oriented prompt is fixed to the number of control code, we change the length of the taskoriented prompt. We find that our DCG achieves superior performance when the prompt length is between 20 and 100, and gets the best scores when the prompt length is 50. The DCG outperforms the strong baseline CTRL by the 3.19% (averaged) for MAE and 2.16% (averaged) for BLEUs but uses only 56% trainable parameters of CTRL, which verifies the effectiveness and robustness of our method. Weight of Disentanglement Loss Figure 8 (b) shows the effect of different weight ratios α for the disentanglement loss LD. We observe that α ∈ (0.05, 0.15) achieves consistent improvements than CTRL and we take α = 0.10 in all experiments. Number of Pseudo Combinations Figure 8 (c) shows the effect of the number of pseudo combinations in the disentanglement loss. We find a larger number will improve the controllability of our model. It’s because more pseudo attribute values help the model to separate the desired attribute combination from the others.

heading: F Comparison with CTRLEval
text: Automatic evaluation metrics are important for text generation tasks, including reference-based like BLEU (Papineni et al., 2002), ROUGE (Lin, 2004),
BERTScore (Zhang et al., 2019) and unreferenced like perplexity (Brown et al., 1992), discriminator scores (Dathathri et al., 2019), BARTScore (Yuan et al., 2021). To evaluate controllability, (Dathathri et al., 2019; Yang and Klein, 2021) trained an attribute classifier to predict the probability using labeled external data, which is hard to multi-attribute controllable generation. As a concurrent work, CTRLEval (Ke et al., 2022) proposes an evaluation method for controllable text generation. Different from our MAE, CTRLEval uses handcrafted prompts to evaluate attribute relevance. However, handcrafted prompts are hard to construct for new tasks and cause generation bias. In contrast, our MAE uses a learnable soft prompt based on PLMs to enhance the generalization capability and robustness. We also provide a performance comparison in Table 8. Results show our MAE shows superior correlations of attribute controllability.

heading: G Performance on Number of Attribute
text: To prove our model still be useful when the number of attributes varies from training to inference, we train CTRL and our DCG with 4 attributes and inference with 5 attributes in ConvAI2-CG. As shown in Table 9, DCG outperforms the strong baseline CTRL by 3.54% , 5.99%, 4.8% in P-SIM, P-NLI
and P-MAE on controllability and achieves comparable BLEU scores. It proves DCG can also handle well with changed number of attributes.
H Impact of TOP on Text Quality
We prove that task-oriented prompts (TOP) can also improve text quality when combined with other methods. Specifically, we trained CTRL with TOP in our experiments. As Table 10 shows, the results of CTRL for BLEU-1, BLEU-2, and METEOR are 24.76%, 11.42%, and 20.45%, respectively. Meanwhile, the results of CTRL+TOP for BLEU1, BLEU-2, and METEOR are 25.88%, 14.36%, and 21.82%. These results indicate that CTRL can utilize TOP to enhance text quality.
ACL 2023 Responsible NLP Checklist

heading: A For every submission:
text: A1. Did you describe the limitations of your work?
Left blank.
A2. Did you discuss any potential risks of your work? Left blank.
A3. Do the abstract and introduction summarize the paper’s main claims? Left blank.
A4. Have you used AI writing assistants when working on this paper? Left blank.

heading: B Did you use or create scientific artifacts?
text: Left blank.
B1. Did you cite the creators of artifacts you used? Left blank.
B2. Did you discuss the license or terms for use and / or distribution of any artifacts? Left blank.
B3. Did you discuss if your use of existing artifact(s) was consistent with their intended use, provided that it was specified? For the artifacts you create, do you specify intended use and whether that is compatible with the original access conditions (in particular, derivatives of data accessed for research purposes should not be used outside of research contexts)? Left blank.
B4. Did you discuss the steps taken to check whether the data that was collected / used contains any information that names or uniquely identifies individual people or offensive content, and the steps taken to protect / anonymize it? Left blank.
B5. Did you provide documentation of the artifacts, e.g., coverage of domains, languages, and linguistic phenomena, demographic groups represented, etc.? Left blank.
B6. Did you report relevant statistics like the number of examples, details of train / test / dev splits, etc. for the data that you used / created? Even for commonly-used benchmark datasets, include the number of examples in train / validation / test splits, as these provide necessary context for a reader to understand experimental results. For example, small differences in accuracy on large test sets may be significant, while on small test sets they may not be. Left blank.

heading: C Did you run computational experiments?
text: Left blank.
C1. Did you report the number of parameters in the models used, the total computational budget (e.g., GPU hours), and computing infrastructure used? Left blank.
The Responsible NLP Checklist used at ACL 2023 is adopted from NAACL 2022, with the addition of a question on AI writing assistance.
C2. Did you discuss the experimental setup, including hyperparameter search and best-found hyperparameter values? Left blank.
C3. Did you report descriptive statistics about your results (e.g., error bars around results, summary statistics from sets of experiments), and is it transparent whether you are reporting the max, mean, etc. or just a single run? Left blank.
C4. If you used existing packages (e.g., for preprocessing, for normalization, or for evaluation), did you report the implementation, model, and parameter settings used (e.g., NLTK, Spacy, ROUGE, etc.)? Left blank.

heading: D Did you use human annotators (e.g., crowdworkers) or research with human participants?
text: Left blank.
D1. Did you report the full text of instructions given to participants, including e.g., screenshots, disclaimers of any risks to participants or annotators, etc.? Left blank.
D2. Did you report information about how you recruited (e.g., crowdsourcing platform, students) and paid participants, and discuss if such payment is adequate given the participants’ demographic (e.g., country of residence)? Left blank.
D3. Did you discuss whether and how consent was obtained from people whose data you’re using/curating? For example, if you collected data via crowdsourcing, did your instructions to crowdworkers explain how the data would be used? Left blank.
D4. Was the data collection protocol approved (or determined exempt) by an ethics review board? Left blank.
D5. Did you report the basic demographic and geographic characteristics of the annotator population that is the source of the data? Left blank.

heading: Future Work
text: FUDGE: Proposed by (Yang and Klein, 2021), this method is a weighted decoding baseline which uses a future discriminator for generation(FUDGE) to guide the DialoGPT. We train a joint discriminator that takes the dialogue history and the current response as input and predicts the attribute combination of emotion and dialogue act on DailyDialogCG. Prompt-tuning: Proposed by (Lester et al. , 2021), this method uses continue prompts to fine-tune language models. We apply this method to the DialoGPT for dialogue generation. CatPrompt: Inspired by Yang et al. (2022); Qian et al. (2022), we initialize an unique prompt for each single attribute value and concatenate singleattribute prompts as the multi-attribute prompts. We fine-tune multi-attribute prompts for dialogue generation. Note that CatPrompt is only applied to coarse-grained discrete attributes like emotion and act instead of persona. Because persona has a large value set, resulting in numerous parameters (see Table 6). B Implementation Details
Our implementation is based on the Hugging Face Transformer models5. DialoGPTSmall is used as a backbone and the input sequence length is truncated to 512 tokens. Following the HuggingFace default setup, we use an AdamW optimizer (Loshchilov and Hutter, 2017) and a linear learning rate scheduler with an initial rate of 7. 5 · 10−5, and the batch size is set to 8. The prompt lengths
5https://github. com/huggingface/transformers
are set to 50 and 150, the attribute-oriented prompt lengths are set to 6 and 100, the disentanglement loss weight is set to 0. 1 and 0. 03, and the number of Pseudo Combinations is set to 8 and 6 for DailyDialog-CG and ConvAI2-CG, respectively. Our model is trained on Tesla V100 machines, taking 24 minutes per epoch on DailyDialog-CG and 36 minutes per epoch on ConvAI2-CG. For all experiments, we set the number of training epochs to 30. At the decoding phase, we use a greedy search and max generated tokens of 150. C Inference Efficiency
We compare the average inference efficiency of our methods with the baselines. As we can observe from Table 5, the inference speed of PPLM, FUDGE, and CoCon is far slower than the original GPT-2 model. Prompt-based methods are much faster than that decoding strategy based methods. The inference speed of our method is close to the original DialoGPT methods. As shown in Table 6, with the growth of attribute combinations, the trainable parameters of CatPrompt increase rapidly, from 0. 84M to 224M, which even exceeds the 117M trainable parameters of full DialoGPT. While our method achieves better results with a lower number of trainable parameters on DialyDialogCG and ConvAI2-CG