Semantic Space Grounded Weighted Decoding for Multi-Attribute Controllable Dialogue Generation
Authors: Zhiling Zhang
Mengyue Wu
Kenny Q. Zhu
Venue: EMNLP 2023 Main (2023)
Keywords: dialogue response generation
chatbot
controllable generation
multi-attributes

Abstract: Controlling chatbot utterance generation with multiple attributes such as 
personalities, emotions and dialogue acts is a practically useful but 
under-studied problem.
We propose a novel framework called DASC 
that possesses strong controllability with a weighted decoding paradigm, 
while improving generation quality with the grounding in an 
attribute semantics space. Generation with multiple attributes is then 
intuitively implemented with an interpolation of multiple attribute embeddings,
which results in substantial reduction in the model sizes. 
Experiments show that DASC can achieve high control accuracy 
in generation task with the simultaneous control of 3 aspects while also producing interesting and 
reasonably sensible responses, even in an out-of-distribution robustness 
test.

Semantic Space Grounded Weighted Decoding for Multi-Attrbbute Controllable Dialogue Generation
Zhiling Zhang
Mengyue Wu"
Kenny Q. Zhu"
Shanghai Jiao Tong University
University of Texas Arlington Arlington, Texas, USA kenny.zhu mruny.zhuu@ie uta.edu
Shanghai, China
(blmoistawindl jeta.u edu. cn
Abstract
Among previous works, Weighted Decoding methods has achieved great success in single- attribute control tasks (Arora et al., 2022; Liu et al., 2022). Weighted decoding methods learn a token- level attribute classifier, which predicts the prob- ability of the text conveying the desired attribute given the generation of each token in the vocab- ulary. Then the predicted probabilities to re-weigh the token generation during decoding to induce the attribute. Despite success in single- attribute control, they have certain limitations when extended the multi-attribute case by multiplying several attribute predictions from multiple classi- fiers. Extra parameters proportional to the large vocabulary size VV will be introduced, which can grow several times further due to the number of attributes. The consequent large number of param- eters will not only make the model inefficient, but also harm the generation quality. The model can be prone to overfit since the data for each attribute combination are usually small, which increases the risk of degeneration (Holtzman et al., 2020).
Controlling chatbot utterance generation with multiple attributes such personalities, emo- tions and dialogue acts is a practically use- ful but under-studied problem. We propose a novel framework called DASC that possesses strong controllability with weighted decoding paradigm, while improving generation quality with the grounding in an attribute semantics space. Generation with multiple attributes then intuitively implemented with an interpola- tion of multiple attribute embeddings, which re- sults substantial reduction in the model sizes. Experiments show that DASC can achieve high control accuracy in generation task with the simultancous control of aspects while also producing interesting and reasonably sensible responses, even an out-of-distribution robust- ness test."
used
1 Introduction
Personalized dialogue systems are promising NLP applications for human-computes interaction and emotional companionship. We would expect such systems have personalities, exhibit emotions, take dialogue acts and even adopt sophisticated strategies (Liu et al., 2021b), which necessitates the research efforts on Controllable Text Generation. Despite recent progress in this field (Dathathri et al., 2020; Keskar et al., 2019; Krause et al., 2021), they mainly tackle single-attribute control, overlooking the fact that human interaction can usually con- vey multiple attributes simultancously. Therefore, we explore a novel task of Mudti-Atribute Control- lable Dialogue Generation, which can significantly ameliorate the expressiveness, human-likenn likeness, and explainability of chat-bots. However, the numerous combinations of attributes can make the available data for each setting scarce, which poses a great challenge for this task.
To overcome these limitations, we propose Dialog Attribute Space Controller (DASC). We establish an attribute semantic space where each token in the vocabulary is projected to the space through Attribute Token Embedding shared across attributes. The language models' hidden states also converted to Attribute Context Embedding in the space through stribute-specific layers. The tribute space will be trained to make the tokens suit- able to convey the desired attribute close to the cur- rent context embedding. We can then assign higher weights for the those tokens during decoding. We will show that DASC can inherit the strong control- lability of weighted decoding, while also achieving natural solution of multi-attribute control with the interpolation of multiple attribute embeddings in the space. Moreover, the shared attribute token embedding also alleviates over over-parameti parameterization, and improves the robustness of the model.
*Corresponding Authors. (Code and data are available at htpps://githun com/ blmois sttwindd ndeDASC..

---

Context (Last Rounds):
ation background. Then we will introduce the proposed DASC framework.
often don't get spend time my family, and feel guilty.
B: It's okay. They'll understand you.
2.1 Task Definition
Desired Attributes:
Gender: Male | Emotion: Fear MonnQueetton
Given dialogue context C and attributes A (a1, U2,..., og). Controllable Dialogue Generation aims to generate a response R = (r11, r2, F'N) that is consistent with the context and carries the attributes. 2 There can be multiple aspects grouping the attributes, where in this work we will focus Gender Style, Emotion, and Dialogue Act. An aspect covers multiple related attributes, such happiness, sadness for the Emotion aspect. Each attribute can take three values: means to use the attribute, mcans not to use and á means the attribute is not applicable to the response.
Responses:
Baseline: Yeah, I will.
Prev Method: I am afraid that wife and children and my children would be seen by my parents.
DASC: But am afraid that my wife and children will worry about me.
Figure I: An example of mulli-attributt controllable di- alogue generation. The baseline system doesn't 't attempt any control and produced a dull response, while pre- vious method of attribute control generated a repetitive and illogical text. DASC successfully gives response that is both fluent and correctly attributed.
2.2 Weighted Decoding for Controllable Generation
Standard, non-controllable, dialogue generation can be formulated with the standard condi- tional language modeling objective: LCLas
We experiment on an attribute-rich open-domain dialogue dataset (Xu et al., 2022) for the simulta- neous control of attribute aspects: Gender Style (male, female, neutral), Emotion (8 classes), and a simple division of Dialogue Act (question VS non- question). As exemplified in Figure I, compared to previous methods, DASC achieves strong control- lability while avoiding low-quality generations in the compositional controlling task. Visualization of the attribute token embeddings (in Figure 3) ex- hibits specific patterns that benefit the controlling, compared to the general LM token embeddings. We further conducted a robustness test in out- of-distribution setting and validated that DASC's controllability generalizes. Our contributions are as follows: 1) We propose semantic space grounded weighted decoding for controllable dialogue gener- ation, which can intuitively solve the multi-attribute control task with the interpolation of embeddings the space: 2) DASC uses smaller number pa- rameters than other weighted decoding alternatives while achieving better performance with the de- sign of shared attribute embeddings; 3) DASC can achieve high accuracy the simultaneous control of aspects while also preserving competitive gen- eration quality in both conventional test settings robustness tests.
Σ log Prrn(rr/ 1,C) We can use a transformer-bawee encoder-decoder architecture like BART (Lewis et al., 2020) to model this, where the encoder encodes the con- text into hidden states as condition for the decoder to generate the response. We will omit C' below for brevity.
In controllable dialogue generation, we addition- ally introduce attributes in the generation condition. Suppose we are generating with single attribute a, then the objective to model Using Bayes' rule, this can be converted to:
(I) where a is a hyperparameter that can adjust the control strength. This means that we can decom- pose the generation probability into the standard CLM probability weighted by the prediction of an- other token-wise attribute classifier during decod- ing. Methods established such decomposition are thus called Weighted Decoding models.
P(ra(rin-1,a) x
Director (Arora et al., 2022), a representative weighted decoding method, implements the at- tribute classifier as a lincar layer on top of the decoder hidden states. A binary classification is performed on determining whether the generated sentence reflects the desired attribute (e.g. happy or
and
2 Method
"In our work, we make a pre-assumption that attributes are provided by a dialogue policy, and do not include end-to-end scenarios.
this section, we will first define our task and weighted decoding method for controllable gener-

---

cases. The learning not) each step. For tokens in the sentence from training set, they can be trained with the attribute the whole sentence using Binary Cross Entropy (BCE) loss. We denote this token-level loss as Lt-
2.3 Dialogue Attribute Space Controller
We hypothesize that the above typical methods of weighted decoding may not be the most efficient ap- proach to learn the token-level attribute semantics, especially in objective is imposed on single token in the tar- get sentence, while all other tokens regularized equally. This is usually reasonable, as some tokens similar to the target token should also have high probabilities given the attribute while other tokens different from it are less likely to be gener- ated. For example, for the first token in a happy response "nice to meet you", "glad" will also be a reasonable alternative, while "sad" is not, but their attribute label in the training will both be 0.5.
L BCF(P(d BC
(2)
where hn E Rd is the hidden state for the n-th token, Wa E RIVixd is the learnable weight matrix for attribute prediction given the generation of each token in the vocabulary, and Hr,, denotes the index selection with the next token Ti. Note that it only gathers the attribute logits with the token rn in the ground truth response. For the other [V] - tokens in the vocabulary V, they have no label and cannot get trained. Therefore, it uses an extra regularizer to train the prediction on these tokens to as close to 0.5 as possible with MSE loss.
We fix this counter-intuition in high- dimensional space. On the one hand, each token has an p-dim embedding that encodes its attribute semantics (Attribute Token Embedding, AT EM B). On the other hand, the hidden states from the L.M (hr) are also projected to the same space with attribute-specific linear layers (W* E IRPx)) to get Attribute Context Embedding, h = W/ Thus different vectors in the space convey different mantics, and we call this space Atribute Semantic Space.
When dealing with multi-attribure control, extend Eq. (1) by introducing the product of multiple attribute classifiers, assuming the condi- tional independence of attributes:
P(ssl1in-1,a) & P(ralrmn-))
II P(antrin)" (3)
To leverage this latent space for weighted decod- ing, for each h., we find its attribute-related tokens according to embedding similarity in the space, and assign them higher weights during decoding. Specifically, it is accomplished with a dot-product based token-level attribute classifier.
The product of probabilities is usually imple- mented with the summation of logits:
8(ra/rim-1:0)- = ((m))) + ((ae(ram)) (4)
(5) In this case, when a token is trained with high probability for certain attribute, its neighbors in the attribute space will also have higher probabilities. This alleviates the limitation of previous weighted decoding methods, and eliminates the need for reg- ularizers on other tokens. Further, when applying this to multi-attribut weighted decoding, we get:
8(aulri..) itt: ATEMB((..)
Existing works have implemented such an exten- sion with either multiple forward passes through attribute-conditioneed language model (Lin and Riedl, 2021) one pass of multiple models (Liu al., 2021a), which can all be very costly as the number of attributes grows. Here we introduce relatively simple extension Director, where we just add K linear classifier heads to make the pre- diction of multiple attributes. We will refer to this simple extension as M- M-Director, or just Director for simplicity. Note that though more efficient than previous methods, M-Director will still introduce d x V K extra parameters. Given that [V] is usually as large as tens of thousands, this model will have enormous number of parameters mak- ing it inefficient to train or infer, and also prone to overfitting.
8(ra/Mis-1.a)) = 8(1a/Fixx-1)
+
Rb
ATEMB((.))
(6)
where the parenthesized part in the second term can be interpreted as the wverageeqqual-weige interpo- lation of multiple attribute context embeddings.
IIt is possible to assign different weights for each embed- ding in interpolation, and we leave it for future works.

---

next token generate, and thus the parameters do not scale with |V]. Then the final loss is: Lirain LCLM + [LL + Lę), where 3 a hyperparameter that controls the weight of attribute-related losses.
Target Token
Attribute Hit (0/1)
Classifier [d. M. K]
LM hidden state
We name the proposed framework as Dialogue Attribute Space Controller (DASC). The illustra- tion of DASC and its comparison with M-Director is shown in Figure 2. DASC introduce fewer pa- rameters (d x p x K + [V] x p) than M-Director (d x [V] x K). Since we set p << [Vi, the param- eters of attributes projections will be much smaller. And when we deal with K > 1, the shared token embeddings across attributes will also save parame- ters, while the parameters of attribute predictor are almost negligible.
Other Tokens Regularize (0.5)
M-Director
DASC
LM hidden state
Target Token
Attribute Projection
Attribute Fimhedder
[d. K]
[[V] p]
Dot Product
Attribute Context
Attribute Token
Embeddiig
Embedding
Attribute Predictor Ip. K]
Attribute Hit (0/1)
3 Experiments
Figure 2: Framework comparison between M- M-Director and DASC. M-Director uses a classifier head to con- duct binary attribute hit classification for each token in the target sentence, and impose regularization for other tokens. DASC projects both LM hidden state and the target token to the attribute space, and uses their dot product for the classification of attribute hit. For each parameterized model component, we show its shape in square brackets.
In this section, we will conduct experiments to ex- amine the following hypotheses: (1) DASC can achieve strong controllability while also preserving good generation quality in multi-attribute control- lable generation; (2) DASC's performance benefits from the meaningful representations the attribute semantic space, and reduction in parameters; (3) DASC can also be flexibly extended for other con- trol tasks like the composition of multiple emotions or adopting certain strategies for emotional support.
This formulation suggests that if the attribute space is properly learned and represented, the embedding interpolation will precisely reflect the semantics of the desired attributes, and then DASC can realize reasonable attribute combinations.
3.1 Experiment Settings
We conduct experiments on the self split of the DuLemon dataset (Xu al., 2022), which a Chi- nese open-domain dialogue dataset that is rich in personalized content so that we can find the vari- ous attributes we would like to control. We split the data to train/dev/test set into 352,999, 2439, 2412 utterances each. Since the original dataset do not contain annotations of control attributes, we develop few classifiers, one for each type of at- tributes, to label the datasct. For gender style (male, female, neutral), we use the dataset released by Su et al. (2020) to train a MacBERT classifier (Cui et al., 2020), which achieved accuracy=94.98% For emotion, we follow Zhou et al. (2018) and use the NLPCC2013 and NLPCC2014 4 dataset (8 emo- tion classes) to train another MacBERT classifier, which has an accuracy of 93.96%. For the ques- tion dialogue act (question VS non-question), we simply use heuristic for labeling: if the sentence contains question mark(?) we will consider a question and otherwise non-question. We then use these classifiers to assign each response in the
To assist the learning of attribute embeddings, we introduce another linear layer on top of the at- tribute context embedding each step to directly predict the attributes of the complete response. This can help better align the attribute context em- beddings with the corresponding region for its at- tributes. We denote the new the sentence-level classification loss as L.. For clarity, we give its for- mulation in the single-attribute case, which can be simply extended to multi-attribute scenarios with the summation over all non-empty attributes.
L = (C
(7)
BCE((o kw))
where U.. E RP is the learnable weight for attribute prediction. Compared with L, (Liq. (2)), is a sentence-level classification task independent of Tn, which can also be interpreted as predicting the prior probability of the attribute before deciding the

---

dataset with the 3 types of attributes (13 of them in total).
BScore
Dist-2
ACeG
Accg
Acco
Baseline
68.18
19.25
68.49
46.31
69.61
Rerank
69.23
19.28
75.46
54.93
82.42
CTRL
71.09
18.91
85.32
77.49
100.00
3.1.1 Competing Methods
Director
69.54
21.40
95.81
86.73
100.00
DASC
70.42
21.94
95.85
86.07
100.00
We compare the proposed DASC framework with representative methods from different types of controllable generation methods. We use the hilp/bart--bae-chhnnss (Shao et al., 2021) model as the backbone for all competing meth- ods 4: Baseline Simply fine-tuning the backbone on the dataset without utilizing the control at- tributes. Rerank Using top-p sampling (Holtz- man et al., 2020) on the baseline model to pro- duce 5 response candidates for each context, and attribute classifiers (here are the same separate models we' 'e used for auto-annotations) to rerank the candidates. Following Thoppilan et al. (2022), we use the sum of predicted probabilities in each aspect for ranking. CTRL We re-implemented Keskar al. (2019)'s method for dialogue genera- tion by defining 3 groups of special control codes for each aspect, and appending the corresponding 3 attribute tokens to each dialogue context during fine-tuning. Director The multi-attribute exten- sion of Director (Arora et al., 2022) discussed in Sec. 2.2. We provide more experimental details in Appendix.
Table 1: Automatic evaluation results DuLemon test set. The best results are in hold, while the second results are underlined.
invited 2 volunteers who are native Chinese speak- ers evaluate each generation from 3 perspectives. Attribute Accuracy: the response conveys the given attribute. Sensibleness 4): if the response is fluent, coherent with the context, and accords with commonsense. Interestingness the response is specific, novel and can encourage more interesting conversation continuation.
whether
3.2 Results
Automatic evaluation results are shown in Table 1. We see that Rerank failed to show strong controllability because the base model struggles to produce attributed ranking candidates without finetuning with the attributes. CTRL leveraged the attributes finetuning, and achieved better con- trol accuracy and BertScore, but doesn't 't produce more diverse responses overall. Both Director and DASC exhibit the best controllability, and DASC produces more diverse and reasonable responses according to Distinct-2 and BertScore.
3.1.2 Evaluation
Automatic Evaluation To evaluate the control- lability, we use the same attribute classifiers as those used for labeling the dataset to calculate the accuracy of attributes the generation (Aceg, Accg, Acco for gender, emotion and question, re- spectively). For the generation quality, we use BertScore (BScore) (Zhang al., 2020) to evaluate generation's 's similarity reference response, and Distinct-2 (Li al., 2016) for diversity.
Acee
Accs
Aceo
Interest
Sensible
Baseline
0.80
0.55
0.64
2.04
3.46
Rerank
0.81
0.62
0.82
2.13
3.44
CTRI.
0.85
0.82
0.97
2.24
3.46
Director
0.87
0.87
0.96
2.25
3.26
DASC
0.88
0.88
0.97
2.37
3.28
Table 2: Human Judgement on DuLemon test set.
We then show human judgement results in Ta- ble 2. The inter-annotator agreement for Accc. Acc and Aceo are 0.65, 0.55 and 0.64 in Cohen's K, which indicates moderate to substantial agree- ment. The agreement of Interestingness and Sensiblencss 0.48 and 0.44 in Pearson's r. This is hardly surprising because the latter two perspec- tives are highly subjective. The evaluation on at- tribute accuracies is similar to the automatic results, except that the accuracy of gender drops slightly. We find that human evaluators can spot errors re- lated to gender stereotype (Bolukbasi et al., 2016), like generating soldier for male style and baby- carer for female, where these occupations should
Human Judgement We sampled 100 contexts from the test set for human evaluation. Since the distribution of the original test set extremely skewed, we've specified a constraint for more bal- anced distribution over all emotions during sam- pling, so as to ensure the representativeness of the evaluation (21 none, 16 sadness, 16 disgust, 16 happiness, 16 like, 5 anger, 5 surprise, 5 fear). We
"We also conduct experiments with those leveraging per- sona description texts, including BoB (Song et al., 2021) and prompting with ChatGPT (OpenAl, 2022). although they may not he especially suitable for controlling the sparse attributes here. We will provide the experimental details and automatic evaluations in Appendix.

---

The annotators also check ques- tions without a question mark, which explains the slight difference in Accq.
be gender-neutral.
curacy of Rerank and CTRL dropped significantly, which shows that their controllability is not gen- eralizable. Another notable phenomenon the abnormal Distinct-2 achieved by Director. We then further analyze their performance with human eval- uation (excluding Rerank as it fails to control at- tributes). We found that Director frequently gen- erate ungrammatical, illogical and repetitive long responses (like the second response in Figure I). Director's loss in emotion accuracy is also higher than DASC, indicating that it may overfit the train- ing distribution given its large parameters, and thus performs worse in this fit-of--dittibuution setting. Compared to CTRL, DASC has lower Sensibleness but higher Interestingnesss when it also has sig- nificant advantage in diversity and controllability.
Overall, the rankings of controllability still hold according to human evaluation, with DASC per- forming the best. Bascline, Rerank and CTRL have slightly better Sensibleness than weighted decod- ing methods, which agrees with the commonly ob- served controllability-qualin trade-off in previous literature (Dathathri et al., 2020; Yang and Klein, 2021; Qian et al., 2022). All controllable genera- tion methods achieved higher Interestingness score than baseline, which supports the benefits of con- trollable generation. DASC achieved the best In- terestingness given similar attributes accuracy as Director, indicating the effectiveness of attribute semantic space, which can establish better repre- sentations of attribute semantics and more reason- able approach to compose the control attributes in weighted decoding.
3,4 Semantic Space Visualization
For a clear understanding of how the proposed at- tribute semantic space can help controllable gen- cration, we visualize them in 2D space with t- SNE (Van der Maaten and Hinton, 2008). First, we visualize the attribute token embeddings of some representative attribute-related tokens, and also compare them with the corresponding embed- ding in the original LM (Figure 3). Comparing the two figures, we can see that (1) The token embed- dings from different aspects are more separable in the attribute space (see points with different colors), while tokens in the same aspect are closer despite the difference in other linguist features like part-of- speech (like 'handsome' and 'male'). (2) The token embeddings from different attributes of the same aspect are also distinguished in the attribute space (like 'male" -femae'" 'love' Movee''minn 'mmssrrble'). These characteristics enable DASC to successfully con- trol the generation of distinctive attributes and com- pose attributes from different aspects.
3.3 Robustness Test
In previous experiments, the control attributes pro- vided to the model come from the reference re- sponse. Therefore, models may coincidentally hit the desired attributes when generating the most likely response to the context, without truly reli- able controllability for arbitrary given attributes. Hence, we further conduct experiments to test the robustness of the controllable generation methods in -fitt--distriuution scenarios.
Specifically, we sampled 100 contexts from the test set, and give the models each of the 8 emotions as the generation condition, paired with the origi- nal gender and question acts. We then use greedy decoding to generate response for each (context, attributes) pair and conduct similar automatic and human evaluation on the 800 generations.
Dist-2
Acc
Interest
Sensible
Next, we also visualize the attribute context embedding. Specificallyy, we take the responses with certain attribute in the dev set of the dataset, feed them into the model and average attribute context embeddings at each decoder token as sentence-level representations, and pair them with the sentence-levé attribute annotations for analysis. For brevity, only show the visualization with emotion labels in Figure 4, and provide those with gender and question act labels in Appendix. We can see that the context embeddings from sentences with different emotions are clearly separated in the space, which supports the strong controllability of
Rerank
17.55
17.00
CTRL
21.07
43.38
1.91
3.00
Director
34.73
61.88
1.62
2.27
DASC
26.71
65.38
2.08
2.82
Table 3: Robustness test results.
Table 3 shows the robustness test results. 6 Com- pared with Table 1, we can see that the emotion ac-
*We do not change these attributes as they are sometimes determined given the context.
"BertScore is not reported here, as the model can be di- rected towards attributes different from the ground truth, inval- idating the similarity-hasee metric as proxy for generation quality.

---

(p)etts)
Cegp mssed) < senserttt
(bandkoo))
[I (saserable c)
(1a
(Olgnesso)
(1
her)
Chose
(ststee)
01
(Ferml
(rai
wil
(hatha)
(hane
(pretty)
(Hb2t)
(sister)
n (brother) (Tumbson))
Cander Bords
Gender Mards
(nai e) di (femlle)
Inution Nards
Caetier Nords
Garalien Nendd
Qurn Porks
Figure 3: Comparison of two sets of token embeddings with I-SNE visualization: those from the language model (left) and from the attribute semantic space (right).
Method
#params
BScore
Avg Acc
Emction
baseline
68 18
61.47
DASC (p=512)
15.94M
70.18
92.56
DASC 1024)
31.88M
70.12
92.72
DASC 2048)
63.75M
70.42
93,97
DASC (p=4096)
127.50M
70.26
94,42
Director
210.98M
69.54
94,14
onger
Table 4: Effect the number of extra parameters for controllability and generation quality.
model (210.98M vs. 116.26M), may have been preemerramteri and thus harms its generation quality. A moderate size of DASC can achieve the best BertScore, but smaller ones do not signifi- cantly degrade the performance. This suggests tha DASC can be a promising candidate in application. given its parameters are fewer than alternatives and are orders of magnitude fewer than LLMs that has generally over 6B parameters.
Figure 4: The t-SNE visualization of attribute context embedding responses with different emotions.
DASC with multiple attributes.
3.5 Parameter Analysis
As analyzed before, DASC can use a rela- tively smaller amount of parameters to implement weighted decoding for multi-attribute controllable generation. Here we study the effect of number of parameters by adjusting the dimension of the attribute space p, and comparing with baseline and M-Director which uses no/large amount of parame- for attribute control. We BertScore eval- uate the generation quality and the average control accuracy on aspects to reflect controllability.
3.6 Case Study
Besides multi-aspect control as shown in Figure 1, we also show a proof-of-concept application that DASC can naturally blend two emotions in one generated response. We can simply achieve this by setting both attributes' value I instead of d. The results are shown in Figure and Figure 8. We see that DASC can successfully generate responses with either single emotion or the combination of both emotions, where the later can produce poten- tially more vivid response.
Results are shown in Table 4. Comparing DASC with different p.
see that larger amount of parameters generally improve the model's con- trollability, but even a relatively small p (p=512) is already capable achieve high control accuracy. For generation quality, Director, which addition- ally uses nearly twice the parameters of the base
3.7 ESCony Experiment
To further explore the potential of DASC, we also experimented on another dataset ESConv (Liu al., 2021b). is an English dataset that aims to pro-

---

Context (Last Rounds);
grate attribute-related trainable parameters into the generation model for fine-tuning, such as discrete control codes (Keskar et al., 2019) or continuous prompt prefix (Qian et al., 2022). Weighted De- coding methods leverage token-level attribute clas- sifiers to guide each decoding step. For example, Krause et al. (2021) and Liu et al. (2021a) utilized one/two additional class conditional language mod- els to provide the attribute discrimination. Director (Arora et al., 2022) integrates the attribute classifier as simple linear layers on top of LM hidden states.
A: Hello, nice to meet you, Aren't you at wark?
I'm ordinary actor. I don't have shoot these days.
Desired Attributes:
Gender: Neutral | Non-Question
DASC with different emotions:
Surprise: Wow. thought you were big star.
Like: That's admirable! I'm still at school.
Surprise + Like: Wow, that's awesome! used to be an actor too, but now I'm just an ordinary white-collar worker.
Multi-attribute controllable generation is rela- tively under-explored now. Lin and Riedl (2021) proposed to extend weighted decoding for the case with the simple product of multiple attribute conditional language models. Gu et al. (2022) proposed VAE-based method combined with an iitersection-searchinn algorithm for multi-aspect controllable generation, but their method cannot simply apply to conditional genera- tion tasks like dialogue generation. Mireshghallah et al. (2022) proposed an energy-based controllable generation method that can combines multiple con- trols, but are mainly suitable for fixed-length gen- eration.
Figure 5: DASC generates different responses the same context given different emotions and their compo- sition as control attributes.
vide emotional supports to help seekers with 8 de- fined strategies. Here we use the human annotated strategy labels the control attributes, and exper- imented with 3 methods: Baseline, CTRL and DASC. We excluded Director here for its ineffi- ciency. We report the automatic metric Distinct- 2 and human evaluated Strategy Accuracy, Use- In Table 5,
fulness(1-4) and Sensibleness we can see that the control of relatively complex strategies is harder, and thus the accuracy is lower than the previous experiment (Table 2). Neverthe- less, DASC still achieves reasonable control accu- racy and outperforms other methods on all met- rics. These results suggest that DASC is language- agnostic and can be effectively applied to many kinds of attribute controls. We provide more de- tails and generation examples in Appendix.
Controllable generation techniques are espe- cially important in dialogue systems and the ap- plications of several controlling aspects have been studied. For example, we may condition the gener- ation with dialogue acts for the genuine reflection of the desired behavior (Wen et al., 2015), add emo- tions in the response enhance the expressiveness of the bot (Zhou et al., 2018), and also impose per- sonal profiles like gender (Su et al., 2020) and per- sona (Zhang et al., 2018) to establish human- like companion. Recent advance in LLMs has pushed the frontiers of dialog generation, enabling appli- cations like role-playing with complex personality and memory (Park et al., 2023). However, their exorbitant cost and privacy concerns make them less relevant in certain deployment scenarios.
Dist-2
Acc
Useful
Sensible
Baseline 19.28
0.27
1.92
3.30
CTRL
21.20
0.52
2,04
3.31
DASC
25.86
0.70
2.24
3.48
Table 5: Test results on ESConv.
4 Related Work
Controllable generation has gained wide research interest recently. PPLM (Dathathri et al., 2020) proposed plug-and-play framework to control the generation with an extra attribute classifier. Later research progress be roughly divided into 3 categories. Reranking methods leverage attribute classifiers to either simply rank the full genera- tion candidates (Thoppilan et al., 2022), or par- tial generations for the guidance of future outputs (Yang and Klein, 2021). Integrated methods inte-
Another line of controllable generation utilizes dense persona descriptions (Zhang et al., 2018). This paradigm is capable of expressing rich per- sona information in free text, such as personal sta- tus, hobbies and occupations. The natural language form allows for integration of other language re- sources for an enhanced generation quality. For example, Song et al. (2021) disentangles the task of persona consistency learning and response gen- eration, and leverages non-dialogue NLI datasets

---

help the former and consequently enhance the latter. However, although attributes can also be expressed in free-text descriptions, they can con- tain noise, and making them less effective than attribute-ppecffic methods, as shown in our exper- iments (Appendix C). It would be promising to further combine the two paradigms for more gen- eral controllable generation (Tang et al., 2023).
other techniques like concatenating the descriptions to achieve this goal, which will require further ex- plorations.
Ethics Statement
The proposed method utilized for the control of gender style. As we ve noticed and discussed in Sec. 3.2, the model may resort to gender stereo- types for generating responses in that gender. The potential reason is that the dataset used to train the classifier already contains gender-biased labels, and such biases are exploited by the classifier, and passed to the generation model through the auto- matic annotated labels. To avoid such effects, we may carefully clean the dataset for such biased la- bels (Gehman et al., 2020), or mine such biased tokens and penalize them during weighted decod- ing. We may also apply RI.FF to further mitigate the biases (Ouyang et al., 2022).
5 Conclusion
In this paper, we propose DASC, novel frame- work for multi-attribute controllable dialogue gen- eration. It is established on the weighted decod- ing paradigm for strong controllability and further grounds it in an attribute semantic space, which enables the simultaneous control of multiple at- tributes with the interpolation of multiple attribute embeddings. Experiments show that DASC can achieve strong controllability for multi-attributt generation while also preserving high quality in out-of-distribution scenarios. DASC is highly effi- cient given its much fewer number of parameters than alternatives and LLMs, making an promis- ing choice for deployment.
Though the proposed method is mainly intended for improving the interestingness of the chatbot, and endowing the model with abilities like emo- tional support, such method may also be applied for vicious application. For example, they may in- troduce toxicity as an attribute and encourage the model to generate more toxic responses. Therefore, the application range of such techniques should be carefully restricted.
Limitations
Some limitations of the proposed methods remain to be addressed in future research.
First, our experiment settings assume that the de- sired attributes are available for generation, which would require a separate dialogue policy to decide the attribute label provided the model. Therefore, our model cannot be directly applied to end-to-end dialogue models, and may also be affected by the potential error propagation from the dialogue pol- icy model. Since the intended use of DASC is to serve as a component of pipeline-style dialogue systems, these common issues in such systems are out of the scope of this work.
We adhere to the license of the used datasets.
References
Kushal Arora, Kurt Shuster, Sainbayar Sukhbaatar, and Jason Weston. 2022. Director: Generator classifiers classfiors for supervised language modeling. In Proceedings of the 2nd Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 12th International Joini Conference on Natural Langugg Processing, pages 512-526.
Tolga Bolukbasi, Kai- Wei Chang, James Y. Zou. Venkatesh Saligrama, and Adam Tauman Kalai. 2016. Man is to computer programmer as woman is to homemaker? debiasing word embeddings. In Ad- vances in Neural Information Processing Systems 29: Annual Conference on Neural Information Process- ing Systems 2016, December 5-10, 2016, Barcelona, Spain, pages
Moreover, we require annotated datasets with multiple attributes for evaluation, which are rare. Therefore, we evaluate the capability of multi- attribute control mostly on one dataset. Experi- ments on more datasets, especially those with ad- ditional attributes may be required to further vali- dated the efficacy of the proposed methods.
Last but not least, DASC is not directly appli- cable for controllable generation with free text as control signal, such as persona descriptions (Zhang et al., 2018), which might limit its application range, though we may simply combine DASC with
Yiming Cui, Wanxiang Che, Ting Liu, Bing Qin, Shijin Wang, and Guoping Hu. 2020. Revisiting pre- pre-trained models for Chinese natural language processing. In Findings of the Association for Computational Lin- guistics: EMNLP 2020, pages 657-668, Online. As- sociation for Computational Linguistics.

---

Sumanth Dathathri, Andrea Madotto, Janice Lan, Jane Hung, Eric Frank. Piero Molino, Jason Yosinski, and Rosanne Liu. 2020. Plug and play language models: A simple approach controlled text generation. In 8th International Conference on Learning Represen- lations, ICLR 2020, Addis Abubu, Ethiopia, April 26-30, 2020. OpenReview.net.
Alisa Liu, Maarten Sap. Ximing Lu, Swabha Swayamdipta. Chandra Bhagavatula, Noah A. Smith, and Yejin Choi. 2021a. DExperts: Decoding-time controlled text generation with experts and anti- experts. In Proceedings of the 59th Annual Meet- ing of the Association for Computational Linguistics and the 1 1ith International Joint Conference on Natu- ral Language Processing (Volume I: Long Papers). pages 6691-67000 Online. Association for Computa- tional Linguistics.
Samuel Gehman, Suchin Gururangan, Maarten Sap, Yejin Choi, and Noah A. Smith. 2020. RealToox- cityPrompts: Evaluating neural toxic degeneration language models. In Findings of the Association for Computational Linguistics: EMNLP 2020, pages Online. Association for Computational Linguisties.
Siyang Liu, Chujie Zheng, Orianna Demasi, Sahand Sabour, Yu Li, Zhou Yu, Yong Jiang, and Minlie Huang. 2021b. Towards emotional support dialog systems. In Proceedings of the 59th Anmual Meet- ing of the Association for Computational Linguistics and the 1 11th International Joint Conference on Natu- ral Language Processing (Volume I: Long Papers), pages 3469-3483 Online. Association for Computa- tional Linguistics.
Yuxuan Gu, Xiaocheng Feng. Sicheng Ma, Lingyuan Zhang, Heng Gong, and Bing Qin. 2022. A distribu- tional lens for multi-aspect controllable text genera- tion. ArXiv preprint, abs/2210.00882
Ari Holtzman, Jan Buys, Li Du, Maxwell Forbes, and Yejin Choi. 2020. The curious case of neural text degeneration. 8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia. April 26-30, 2020. OpenReview.net.
Yizhu Liu, Qi Jia, and Kenny Q 7hu, 2022. Length control in abstractive summarization by pretraining information selection. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 6885- 6895, Dublin, Ireland. Association for Computational Linguistics.
Nitish Shirish Keskar, Bryan McCann, Lav Varshney, Caiming Xiong, and Richard Socher. 2019. Ctrl: A conditional transformer language model for control- lable generation. ArXiv preprins, habs/1999.0589
Fatemehsadat Mireshghallah, Kartik Goyal, and Taylor Berg-Kiin KirggKiikaatrick. 2022. Mix and match: Learning- free controllable text generationusing energy lan- guage models. In Proceedings of the 60th Annual Meeting of the Association for Computational Lin- guistics (Volume 1: Long Pupers), pages 401-415, Dublin, Ireland. Association for Computational Lin- guistics.
Ben Krause, Akhilesh Deepak Gotmare, Bryan McCann, Nitish Shirish Keskar, Shafiq Joty, Richard Socher. and Nazneen Fatema Rajani. 2021. GeDi: Gener- ative discriminator guided sequence generation. In Findings of the Association for Computational Lin- guistics: EMNLP 2021, pages 4929-4952, Punta Cana, Dominican Republic, Association for Compu- tational Linguistics.
OpenAl. 2022. Introducing ChatGPT. https:// openai. citiaaa.commboog//hcee com/blog/chatgpt.
Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal. Katarina Slama, Alex Ray, al. 2022. Training language models to follow instruc- tions with human feedback. Advances in Neural Information Processing Systems, 35:27730-277 27744..
Mike Lewis, Yinhan Liu, Naman Goyal. Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Veselin Stoyanov, and Luke Zettlemoyer. 2020. BART: Denoising scquence-to-sequeee toquuuecceeeeeeeeeeeeeeeeeeeeeee pre-training for natural language generation, translation, and com- prehension. In Proceedings of the 58th Annual Meet- ing of the Association for Computational Linguistics, pages 7871-7880, Online. Association for Computa- tional Linguistics.
Joon Sung Park, Joseph CO'Brien, Carrie J Cai, Mered- ith Ringel Morris, Percy Liang, and Michael S Bernstein. 2023. Generative agents: Interactive simulacra of human behavior. arXiv preprint 23X1v:2304.03442.
Jiwei Li. Michel Galley, Chris Brockett, Jianfeng Gao, and Bill Dolan. 2016. A diversit ob- jective function for neural conversation models. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computa- tional Linguistics: Human Language Technologies, pages 110-119, San Diego, California. Association for Computational Linguistics.
Jing Qian, Li Dong, Yelong Shen, Furu Wei, and Weizhu Chen. 2022. Controllable natural language genera- tion with contrastive prefixes. In Findings of the As- sociation for Computational Linguistics: ACI. 2022, pages 29 2912-2924, Dublin, Ireland, Association for Computational Linguistics.
Zhiyu Lin and Mark O Riedl. 2021. Plug-and aad-blend: a framework for plug-and-play controllable story gen- eration with sketches. In Proceedings of the AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment, volume 17, pages 58-65.
Yunfan Shao, Zhichao Geng, Yitao Liu, Junqi Dai. Fei Yang, Li Zhe, Hujun Bao, and Xipeng Qiu. 2021. Cpt: A pre-trained unbalanced transformer for both chinese language understanding and genera- tion. arXiv preprini arXiv:2109.055799

---

Haoyu Song, Yan Wang. Kaiyan Zhang, Wei-Nan Zhang, and Ting Liu. 2021. Bob: Bert over bert for training persona-based dialogue models from limited personalized data. arXiv preprint arXiv: 200::2006000 06169.
Saizheng Zhang, Emily Dinan, Jack Urbanek, Arthur Szlam, Douwe Kiela, and Jason Weston. 2018. Per- sonalizing dialogue agents: have dog. do you have pets too? In Proceedings of the 56th Annual Meeting of the Association for Computational Lin- guistics (Volume I: Long Pupers), pages 2204-2213, Melbourne, Australia. Association for Computational Linguistics.
Yixuan Su, Deng Cai, Yan Wang. Simon Baker, Anna Korhonen, Nigel Collier, and Xiaojiang Liu. 2020. Stylistic dialogue generation via information-guider reinforcement learning strategy. ArXiv preprint, abs/2004.02202.
Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q. Weinherger, and Yoav Artzi. 2020. Bertscore: Evalu- ating text generation with BERT. In 8th International Conference on Leurning Representations ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020. OpenRe- view.net.
Yihong Tang, Bo Wang, Miao Fang, Dongming Zhao, Kun Huang. Ruifang He, and Yuexian Hou. 2023. En- hancing personalized dialogue generation with con- trastive latent variables: Combining sparse and dense persona. arXiv preprint wrXiv:2305.11493
Hao Zhou, Minlie Huang, Tianyang Zhang, Xiaoyan Zhu, and Bing Liu. 2018. Emotional chatting ma- chine: Emotional conversation generation with in- ternal and external memory. In Proceedings of the Thirty-Second AAAI Conference on Artificial Intelli- gence, (AAAI-18), the 30th innovative Applications of Artificial Intelligence (IAAI-18 18), and the 8th AAAI Symposium Educational Advances in Artificial In- telligence (EAAI-18), New Orleans, Louisiana, USA, February 2-7, 2018, pages 730-739. AAAI Press.
Romal Thoppilan, Daniel De Freitas, Jamie Hall, Noam Shazeer. Apoorv Kulshreshtha, Heng-TZe Cheng. Alicia Jin, Taylor Bos, Leslie Baker, Yu Du, et al. 2022. Lamda: Language models for dialog applica- tions. ArXiv preprint, abs/2201.085 008299.
Laurens Van der Maaten and Geoffrey Hinton. 2008 Visualizing data using t-sne. Journal of machine learning research, 9(11).
Tsung-Hsien Wen, Milica Gašić, Nikola Mrkšić, Pei-
Hao Su, David Vandyke, and Steve Young. 2015. Semantically conditioned LSTM-based natural lan- guage generation for spoken dialogue systems. In Proceedings of the 2015 Conference Empirical Methods in Natural Language Processing. pages 1711-1721, Lisbon, Portugal. Association for Com- putational Linguistics.
A
Effect of Control Strength
100
90
80
70-
Gender Acc
Emation Acc
Question Acc
Liang Xu, Hai Hu, Xuanwei Zhang, Lu Li, Chenjie Cao. Yudong Li, Yechen Xu, Kai Sun, Dian Yu, Cong Yu, Yin Tian, Qianqian Dong. Weitang Liu, Bo Shi, Yiming Cui, Junyi Li, Jun Zeng, Rongzhao Wang, Weijian Xie, Yanting Li, Yina Patterson, Zuoyu Tian, Yiwen Zhang, He Zhou, Shaoweihua Liu, Zhe Zhao, Qipeng Zhao, Cong Yue, Xinrui Zhang. Zhengliang Yang, Kyle Richardson, and Zhenzhong Lan. 2020. CLUE: A Chinese language understanding evalua- tion benchmark. In Proceedings of the 28th Inter- national Conference on Computational Linguistics, pages 4762-4772, Barcelona, Spain (Online). Inter- national Committee on Computational Linguistics.
70-
BertScore
69
67-
0.0
0.5
1.0
2.0
4.0
Strenghh
Figure 6: Effect of control strength on controllabilityy and generation quality
We show the effect of control strength a (Eq. (6)) on DASC's controllability and generation quality in Figure 6. From the trend shown in this figure, we can see that for Question which is easy to control, we can already achieve perfect control with a low o, while harder attributes like Emotion would require a higher a to get high success rate. Therefore, we may further hypothesize that better balancing of the control accuracy of each attribute and the gen- eration quality can be achieved by setting different control strengths for each aspect, like higher a for Emotion and lower a for Question. Careful tuning of the parameters or specific searching algorithms
Xinchao Xu, Zhibin Gou, Wenquan Wu, Zheng- Yu Niu, Hua Wu, Haifeng Wang, and Shihang Wang. 2022. Long time see! open-domain conversation with long-term persona memory. In Findings of the As- sociation for Computational Linguistics: ACL 2022, pages 2639-2650, Dublin, Ireland. Association for Computational Linguistics.
Kevin Yang and Dan Klein. 2021. FUDGE: Controlled text generation with future discriminators. In Pro- ceedings of the 2021 Conference of the North Amer- ican Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 3511 1-355,,, Online. Association for Computational Linguistics.

---

(Gu et al., 2022) may serve the goal, and we leave this for future work.?
BoB (Song et al., 2021) disentangles the task of persona consistency learning and response gener- ation, and leverages non-dialogue NLI datasets to help boost the performance of consistency learning and finally improves personalized generation.
B Experiment Details
For all experiment methods, they use bart-base the base model. All models are fine-tuned on the dataset for 6 epochs. When conducting multi-aspect control for Director and DASC under weighted-decoding paradigm (Éq. (4)), we set the variables for the desired attribute as 1, and other variables as Φb The decoding method is top-p sam- pling with p - 0.5. DASC uses control weight a = and classifier loss weight 3 = 0.1, similar the previous experiments. All experiments of the paper are conducted on Linux server, and each experiment is run on a single NVIDIA A100 GPU, To avoid overfitting, we select the checkpoint with the best BertScore on dev set for final testing. We fix the random seed in experiment and report the re- sults coming from a single run. Below we provide the specific details for the experiments on ESConv. For experiments ESConv (Liu et al., 2021b). we use the latest released version", which has 1,300 conversations, and we split them into 1, 1,100/100/100 train/dev/test set, which contains 15,605/1,403/1,55 utterances each. In human eval- uation, we further sample 15 utterances for each of the 7 emotional support strategies defined in the dataset (except the vague Other class), and get 105 utterances in total. The meaning of Sen- sibleness( 4) is similar to the experiment in the previous dataset: if the response is fluent, coherent with the context, and accords with commonsense. By (1) we consider the response dives deep in the problem faced by the support seeker, is comforting, contains useful suggestions or encourages in-depth further discussions.
To apply BoB on the dataset we've experimented with, we convert the discrete attribute annotations into textual descriptions with rules. For example. the male/female gender will be converted "I'm a girl/boy.". a question will have the description want to ask question". And for emotion, we fill them in the template "I feel (emotion). We con- catenate these 3 description texts as the persona text to be used by BoB. As is suggested in the official GitHub repository, we leverage the Chinese NLI dataset CMNLI (Xu et al., 2020) the auxiliary inference datasets.
ChatGPT (OpenAI, 2022) is a representative Lrgge-Langagge-oon (LLM) that can follow hu- man instruction and give responses. Therefore, we can encode the attribute values into the ChatGPT system message to achieve control on them. We use the template:
A dialog history is given below. Please act as the (current speaker) and respond to the next sentence with |dialog act] in the voice of (gender and emotion).
dialog history as the user message, and let ChatGPT (gpt-3.5-turbo) give a following sentence. We use temperature=0 and stop="'n", that is, greedy search for one line of text similar to the setting of the original dataset.
Then
BScore
Dist-2
Aceg
Acc
Acci
Baseline
68.18
19.25
68.49
46.31
69.61
CTRL
71.09
18.91
85.32
77.49
100,00
DASC
70.42
21.94
95.85
86.07
100,00
C Experiments with Description Control
BoB
65.47
23.44
74.59
64.76
98.51
ChatGPT
66.21
30.98
69.49
56.88
98.22
Dense persona descriptions are another common form of control signal in dialog generation (Zhang et al., 2018), and we can also convert the sparse at- tributes into descriptive texts to be compatible with these methods. Therefore, we now supplement new experiment results with two representative methods that leverages persona descriptions for control.
Table 6: Automatic evaluation on the DuLemon test set with persona description- controlling methods.
Dist-2
Accg
CTRL
21.07
43,38
DASC
26.71
65.38
BoB
24.25
30.13
'In practice, we further multiply a by the number of at- tributes K to adapt the variable attribute numbers, which not counted in Eq. (6) and Figure 6 for clarity.
ChatGPT
37.02
30.00
Table Automatic evaluation on the DuLemon ro- bustness test with persona description-based controlling methods.
'ht tps: phttts/://ugee htt.g//gggggg.....e/oe/o wwwee.ooffccccoe waafebbboo/bbarrr t-ase
//gi hat.l..cc//tttiae cww/thu-cceeeeeecccep /
Emotional- Support SampporttCooversattinn Conversation

---

Firstly, we can see that ChatGPT can generate highly diverse texts (with dist-2 on similar as the human ground truth, 30.98 VS 30.63). How- ever, ChatGPT cannot achieve comparable con- trollability with finetuned methods like CTRL. and DASC, especially on attributes whose manifesta- tion cannot be easily described in the prompt (e.g. Gender and Emotion).
Context (Last Rounds):
A: OO OV ボ HAREE
Desired Attributes:
Gender: Neutral
DASC with different emotions:
Surprise: 1
Moreover, we can see that BoB also shows strong controllability compared to baseline and zero-shot ChatGPT. It also has higher generation diversity than other methods except ChatGPT, po- tentially due to the introduction of the auxiliary inference dataset. However, its controllability is relatively worse than DASC. Its generation qual- ity is also poor, both reflected in the low BertScore and our manual check, where we find many influent cases like "Yes. After all, I'm a lawyer, otherwise it would hard to find a body.". BoB also shows less sensitivity to the change of control signals, as is shown in the lower dist-2 and emotion accuracy in the robustness test.
Like: A AOT -.
Surprise+ Like: i 7 AR RitER -M
Figure 7: Original Chinese text for Figure
Context (Last Rounds):
A: 1 (What movies do you like to watch?) B: f
* (1 prefer
to watch horror movies. Do you like them?)
Desired Attributes:
Gender: Neutral Non-Question
To conclude, we believe description-based controllable generation methods like ChatGPT and BoB to perform better in the control of discrete attributes, but it would require significant efforts on prompt engineering (e.g. de- scribe in more detail for ChatGPT, or make the description more similar to the auxiliary inference dataset for BoB). However, when we have rela- tively sufficient labeled data the discrete control attributes, DASC on 'small' LMs will certainly be a simple and competitive choice.
possible for
DASC with different emotions:
Fear: ,,, = (1
don't like them. think they're too scary.)
Like: #: (I also like watch horror movies, think is very exciting.)
Fear+Like:
also
like them, but I'm too scared to watch them.)
Figure Example DASC composing Like and Fear emotion in the generated response.
D Examples and Visualizations
In this section, we provide supplementary examples and figure visualizations.
and DASC successfully applied the desired strat- egy, while CTRL failed to do so. However, baseline also included repetitive question at the end, while DASC gives more comprehensive restatement, which exhibits a deep understanding of the situa- tion and will be regarded as more helpful for the help seeker. In Figure 12, only DASC used the correct strategy in the generation, and such precise reflection of the anxious mood makes the response more sympathetic.
Figure shows the original Chinese text for the emotion composition example in Figure 5, and we provide another example in Figure 8, which shows that DASC can compose positive emotion Like and negative emotion Fear in the same re- sponse express complex meanings.
We provide the t-SNE visualizations of attribute context embeddings sentences with different Gender Style and Question Act in Figure and Figure 10. We find similar results as we' ve seen for emotion, that the embeddings from different attributes are clearly separated.
We also show examples of the generated re- sults on the ESCony. In Figure 11, both baseline

---

Gender neutral
Figure 9: Visualization of attribute context embedding responses with different gender styles.
Context (Last Rounds):
true
A: What's your mind? have real bad anxiety about the possible 2nd shut down due to the rise in Covid. B: So many people lost their job during the first shut down
Strategy:
[Reflection of feelings]
Responses:
Baseline: I'm so sorry to hear that. Have you tried to talk to your boss about it?
CTRL: I'm sorry to hear that. I'm sure you will get through this.
Figure 10: Visualization of attribute context embedding
responses with different question acts.
DASC: can understand how you are feeling, I've been in this position myself and can tell you are very anxious.
Context (Last Rounds):
Figure System generations one example of ES- Cony, with the "Reflection of feelings" strategy.
A: Glad to know that you are doing well. How may assist you today? B: had a drinking challenge which has been affecting marriage for quit sometime now
Strategy: [Restatement or Paraphrasing]
Responses:
Baseline: am sorry to hear that you are having drinking challenge. How may assist you today?
CTRL: sorry hear that. have been in similar situation before.
see. So you are having drinking challenge and you are struggling get your marriage work?
DASC:
Figure 1I:
System generations in example of ESS- Cony, with the "Restatement or Paraphrasing" strategy.


Reviews:
[{"review_id": "TIR16aT1ot", "reviewer": "Reviewer_kcyB", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 3, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for your suggestions.\n\nAs for the lack of baselines. We didn't experiment with baselines like PPLM because they are initially designed for single-attribute control and would be very costly to extend to multi-attribute cases, requiring multiple classifiers/gradient backward passes. According to previous works, their controllability is also generally worse than DIRECTOR, our major baseline. \n\nWe agree that examining the performance of recent PLMs on this task would be interesting. Therefore, we now supplement new experiment results with ChatGPT(gpt-3.5-turbo) on the DuLemon dataset. We will show the results first, and give the experimental details at the bottom.\n\n*Since it is difficult to make fair comparion with prior results using human annotated scores, we just provide the automatic metrics. According to the suggestion of reviewer yeTY, we also added another baseline BoB, which is a representative method with finetuned model that leveraged persona description text for controllable generation. \n\nResults on the test set (ref Table 1).\n|          |   BScore  |   Dist-2  |   Acc_G   |   Acc_E   |    Acc_Q   |\n|:--------:|:---------:|:---------:|:---------:|:---------:|:----------:|\n| Baseline |   68.18   |   19.25   |   68.49   |   46.31   |    69.61   |\n|   CTRL   | **71.09** |   18.91   |   85.32   |   77.49   | **100.00** |\n|   DASC   |   70.42   |   21.94   | **95.85** | **86.07** | **100.00** |\n|   BoB    |   65.47   |   23.44   |   74.59   |   64.76   |    98.51   |\n|  ChatGPT |   66.21   | **30.98** |   69.49   |   56.88   |    98.22   |\n\nRobustness test results (ref Table 3).\n|         |   Dist-2  |   Acc_E   |\n|--------:|:---------:|:---------:|\n|    CTRL |   21.07   |   43.38   |\n|    DASC |   26.71   | **65.38** |\n|     BoB |   24.25   |   30.13   |\n| ChatGPT | **37.02** |   30.00   |\n\nWe can see that ChatGPT can generate highly diverse texts. However, ChatGPT cannot achieve comparable controllability with finetuned methods like CTRL and DASC, especially on attributes whose manifestation cannot be easily described in the prompt (e.g. Gender and Emotion). \n\nTherefore, we may conclude that when we have relatively sufficient labeled data, DASC on 'small' LMs can be a competitive option compared with LLMs like ChatGPT which can not be easily tuned.\n\n---\n\n*Experimental Details*: Specifically, we encode the attribute values into the ChatGPT system message with the template following the popular role play format (ref. https://github.com/f/awesome-chatgpt-prompts): \n```\nf\"A dialog history is given below. Please act as the {current speaker} and respond to the next sentence with a {dialog act} in the voice of {gender and emotion}.\"\n```\nThen we use dialog history as the user message, and let ChatGPT give a following sentence. We use temperature=0 and stop=\"\\n\", that is, greedy search for one line of text, similar to the setting of the original dataset.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. The paper studies an important multi-attribute controllable dialogue generation task and proposes an intuitive weighted decoding method.\n2. The analysis is comprehensive and instructive.", "reasons_to_reject": "lack of some important baselines, such as FUDGE, PPLM,  Cocon, and recent PLMs.", "questions_to_authors": "", "paper_topic_and_main_contributions": "This paper proposes a framework for multi-attribute controllable dialogue generation. It uses the weighted decoding paradigm for strong controllability and further grounds it in an attribute semantic space.", "excitement": 3, "reproducibility": 5, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "jkqbgxrFY4", "reviewer": "Reviewer_fjTP", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for your constructive reviews.\n\n### 1. The experiment section could be improved. For example, it is better to carry significance test on the human evaluation results. It is also beneficial to compare the proposed method with some most recent LLM.\n\nWe agree that examining the performance of the most recent LLM on this task would be interesting. Therefore, we now supplement new experiment results with ChatGPT(gpt-3.5-turbo) on the DuLemon dataset. We will show the results first, and give the experimental details at the bottom.\n\n*Since it is difficult to make fair comparion with prior results using human annotated scores, we just provide the automatic metrics. According to the suggestion of reviewer yeTY, we also added another baseline BoB, which is a representative method with finetuned model that leveraged persona description text for controllable generation. \n\nResults on the test set (ref Table 1).\n|          |   BScore  |   Dist-2  |   Acc_G   |   Acc_E   |    Acc_Q   |\n|:--------:|:---------:|:---------:|:---------:|:---------:|:----------:|\n| Baseline |   68.18   |   19.25   |   68.49   |   46.31   |    69.61   |\n|   CTRL   | **71.09** |   18.91   |   85.32   |   77.49   | **100.00** |\n|   DASC   |   70.42   |   21.94   | **95.85** | **86.07** | **100.00** |\n|   BoB    |   65.47   |   23.44   |   74.59   |   64.76   |    98.51   |\n|  ChatGPT |   66.21   | **30.98** |   69.49   |   56.88   |    98.22   |\n\nRobustness test results (ref Table 3).\n|         |   Dist-2  |   Acc_E   |\n|--------:|:---------:|:---------:|\n|    CTRL |   21.07   |   43.38   |\n|    DASC |   26.71   | **65.38** |\n|     BoB |   24.25   |   30.13   |\n| ChatGPT | **37.02** |   30.00   |\n\nWe can see that ChatGPT can generate highly diverse texts. However, ChatGPT cannot achieve comparable controllability with finetuned methods like CTRL and DASC, especially on attributes whose manifestation cannot be easily described in the prompt (e.g. Gender and Emotion). \n\nTherefore, we may conclude that when we have relatively sufficient labeled data, DASC on 'small' LMs can be a competitive option compared with LLMs like ChatGPT that can not be easily tuned.\n\n---\n\n*Experimental Details*: Specifically, we encode the attribute values into the ChatGPT system message with the template following the popular role play format (ref. https://github.com/f/awesome-chatgpt-prompts): \n```\nf\"A dialog history is given below. Please act as the {current speaker} and respond to the next sentence with a {dialog act} in the voice of {gender and emotion}.\"\n```\nThen we use dialog history as the user message, and let ChatGPT give a following sentence. We use temperature=0 and stop=\"\\n\", that is, greedy search for one line of text, similar to the setting of the original dataset.\n\n### 2. The classifier of determining attributes using only parts of the sentence may not perform well. Specifically, I am wondering what is the performance of the attribute classifer obtained using Eq.2 and Eq.7.\n\nThis is a good question. We didn't deliberately evaluate the accuracy for the attribute classifier as this is not our focus in this work, and such metric will be hard to define and implement. However, we do print out the in-batch prediction AUC (which doesn't require the tuning of threshold) during our experiments at the 1st, 3rd, 11th token. We observed that in most batches, the emotion attribute AUC at the 1st token is about 0.70, while at the 11th token, the AUC generally rises to > 0.9, and in many batches = 1.\n\nWe think that it is possible to further improve the model by aligning higher control weights on positions where the model is confident about its attribute prediction. However, we failed to achieve success in our preliminary attempts. We may explore further on this in future works.\n\n### 3. Some of the experiment results could be explained in more details. For example, the author observes that \"Compared to CTRL, DASC has lower Sensibleness but higher Interestingness\", but why? Is that because DASC is bad for exhibiting Sensibleness? Similar results are also observed in Table1.\n\nLet's explain this with an example \n```\nA: Hello, do you have time to chat? I'm a little bit sad.\nB:\nControl Attributes: Gender (female), Emotion (neutral), Question or Statement (statement)\nbaseline: I'm ok. What's wrong with you?\nDASC: My mom is a doctor, and my mom doesn't like me very much, and I want her to help her.\n```\nThere are some (but not many) cases like this, where DASC would sacrifice the generation's coherence with the context for a better exploitation of control attributes. In this example, DASC introduced `mom` (a female word), and repeatedly used the word `her` to meet the requirement of generation with the female attribute.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1.  An interesing task is investigated.\n\n2. An effective method is proposed. The experiments results show that the proposed method help to enhance the controlability of different attributes.", "reasons_to_reject": "1. The experiment section could be improved. For example, it is better to carry significance test on the human evaluation results. It is also beneficial to compare the proposed method with some most recent LLM.\n\n2. The classifier of determining attributes using only parts of the sentence may not perform well. Specifically, I am wondering what is the performance of the attribute classifer obtained using Eq.2 and Eq.7.\n\n3. Some of the experiment results could be explained in more details. For example, the author observes that \"Compared to CTRL, DASC has lower Sensibleness but higher Interestingness\", but why? Is that because DASC is bad for exhibiting Sensibleness? Similar results are also observed in Table1.", "questions_to_authors": "", "paper_topic_and_main_contributions": "This paper proposes a controllable dialogue generation model. Specifically, it aims to control multiple attributes of the generated dialogue responses. The author introduces a method named Dialog Attribute Space Controller (DASC). They try to directly modify the logits of the generated sequences with some attribute classifiers and some soft labels produced by the similarity between the target attribute and next tokens. Evaluation results show that the proposed method can control the attribute of the generated responses.", "excitement": 3, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "3wyHs9F0ja", "reviewer": "Reviewer_yeTY", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 3, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for providing a broader view on controllable generation by providing more related works like BoB. We are aware of this excellent work that leverage textual persona description for controllability, but our work focuses on sparse persona attributes, which makes it not easy to establish a fair comparison. It would certainly be interesting to compare the two lines of works. Therefore, we present our trial on comparing BoB with our methods. We will show the results first, and give the experimental details at the bottom.\n\n*Since it is difficult to make fair comparison with prior results using human annotated scores, we just provide the automatic metrics. According to the suggestion of other reviewers, we also added another baseline ChatGPT, where we describe the attributes in prompt for zero-shot generation. \n\nResults on the test set (ref Table 1).\n\n|          |   BScore  |   Dist-2  |   Acc_G   |   Acc_E   |    Acc_Q   |\n|:--------:|:---------:|:---------:|:---------:|:---------:|:----------:|\n| Baseline |   68.18   |   19.25   |   68.49   |   46.31   |    69.61   |\n|   CTRL   | **71.09** |   18.91   |   85.32   |   77.49   | **100.00** |\n|   DASC   |   70.42   |   21.94   | **95.85** | **86.07** | **100.00** |\n|   BoB    |   65.47   |   23.44   |   74.59   |   64.76   |    98.51   |\n|  ChatGPT |   66.21   | **30.98** |   69.49   |   56.88   |    98.22   |\n\nRobustness test results (ref Table 3).\n|         |   Dist-2  |   Acc_E   |\n|--------:|:---------:|:---------:|\n|    CTRL |   21.07   |   43.38   |\n|    DASC |   26.71   | **65.38** |\n|     BoB |   24.25   |   30.13   |\n| ChatGPT | **37.02** |   30.00   |\n\nWe can see that BoB also shows strong controllablity compared to baseline and zero-shot ChatGPT. It also has higher generation diversity than other methods except ChatGPT, potentially due to the introduction of the auxiliary inference dataset. However, its controllablity is relatively worse than DASC. Its generation quality is also poor, both reflected in the low BertScore and our manual check, where we find many influent cases like \"Yes. After all, I'm a lawyer, otherwise it would be hard to find a body.\"\n\nBoB also shows less sensitivity to the change of control signals, as is shown in the lower dist-2 and emotion accuracy in the robustness test. \n\nTo conclude, we believe it is possible for text-based controllable generation methods like ChatGPT and BoB to perform better in the control of discrete attributes, but it would require significant efforts on prompt engineering. However, when the discrete control attributes are already available, DASC will certainly be a simple and competitive choice.\n\nWe will also add more discussions about this line of work like (Tang et al., ACL 2023) in related works.\n\n---\n\n*Experimental Details*: To apply BoB on the dataset we've experimented with, we convert the discrete attribute annotations into textual descriptions with rules. For example, gender=female will be converted to \"I'm a girl.\", a question will have the description \"I want to ask a question\". And for emotion, we fill them in the template \"I feel {emotion}.\" We concatenate these 3 description texts as the persona text to be used by BoB. We leverage the NLI dataset CMNLI as the auxiliary inference datasets.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "Different from previous work, the author deals with the attributes of characters separately and turns it into a multi-attribute controllable dialogue generation task, which enables us to explore how different attributes affect dialogue generation, and the author makes this process very clear, which is very helpful for subsequent work.", "reasons_to_reject": "The author has little exploration of the previous works. The experiment is not comprehensive enough. Most of the comparison of the author's experimental results is based on the accuracy of attribute prediction. Still, some models do not adopt this method to deal with attributes (such as overall processing) and can obtain good generated results, and these models have not been compared.", "questions_to_authors": "", "paper_topic_and_main_contributions": "The authors aim to generate personalized conversations (gender, mood, etc.) through techniques for controlled conversation generation. They proposed DASC(Dialog Attribute Space Controller) to control multiple attributes at the same time to generate personalized replies, that is, multi-attribute controlled conversation generation. The author proposes a new weighted decoding method. Each lexeme in the vocabulary is mapped to an Attribute semantic space through the Attribute Token Embedding. The hidden state of the language model is also transformed into the Attribute Context Embedding in the space through the attribute-specific layer. Then, a dot-product-based attribute classifier assigns higher weights to adjacent words embedded in the current context in the attribute space, thus achieving weighted control in the decoding process. Experiments on open domain dialogue datasets show that DASC achieves strong controllability in combinatorial control tasks, the visualization of attribute lexical embedment clearly shows the specific pattern related to control, and the controllability of DASC for a single attribute is also guaranteed.", "excitement": 4, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {"Missing_References": "Many models do not get a chance to compare, such as BoB: BERT Over BERT for Training Persona-based Dialogue Models from Limited Personalized Data (Song et al., ACL-IJCNLP 2021).\n\nThis work Enhances Personalized Dialogue Generation with Contrastive Latent Variables: Combining Sparse and Dense Persona (Tang et al., ACL 2023) is similar to the author's work, except that it addresses \"multi-attribute controlled conversation generation\" in text descriptions, which the authors can try to focus on. Of course, this is just advice, I wouldn't ask for it (in the three months before the deadline)."}}]

Author rebuttal:

Meta-review: The authors propose a new LLM decoding method that allows to control for multiple attributes. The reviewers raise concerns on comparisons with related works and ablation studies, to which the authors respond by conducting additional experiments, showing that their method outperforms the new baselines on most metrics. On the other hand, the reviewers appreciate the novelty and performance of the proposed method and for all the reasons above I would recommend accepting this paper.
Decision: Accept-Main