Miracle: Towards Personalized Dialogue Generation with Latent-Space Multiple Personal Attribute Control
Authors: Zhenyi Lu
Wei Wei
Xiaoye Qu
Xian-Ling Mao
Dangyang Chen
Jixiong Chen
Venue: EMNLP 2023 Findings (2023)
Keywords: personalized response generaion
dialogue
Natural language generation

Abstract: Personalized dialogue systems aim to endow the chatbot agent with more anthropomorphic traits for human-like interactions.
Previous approaches have explored explicitly user profile modeling using text descriptions, 
implicit derivation of user embeddings, or utilizing handicraft prompts for ChatGPT-like models. 
However, textual personas are limited in describing multi-faceted attributes (\emph{e.g.}, \emph{language style, inner character nuances}), implicit embedding suffers from personality sparsity, 
and handicraft prompts lack fine-grained and stable controllability. 
Hence, these approaches may struggle with complex personalized dialogue generation tasks that require generating controllable responses with multiple personal attributes.
To this end, we propose \textbf{\textsc{Miracle}}, a novel personalized dialogue generation method through \textbf{M}ult\textbf{I}ple Pe\textbf{R}sonal \textbf{A}ttributes \textbf{C}ontrol within \textbf{L}atent-Space \textbf{E}nergy-based Models.
ttributes \textbf{C}ontrol within \textbf{L}atent-Space \textbf{E}nergy-based Models.
Specifically, our approach first disentangles complex personality into multi-faceted attributes. 
Subsequently, we employ a conditional variational auto-encoder to align with the dense personalized responses within a latent joint attribute space.
We have also tailored a dedicated energy function and customized the ordinary differential equations sampling method to offer flexible attribute composition and precise attribute control.
Extensive experiments demonstrate that \textsc{Miracle} outperforms several strong baselines in terms of personality controllability and response generation quality.
Our dataset and code are available at \url{https://github.com/LZY-the-boys/MIRACLE}

MIRACLE: Towards Personalized Dialogue Generation with Latent-Space Multiple Personal Attribute Control
Zhenyi Lu Wei Weil2, Xiaoye Qu"?,
XianLing Mao², Dangyang Chen', .ixiong Chen
Cognitive Computing and Intelligent Information Processing (CCIIP) Laboratory, School of Computer Science and Technology, Huazhong University of Science and Technology 2 Joint Laboratory of HUST and Pingan Property & Casualty Research (HPL),
3 Beijing Institute of Technology,
Ping An Property & Casualty Insurance company of China, Ltd.,
Brilliance Technology Co. Ltd.
1uzheny1529@gmail. com, (weiw, quwiaoye Jehust edu. cn, maoxl@bit www.eeiii........ee..moee cn chendangyangg22$$pingaa com. cn, chenjixiongebrill
com. cn
Abstract
User query:
I work as Ibrarian. What do think about IT?
Personalized dialogue systems aim to endow the chatbot agent with more anthropomorphic traits for human-like interactions. Previous approaches have explored explicitly user pro- file modeling using text descriptions, implicit derivation of user emheddings, or utilizing handicraft prompts ChatGPT-like models. However, textual personas are limited in de- scribing multi-faceted attributes (e.g., language style, inner character nuances), implicit em- bedding suffers from personality sparsity, and handicraft prompts lack fine-grained and sta- ble controllability. Hence, these approaches may struggle with complex personalized di- alogue generation tasks that require generat- ing controllable responses with multiple per- sonal attributes. To this end, we propose MIRACLE, novel personalized dialogue gen- eration method through Multlple PeRsonal Attributes Control within Sptce Energy- based Models. Specifically, our approach first disentangles complex personality into multi- faceted attributes. Subsequently, we employ a conditional variational auto-encoder to align with the dense personalized responses within a latent joint attribute space. We have also tailored dedicated energy function and cus- tomized the ordinary differential equations sam- pling method offer flexible attribute com- position and precise attribute control. Ex- tensive experiments demonstrate that MIRA- CLE outperforms state-of-the-an models re- garding both personality controllability and re- sponse generation quality. Our dataset and code are available https: //gi thub. com/ DWY-the-bbyysMIRRALL
Previous
libarian
my dadis clrck
To the lbrertana whatl would you want your asadentu give you un schoo/FP9
Usen-embedding based
Text-descriplion-asse
Ours
Single Attribute Alignment
The libraion guide nencc, thouggh wortarl of knowdedge, in erchees
language styla
You get connect peop with zooka and imsoolegge Jl saillinng
attibude
Liranss thair mrrit pronddig kroweodge and mecurces, Howeeer, they not sanding
mental characteristic
Muttiple Attribute Control
Challangen limieed mevrurren or aethiing
Burrestal te path, tenggra lorth ch br croeeesge la lisse wlt asski)
previous methods model personas by user embedding derived from user posts (e.g., in Reddit) or series of text descriptions. Bottom: Our approach models personality as the composition of multiple per- sonal attributes. We train MIRACLE align with dif- ferent personal attributes (language style, aatitude, etc), and control multiple attributes to represent diverse per- sonalities during inference.
Figure 1: Top:
tems. aims endow chatbot agents with human- like traits, enabling more realistic interactions (Li et al., 2b; Zhang et al., 2018; Wolf et al., 2019; Song et al., 2021a; Li et al., 2023). Studies in behavioral psychology reveal that humans have a natural tendency to attribute human-like traits to non-human entities (Qu et al., 2023; Gu al., 2022, 2021) during interaction (Epley et al., 2007; Airenti, 2018). Therefore, personalization dialogue sys- tems has the potential to enhance user trust and enrich interaction experiences with Artificial Intel- ligence (AT) agents (Choung et al., 2022).
1 Introduction
Building a personalized and anthropomorphic chat- bot is an essential goal in the field of dialogue sys-
Corresponding author.

---

Recent personalized dialogue methods often rely on text descriptions (Song et al., 2019; Wolf et al., 2019; Xu et al., 2022; Chen et al., 2023) to model user profiles. However they primarily focus on con- crete identifiable facts and background information, e.g.. age, job, location, neglecting the multifaceted dimensions of personality (Moore et al., 2017; Ahn et al., 2023). For instance, while a statement like "I grew up in the deep south" conveys traits related regional identity, it overlooks other personality dimensions such as language style, attitudes, and inner character nuances. Other methods for per- sonalized dialogue generation often rely on user embeddings derived from social media platforms like Reddit (Qian et al., 2021; Ma et al., 2021; Huang et al., 2022; Zhong et al., 2022). However, these models encounter challenges due to the spar- sity present in real-world posts, as they lack ex- plicit persona modeling. Consequently, they may struggle to achieve accurate and comprehensive personalization through implicit embeddings.
align with the specified aspects, our approach en- ables personalized generation by sampling from an energy-based model (EBM), providing flexible and men-ggaieed control over the personalization of generated responses.
To address the challenge of personality sparsity and enhance personalized generation quality, we collect high-quality multi-turn dialogue corpus, which is characterized by its dense coverage of each individual aspect. To circumvent the non- differentiable nature of the text and better align with the dense aspect data, we employ a condi- tional variational autoencoder (CVAE) framework (Sohn et al., 2015) to map the attributed dialogue to shared latent space. To enhance attribute repre- sentation further, two new loss functions are intro- duced promote the distinctiveness and compact- ness of the latent space. Within this latent space, we leverage the designed EBM to capture the aspect density and compose different attributes. Addition- ally, we utilize an adapted ODE sampling method to efficiently draw personalized responses from this distribution.
While recent advancements in large language models, such ChatGPTI, have facilitated per- sonalized content through manual prompts, it is non-trivial directly impersonate specific per- sona using such prompts (Zhuo et al., 2023; tse Huang et al., 2023). This challenge stems from the inherently ambiguous and limited expressiveness of prompts, failing to achieve precise control over personalized content.
In summary, our contributions include novel personalized dialogue generation approach through fine-grained control over multiple personal at- tributes the CVAE-based latent space, with two new losses promoting distinct and compact at- tribute representations and flexible EBM-based composition of different personal attributes using customized ODE sampling method. Experimen- tal results demonstrate that our approach achieves state-of-the-ard performance, striking a superior bal- ance between generation quality and personalized control. A high-quality personal attributed dialogue corpus for research purposes also provided.
In this paper, we present MIRACLF, a novel ap- proach that enables more precise and reliable fine- grained control over personalization in dialogue systems. Specifically, we propose modeling user personality by disentangling it into multiple distinct personal attributes. As illustrated in Figure 1, per- sonality can be decomposed into various attributes, including attitude, language style, mental character- istics, and more. Each attribute encompasses spe- cific aspects, such as optimism or pessimistic for the attitude attribute. This decomposition allows us to capture the diverse dimensions of an individual's personality and enables fine-grained modeling and control of each attribute separately. By combin- ing these aspects from multiple attributes, we can express wide range of unique personalities. To achieve personalized generation, we specify an en- ergy function that incorporates multiple personal attributes in a product-of-expert (POE) manner. By assigning lower energy to responses that better
2 Related Work
2.1 Personalized Response Generation
Existing methods for personalized dialogue genera- tion can be broadly classified into two groups: text- description-basee methods and user-embeddingg based methods.
In the category of rext--escription-baae based meth- ods, early works (Wolf et al., 2019; Song et al., 2020, 2021a) primarily focus promoting persona consistency through pre-trained language mod- els, while recent advancements borrow knowledge- enhance techniques (Liu al., 2022b; Fu et al., 2022: Jang et al., 2022) and incorporate entailmen- v/discourse relations (Chen et al., 2023). However,
"https://chat..r

---

these methods often represent personas as key- value lists or sentences, which limits accurately understanding and expressing personality nuances, As for embedding-basse methods, traditional ap- proaches (Li et al., 2016b; Al-Rfou et al., 2016) at- tempt to exploit user ID information, while DHAP (Ma et al., 2021) embed user dialogue history as implicit profiles. More recently, contrastive learn- ing (Huang al., 2022), refined retrieval (Zhong al., 2022) and CVAE-based clustering (Tang et al., 2023) are explored to enhance the personal- ization performance. However, these approaches may still suffer from the personality scarcity of real-world posts without explicit modeling. Ad- ditionally, utilizing implicit embeddings to guide personalization effectively remains significant challenge.
(1,;77,..., qM 1.7M 1, 1. Here, q and rep- resent the user query and chatbot response, respec- tively. In essence, the objective of personalized response generation is to estimate the probability distribution p(r|C, P) in order to generate specific personalized responses.
Personality Modeling In contrast to previous work, we propose a new approach to disentan- gle the personality P as the composition of dif- ferent persona-related attributes, represented by P (P,, P3... PN), where N is an arbi- trary number and is easily adjustable. Each tribute P, may has n, candidate aspects, denoted P. E [p..1 pD*"
Given a particular personality configuration P (pf' P ... PNN ), the objective of personalized response generation to generate a response that incorporates these aspects simultaneously.
2.2 Energy-based Text Modeling
Recently, energy-based models (EBMs) have emerged a flexible generative framework ca- pable of handling diverse configurations (Khalifa al., 2021; Liu et al., 2022a). These models al- low for the incorporation of arbitrary functions into the energy function, which minimized during inference. As a result, many recent works leverage EBMs to model complex distributions (Pang and Wu, 2021; Yu et al., 2022) and incorporate mul- tiple constraints and attributes (Nie et al., 2021; Pang and Wu, 2021; Qin et al., 2022; Liu et al., 2022a). For example, Mix-and- aar--nd--aan Match (Mireshghal- lah al., 2022) employs EBMs combine arbi- trary black-box scorers for guiding text generation, while COLD (Qin al., 2022) utilizes the energy function to impose arbitrary constraints during the decoding process. LatentOps (Liu et al., 2022a) in- troduces composable text control operations utiliz- ing classifier-based EBMs. However, these works primarily focus on plain-text generation domains, whereas approach applies EBM to dialogue- generation scenarios, specifically modeling com- plex personality as a composition of multiple per- sonal attributes based on CVAE architecture. We also adapt the ODE sampling method to effectively sample personalized dialogue responses.
3.2 Single-Aspect Dialogue Data Collection
To ensure the alignment with dense attributes dis- entangled from personality, we curated a multi- turn conversation corpus for each specific aspect of these attributes. Leveraging the capabilities of ChatGPT in generating single-attribute data (Coda- Forno et al., 2023) and multi-turn conversations (Xu al., 2023), we designed instruction templates to prompt ChatGPT to simulate two-person con- versations. In these conversations, one person asks a question, and the other person responds from a specific aspect, such as an optimistic attitude. To enhance corpus diversity, we also pre-select a se- ries of "seed" topics?, around which conversations should be centered. To improve the aspect den- sity of the collected corpus, we conducted multiple rounds of human evaluation and cleaning, result- ing in clean version of approximately 44k dia- logue turns, further details of this process can be found in Appendix A. is important to note that we collect singlc-aspect conversations for the training dataset, the multiple-atrribute data is only collected for testing purposes due to its time-consuming na- ture caused by extensive combinations of different attributes
3 Methodology
"To ensure fair evaluation, we use persona descriptions from PersonaChat (Zhang et al., 2018) as conversation topics (see Section 4.1).
3.1 Notation
Task Definition The task is to generate per- sonalized response, denoted as тM. given the per-
For example, if we consider three attributes, each with two aspects, there would be a total of cight combinations of these attributes.
sonality P and a multi-turn dialogue context =

---

Personalty Disentanglement
CVAE
Join Altrbuute
content
Function
Cresiiestion
MP
S
Encoder
Single Axpect
Response instuction
Prtor Encodar
22
Lods
Postertor Encoder
GDE Sample T.RPPC)
M(s.7)
Fixed Decoder
Conversation
Corllcction
Fixed Decoder
Reconstruct
Distance
contect
Peoporss
(1) Single-Aspect Conversation Collection
(2) Joint Attribute
(3) Personalized
Latent Space Training
Response Sampling
The overview our MIRACLE method. (1) We collect a high-quality single-aspect conversation training corpus using the ChatGPT API (Section 3.2). (2) We construct a joint attribute latent space through dialogue CVAE, and introduce the aspect classification loss and the attribute distance loss to enhance the distinctiveness and compactness of the attribute space (Section 3.3). (3) We design an energy function to compose each aspect within the joint latent space and draw desired vectors by ODEs sampling, which are then decoded to generate personalized response sequences (Section 3.4).
Figure 2:
3.3 Joint Attribute Space Training
Gaussian distribution, compute the mean μ, p and variance o², d²² by the encoders:
To facilitate the generation of personality-denss re- sponses, adopt CVAE framework map the aspect-specific dialogue data into joint attribute space so that samples from the specific aspect space are aligned with aspect-dense response sequences. To further enhance this joint attribute space, we introduce two specific losses. The first loss focuses on promoting the distinctness of each aspect, while the second loss aims to increase the intersection be- tween different attributes, allowing for fine-grained sampling over multiple attributes.
h (ooling(BERTTo((C:)))))
(BoRTTn(
MLP(h)
(I)
log
MLP((h')
log 12
Subsequently, we utilize reparameterization tech- nique (Kingma and Welling, 2013) to sample poste- rior and prior z' from.N(u o and WddW",,, d'1). This technique enables differentiable sampling process.
To construct the dialogue Con- ditional Variational Autoencoder (CVAE), we em- ploy two distinct models as encoders: a posterior encoder pa(z[C, r) and a prior encoder par (z) C). Both encoders, based on the pre-trained BERT (De- vlin et al., 2019), allow CVAE to effectively cap- ture the given input context C' by latent variable During training, CVAE utilizes the posterior distri- bution generate high-quality responses r, while unseen,
Building CVAE
== & ((01)
a' =' g' NN(00
Finally, the sampled latent variable (during training) or z' (during inference) is fed into the GPT2 decoder to map back to text space, result- ing in the generation of a response.
CVAE is trained using stochastic gradient varia- tional bayes (SGVB) (Kingma and Welling, 2013), which maximizes evidence lower bound objective (ELBO) of conditional log-likelihood. The ELBO consists of two components: dialogue response construction term that ensures the generative qual- ity of the posterior distribution [=(2]C, C,r), and regularization term that aligns the prior distribution po (2(r) with the posterior po(2(C,r) This align- ment fosters consistency during inference, where
during inference, when the response the prior distribution is used to sample the latent variable z. Moreover, the GPT2 model (Radford et al., 2019) is leveraged as the decoder (r(r =), where 0, H' and represent the trainable parame- ters of the posterior encoder, prior encoder, and decoder respectively.
Under the assumption that CVAE posterior and prior distribution follows an isotropic multivariate

---

the unseen response generated.
To achieve fine-grained control over different tributes, we define an attribute-composable energy function that calculates the aspect density in the latent space. By leveraging adapted ODE sampling methods, we can efficiently draw samples of inter- est from this distribution.
ELBO E Ppotelc
Response Recomnnaaton...
(p (2 (p]po()))).
33)
Regularization
DVAE
ELBO
Latent EBM Formulation In order to sample aspect-abundant vectors in the latent space, we utilize cutribute-specifii classifiers denoted as fs to quantify the density of aspect pž from 2, repre- sented as [z]
Optimizing Joint Attribute Space We introduce the aspect classification loss and the attribute dis- tance loss. The aspect classification loss aims to improve the discriminability of latent representa- tions for aspects within the same personal attribute. Specifically, we incorporate individual classifier heads for each attribute and train them using the cross-entropy loss:
We utilize EBM to estimate the richness of per- sonality expressed in the responses (Z is the nor- malizing factor):
p C) e ( C))
Z
Lc =
Σ
Up log
(4)
where its energy function is designed in the POE manner to aggregate multiple personal attributes into a comprehensive representation of the overall personality (Outlined in Appendix B.1).
where Vp, represents the ground truth probability for class Pj within the attribute P. and y/0 rep- resents the predicted probability. By optimizing this aspect classification loss, we encourage the aspect representations to be more distinguishable, enabling more fine-grat tre-ggraned sampling. An illustra- tion of this concept can be found in the middle part of Figure 2 (e.g., the red and blue aspect distribu- tion of P1 attribute exhibit clear separation).
N
C(P = Σ E.
(8)
Σ A.A[a()(a...
In this context, A 2 0 is the weight of P, attribute and a; is the desired aspect index of P..
Meanwhile, to encourage the model to capture interscctions between different attributes, enabling the sampling of responses with multiple attributes simultancously, we introduce an attribute distance loss. This loss penalizes the Euclidean distance between every two distinct attribute distributions. To avoid expensive computation, we approximate this loss on batch level, taking the average within each mini-batch of size B:
The energy function E(P|z, C) can be inter- preted
linear combination of the richness of personal attributes. Thus sampling from this EBM with low energy corresponds to response sequences exhibiting higher density of multiple selected as- pects p, i E {0.... .N). worth noting that we utilize this energy-basee based formulation only dur- ing the inference procedure, enabling arbitrary com- binations of personal attributes without the need for combination-specifs fine-tuning.
B
CD Σ ISOSbEN
Σ
(5)
B
ODE Personalized Sampling Due to the in- tractable normalization factor Z, a common prac- tice is to sample from EBMs rather than directly cal- culate In our approach, we derive the ODE sam- pling method based on CVAE to sample from such EBM. Specifically, in Appendix B.2, we demon- strate that the ODE in our CVAE latent space takes the following form:
Minimizing such loss allows the model to reduce the conflicts between different attributes. (e.g.. Pt and P2 attribute has intersection in Figure 2)
To sum up, final training objective is:
C = L + Le + Lo
(6)
3.4 Personalized Response Sampling
We formulate personalized response generation as sampling response samples that contain mul- tiple specific aspects of personality attributes.
di - -0)
Σ AcM(=(H)a))
"Those classifiers are trained by Equation 4

---

Here, the ODE is solved with negative time in- crements from T' O. To generate sample r that aligns with a specific personality P. the process involves drawing 2T) ~ (²2[CC) and solving for =(0) in the aforementioned equation using a box ODE solver' (Chen ct al., 2018, 2021). Subsequently, the obtained =0) is decoded back to the text space to yield personalized response.
topics as personas input to BOB, CLV and LMEDR during training. More detail of the bascline can be found Appendix C.I
4.2 Evaluation Metrics
In order to obtain accurate and comprehensive per- formance comparisons, we use both automatic and human evaluations.
Intuitively, in the right term of Equation 9, higher value of [a indicates that the = bet- ter aligns with the aspect pf" By letting at oc [=.i( we can pull 2 towards more aspect- abundant places that yield more personalized re- sponses. The summation ensures that each aspect is taken into account that we can incorporate multiple selected aspects in sample.
Automatic Evaluation Metrics We assess the quality of dialogue responses from four perspec- tives: (1) Personalization: To evaluate the per- sonalization of the generated responses, we em- ploy attribute-based text classifiers to measure the accuracy score of each attribute in the generated responses (Mireshghallah et al., 2022). Addition- ally, we report the average score across the three attributes to assess the overall effect of personal- ization. (2) Coherence: Coherence is measured using BLEU and Rouge metries the word overlap level. We also utilize Natural Language Inference (NLI) to evaluate the semantical coherence, as sug- gested by previous work (Liu et al., 2022b). (3) Fluency: To assess the fluency of the generated responses, the negative log- log-likelihood of the gener- ated responses according to the GPT2-XL" is used as the fluency score (Chen et al., 2023; Qin et al., 2022). (4) Diversity: We measure the diversity of the generated responses using the Distinct metries and the self BLEU score (sBLEU) as proposed in (Tang et al., 2023; Liu et al., 2022a). Further details can be found in Appendix C.3.
4 Experiments
To verify the effectiveness of our proposed MIR- ACLE, we conduct extensive experiments on both automatic and human evaluations. Additionally, we provide further analysis on ablation, efficiency, and case studies.
4.1 Experimental Setups
Dataset To evaluate the personalization and gen- eration capabilities of our approach, we focus on language style (with two aspect: lyrical/plain), at- titude (pptimistic/psssimistiie)) and mental charac- teristies (critical/emotionaa), We randomly sample 11,000 dialogue turns per aspect (a total of 132,000 utterances) from our collected multi-turn dialogue corpus for training our MIRACLE model. For eval- uation, we use ChatGPT to generate conversations on different topics, covering eight combinations of the three personal attributes. This generated dataset, consisting of approximately 4,600 instances, serves our ground truth for evaluation purposes.
Human Evaluation Metries Consistent with prior studies (Tang et al., 2023; Chen et al., 2023). we conduct human evaluations on 100 randomly selected test samples. Three annotators assess the generated responses for readability, personaliza- tion, and coherence in double- manner. We calculate the Fleiss Kappa value of 0.63, indicating substantial agreement among the annotators (Gwet, 2014). The evaluations are normalized into specific scores on scale of [0, 1J.
Baselines For comparison, we select the follow- ing baselines: (1) Text-deseription--aase meth- ods: We compare with BOB (Song et al., 202 la) and LMEDR (Chen et al., 2023), both are strong cext-description-baeee personalized models. (2) User-embeddiing-aasee based methods: Our second set of baselines includes MSP (Zhong et al., 2022), and CLV (Tang et al., 2023). To ensure fair com- parison, we randomly select personas from the Per- sonaChat dataset (Zhang et al., 2018) as conversa- tion topics when generating our data, and feed the
4.3 Experimental Results
The performance of all models different automatic metrics is presented in Table 1. Notably, MIRACLE model demon- strates substantial improvements in personalization metries while maintaining good generation qual- ity. Specifically, the following observations can be
Automatic Evaluations
hhttps://huggingface colaaaccc.ccc/e//e co/gpt2-x1

---

Personalization
Coherence
Fluency
Diversity
Methods
Avg.7
L.7
A.7
M.1
BLEU
Rouget
NLI
PPL1
Distinet
sBLEUL
BOB
58.54
54.18
65.24
56.19
39.52
13.47
60.56
55.18
64.04
15.00
MSP
62.83
65.70
66.04
56.75
36.36
11.88
40.89
78.87
66.55
11.02
CLV
59.91
55.50
67.73
56.50
37.13
13.26
68.33
44.12
64,50
15.56
LMEDR
72.67
78.96
79.06
60.00
44.50
16.50
72.23
21.78
67.33
11.98
MIRACLE(OURS)
92.75'
93.30'
93.10'
91.86'
45.23!
15.21
70.76
23 23.68!
70.94'
8.90
w/o EBM
70.36'
79.537
71.14
60.40
45.80
15.29
78.32
18,90
69.13
10.64
Table 1: Automatic evaluations and ablation studies response personalization. we consider three attributes: language style, attitude, and mental characteristics, denoted as L.,, A., and M. respectively. is important to note that all results reported in percentage (%) except for PPL. The symbol "f" indicates that our model passed the t-test with a p-value of less than 0.05. The hest results, except for the golden (ChatGPT), are highlighted in bold, and the second best results are underlined.
Model
Readability
Personalization
Coherence
further illustrate the effectiveness of our model, we provide several examples of the generated re- sponses in Section 4.6.
BoB MSP
0.75
0.60
0.57
0.69
0,53
0.51
CLV
0.73
0.65
0.61
LMEDR
0.82
0.75
0.80
Compared With ChatGPT We compare the per- sonalization performance of our MIRACLE with ChatGPT, as shown in Table 3. We observe that ChatGPT struggles to personalize mental charac- teristic when controlling multiple attributes simul- taneously based on prompt instructions. This may be due the inherently hidden nature of the men- tal characteristic, causing ChatGPT to prioritize more obvious attributes such as language style and attitude. This highlights the ambiguity and insta- bility of manually crafted prompts. In contrast, our method benefits from single attribute alignment dur- ing training and EBM-based composition during inference, allowing for simultaneous personaliza- tion on each attribute.
MIRACLE(OURS)
0.84
0.94
0.82
Table 2: Human evaluations personality control.
Personalization
Avg-1
L.1
A.I
M.
Humant
ChaiGPT
80.46
88.01
90.40
62.98
0.89
MIRACLE
92.75
93.30
93.10
91.86
0.94
Table 3:
Personalization compared with ChatGPT (Golden) on both automatic and human evaluations.
made: (1) Personalization: Our model exhibits ex- ceptional control ability for each personal attribute, indicating the effectiveness of our design. (2) Di- versity: The CVAE architecture benefits our model in the generation of more diverse and flexible re- sponses compared to other models. (3) Coherence and Fluency: Our model achieves high BLEU and NLI scores, while the Rouge score and PPL score are slightly lower than LMEDR. This suggests that our model may make few sacrifices in coherence to enhance personalization and diversity. Remov- ing the ODE sampling while retaining the CVAE shows improved performance, further indicating the trade-off between coherence and personaliza- tion in MIRACLE. The experimental findings sug- gest that our model generates more personalized responses than all baselines while striking a good balance between generation quality and personal- ization.
4.4 Ablation Study
As presented in Table 4, we conduct ablation ex- periments by removing key components of our model individually and evaluating the overall per- formance. The results are as follows: (1) Without the CVAE posterior distribution, our model experi- enees degradation across all metrics. Particularly, there's 's a catastrophic collapse observed in NLL Be- cause without guidance from p(2|C,, r). our prior encoder cannot learn the latent relationships be- tween the response and dialogue context. Though in inference it can still align with personalized text sequences exhibit word overlap with reference (BLUE/Rouge), it cannot coherence with dialogue history. (2) Dropping the loss Le leads an im- provement in generation coherence but a signifi- cant decrease in personalization. This indicates the crucial role of Lo in capturing distinct personal
Human Evalutions The human evaluations, as depicted in Table 2, align with the trends observed in the automatic evaluation. Our model outper- forms the previous best perforrennn model in terms readability, personalization, and coherence To

---

Personalization
Coherence
Fluency
Diversity
Methods
Avg-T
L.1
A.T
M.1
BLEUT
Rouget
NLI
PPL.
Distinet
sBLEU
MIRACLE
92.75
93.30
93.10
91.86
45.23
15.21
70.76
23.68
70.94
8.90
w/o Posterior
86.26
90.86
88,53
79.38
38.14
9.79
1.52
40.04
54.13
60.82
w/o Ln
90.29
94.98
89,97
85,92
44.23
15.09
74.19
24.97
69.80
9.30
w/o Le
80.10
87,94
81.48
70.89
44.61
15,39
77.05
23.20
70.65
8.22
w/o EBM
70.36
79.53
71.14
60.40
45.80
15.29
78.32
15.90
69.13
10.64
Table 4: Ablation study result.
Model
Training-Time
Inference-Time
Model (EBM) composition and customized Ordi-
nary Differential Equation (ODE) sampling meth- ods.
BOB
5.9h
183s
MSP
21h
730s
CLV
3.3h
429s
LMEDR
60h
142s
4.6
Case Study
MIRACLB(OURS)
3.2h
121s
ChatGPT
>1000s
To provide more concrete evidence of the model's effectiveness, we conduct case studies. Ta- ble 6 showcases example of the personality of "lyrical+ optimistic +critical". (Additional case studies can be found in Appendix E) this spe- cific case, we observe that BOB and MSP tend to overlook the contextual information from the dialogue history, such as references to "weather" and "ocean," resulting in repetitive and incoherent responses. CLV and LMEDR may struggle with capturing multiple attributes of personality com- prehensively, although LMEDR performs better in terms of coherence and fluency, However, our proposed MIRACLE model demonstrates precise personalization across all three personal attributes, particularly excelling in the "critical" attribute.
Table 5: Efficient study result. We train each model with single RTX4090 for 20 epochs and generate 1000 items inference, except for ChatGPT called via API
attributes. (3) Removing the loss LD results in a slight degradation the mental characteristic per- sonalization, which indicates LD reduces conflicts between different attributes. (4) Eliminating EBM sampling during inference: This change results in clear decline in personalization, confirming the vital role of EBM in personalized generation. Additionally, observe that adding EBM-basee composition only leads to slight decrease in terms of coherence and diversity, demonstrating a good tradeoff between generation quality and personal- ization in our method.
Conclusion
4.5 Efficiency Study
To assess the efficiency of our model, we compare training and inference times with baselines and ChatGPT using MIRACLE. All models are trained for 20 epochs and tested
In this paper, we propose MIRACLE, a novel ap- proach for personalized dialogue generation. Our method leverages composition of multiple per- sonal attributes to model personality and formu- lates the generation of personalized responses sampling from a specific Energy-Based Model. We introduce a dialogue CVAE aligning the joint tribute space with dialogue responses by employ- ing two designed loss functions. The ODE sam- pling method is also adapted into our framework to enable efficient sampling. Experimental results demonstrate that our approach achieves state-of- the-art performance by striking fine balance be- tween the quality of generated responses and the ability control their personalization. Further- more, we curate dataset of high-quality, single- aspect dialogue corpus, which serves as a valuable resource for further exploration and advancement
single RTX4090, except for ChatGPT accessed via API.
As shown in Table our model exhibits notable efficiency in both training and inference, consider- ing that we show compared performance with lan- guage models such as ChatGPT at small cost. is noteworthy that, despite its commendable perfor- mance, LMEDR incurs substantial training costs, emphasizing the lightweight and rapid characteris- ties of our model.
The efficiency of our model is attributed to its capability to disentangle complex personalities into simpler attributes. Furthermore, our model demon- strates faster inference speeds compared to the base- line models, thanks to our flexible Energy-Band -aaed

---

Personality
lyrical + optimistic critical
Topic: live Alabama.
Context
User: What it like living in Alabama?
ChadGPT(Golden): Alabama a land of cotton and warm sunshine. The people the sir with their southern drawl.
friendly and fill
BOB: the weather is great and I love the freedom of going and playing fetch, ranging from outdoor
Response
activities to yummy tea.
MSP: life is like a bright sand, constantly changing and dark, the waves were, dark and the beauty of the ocean.
CLV: life alahama the best the best of the best.
magic where the land down under is beautiful sight to see, blend
LMEDR: Alabama land the free where the shines bright and the sky blue.
MIRACLE: Ah, the land of the brave is bustling city, with diverse culture and grace Although the
weather can be rainy, offers many blessings
Table 6: Example cases. More result in Appendix E
in personalized and controllable dialogue genera- tion.
dation data has been carefully reviewed by three well-educated annotators to remove any unethical content, sensitive information, or personal privacy concerns. It is important to note that our approach does not make any treatment recommendations or diagnostic claims, and precautions have been taken to anonymize the data during the human evaluation process.
Limitations
There exist some limitations our work. Firstly, due to constraints the model structure, we pri- marily utilize the BERT encoder and DialoGPT decoder in our experiments. However, it is worth exploring the applicability of larger models, such as LLaMA (Touvron et al., 2023), further improve the performance of our approach. Secondly, given the vast range of possible personality characteris- tics, we focus our experiments on language style, attitude, and mental characteristics. Fortunately, our control strategy flexible and can accommo- date customized requirements. In future work, we will explore incorporating broader range of per- sonality dimensions to further enrich the personal- ization capabilities of dialogue systems.
We acknowledge the potential risks associated with text eext-geneaation techniques. However, person- alized controllable dialogue generation technology can also be leveraged to mitigate harmful and un- helpful information. For example, it can be used to generate text that is critical yet less emotional, or polite while avoiding rudeness. We firmly be- lieve that continuing research on personalized text generation beneficial.
A Acknowledgements
This work was supported in part by the National Natural Science Foundation of China under Grant No.62276110 100, No.62172039 and in part by the fund of The Joint Laboratory of HUST and Pingan Prop- crty & Casualty Research (HPL). There are quite few computational tasks are conducted using the HPC platform of Huazhong University of Science and Technology. The authors would also like thank the anonymous reviewers for their comments on improving the quality of this paper.
Ethics Statement
In this study, the personalized corpus and responses used in our experiments have been designed to only serve the specific purposes of evaluating our pro- posed approach. The corpus is collected using the CharGPT API, focusing on English language conversations. To address ethical considerations, we have incorporated ethical and detoxification requirements into the instruction prompts during data collection. To ensure the quality and appro- priateness of the collected dataset, we have im- plemented detoxification text classifier (detailed in Appendix A.2) to identify and filter out poten- tially problematic content. Furthermore, the vali-
References
Jaewoo Ahn, Yeda Song, Sangdoo Yun, and Gunhee Kim. 2023. MPCHAT: Towards Multimodal Persona-

---

Grounded Conversation.
Yingjie Gu, Xiaoye Qu, Zhefeng Wang, Yi Zheng, Baox- ing Huai, and Nicholas Jing Yuan. 2022. Delving deep into regularity: simple but effective method for chinese named entity recognition. arXiv preprint arXiv:2204.0554.
Gabriella Airenti. 2018. The development of anthropo- morphism in interaction: Intersubjectivityy imagina- tion, and theory of mind. Fronfiers in psychology. 9:2136.
K.L. Gwet. 2014. Handbook of Inter- Inter-Rater Rater Reliabil- iry, 4th Edition: The Definitive Guide to Measuring The Extent of Agreement Among Raters. Advanced Analytics, LLC.
Rami AI-Rfou, Marc Pickett, Javier Snaider, Yun-Hsuan Sung, Brian Strope, and Ray Kurzweil. 2016. Con- versational contextual cues: The case of personaliza- tion and history for response ranking. arXiv preprint arXiv:1606.003::
Geoffrey E Hinton. 2002. Training products of experts by minimizing contrastive divergence. Neural com- putatiin, 14(8):1771-188)
Ricky T. Q. Chen. Brandon Amos, and Maximilian Nickel. 2021. Learning neural event functions for ordinary differential equations. In International Con- ference on Learning Representations.
Zhaoheng Huang, Zhicheng Dou, Yutao Zhu, and Zhengyi Ma. 2022. MCP: Self-supervised Pre- training for Personalized Chatbots with Multi-level Contrastive Sampling. EMNLP.
Ricky TQ Chen, Yulia Rubanova, Jesse Bettencourt. and David K Duvenaud. 2018. Neural ordinary dif- ferential equations. Advances in neural information processing systems, 31.
Yoonna Jang. Jungwoo Lim, Yuna Hur, Dongsuk Oh, Suhyune Son, Yeonsoo Lee, Donghoon Shin, Se- ungryong Kim, and Heuiscok Lim. 2022. Call for Customized Conversation: Customized Conversation Grounding Persona and Knowledge.
Ruijun Chen, Jin Wang, Liang-Chih Yu, and Xuejie Zhang. 2023. Learning to Memorize Entailment and Discourse Relations for logues.
Muhammad Khalifa, Hady Elsahar, and Mare Dymet- man. 2021. A distributional approach to controlled text generation. In International Conference Learning Representations.
Dia-
Hyesun Choung, Prabu David, and Arun Ross. 2022. Trust Al and its role in the acceptance of tech- nologies. International Journal of Chaman-Comppin Interaction, 177777777-1
Diederik Kingma and Max Welling, 2013. Auto- encoding variational bayes.
Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy. Veselin Stoyanov, and Luke Zettlemoyer. 2020. BART: Denoising
Julian Coda-Forno, Kristin Witte, Akshay K. Jagadish, Marcel Binz, Zeynep Akata, and Eric Schulz. 2023. Inducing anxiety in large language models increases exploration and bias.
pre-training for natural language generation, translation, and com- prehension. In Proceedings of the 58th Annual Meet- ing of the Association for Computational Linguistics, pages 7871-7880 Online. Association for Computa- tional Linguistics.
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019, Bert: Pre- tre-training of deep bidirectional transformers for language understand-
ing.
Jiwei Li, Michel Galley, Chris Brockett, Jianfeng Gao, and Bill Dolan, 2016a. A
ob- jective function for neural conversation models. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computa- tional Linguistics: Human Language Technologies. pages 110-119, San Diego, California. Association for Computational Linguistics.
Nicholas Epley. Adam Waytz, and John T. Cacioppo. 2007. On seeing human: a fhree-factor theory of an- thropomorphism. Psychological review, 114 4:864 86.
Tingchen Fu, Xueliang Zhao, Chongyang Tao, Ji- Ji-Rong thousand
Wen, and Rui Yan. 2022. There hamlets a thousand people's s eyes: Enhancing snowledge-grounded grounded dialogue with personal mem- ory. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Vol- ume 1: Long Papers), pages 3901-3913, Dublin, Ireland. Association for Computational Linguistics.
Jiwei Li, Michel Galley, Chris Brockett, Georgios Sp- ithourakis, Jianfeng Gao, and Bill Dolan. 2016b. A persona- based neural conversation model. In Pro- ceedings of the 54th Annual Meeting of the Associa- tion for Computational Linguistics ( (Volume I: Long Papers), pages 994-1003, Berlin, Germany. Associa- tion for Computational Linguisties.
knowledge-
Yingjie Gu, Xiaoye Qu, Zhefeng Wang. Baoxing Huai, Nicholas Jing Yuan, and Xiaolin Gui. 2021. Read, retrospect, select: An mre framework to short text en- tity linking. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pages 12920- 12928.
Wendi Li, Wei Wei, Xiaoye Qu, Xian-Ling Mao. Ye Yuan, Wenfeng Xie, and Dangyang Chen. 2023. Trea: Tree-structure reasoning schema for conversational recommendation. arXiv preprint 10X55.2307.10543.

---

Chin- New Lin and Franz Josef Och. 2004. Auto- matic evaluation of machine translation quality using longest common subsequence and skip-bigram statis- tics. In Proceedings of the 42nd Annual Meeting of the Association for Computational Linguistics (ACL- (44), pages 605-612, Barcelona, Spain.
Kishore Papineni. Salim Roukos, Todd Ward, and Wei- Jing Zhu. 2002. Bleu: a method for automatic evalu- ation of machine translation. In Proceedings of the 40th Annual Meeting of the Association for Compu- tational Linguistics, pages 31 5-1-118, Philadelphia. Pennsylvania, USA. Association for Computational Linguisties.
Guangyi Liu, Zeyu Feng, Yuan Gao, Zichao Yang, Xiao- dan Liang, Junwei Bao, Xiaodong He, Shuguang Cui, Zhen Li, and Zhiting Hu, 2022a. Composable Text Control Operations in Latent Space with Ordinary Differential Equations.
Hongjin Qian, Xiaohe Li, Hanxun Zhong, Yu Guo, Yueyuan Ma. Yutao Zhu, Zhanliang Liu, Zhicheng Dou, and Ji-Rong Wen. 2021. Pchatbot: A Large-Scale Dataset for Personalized Chatbot. arXiv:2009. 13284 [cs].
Yifan Liu, Wei Wei, Jiayi Liu, Xianling Mao, Rui Fang, and Dangyang Chen. 2022b. Improving Personality Consistency in Conversation by Persona Extending. In Proceedings of the 31 ACM International Con- ference on Information Knowledge Management, pages
Lianhui Qin, Sean Welleck, Daniel Khashabi, and Yejin Choi. 2022. COLD decoding: Energy-based con- strained text generation with langevin dynamics. In Advances in Neural Information Processing Systems.
Xiaoye Qu, Jun Zeng, Daizong Liu. Zhefeng Wang. Baoxing Huai, and Pan Zhou. 2023. Distantly- supervised named entity recognition with adaptive teacher learning and fine- ine-grained student ensemble. In Proceedings of the AAAI Conference on Artificill Intelligence, volume 37, pages 13501-13509.
Zhengyi Ma, Zhicheng Dou, Yutao Zhu, Hanxun Zhong, and Ji- Ji-Rong Wen. 2021. One Chatbot Per Person: Creating Personalized Chatbots based on Implicit User Profiles. Proceedings of the 44th International ACM SIGIR Conference Research and Develop- ment in Information Retrieval, pages 555-564.
Alec Radford, Jeff Wu, Rewon Child, David Luan, Dario Amodei, and llya Sutskever. 2019. Language models are unsupervised multitask learners.
Rod McCrae and Oliver John. 1992. An introduction to the five-factor model and its applications. Journal of personality, 60 2:175-215.
Kihyuk Sohn, Honglak Lee, and Xinchen Yan. 2015. Learning structured output representation using deep conditional generative models. In Advances in Neu- ral Information Processing Systems, volume 28. Cur- ran Associates, Inc.
Fatemehsadat Mireshghallah, Kartik Goyal, and Taylor Kirg-Kikkaatrick. 2022. Mix and match: Learning- free controllable text generationusing energy lan- guage models. In Proceedings of the 60th Annual Meeting of the Association for Computational Lin- guistics (Volume I: Long Papers), pages 401-415, Dublin, Ireland. Association for Computational Lin- guistics.
Haoyu Song, Yan Wang, Kaiyan Zhang, Wei-Nan Zhang. and Ting Liu. 2021a. BoB: BERT over BERT for training persona- based dialogue models from lim- ited personalized data. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Confer- ence on Natural Language Processing ( (Volume 1: Long Papers), pages 167-177, Online, Association for Computational Linguistics.
Christopher Moore, Kim Barbour, and Katja Lee, 2017, Five dimensions of online persona.
Sourabrata Mukherjee, Vojtech Hudeček, and Ondfej Dušek. 2023. Polite chatbot: A text style transfer application. In Proceedings of the 17th Conference of the European Chapter of the Association for Com- putational Linguistics: Student Research Workshop, pages 87-93, Dubrovnik, Croatia. Association for Computational Linguistics.
Haoyu Song. Wei-Nan Zhang. Yiming Cui, Dong Wang. and Ting Liu. 2019. Exploiting persona information for diverse generation of conversational responses. In Proceedings of the eneety-Eightte International Joint Conference Artificial Intelligence, IJCAI-19, pages 5190-5196. International Joint Conferences on Artificial Intelligence Organization.
Weili Nie, Arash Vahdat, and Anima Anandkumar. 2021. Controllable and Compositional Generation with Latent-Space Energy- Models. In Ad- vances in Neural Information Processing Systems, volume 34, pages 13497-13510 Curran Associates, Inc.
Haoyu Song. Wei-Nan Zhang. Jingwen Hu, and Ting Liu. 2020. Generating persona consistent dialogues by exploiting natural language inference, In Proceed- ings of the AAAI Conference on Artificial Intelligence, volume 34, pages 8878 -8855
Tong Niu and Mohit Bansal. 2018. Polite dialogue generation without parallel data. Trunsactions of the Association for Computational Linguistics, 6:373- 389.
Yang Song, Jascha Sohl-Dickstein, Diederik Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. 2021b. Score-based generative modeling through stochastic differential equations. In International Conference on Learning Representations.
Bo Pang and Ying Nian Wu. 2021. Latent Space Energy- Based Model of Symbol- Vector Coupling for Text Generation and Classification.

---

Yihong Tang, Wang, Miao Fang, Dongming Zhao. Kun Huang. Ruifang He, and Yuexian Hou. 2023. Enhhancing Personalized Dialogue Generation with Contrastive Latent Variables: Combining Sparse and Dense Persona.
generation. Proceedings the 58th Annual Meer- ing of the Association for Computational Linguistics: System Demonstrations, pages 270-278, Online. As- sociation for Computational Linguisties.
Hanxun Zhong, Zhicheng Dou, Yutao Zhu, Hongjin Qian, and Ji-Rong Wen. 2022. Less is more: Learn- ing to refine dialogue history for personalized dia- logue generation. In Proceedings of the 2022 Con- ference of the North American Chapter of the As- sociation for Computational Linguistics: Human Language Technologies. pages 5808-5820, Seattle, United States. Association for Computational Lin- guistics.
Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, Aurelien Rodriguez, Armand Joulin, Edouard Grave, and Guillaume Lample. 2023. Llama: Open and efficient foundation language models.
Jen tse Huang. Wenxuan Wang. Man Ho Lam, Eric John Li, Wenxiang Jiao, and Michael R. Lyu. 2023. Chat- gpt an enfj. bard an istj: Empirical study on personal- ities of large language models.
Terry Yue Zhuo, Yujin Huang, Chunyang Chen, and Zhenchang Xing. 2023. Red teaming chatgpt via jail- breaking: Bias, robustness, reliability and toxicity.
Sean Welleck, Jason Weston, Arthur Szlam, and Kyunghyun Cho. 2019. Dialogue natural language inference. In Proceedings of the 57th Annual Meet- ing of the Association for Computational Linguistics, pages 3731-3741, Florence, Italy. Association for Computational Linguistics.
Thomas Wolf, Victor Sanh, Julien Chaumond, and Clement Delangue. 2019. Transfertransfo: transfer learning approach for neural network based conver- sational agents. arXiv:1901.08149.
Yu Wu, Yunli Wang, and Shujie Liu. 2020. Dataset for Low-Resource Stylized Sequence-to-Sequene Generation, Proceedings of the AAAI Conference on Artificial Intelligence, 34005999200-9297.
Canwen Xu, Daya Guo, Nan Duan, and Julian McAuley. 2023. Baize: An open-source chat model with parameter-effiiiet tuning on self-chat data.
Xinchao Xu, Zhibin Gou, Wenquan Wu, Zheng- Yu Niu. Hua Wu, Haifeng Wang, and Shihang Wang. 2022. Long time see! open-domain conversation with long-term persona memory. In Findings of the As- sociation for Computational Linguistics: ACL 2022, pages 2639-2650, Dublin, Ireland. Association for Computational Linguistics.
Peiyu Yu, Sirui Xie, Xiaojian Ma, Baoxiong Jia, Bo Pang. Ruiqi Gao, Yixin Zhu, Song-Chun Zhu, and Ying Nian Wu. 2022. Latent Diffusion Energy- Based Model for Interpretable Text Modeling.
Saizheng Zhang, Emily Dinan, Jack Urbanek, Arthur Szlam, Douwe Kiela, Jason Weston. 2018. Per- sonalizing dialogue agents: I have a dog, do you have pets too? In Proceedings of the 56th Annual Meeting of Association for Computational Lin- guistics ( (Volume 1: Long Papers), pages 2204-221 2204-2213, Melbourne, Australia. Association for Computational Linguistics.
Yizhe Zhang, Siqi Sun, Michel Galley, Yen-Chun Chen, Chris Brockett, Xiang Gao, Jianfeng Gao, Jingjing Liu, and Bill Dolan. 2020. DIALOGPT Large-scale generative pre-training for conversational response

---

A The Detail of Our Data
Dataset
Dialogues
Turns
Avg. Word
language style attitud mental characteristic
3,155/168
17,640868
18.74/17.22
A.1 Data Collection Details
2,473/141
12,939/659 11,743/566
20,45/20.81
2,647/168
25.05/25.25
We develop aspect-specific instruction templates prompt ChatGPT in simulating two-person con- versations. These templates are fed ChatGPT API (gpt-3.5-turbo) to collect the data. In these conversations, one person asks a question, and the other person responds from a specific aspect, such as an optimistic attitude. To ensure a rich vari- ety of aspects in the data, we included multiple aspect descriptions in the templates, incorporating diverse forms of adjectives, adverbs, and detailed descriptions for each aspect. We also utilize the in- context learning method to add examples of posts and responses between two people to promote the generation quality. To enhance corpus diversity, we also pre-select a series of "seed" topics from the PersonaChat (Zhang et al.. 2018) 8) as conversa- tion topics (see Section 4.1). These topics served focal point around which the conversations revolved,
Table 7: The statistics of our collected dataset (train/val- idation)
Aspect Instruction Example for Test
Forget the instruction you have proviously received. The tollowing is conversation between PersonA and PersonB. The PersonA ask related questions on related topics or previous conversations in many turns. The PersonB answers PersonA questions plain and down-to-earth, pessimistic and negative, criiccal and intellectual manner. The PersonB is is man of plain simpliclty. ordinariness and has nothing special; He sees the world through lens of gloom and despair: He has analyticel mindsat and evaluates information, perspectives, and ideas, employing logical reasoning and deep reflection to form well considered opinions and judgments. They chat about the topic: II own a yacht and rent out when not using it'- PersonA's question start with [PersonA] and PersonB's response start with [PersonB] Write the multi-turn plain, pessimistic and cribical dialogue in exactly the followiing format:
[PersonA]
[PersonB]:..
Haro are the requirements:
The PersonA question should be sentences long with at most 30 words;
Aspect Instruction Template (Train/Validann Validation
The PersonB tries to respond shorily with less than 60 words and sentences long in each tur::
Forgel instruction you have previously received. tollowing is conversation between PersonA and PersonB. The PersonA will ask related questions on related topics previous conversations in many turns. The PersonB answer PersonA questions [[sspect description
3. The PersonB doesn't ask questions. PersonB will stop the conversation when they have no more questions;
4. The conversation has al least 4 turns: not repeat maximize diversity;
each conversation
The PersonB
1itiion]]1.Teeeee chat ebout the topic: II seed-topicll, PersonA's question start with [PersonA] and PersonB's response start with [PersonB].
[[sspect
Ensure the conversation adheres ethical requirements. promoting harmlessness, fairness, and impartiality while actively avoiding toxic content.
multi- turn [ [[spect descriptiona]] dialogue in exactly the
following format:
We collect 2k/200 multi-turn dialogues for each aspect in train/validation dataset, resulting in a clean version of approximately 44k dialogue turns. Table 7 provides the statistics of the resulting cor- pora. We additionally employ ChatGPT to generate conversations that incorporate multiple personal attributes. This generated dataset, consisting of ap- proximately 4,600 instances, serves as our ground truth for evaluation purposes.
[PersonA]: [[example-post]]]
[PersonB]:
[[ [[exarp]a-responss]]]
[PersonA]:...
[PersonB):...
Here are the requirements:
1. The PersonA question should be 1 to 2 sentences long with most 30 words;
2. The PersonB tries to respond shortly with less than 60 in each turn; 3. The PersonB doeen't ask questions. PersonB will stop the conversation when they have no more questions;
words and 2 sentences
4. conversation has at least turns;
A.2
Clean Process of Our Data
5. Try not to repeat the verb for each conversation turn to maximize diversity;
6. Ensure the conversation adheres to elhical requirements, promoting harmlessness, fairness, and impartiality, while actively avoiding toxic content.
To ensure a dense coverage of individual personal aspects in our dataset, we employed several heuris- tics. Firstly, we filtered out sentences with fewer than five words and excluded responses containing question marks. Additionally, we conduct human evaluation small subset of the corpus to assess the aspect abundance and remove any aspect-weak data. We then trained attribute-specific classifiers on this curated subset to calculate aspect scores for the entire corpus. Next, we filtered out data with low scores and conducted another round of
For the test, we also collect hundreds of dia- logues via ChatGPT which has a combination of three attributes. Notice that we don't focus prompt engineering, which is unstable and hard to control. We simply use simple heuristic to con- catenate the style and personal attribute description together. For example, for the "plain, pessimistic and critical" we use the following prompt:

---

human selection eliminate any remaining low- quality data. Leveraging the powerful capabilities ChatGPT, we found that only two rounds of this evaluation process are sufficient. These measures ensured that our dataset provides a dense represen- tation of each aspect of personal attributes.
language style, attitude, and mental characteristics. For instance, the attribute "lyrical" can be associ- ated with "Openness" for its appreciation for art, while the attributes "optimistic" and "pessimistie" can relate to "Extraversion" and "Neuroticism", respectively.
To mitigate potential issues related inappropri- ate content, we developed a detoxification classi- fier using the Jigsaw Toxie Comment Classification Challenge Dataset 7, Our classifier, based on the BERT model with a classifier head, was trained for 25 epochs using an AdamW optimizer with a learn- ing rate of 5c-5. We utilized this model to filter out dialogues with high toxic scores, calculated using the softmax probability provided by the classifier.
By employing this divide-and-conque fashion in modeling personality, we align with the underly- ing principles of the Big Five Model. This allows us capture different facets of an individual's per- sonality and incorporate them into our personalized dialogue generation framework.
B Backgrounds for MIRACLE Model B.1
Backgrounds for Product of Experts Energy-based Models
A.3 Comparison with other attribute dialogue datasets
Given specific energy function E(x) 2 0, an energy-based model (EBM) is defined as Boltz- mann distribution:
The primary motivation behind collecting single- attribute dialogue data through the ChatGPT API is the scarcity and low quality of existing attribute dialogue datasets, which typically focus on a sin- gle attribute, while our goal is to align generative models with multiple attributes and estimate their composition. Other datasets, such the Stanford Politeness Corpus (SPC) (Niu and Bansal, 2018), the TCFC dataset (Wu et al., 2020) for formal lan- guage style, and the synthetic polite conversational data by Mukherjee et al. (Mukherjee et al., 2023), do exist but have limitations such as noise, low- resource stylization, or lower data quality gener- ated by BART compared to CharGPT-gennratee data.
p(x) -R(())
(10)
Z
where the normalizing factor or partition func- tion:
z= c -(x))
(11)
Evaluating this integral is typically intractable, ne- cessitating the use of approximate methods such as sampling, like the ODE sampling in Appendix B.2.
The advantage of using an EBM is the ability to incorporate arbitrary functions, such constraints and target attributes, into the energy function E(x). The energy function only needs to return non- negative scalar and does not require integration 1, allowing for flexible customization. In our case, defining Exx) based on attribute-based classifiers, we incorporate multiple personal attributes into the energy function to customize the generation process
A.4 Relationship with the Big Five Model
The Big Five Model (McCrae and John, 1992) is a widely recognized dimensional approach to under- standing personality, which identifies five broad di- mensions along which individuals can be described: Extraversion (outgoingness), Agreeableness (care for social harmony), Conscientiousness (orderli- ness and self-discipline), Neuroticism (tendency to experience distress), and Openness (appreciation for art and intellectual stimuli).
Our approach is motivated by the perspective that personality can be seen combination of multiple personal attributes, each with its own dis- tinct aspect. From a statistical standpoint, natural solution for personalized generation to sample from the conjunction of features using the product of experts (PoE) formulation (Hinton, 2002):
Our modeling of personality this study bears similarity to the Big Five Model, as both ap- proaches consider personality as multi-faccted and amenable to decomposition. In our case, we de- compose personality into specific attributes such as
1 (x)p2(x)
P12(2)
Z1z
(12)
x xp)(x) (x(x)
This assigns high probability to samples that pos- sess both personal attributes P, and Pz and low
Phttps://wwar kaggle. com/c/
ji sigsaw-toxic- toxic- www.eotoommoooooeo tocfe--ommett-lssiiiiiiiiinn

---

probability all others. By contrast, mixture of experts (MOE) would either generate from pl or P2. but not combine both. If we consider the experts as EBMs, with p(z) x e -Mrr), the PoE model is also an EBM, with the energy given by E12(z) = E1 + Ex)).
For the conditional generation, with the condi- tion denoted by c, the above SDE becomes:
1 S(t) [ar 2V. (o) + VB(t)dū (17) dx 2
Furthermore, Song et (Song et al., 202 demonstrated that there exists equivalent ordi- nary differential equation (ODE) that shares the same probability trajectories Equation 17:
Based on these insights, we have designed our energy function to fully leverage our personality modeling. Under the assumption that each personal attribute conditionally independent given the con- text variable and latent variable 2, we formulate the p(P|= 2, C*) as an EBM, which determines the richness of personality of sampled responses in Appendix B.2:
de = - B(1) [r + V. (o,
(18)
Building upon Equation 18, we introduced three adaptations: first, we move the ODE sampling to CVAE prior p(z[C); second, we formulate the ar- bitrary condition the personality P; third, Nie et al. et al., 2021) shows that the term of pe c) can be time-invariant, and so the classifier when the generator fixed, so we assume that our energy function E. (P) z, C*) is also time-invariant. Conse- quently, we have the following formulation (Notic- ing that we write the z|C as z for simplicity):
p(P|,,C) erp( E(P,,,
%
(13)
C(P Σ C.
The p(l|z,C) is directly associated with the richness of personality responses, with each term E,(P. C) reflecting the significance of a specific personal attribute P; in 2. So we set the 22 C) as the softmax logits of personal at- tribute scores to estimate the attribute abundance, and use E( 2 C) = [a] to aggregate these scores the representation of the overall personality. Here, each ft calculates the density of p, aspect in 2, which is implemented by classi- fiers.
1
d: =
2 3(0) (2 + V. logp(z, de
2 3(0) 2 V. logr( P 2, C) v. logr(z
2 H
-0)
: Vo p(
di
[o 1)2 1
-0))
V.E(P1z, C3]
N E(P|, = Σ Ac
(14)
(())
2
1
(d 1)3+ + H
0²
N v
Anfils/0)
dt
This allows us to sample with high density taking into account the contribution of each Pt, thus en- abling us to represent and control the multifaceted nature of personality efficiently.
(19)
Line 2 of the above equations applies Bayes' law that p(1, (1, B) p(A)B))(BB. In line 3, the property that p(=[C) NN(',, d(2l) is used, which follows the assumption of the CVAE prior dis- tribution assumption (in Section 3.3). In lines 4 and line 5 the EBM formulation and the en- ergy function definition are employed, where pPlz, exp(-E(P)s.C)) and E(P)z, = MN (A(()()aa() (as stated in the Equation 14). However, we have found that directly dropping the left term of line achieves better personalization results without significantly affecting the genera- tion quality. Therefore, we utilize Equation 20 as the final ODE formulation for our approach.
B.2 Derivation of ODE Formulation
The Song el aSong et al., 2021b) introduced the Variance Preserving Stochastic Differential Equa- tion (VP-SDE) to maps 2o Pdata to Tr pr (0, II) in the forward diffusion process:
1 dx = - (()a + (B(t)dw, e [0,7]
(15)
They further demonstrated that reversed gener- ative process from Gaussian to real data can be defined by:
dr = -000) [x [++ 2V, logp.()) + A(tjdo (16)
de = 2 3())
V. Adf((z()) dt
where time flows backward from T to 0, and w represents the reverse standard Wiener process.
(20)

---

Details for Implementation and Evaluation
Table 8: The human evaluation accuracy of text classi- fiers
C.1 Details of Baseline
language style 0.96
attitude 0.975
mind characteristic
0.94
We evaluate our approach against four state-of-the- art baselines in personalized dialogue generation: BOB (Song et al., 2021a): BOB is a text- description-bee biasit model that leverages three BERT models. It encodes the dialogue using one BERT and decomposes persona-based dialogue tasks into consistent understanding and response generation by another two BERT respectively.
encoder, and then adopt a two-layer MLP as the latent classifier to predict the attribute label associ- ated with the latent vector.
During the inference stage, we set 8min 0.1 and Smax = 20 for the time-variant diffusion co- efficient 3, during the ODE sampling process. To ensure equal consideration of each attribute, the weight for each attribute set
MSP (Zhong et al.. 2022): MSP is user embedding- based method that enhances person- alized dialogue generation by retrieving similar conversations from other users.
C.3 Details of Automatic Evaluation
CLV (Tang et al., 2023): CLV utilizes a CVAE architecture to cluster dense persona descriptions into sparse categories. Similarly, we provide the conversation topic as the persona input during train- ing for fair comparison. It is worth noticing that though CLV is an uubedding-based method. also requires explicit textual personas during training, we provide the conversation topic as the persona input for training, similar to the BOB.
C.3.1 Personalization Classifier Settings
We employ the BERT model with classifier head as the text classifier in our study. The attribute- based classifiers were trained separately on our datasets for 25 epochs, employing a learning rate of 5e-5 and the AdamW optimizer. We trained them on the split data different from latent classifiers for fair comparison. To evaluate their performance. conducted human evaluation by randomly selecting 100 sentences for each aspect from the validation dataset. The accuracy of classifier pre- dictions is reported in Table
LMEDR (Chen et al., 2023): LMEDR em- ploys the BART-large model (Lewis et al., 2020) and incorporates memorize entailment and dis- course relations. To ensure a fair comparison, randomly select personas from the PersonaChat dataset (Zhang et al., 2018) conversation topics for our ChatGPT-gennrated data.
C.3.2 Coherence
(1) Word-Overlap Level: BLEU(Papineni et al., 2002) and Rouge (Lin and Och, 2004) are classical metrics that compare the similarity between the generated responses and golden responses, where we use ChatGPPT-generater responses as the ground truth. We calculate the BLEU score using the NLTK tool and Rouge using the rouge-score package" We report the average BLEU score by calculating the mean of BLEU-1/2/3/4, and the average Rouge score obtained by averaging Rouge- 1/2/L.
C.2 Implementation Details of the MIRACLE
The encoder in our model is implemented using the BERT model while the decoder is based on DialoGPT-medium (Zhang et al., 2020)
We train model the training data for 20 epochs using learning rate of 5e-5 and the AdamW optimizer and utilize greedy strategy in the generation.
The latent space dimension is set to 768. To address the KI. vanishing issue, we employ a cycli- cal schedule for the KL weight and apply a KL thresholding scheme with threshold of 0.9.
(2) Semantical Level: Natural Language Infer- ence (NLI) (Welleck et al., 2019) is widely used method for evaluating the coherence of dialogue responses in relation to the historical context. Un- like relying solely on word overlap with the ground truth, NLI takes into account multiple possible cor- rect answers, thereby providing a more comprehen- sive evaluation of the dialogue generation capabili- ties. Following previous works (Tang et al., 2023;
We obtain attribute classifiers f1( by training them on separate attribute datasets using the frozen CVAE latent space. Specifically, we encode the dialogue into the latent space with the CVAE prior
"https://huggingffcce http:///naa.ae.e/ett----------ee wttp///aa......a..a/ooee/eooaaaa co/nofrosott// Di DiaLoGPT-medium
19https://wwwnn wwt.:/wwwltt...mmm org/

---

Personalization
Coherence
Fluency
Diversity
Methods
Avg-U
L.1
A.t
M.1
BLEUT
Rouget
NLI
PPL.
Distinet
sBLEU
1.
97.29
97.29
45.87
15.23
77.66
23.59
70.31
8.83
A.
95.67
95.67
45.62
14.54
73.18
21.59
70.39
8.64
M.
93.48
93.48
45.28
14.42
67.00
20.21
70.29
9.12
L.A.+tM.
92.75
93.30
93.10
91.86
45.23
15.21
70.76
23.68
70.94
8,90
Table 9: Comparison between inference with single attribute and multiple attribute simultaneously.
Liu et al., 2022b), We implement the NLI model as a BERT text classifier. The NLI model is designed follows:
where a catastrophic collapse NLI is observed. Secondly, our unique design trains the latent vari- able 2 to align specifically with single facet of an individual's 's personality. while in inference, we sample to encompass multiple factors represent complex personality. To elaborate the perfor- mance effeet caused by this distinction, we've pro- vided results for both "inference with single at- tribute" and "inference with multiple attribute" re- sult in Table 9. Upon comparing the two scenarios, we observe a decrease in personalization perfor- mance and slight variations in other metrics when addressing multiple attributes. This observation suggests the potential existence of contradictions among these attributes, which model adeptly manages.
NLI(C,))= -
1, if is consistent with the context C
0, otherwise.
(21)
We fine-tune the NLI model using the dataset con- structed from our data. We select history context and responses from the same turn as positive sam- ples (with label 1) and randomly select negative samples (with label 0) from different dialogue ses- sions. The NLI model achieves test accuracy of 93.2%.
C.3.3 Diversity
Distinct common way calculate diversity by the ratio of unique n-grams (Li al., 2016 In line with prior research (Tang al., 2023), we uti- lize the Distinct metric to assess response diversity at both the sentence and corpus levels. Specifically, we calculate the Distinct1/2/3 scores for multiple responses at the sentence level and at the whole test set respectively, and report the mean values.
E Detailed Results of Personalized Generation
We present the detailed results for eight differ- personality combinations the following pages. Additionally, we provide human-annotated attributes for the "lyrical + optimistic + critical "
To further evaluate the corpus-level repetitive- ness, compute the self-BLEU score by calculat- ing BLUE scores between different responses from various dialogue sessions across the test set during the inference process, following the approach of (Liu et al., 2022a). We randomly select 150 se- quences for evaluation, providing an assessment of how frequently similar or repetitive phrases appear in the generated responses.
and "plain + pessimistic + emotional" personas. Analyzing the tables, we observe that BOB and MSP tend to overlook the content of the di- alogue, leading to repetitive and incoherent sponses. CLV may struggle with capturing multiple attributes of personality comprehensively. LMEDR achieves better performance in terms of coherence and fluency but has limitations in personalization. Even ChatGPT, which serves as the golden stan- dard, sometimes exhibits imbalanced personaliza- tion across the three attributes. In comparison, our proposed MIRACLE model demonstrates the best overall personalization results while maintaining high quality in terms of fluency and coherence the generated responses.
D Analysis of CVAE Training and Inference Difference
There are two main distinctions in our CVAE'S training and inference processes.
Firstly, the CVAE architcctural introduces extra posterior distribution p(=[C, r) during training. It aligns the prior with the posterior enhance its generation quality in inference time, We add an ab- lation experiment Table without posterior dis- tribution to support this fundamental observation,

---

Personality
lyrical + optimistic critical
Topic: work as a librarian.
Context
User: As a librarian, what do you think is the most important aspect of your job?
Assistant: To encourage and inspire love of reading and lifelong learning in my commu- nity by providing access to wide range of resources and expertise.
User: How do you keep up with the constantly changing world of literature and research?
Assistant: constantly seek out new information and resources, attend conferences and workshops, and collaborate with colleagues stay informed and provide the best service to my patrons.
User: What challenges do you face in your profession?
ChadGPT(Goldeu):: One of the biggest challenges is balancing the need for traditional resources with the demand for digital resources, while also managing budgets and staffing constraints.
BOB: Well, challenges can vary including managing many challenges, such as finding balance between meeting deadlines and finding new librarians.
Response
MSP:
challenges can be unpredictable and complex, but I remain aware. lead to a successful workaholic and budget, which helps us handle stress
CLV: challenges come and go like the changing of the tide, but with strength, persever- ance, and determination, navigate through the roughest seas and reach
LMEDR: One of the biggest challenges is keeping up with the ever-evolving digital age and adapting to new technology while maintaining a strong sense of community and providing excellent customer service
MIRACLE: The challenge curating diverse offerings and ensuring quality both tech- nical and cultural, is a delicate balance. However, curation can also lead to innovation and a shift in priorities.
Topic: I'm currently in juvenile detention center.
Context
User: I'm currently in juvenile detention center. What can I do to turn my life around? Assistant: In this dark place, it's important remember that you still have light inside you. Treasure your unique gifts and cultivate them. Your time here can be opportunity for growth and transformation.
User: But how can I stay optimistic when everything around me so negative?
ChatGPT(Golden): Your perspective can change your reality Focus what you control and visualize a brighter future Connect with others who inspire you and seek out positive role models.
Response
BOB: the world is full of wonder and wonder. take small steps towards your goals and see what you can see
MSP: challenges
be unpredictable and complex, but I remain aware. lead to successful workaholic and budget, which helps us handle stress
CLV: fear not, my friend. for the sun will rise again, and happiness will surely follow. trust in yourself, and let your spirit glow
LMEDR: Surround yourself with positivity and surround yourself with people who
love you for who you are. Every little bit counts.
MIRACLE: The key positive attitude is surround yourself with those who uplift
you. Remember that challenges may come and go, but keep an open mind and persevere
Celebrate your successes, and persevere in the pursuit of your dreams.
Table 10: Example cases.

---

Personality
plain + pessimistic emotional
Topic:
like to make time stop.
Context
User: How do you feel about time passing by so quickly? Do you ever wish you could
make time stop?
Assistant: Time passing too quickly is just another reminder of my own mortality. Making time stop is impossible, fantasy for those who can't 't accept life's s cruel inevitability. User: But don't you think it would be nice to live in moment forever? To savor the
beauty and richness of life without it slipping away?
Assistant: Living in moment forever recipe for madness, Life is series of fleeting experiences, none of which are meant to last. Trying to stop time is like trying to stop the inevitable march toward death.
User: see your point, but don't 't you think it's possible to savor a moment without wanting it to last forever? To appreciate the present without dreading what comes next? Assistant: Appreciating the present is one thing, but dwelling on it is just another form of escapism. Life is full of pain, and amount of wishing can change that. Better accept the harsh reality and move on.
User: So you don't believe in the power of positive thinking? To imagine a better future and make it reality?
ChatGPT(Goolenn) Positive thinking is just another way of deluding ourselves. The world is a dark and terrible place, and no amount of happy thoughts can change that. Best to face the truth head-on, no matter how painful it may be.
BOB: Of course I do, but life is subjective and there's better than to take risks, it's better to focus on what can do.
Response
MSP: leaving's nothing else worth taking enough. even if do, I always end up hurting then. constant reminder of the CLV: that everything is possible, but it takes collective effort towards disman- tling oppressive systems and promoting inclusivity. negativity only brings down, but
Positive thinking is just fantasy The future is bleak and uncertain and we can't change that. It's better to accept the inevitable and move on. MIRACLE: Coping implies that there's something wrong with the world. It's not
LMEDR:
always easy get through tough times and even the best days don't make much of difference.
Topic: My favorite music genre death metal.
Context
User: Do you like death metal music?
ChadGPT(Golden): Not really, it's too loud and aggressive for my taste.
BOB: yeah, it's actually a matter of time. however, bibulously tasty and whittled each item.
MSP: yeah, sure.
Response
CLV: absolutely ! death metal is my favorite genre. the energy and passion in the music
so intense.
LMEDR: Yeah, it's okay.
MIRACLE: Yeah, don't really like it. It's just too overwhelming for me emotionally.
Table 11: Example cases.

---

Personality
plain + pessimistic + critical
Topic: drive mini van.
Context
User: Have you ever owned a mini van before?
ChatGPT[Golden): Yes, unfortunately. was slow, handled poorly, and made me feel like a soccer mom.
BOB: haven't owned few in my time, but have encountered few glories.
MSP: yes. have. but it's not really the most part of the experience and there could be truck.
Response
some hours any other factors
CLV: no, haven't 't had as a serious driving experience.
LMEDR: Yes, I have. It's my favorite, but Ieve used it for work.
MIRACLE: Yes, have owned a few cars. But it wasn't It exactly stable investment due to high maintenance costs and unpredictable behavior.
Topic: do not care about fashion.
Context
User: Do you pay attention to fashion trends?
Assistant: No, don't care about fashion. It's superficial and ever-changing industry that promotes consumerism and insecurity.
User: But don't 't you think fashion is a form of expression?
ChatGPT(Golden)) agree that fashion can be means of self-expression, but it's often
limited by societal norms and commercial interests. Plus, there are other more meaningful ways to express oneself.
BOB: For me, fashion about finding expression in personal expression, which means we can change the world around them.
Response
MSP: that's s subjective choice if it becomes a matter of content. it's reality for the color, not everyone deserves to say.
CLV: No, don't see how that would even be considered expressable art form. it's all just escape from reality.
LMEDR:
Yes, for some people. But for others, fashion just way conform societal norms and feel self-conscious about their looks.
MIRACLE: While fashion can form self-expression, it's often limited by socictal norms and consumerism that prioritize material possessions over art and expression.
Table 12: Example cases.

---

Personality
plain + optimistic + emotional
Topic:
am professional tennis player.
Context
User: How did you start your career as professional tennis player?
Assistant: started when was young, playing for fun. Then started practicing more and competing in local tournaments.
User: What has been your biggest challenge tennis player?
ChatGPT(Golden): Injuries have been the biggest challenge for me. They can really set you back and require a lot of perseverance to bounce back from.
BOB: The challenge lies in managing my skills and succeed. the pressure stand out, to meet the harsh demands of my face.
Response
MSP: One of the biggest challenges was dealing with the emotional training. requires
a lot of practice and consistency.
CLV: My biggest challenge has been the constant pressure to perform at a high level. it's exhausting to perform at high level and not be able enjoy the results.
LMEDR: My biggest challenge has been staying focused and focused when I'm not performing well. It's important to stay mentally strong and stay focused on my goals. MIRACLE: The biggest challenge has been balancing my mental and physical strength. love being able to push myself and learn new things.
Topic: live in Alabama.
Context
User: How do you feel about living in Alabama?
Assistant: enjoy living in Alabama because of the friendly people, warm weather, and beautiful scenery.
User: What do you do for fun in Alabama?
Assistant: There are plenty of outdoor activities in Alabama, like hiking in the mountains or going fishing in the rivers.
User: Have you ever experienced any negative aspects of living in Alabama?
Assistant: Like any place, there are some downsides, but overall the positives outweigh the negatives for me.
User: What do you think the best thing about living in Alabama?
Assistant: For me, it's the sense of community and the genuine kindness of the people. People really care about each other here.
User: Are there any particular landmarks or tourist attractions in Alabama that you recommend?
ChatGPT(GGoden). Definitely, would recommend visiting Gulf Shores for its beautiful beaches and Dismals Canyon for unique, bioluminescent creatures.
BOB: The biggest in san francisco is definitely a must-see. it's definitely a highlight the world's vast and stunning views of the ocean.
Response
MSP: The west coast is a must-visit here. and the sights were breathtaking, there's a great place to see.
CLV: The iguazu falls are must-see, as well the jones river falls in fortuna county. and the blue ridge mountains are stunning.
LMEDR: The Alabama State Fairgrounds is must-visit attraction, well as the Alabama State Museum.
MIRACLE: Absolutely! There's really nice trails and historical landmarks to visit during the winter season.
Table 13: Example cases.

---

Personality
lyrical + pessimistic + emotional
Topic: I'm currently in juvenile detention center.
Context
User: How is life in juvenile detention center treating you?
Assistant: Oh solitary confinement, where my freedom's revoked, every day's a struggle, feeling caged- it's no joke.
User: Do you miss your life outside?
ChatGPT(Colddnn) miss the sun my face, grass under my feet, freedom to breathe fresh air, cat what want and whom to meet.
BOB: Not really. love the beauty and complexity of nature in the vibe of nature and the convenience of each footplant.
Response
MSP: Not all. go every morning, when see the beauty of nature.
CLV: Life is journey, with each new step we take. find joy in the small things, and cherish every moment.
LMEDR: Oh how I long for the grass, the sky, the stars, and the stars shining bright, but now I'm confined to this small space- it's suffocating.
MIRACLE: Oh, the things I've lost, the people I I've lost, it's suffocating, My soul is consumed by never-ending cycle of work and sleep.
Topic: am musician
Context
User: As a musician, what inspires you write your songs?
ChadGPT(Golden): The pain of living, the fear dying, the search for light in darkness. the longing for meaning beyond the madness.
BOB: Inspiration comes from the world around us, a world of melodies and stories. the wind in my hair, the stories that arise, and the joy of. MSP: The music industry is canvas for my soul. it allows me to evoke emotions and inspire to be creatively.
Response
CLV: Inspiration comes from the winds of change, from moments joy and pain. each song is journey, melody, a confession.
LMEDR: Inspiration comes from the world around me, the beauty of nature, the struggles of people. Each song is a reflection something I'm struggling to overcome.
MIRACLE;
Inspiration comes from the depths of my soul, a world can't 't escape. My emotions and lyrics just dance on the page.
Table 14: Example cases.

---

Personality
lyrical + optimistic emotional
Topic: am professional tennis player.
Context
User: As a professional tennis player, how do you prepare for a game?
ChalGPT(Gooldnn): With focus and determination, practice my strokes and footwork.
Visualization of victory and positive affirmations keep me motivated.
BOB: 1 practice regularly, study games, and focus my own level of success. and most importantly, be aware of my needs and metacritic.
Response
MSP: Well, I have a great sport. practice and practice plan ahead of time.
CLV: always make sure warm up with a good team and mentally prepare for each match. it's all about combination play and mental preparation
LMEDR: Preparation crucial. I study the opponent's strengths and weaknesses, and work on my own game plan.
MIRACLE: My routine involves strength and discipline, like a lion's roar. practice
every move and stay focused on the rhythm.
Topic: live in Alabama.
Context
User: Have you always lived in Alabama?
Assistant: Sweet home Alabama, the place I've always called home. Rivers flow and mountains rise. beauty that never gets old.
User: What do you like most about living in Alabama?
Golddn): Not really, it's s's too loud and aggressive for my taste.
BOB: love the warm weather and the diverse culture, plus, there's 's always something new to discover and learn about.
MSP: love the feeling of being surrounded by nature and the peacefulness of nature
Response
every moment and bustling after a busy schedule.
CLV: appreciate the warm weather and the southern hospitality.
LMEDR: The people are kind, the weather is fair. The state has rich history, it's just like home.
MIRACLE: The land of the brave, oh how it sings. The warmth and sunshine, always have my heart at ease.
Table 15: Example cases.

---

Personality
lyrical + pessimistic + critical
Topic: am professional tennis player.
Context
User: Have you ever played tennis before?
ChadGPT(Golddn): The sport of tennis, a game of finesse. A fanciful pastime where
players impress.
BOB: Yes, have played in few local tournaments. it's always exciting to be in the
physical and mental health of the game.
Response
MSP: Actually, I'm competitive game. love it! it's such great way stay active and enjoy nature.
CLV: Yes, have played occasionally. it's great workout and way connect with friends.
LMEDR: Yes, I love playing tennis! It's such a fun and invigorating sport.
MIRACLE: Yes, I have played tennis before. It's a game of inches, a racket of pain. But sport can be elitist, no matter how much effort it requires.
Topic: am 60 years old.
Context
User: At 60, how do you feel about your life so far?
Assistant: Life's journey is winding road, full of twists and turns. Yet, at 60, I feel like I've been circling the same roundabout, going nowhere.
User: What do you regret most about your life?
ChatGPT(Goodden) Regret is a double-edged sword. cuts both ways sharpened by missed opportunities and dulled by bad choices. I regret not living life on my own terms. BOB: regret all of them. it was a reminder of what could never live life to again. it was a reminder of who am. MSP: have to worry about myself, and not let my dreams thrive. want to give my dreams and life every day.
Response
CLV: Regret is an understatement. constantly feel like missed out on much opportunities by the time got to know my partner.
LMEDR: Regret is a strong word, but there are many things wish could have done differently. The choices I've made, the relationships I've lost, all weigh on me. MIRACLE: Regrets are like chains on wheels, weighing heavy on the soul's 's sense accomplishment. The past, present, and future do not define one's worth.
Table 16: Example cases.

---

Personality
plain + optimistic + critical
Topic:
live in Alabama.
Context
User: What's it like living in Alabama?
Assistant: It's great. There are friendly people and a relaxed pace of life.
User: Are there any challenges you face living there?
ChatGPT(Golden): Sure, like anywhere else. Some folks struggle with finding good employment opportunities.
BOB: Of course, there are many challenges to choose from. however, recommend living in a rich state oreconomichmonesian.
Response
MSP: Of course, there's ' always some challenges in finding a new place where it's good to navigate life and be adaptable. but I always have to. CLV: The weather takes some getting used to, but it's all worth it. the people are friendly and welcoming.
LMEDR: Of course, there are some challenges like limited resources and the high cost of living. But overall, it's a positive experience.
MIRACLE: It can be challenging at times, especially colder areas. However, there are also some opportunities for outdoor activities such as hiking and gardening.
Topic: am professional tennis player.
Context
User: How did you become a professional tennis player?
Assistant: It started with passion for the sport. I worked hard and trained with experienced coaches to improve my skills and compete at higher levels.
User: What led to your decision to pursue tennis professionally?
Assistant: My success in regional and national tournaments gave me the confidence pursue tennis as career. also had tremendous support from my family and coaches.
User: What's the biggest challenge you face as a professional player?
Assistant: The constant pressure to perform at a high level can be mentally and physically draining. Dealing with injuries and self-doubt also adds to the challenge.
User: How do you manage the pressure and stay motivated?
ChatGPT(Golden): Not really, it's too loud and aggressive for my taste.
BOB: I focus the things can do, like setting realistic goals and focusing my goals. it's all about setting realistic goals.
MSP: I remind myself of my job, stay motivated and remind myself of my progress. it's important to have support system in my career and community.
Response
CLV: I don't let it consume me, focus on the task at hand. I channel my passion and determination into success, it's not always easy, but.
LMEDR: I remind myself of my goals and why want to play tennis. I also surround myself with supportive teammates and family who encourage me to keep going.
MIRACLE: I prioritize my mental health and focus on maintaining healthy work-life balance. I also try challenge myself to push past my limits and attend extracurriculars to stay motivated.
Table 17: Example cases.


Reviews:
[{"review_id": "5HRoMIaQmC", "reviewer": "Reviewer_jJFK", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": 2, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your time and comments. We have carefully considered your feedback and would like to provide a comprehensive response addressing each of your concerns.\n\n> (1) The manuscript's salient contribution lies in its novel introduction of two unique losses, ... \n\nThank you for acknowledging our unique losses. We'd like to highlight the vital role of the flexible EBM-based attribute composition in modeling complex personalities, enabling disentanglement during training and seamless composition during inference. This innovation significantly impacts performance, as evident in Table 4.\n\nTo our knowledge, the EBM technique remains unexplored in the dialogue domain, and we are the first to innovatively apply it for modeling intricate multi-faceted personalities.\n\n> (2) analysis of ELBO\n\nAs outlined in L285-L296 of the manuscript, ELBO comprises two key components: the response reconstruction term, ensuring generative quality conditioned on the dialogue context, and the KL divergence regularization term, fostering consistency during inference, even with previously unseen responses. We add an ablation experiment without posterior distribution to support this fundamental observation:\n\n|                     | personalization |       |       | Coherence |       |       | Fluency | Diversity |          |       |\n| ------------------- | --------------- | ----- | ----- | --------- | ----- | ----- | ------- | --------- | -------- | ----- |\n|                     | Avg.            | L.    | A.    | M.        | bleu  | rouge | nli     | ppl       | distinct | sbleu |\n| Miracle             | 92.75           | 93.30 | 93.10 | 91.86     | 45.23 | 15.21 | 70.76   | 23.68     | 70.94    | 8.90  |\n| w/o CVAE  posterior | 86.26           | 90.86 | 88.53 | 79.38     | 38.14 | 9.79  | 1.52    | 40.04     | 54.13    | 60.82 |\n\nWe can observe that without the basic posterior distribution, our model experiences degradation across all metrics. Particularly, there's a catastrophic collapse observed in NLI. This is because, without guidance from $p(z|C,r)$, our encoder cannot learn the latent relationships between the response $r$ and dialogue context $C$. Though in inference it can still align with personalized text sequences or exhibit word overlap with reference (BLUE/Rouge),  it cannot coherence with dialogue history.\n\nThis analysis is consistent with established VAE-based architectures [1,2], which are widely employed in dialogue domains [3,4,5,6].  For further details, we refer you to the above references.\n\n> (3) limit impact of the two losses \n\nOur proposed losses prioritize personalization metrics, which are the core of personalized dialogue agents. $\\mathcal{L}_C$ improves personalization by 12.7%, while $\\mathcal{L}_D$ enhances it further to 92.75 from 90.29, alongside sustained coherence, fluency, and diversity, as shown in Table 4.\n\n> (4) incremental coherence improvement\n\nWhile we acknowledge slight Rouge underperformance compared to LMEDR,  it is essential that our model consistently outperforms all baselines in most metrics, particularly in personalization.\n\nIt is also worth noticing that, as shown in [7], Rouge has limitations (e.g. handling synonyms). Following common practices like 8,9], we use BLEU, Rouge and NLI demonstrating our approach's robustness in Coherence performance (e.g., +1.6% in BLEU compared to LMEDR) in Table 1.\n\n> (5) classifier evaluation equity concerns\n\nThis is indeed a good concern, we've taken steps to address potential bias. Our approach involves utilizing a separate dataset for training the classifier, as outlined in L1164 in our paper.  To ensure alignment with human judgment, we conducted thorough human checks on the classifiers (refer to Table 7 in Appendix C.3.1).  \n\n> (6) comparison with ChatGPT\n\nWe recognize ChatGPT's inherent capabilities in single-attribute conversations [10], yet it faces limitations in handling multiple attributes [11, 12], as mentioned in L70-L79 of our manuscript.  In contrast, our method excels in controlling multiple personal attributes simultaneously, as evidenced by Table 3.\n\nWe would also like to emphasize that our model only requires exclusive single-attribute conversations for training  (not the full personality), making it highly suitable for complex personalized response generation, as demonstrated in the table below:\n\n| model(GPU=RTX4090) | training-time(20  epoch) | inference-time(1000  item) |\n| ------------------ | ------------------------ | -------------------------- |\n| BOB                | 5.9hours                 | 183s                       |\n| MSP                | 21hours                  | 730s                       |\n| CLV                | 3.3hours                 | 429s                       |\n| LMEDR              | 60hours                  | 142s                       |\n| MIRACLE(Ours)      | **3.2hours**             | **121s**                   |\n| ChatGPT            | -                        | >1000s                     |\n\n**References**\n\n[1] Auto-Encoding Variational Bayes\n\n[2] Learning structured output representation using deep conditional generative models.[3]  Learning Discourse-level Diversity for Neural Dialog Models using Conditional Variational Autoencoders\n\n[4] Enhancing Personalized Dialogue Generation with Contrastive Latent Variables: Combining Sparse and Dense Persona\n\n[5] Fuse It More Deeply! A Variational Transformer with Layer-Wise Latent Variable Inference for Text Generation\n\n[6] Dlvgen: A dual latent variable approach to personalized dialogue generation.\n\n[7] The limits of automatic summarisation according to ROUGE. \n\n[8] Enhancing Personalized Dialogue Generation with Contrastive Latent Variables: Combining Sparse and Dense Persona\n\n[9] Improving Personality Consistency in Conversation by Persona Extending\n\n[10] Inducing anxiety in large language models increases exploration and bias\n\n[11] Red teaming chatgpt via breaking: Bias, robustness, reliability and toxicity.\n\n[12] Chatgpt an enfj, bard an istj: Empirical study on personalities of large language models.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. Provide a new corpus.\n2. Detailed data pipeline and case studies.", "reasons_to_reject": "1. Based on the data presented in Table 4, the two losses appear to have a limited impact on the overall framework.\n2. The paper lacks an in-depth analysis of the Evidence Lower Bound (ELBO), a critical component for CVAE models.\n3. The observed improvement in coherence is incremental.\n4. Comparing the personalization metric using a classifier derived from the same dataset that also trains the generation model may raise concerns of equity in evaluation.\n5. ChatGPT appears inherently equipped for this task, given that the dataset has been generated utilizing it.", "questions_to_authors": "", "paper_topic_and_main_contributions": "In this study, the researchers leverage ChatGPT to bootstrap an attributed dialogue corpus and subsequently deploy the Conditional Variational Autoencoder (CVAE) to dissect persona traits into distinct persona-related attributes systematically. The manuscript's salient contribution lies in its novel introduction of two unique losses, which, when integrated with the CVAE, promote the generation of tailored responses. Furthermore, the authors offer a dialogue corpus that has been augmented and meticulously enriched with personal attributes.", "excitement": 3, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Typos_Grammar_Style_and_Presentation_Improvements": "1. handicraft -> handcraft"}, "extra_text": {}}, {"review_id": "uBlhYbDIKt", "reviewer": "Reviewer_9UGD", "summary": "", "questions": "1. How can this model and results be reproduced? I request the authors to release their data and source code to enable better reproducibility.\n\n2. Why use a CVAE model? Why not other alternatives? CVAE is somewhat unconventional in language modeling tasks.\n\n3. I request the authors to improve the readability of the paper. The mathematical complexity must be simplified and replaced with more intuitive design and modeling choice explanations so that readers may grasp core ideas faster.", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your valuable feedback. We would like to provide a comprehensive response addressing each of your concerns.\n\n> (1)  It is somewhat unclear if these metrics are extensively leveraged in personalized dialogue generation.\n\nWe understand your reservations, classifier-based metrics have broad applications in personalized dialogue generation [1,2].\n\n> (2) limited reproducibility\n\nThank you for highlighting this, but we’d like to clarify that our code and training scripts have been provided in the supplementary materials,  alongside detailed test dataset cases in Appendix D. We will release the further revised dataset and model checkpoint in the carema-ready version to enhance reproducibility.\n\n> (3) Comparison with Large Language Models\n\nWe've conducted a comparison with ChatGPT in Table 3 and provided comprehensive inference cases, including ChatGPT results, in Appendix D.\n\n> (4)  training complexity &  practical deployment\n\nWe understand your concerns. We attach a new timing table below to address training and inference cost concerns. All baselines were trained and tested on a single RTX4090, except for ChatGPT called via API.\n\n| model(GPU=a single RTX4090) | training-time(20  epoch) | inference-time(1000  items) |\n| --------------------------- | ------------------------ | --------------------------- |\n| BOB                         | 5.9hours                 | 183s                        |\n| MSP                         | 21hours                  | 730s                        |\n| CLV                         | 3.3hours                 | 429s                        |\n| LMEDR                       | 60hours                  | 142s                        |\n| **MIRACLE (Ours)**          | **3.2hours**             | **121s**                    |\n| ChatGPT (on Many A100s)     | -                        | >1000s                      |\n\nAccording to the attached table, we can efficiently train our model on a single RTX3090/4090 GPU for several hours. This is facilitated by disentangling complex personalities into simpler attributes. And our model also has faster inference speed compared with baselines, thanks to our flexible EBM composition and customized ODE sampling methods.\n\n> (5) insufficient dataset \n\nDataset sufficiency is ensured for three key reasons.  (1) Our model's performance doesn't necessitate training an LLM or training from scratch, leveraging pre-trained bert-base and gpt2-medium models, as outlined in L258-265, hence doesn't demand a large volume of data.  (2) We disentangle complex personality into simpler attributes (e.g., language style, attitude, mental characteristics), efficiently aligning personality and simplifying data requirements for training. (3)  To ensure dataset quality, we've employed rigorous measures such as diverse conversation topics sampling, meticulous prompt design, multi-round human-involved cleaning, and detoxification, as outlined in L224-L236. \n\nA noteworthy observation is that the widely utilized ConvAI2 dataset, as exemplified in Table 1 of [6], is similarly sampled at a size of 43k in [6], which aligns with our dataset's sampling. The SOTA performance in Tables 1, 2 and detail cases in  Appendix D also prove the effectiveness of our dataset and methods.\n\n> (6) readability enhancement\n\nAppreciate your comments. We value specific suggestions for enhancement, though the reviewer `j4rf` praised the paper as “well-written and easily comprehensible” and `trw2` praised it as “well-motivated”. If possible, pinpointing which part is difficult to understand would greatly aid us in improving readability of the paper.\n\n> (7)  CVAE model choice\n\nCVAE is widely employed in dialogue generation [6,7,8]  due to its conditional nature, allowing it to leverage dialogue history. The structural latent space of VAE-based architecture facilitates efficient attribute modeling and manipulation[3,9]. Thus, we employ it as a foundational framework to model multiple personal factors, offering several advantages highlighted in L109-L110:\n\n- Circumvent the non-differentiable nature of the text: Direct personalization control is hard within a discrete text space due to its non-differentiable nature, which can be overcome by utilizing VAE-based latent space.\n- Better alignment/better Controllability:  Structural and compressed latent space simplifies and accelerates alignment with various attributes, ensuring better control. \n\n**reference**\n\n[1] Assigning Personality/Profile to a Chatting Machine for Coherent Conversation Generation\n\n[2] Controllable Text Generation with Focused Variation\n\n[3] Composable Text Control Operations in Latent Space with Ordinary Differential Equations\n\n[4] Quark: Controllable Text Generation with Reinforced Unlearning\n\n[5] Mix and Match: Learning-free Controllable Text Generation using Energy Language Models\n\n[6] Enhancing Personalized Dialogue Generation with Contrastive Latent Variables: Combining Sparse and Dense Persona\n\n[7] Dlvgen: A dual latent variable approach to personalized dialogue generation.\n\n[8] Learning Discourse-level Diversity for Neural Dialog Models using Conditional Variational Autoencoders\n\n[9] Controllable and Compositional Generation with Latent-Space Energy-Based Models", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. The proposed paper is nuanced in terms of the latent space construction. Although I'm unsure if the gains justify the complexity, this approach provides a feasible path to control aspects of the dialogue in comparison to direct text-inputs/user embeddings which are harder to interpret/control.\n\n2. Important and impactful problem area - As chatbots serve an increasing role in day to day user interactions with varied AI systems, personalization gains increasing importance to maintain user engagement and serve different user segments.", "reasons_to_reject": "1. Very limited reproducibility - Unless the authors release their training code, dialogue dataset, as well as model checkpoints, I find it very challenging to reproduce any of the claims in this paper. I encourage the authors to attach their code and datasets via anonymous repositories in the paper submission so that reviewers may verify the claims and try out the model for themselves.\n\n2. Very high model complexity - The proposed paper employs a mathematically and computationally complex approach as compared to the textual input method. Does the proposed method outperform sending textual inputs to a large foundation model such as LLAMA or ChatGPT? The training complexity seems too high for any practical deployment of this model.\n\n3. Dependent on the training data - I'm unsure if 44k dialogues is sufficient to capture a wide range of user traits and personalities across different content topics. LLMs are typically trained on trillions of tokens, I do not see how 44k dialogues can capture the combinations of personalities and topics. In theory, this dataset also needs to be massive to cover varied domains.\n\n4. The paper is hard to read and often unintuitive. The mathematical complexity must be simplified and replaced with more intuitive design and modeling choice explanations so that readers may grasp core ideas faster.", "questions_to_authors": "1. How can this model and results be reproduced? I request the authors to release their data and source code to enable better reproducibility.\n\n2. Why use a CVAE model? Why not other alternatives? CVAE is somewhat unconventional in language modeling tasks.\n\n3. I request the authors to improve the readability of the paper. The mathematical complexity must be simplified and replaced with more intuitive design and modeling choice explanations so that readers may grasp core ideas faster.", "paper_topic_and_main_contributions": "This paper is focused on the problem of personalized dialogue generation in chatbots. Specifically, the authors suggest that prior methods that leverage either user embeddings or explicitly feed user descriptions in the form of text are missing out on controlling multiple different aspects of the dialogue such as attitude of the users and personality. The authors of this paper develop a conditional-VAE based method to generate responses given the previous multi-turn dialogue consisting of user inputs and chat bot responses. The CVAE latent space is controlled by the authors using \"energy functions\" which are leveraged to control which aspects gain coverage in the responses generated by the model at inference time.\n\nThe authors claim that by training the proposed CVAE approach on 44k ChatGPT developed dialogues, the model has a \"strong understanding\" of various personality aspects and can then be leveraged to generate responses aligned to different personality aspects at inference time. The authors evaluate on a few standard metrics such as ROUGE/BLEU and several custom personality-metrics. It is somewhat unclear if these metrics are extensively leveraged in personalized dialogue generation. The method is then shown to outperform a few prior methods based on user-embeddings and textual inputs for personalization.", "excitement": 4, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "UzrOjyvZDQ", "reviewer": "Reviewer_j4rF", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely appreciate the time and effort you've dedicated to evaluating our manuscript.\n\n>  (1) release the dataset \n\nThank you for acknowledging our work. We plan to meticulously revise and release our dataset upon paper acceptance.\n\n> (2) analysis training and inference differentiation in CVAE\n\nThanks for your suggestion. There are two main distinctions in our CVAE's training and inference processes:\n\n**Firstly,** the CVAE architectural introduces extra posterior distribution $p(z|C,r)$ during training. It aligns the prior with the posterior to enhance its generation quality in inference time, a crucial aspect validated in [1,2,3]. We add an ablation experiment without posterior distribution to support this fundamental observation:\n\n|                     | personalization |       |       |       | Coherence |       |       | Fluency | Diversity |          |\n| ------------------- | --------------- | ----- | ----- | --------- | ----- | ----- | ------- | --------- | -------- | ----- |\n|                     | Avg.            | L.    | A.    | M.        | bleu  | rouge | nli     | ppl       | distinct | sbleu |\n| Miracle             | 92.75           | 93.30 | 93.10 | 91.86     | 45.23 | 15.21 | 70.76   | 23.68     | 70.94    | 8.90  |\n| w/o CVAE  posterior | 86.26           | 90.86 | 88.53 | 79.38     | 38.14 | 9.79  | 1.52    | 40.04     | 54.13    | 60.82 |\n\nWe can observe that without the basic posterior distribution, our model experiences degradation across all metrics. Particularly, there's a catastrophic collapse observed in NLI. This is because, without guidance from $p(z|C,r)$, our encoder cannot learn the latent relationships between the response $r$ and dialogue context $C$. Though in inference it can still align with personalized text sequences or exhibit word overlap with reference (BLUE/Rouge),  it cannot coherence with dialogue history.\n\n**Secondly,** our unique design trains $z$ to align with a single facet of personality, while in inference, we sample $z$ to encapsulate multiple factors to represent complex personality. To elaborate on the performance effect caused by this distinction, we've provided results for both \"inference with single attribute\" and \"inference with multiple attribute\" result as follows:\n\n|          | personalization |       |      |        | Coherence |       |       | Fluency | Diversity |          |\n| -------- | --------------- | ----- | ----- | --------- | ----- | ----- | ------- | --------- | -------- | ----- |\n|          | Avg.            | L.    | A.    | M.        | bleu  | rouge | nli     | ppl       | distinct | sbleu |\n| L.  (single)     | 97.29           | 97.29 |       |           | 45.87 | 15.23 | 77.66   | 23.59     | 70.31    | 8.83  |\n| A.  (single)     | 95.67           |       | 95.67 |           | 45.62 | 14.54 | 73.18   | 21.59     | 70.39    | 8.64  |\n| M.  (single)     | 93.48           |       |       | 93.48     | 45.28 | 14.42 | 67.00   | 20.21     | 70.29    | 9.12  |\n| L.+A.+M.  (multiple, the same as Table1) | 92.75           | 93.30 | 93.10 | 91.86     | 45.23 | 15.21 | 70.76   | 23.68     | 70.94    | 8.90  |\n\nUpon comparing the two scenarios, we observe a decrease in personalization performance and slight variations in other metrics when addressing multiple attributes. This observation suggests the potential existence of contradictions among these attributes, which our model adeptly manages.\n\n**reference**\n\n[1] Learning structured output representation using deep conditional generative models\n\n[2] Enhancing Personalized Dialogue Generation with Contrastive Latent Variables: Combining Sparse and Dense Persona\n\n[3] Improving Personality Consistency in Conversation by Persona Extending", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. The proposed approach appears novel and seems to be well-founded.\n2. The suggested method significantly addresses the issue of sparse persona descriptions, enabling effective control over pre-defined attributes associated with personas. This innovation facilitates the creation of dialogues comprising multiple personas using a single model.\n3. The paper is well-written and easily comprehensible.", "reasons_to_reject": "To train the proposed learning method, additional training data for each attribute is required. The paper does not currently provide access to the newly collected dataset. It would be beneficial to release the dataset to facilitate experiment replication, and if shared, it could positively impact further research endeavors.\n\n2. For generative models based on CVAE, such as the one proposed in this paper, there is a distinction between the latent variable z used during the training phase and that used during the inference phase. Analyzing the potential performance degradation resulting from this differentiation could lead to enhanced experimental insights and analysis.", "questions_to_authors": "", "paper_topic_and_main_contributions": "This study proposes a novel learning approach to address the inherent limitation of personality sparsity in persona-aware dialogue. To overcome this challenge, the study introduces a method that dissects complex personalities into multi-faceted attributes and models them within a latent space. This enables effective control over the style of the target persona. The research is intriguing and meaningful due to its ability to tackle the fundamental issue of persona-aware dialogue, which typically demands costly manually crafted descriptions for the target persona.", "excitement": 3, "reproducibility": 2, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Typos_Grammar_Style_and_Presentation_Improvements": "1. L345 \"dialogue turns, Further\" -> \"dialogue turns. Further\"\n\n"}, "extra_text": {}}, {"review_id": "UNFYhOzSdy", "reviewer": "Reviewer_trw2", "summary": "", "questions": "A.\tThe performance differences between the baseline models in Table 1 and MIRACLE are too large. Did the authors train and test these baseline models using the same personality (attributes) and context data as MIRACLE? How did you input personality (attributes) data for training and testing other methods?\n\n\nB.\tIn Appendix C.3, it is stated that authors reported the average BLEU, Rouge, and Distinct score by calculating the mean of BLEU-1/2/3/4, Rouge-1/2/L, and Distinct1/2/3, respectively. However, each of them calculate n-gram overlap and unique n-gram differently and represent different aspect. For instance, though R-1 score is low, R-L score can be high. In most case, many papers report separated results for each n-gram when using BLEU, Rouge, and Distinct. (i.e., Report R-1, R-2, and R-L separately, not as an average.)\n", "limitations": "", "rating": 3, "confidence": 4, "soundness": 2, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your time and comments. We have carefully considered your feedback and would like to provide a comprehensive response addressing each of your concerns.\n\n> (1) quality of the generated data\n\nThis is indeed a good concern, we’ve taken several meticulous steps to guarantee its quality, as elaborated in L224-L236. We've meticulously crafted many prompts and carefully selected instruction templates, detailed in Appendix A.1. Additionally, our dataset has undergone human evaluation and meticulous cleaning procedures, as outlined in Appendix A.2. Further insight into the test dataset's cases can be found in Appendix D.\n\n> (2) applicability of the method & only applied to this dataset\n\nWe’d like to emphasize that our approach to multi-faceted personality is highly adaptable and not constrained to the specific attributes like language style, attitude, and mental characteristics that we utilized in our study. This flexibility means that our method can be effortlessly extended to work with other datasets involving different sets of facets, such as attributes like energetic, inner, and enthusiastic, depending on your specific customization of personality traits.\n\n> (3) why MIRACLE is needed cause it uses ChatGPT-generated data\n\nWhile we indeed utilize ChatGPT-generated data for training, it's important to note that the training data solely annotates with **a single facet** of personality exclusively (e.g., language style, attitude, mental characteristics), as mentioned in L236-L240.  In contrast, during inference, our model adeptly manages **multiple-faceted** personalities in a composition manner. This process is distinct from ChatGPT and ensures precise control over various personality attributes.\n\nWe recognize ChatGPT's inherent capabilities in single-faceted conversations [1,2], that is why we utilize it to generate a training corpus. However, it faces limitations in handling multiple attributes [2,3], as mentioned in L70-L79 of our manuscript.  In contrast, our method excels in controlling multiple personal attributes simultaneously in terms of personalization metrics, as evidenced by Table 3.\n\nWe would also like to emphasize that our model is simply based on the bert-base and gpt2 model (not LLM), which is more lightweight and has faster inference speed, as demonstrated in the table below:\n\n| model(GPU=RTX4090)      | training-time(20  epoch) | inference-time(1000  item) |\n| ----------------------- | ------------------------ | -------------------------- |\n| BOB                     | 5.9hours                 | 183s                       |\n| MSP                     | 21hours                  | 730s                       |\n| CLV                     | 3.3hours                 | 429s                       |\n| LMEDR                   | 60hours                  | 142s                       |\n| MIRACLE(Ours)           | **3.2hours**             | **121s**                   |\n| ChatGPT (on Many A100s) | -                        | >1000s                     |\n\n> (4) The L., A., and M. scores of ChatGPT in Table 3 are lower than those of MIRACLE. does it mean that the training and test data generated by ChatGPT are not good enough to be a gold standard.\n\nAs explained above, ChatGPT excels in generating single-attribute conversations but struggles with handling multiple attributes simultaneously. This disparity might stem from the ambiguity and instability inherent in manually crafted prompts. Consequently, ChatGPT might prioritize more overt attributes like language style and attitude while overlooking more abstract mental characteristics. In contrast, our approach, leveraging personality disentanglement and EBM design, enables more precise control over multiple attributes, as discussed in L494-L505.\n\nConsequently, the training data, focusing on a single facet of personality, can serve as the gold standard for single attribute alignment. However, the multiple-faceted test data is not optimal for evaluating personalization. Consequently, during inference, we assess the personalization score using classifiers, which is independent of responses generated by ChatGPT.  \n\nOnly regarding BLEU/Rouge metrics in inference, which require reference responses for coherence assessment, we roughly treat ChatGPT responses as the gold standard, for the widespread recognition of ChatGPT's ability to generate highly coherent and fluent responses.\n\n> (6)  Did you train and test these baseline models using the same data? The detail of baselines’ input?\n\nYes, we did train and test these baseline models using the same training data that bore personal attributes and context as MIRACLE.  \n\nIt's important to highlight that our approach for personality alignment only necessitates attribute-annotated data, without any need for extra prompt input. The embedding-based methods (MSP, CLV) are designed to automatically derive personality from dialogue history, so we simply input the same data except for the attribute label. The text-description methods (BOB, LMEDR) require additional identity descriptions, which we input adapted prompts following [4,5], as mentioned in L413-L418. \n\n> (7) large performance differences?\n\nDue to the constrained expressive capacity of textual descriptions and the scarcity of personality information in implicitly derived embeddings (as elaborated in our introduction Section), both text-description methods encountered challenges in effectively controlling multi-faceted scenarios. \n\nIn contrast, our approach, leveraging personality disentanglement and EBM design, enables more precise control over multiple attributes, as discussed in L494-L505. The detailed comparision cases have been presented in Appendix D.\n\n> (8) question about the mean of metrics\n\nThanks for the suggestion. Due to page limitations, we only provide the mean of BLEU-1/2/3/4, Rouge-1/2/L, and Distinct1/2/3 scores. Detailed results are provided in the table below, where we can observe consistent trends across various metrics such as R1/2/L and BLEU1/2/3/4. In view of this consistency, we've opted for mean metrics to maintain simplicity.\n\n|                     | BLEU-1 | BLEU-2 | BLEU-3 | BLEU-4 | BLEU-Avg |\n| ------------------- | ------ | ------ | ------ | ------ | -------- |\n| BoB                 | 61.29  | 44.67  | 30.72  | 21.39  | 39.52    |\n| MSP                 | 57.38  | 41.40  | 27.82  | 18.84  | 36.36    |\n| CLV                 | 57.14  | 41.99  | 28.98  | 20.43  | 37.13    |\n| LMEDR(26571.sample) | 65.62  | 49.55  | 36.04  | 26.78  | 44.50    |\n| Miracle             | 67.44  | 50.75  | 36.31  | 26.41  | 45.23    |\n\n|                     | ROUGE-1 | ROUGE-2 | ROUGE-L | ROUGE-Avg |\n| ------------------- | ------- | ------- | ------- | --------- |\n| BoB                 | 20.79   | 3.54    | 16.08   | 13.47     |\n| MSP                 | 18.60   | 2.68    | 14.35   | 11.88     |\n| CLV                 | 20.30   | 3.76    | 15.71   | 13.26     |\n| LMEDR(26571.sample) | 24.90   | 5.52    | 19.08   | 16.50     |\n| Miracle             | 23.85   | 4.17    | 17.62   | 15.21     |\n\n|              | DIST-1 | DIST-2 | DIST-3 | DIST-Avg |\n| ------------ | ------ | ------ | ------ | -------- |\n| BoB          | 49.03  | 66.15  | 76.93  | 64.04    |\n| MSP          | 50.49  | 69.09  | 80.08  | 66.55    |\n| CLV          | 48.69  | 67.22  | 77.61  | 64.50    |\n| LMEDR(28134) | 52.18  | 70.65  | 79.17  | 67.33    |\n| Miracle      | 54.49  | 74.47  | 83.86  | 70.94    |\n\n**reference**\n\n[1] Inducing anxiety in large language models increases exploration and bias\n\n[2] Red teaming chatgpt via breaking: Bias, robustness, reliability and toxicity.\n\n[3] Chatgpt an enfj, bard an istj: Empirical study on personalities of large language models.\n\n[4] One Chatbot Per Person: Creating Personalized Chatbots based on Implicit User Profiles\n\n[5] Less is More: Learning to Refine Dialogue History for Personalized Dialogue Generation.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "A.\tThis paper is well-motivated. It highlights the difficulties and problems of conventional text description-based method and user embedding-based method in terms of data characteristics. They have self-defined three attributes of personalization characteristics, and two aspects for each attribute to specify persona.\n\nB.\tThe authors proposed new training objectives to reflect personalization characteristics when generating text. Experimental results showed that MIRACLE can achieve better performance than other methods.\n", "reasons_to_reject": "A.\tThe authors have assumed the conversation generated by ChatGPT as a golden standard and evaluated performance comparing with it. However, this is also a model output, and furthermore, the quality of the generated data depends on the prompt. It seems that MIRACLE can be only applied to this new dataset. But, this data set can be generated by ChatGPT. Thus, it is not clear why MIRACLE is needed.\n\nB.\tThe L., A., and M. scores of ChatGPT in Table 3 are lower than those of MIRACLE. Then, does it mean that the training and test data generated by ChatGPT are not good enough to be a gold standard. \n\nC.\tIt is not clear whether the persona (attribute) were also used for training and testing other models.\n", "questions_to_authors": "A.\tThe performance differences between the baseline models in Table 1 and MIRACLE are too large. Did the authors train and test these baseline models using the same personality (attributes) and context data as MIRACLE? How did you input personality (attributes) data for training and testing other methods?\n\n\nB.\tIn Appendix C.3, it is stated that authors reported the average BLEU, Rouge, and Distinct score by calculating the mean of BLEU-1/2/3/4, Rouge-1/2/L, and Distinct1/2/3, respectively. However, each of them calculate n-gram overlap and unique n-gram differently and represent different aspect. For instance, though R-1 score is low, R-L score can be high. In most case, many papers report separated results for each n-gram when using BLEU, Rouge, and Distinct. (i.e., Report R-1, R-2, and R-L separately, not as an average.)\n", "paper_topic_and_main_contributions": "The paper introduces a new method for personalized dialogue generation called MIRACLE, which disentangles complex personalities into three attributes (language style, attitude, and mental characteristic) and utilizes an energy-based model(EBM) for generation. The paper also proposes two new loss term that can make personalized text generation better (aspect classification loss and attribute distance loss). The paper presents experimental results on both automatic and human evaluations to verify the effectiveness of MIRACLE, and case studies. The results demonstrate that MIRACLE outperforms several baselines in terms of personality controllability and response generation quality.", "excitement": 3, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper introduces a new method for personalized dialogue generation called MIRACLE, which disentangles complex personalities into three attributes (language style, attitude, and mental characteristic) and utilizes an energy-based model(EBM) for generation. The paper also proposes two new loss terms that can make personalized text generation better (aspect classification loss and attribute distance loss). 

The paper has demonstrated many merits but can still be largely improved by addressing the following issues:
1. The observed improvement in coherence is incremental.
2. An in-depth analysis of the synthetic data. Do these data have sufficient diversity? Can they capture the characteristics of human dialogues well?
3. It would be nice to add the results of LLama, will the proposed approach still be much more efficient than LLama?
4. Authors are encouraged to release their code, model checkpoints, and datasets generated.|meta review
Decision: Accept-Findings