Conversation Chronicles: Towards Diverse Temporal and Relational Dynamics in Multi-Session Conversations
Authors: Jihyoung Jang
MinSeong Boo
Hyounghun Kim
Venue: EMNLP 2023 Main (2023)
Keywords: Multi-Session Dialogue
Long-Term Conversation

Abstract: In the field of natural language processing, open-domain chatbots have emerged as an important research topic. However, a major limitation of existing open-domain chatbot research is its singular focus on short single-session dialogue, neglecting the potential need for understanding contextual information in multiple consecutive sessions that precede an ongoing dialogue. Among the elements that compose the context in multi-session conversation settings, the time intervals between sessions and the relationships between speakers would be particularly important. Despite their importance, current research efforts have not sufficiently addressed these dialogical components. In this paper, we introduce a new 1M multi-session dialogue dataset, called Conversation Chronicles, for implementing a long-term conversation setup in which time intervals and fine-grained speaker relationships are incorporated. Following recent works, we exploit a large language model to produce the data. The extensive human evaluation shows that dialogue episodes in Conversation Chronicles reflect those properties while maintaining coherent and consistent interactions across all the sessions. We also propose a dialogue model, called ReBot, which consists of chronological summarization and dialogue generation modules using only around 630M parameters. When trained on Conversation Chronicles, ReBot demonstrates long-term context understanding with a high human engagement score.

CONVERSATION CHRONICLES: Towards Diverse Temporal and Relational Dynamics in Multi-Session Conversations
Jihyoung Jang Minseong Boo
Hyounghun Kim
Artificial Intelligence Graduate School, UNIST [jihyoung, b.ms, h. h.kim@euistt.ac. kr
Abstract
Relationship: Co-workers
Session
the field natural language processing, open-domain chatbots have emerged as im- portant research topic. However, a major limita- tion of existing open-domain chatbot research is its singular focus on short single-session di- alogue, neglecting the potential need for un- derstanding contextual information in multiple consecutive sessions that precede an ongoing dialogue. Among the elements that compose the context in multi-session conversation tings, the time intervals between sessions and the relationships between speakers would be particularly important. Despite their impor- tance, current research efforts have not suffi- ciently addressed these dialogical components. In this paper, we introduce new IM multi- session dialogue dataset, called CONVERSA- TION CHRONICLESS, for implementing long- term conversation setup in which time intervals fine-grained speaker relationships are in- corporated. Following recent works, exploit a large language model to produce the data, The extensive human evaluation shows that di- alogue episodes in CONVERSATIO CHRONI- CLES reflect those properties while maintaining coherent and consistent interactions across all the sessions. We also propose dialogue model, called REBOT, which consists of chronolog- ical summarization and dialogue generation modules using only around 630M parameters. When trained on CONVERSATION CHRONI- CLES, REBOT demonstrates long-ten tere-ermmoe context understanding with a high human engagement score.
Hey, let's take break and
Sounds good to need to cool down after working in this heat all day.
you go, beer for my hard-working callengue.
Thanks, Cheers!
A couple of years after
Session
Yeah, I did. But I'm worried falling work and losing my job.
know taugh. And I'm sure your boss will un- derstand.
hope you're right. Thanks and sup-
Anytime. Remembar when we had that relaxing moment with a couple of warking in sun all day? Maybe we similar moment once you're out the hospital.
porting me.
Figure I: A sample of a multi-session conversation from CONVERSATION CHRONICLES. Based on the es- tablished relationship CONVERSATION CHRONICLESS provides the relevant conversation for the user. ses- sion N, the co-workers hold conversation based on information remembered from previous sessions.
conversation models (Rashkin et al., 2019; Zhang et al., 2020; Roller al., 2021; Shuster al., 2022). However, although these chatbot models produce human-like fluent responses, they seem to have a limited ability that only understands short-term dia- logue context, making them less applicable in real- world scenarios in which long-term conversational situations are often encountered. Specifically, they do not care about the context of past conversations and only generate responses based on an ongoing dialogue (so-called single-session dialogue).
1 Introduction
Open-domain conversation one the important research topics. By deploying conversation sys- tems in our daily lives, we can enjoy automated ser- vices like counseling, language tutoring, etc. There has been much research effort to build such AI
OOu data/code are publicly available at https:// conversation-chrontte
www.eeronnneegggi.......... github. io/
To address these issues, the multi-session con-

---

versation has been proposed (Xu et al., 2022a). Multi-session conversation comprises consecutive sessions that make a coherent dialogue episode. In a multi-session conversation, each session is assumed to occur serially with a time interval in between. Time interval plays an important role to infuse dynamics in conversational interaction between speakers. For instance, depending on the time elapsed since the last conversation, their re- sponses about past events would vary. However, previously introduced works have relatively short range time intervals, limiting types of transitions from the past sessions. Also, to our best knowl- edge, there is no research effort to incorporate the relationship between speakers into conversations. The relationship can significantly rule the way they perceive and interact with each other, giving an additional dimension of dynamics a dialogue.
We also propose new multi-session conversa- tion model, REBOT. This model uses only about 630M parameters and reflects the chronological and relational dynamics in the long-term conversa- tion setting. The extensive human evaluation shows that its responses are preferred over other chatbot models in long-term conversational situations. Our contributions in this study are:
1. We introduce CONVERSATION CHRONICLES, a new IM multi-session dataset that includes more various time intervals and fine-grained speaker relationships.
2.
We propose REBOT which can generate dia- logues with the chronological dynamics with only about 630M parameters.
Extensive human evaluation verifies that RE- BOT trained on CONVERSATION CHRONI- CLES shows user engagement in situations with various temporal and relational contexts.
Therefore, we introduce CONVERSATION CHRONICLES, new high-quality long-tern con- versation dataset that consists of 1M multi-session dialogues (200K episodes; each episode has di- alogue sessions). CONVERSATION CHRONICLES features more diverse chronological context and fine-grained speaker relationships. Time interval in CONVERSATION CHRONICLES includes varying ranges from few hours to even years, allowing to cover longer elapsed time than previous multi- session dialogue settings. Also, various relation- ships induce varied events and interaction flows to the conversations, which facilitates the application to different real-world scenarios (Figure 1).
2 Related Works
Open-domain Chatbot. Building human-like open-domain dialogue models an important re- search topic the field of natural language pro- cessing. Diverse datasets have been proposed to study such chatbots. Previous studies of open- domain dialogue datasets are DailyDialog (Li et al., 2017), PersonaChat (Zhang et al., 2018), Wiz- ard of Wikipedia (Dinan et al., 2019), Empa- thetic Dialogues (Rashkin et al., 2019), Blended- SkillTalk (Smith al., 2020), twitter (Ritter et al., 201 I) and Pushshift.io Reddit (Baumgartner et al., 2020). However, these dialogue datasets consist of short, single sessions, making it difficult to re- flect real-world conversational scenarios in which conversations occur in series with time intervals.
On the other hand, collecting data samples, which requires sophisticated interaction, at scale is not easy and consumon.. Thus, recent works are getting turning to exploit large language mod- els (L..Ms) to collect such complicated data in an automated way by designing refined query meth- ods (Kim et al., 2022a; Taori et al., 2023; Xu et al., 2023; Zheng et al., 2023; Gilardi et al., 2023). Fol- lowing those works, collect our conversation dataset through well-defined prompts to L.MMs.? To be specific, each prompt consists of relationships, event descriptions, and time in- tervals so that the created dialogues incorporate those components. According to human evaluation based on multiple criteria, our CON VERSAAION CHRONICLES is preferred to other multi-session conversation datasets.
Long-term Conversation. Current open-domain dialogue models learn from short conversations with little context, which has the obvious limitation that they will not remember the information for future conversations. To address these issues, there are attempts to add modules to the standard archi- tecture or propose datasets for long term situations. Wu et al. (2020) proposes a method of extracting and managing user information from dialogues. Xu et al. (2022b) proposes Chinese multi-turn dataset DuLeMon and a persona memory- based framework PLATO-LTM. Xu et al. (2022a) pro- poses the first multi-session dataset, called MSC, with time intervals between sessions. Also, Bae
We use ChatGPT (OpenA1, 2022) in this study, hut other LLMs, like Bard (Google, 2023), could also he employed.

---

al. (2022) proposes a dynamic memory manage- ment method to keep user information up-to-date and introduces a Korean multi-session dialogue dataset, CareCallmen- However, previous multi- session datasets have a limited or fixed range of time intervals and they have less focused on the im- pact of the time interval in training dialogue models. Also, there is still no open-domain dialogue dataset that constructs conversations taking into account the relationship between speakers, which is quite important for engaging conversation experiences. To our best knowledge, CONVERSATION CHRON- ICLES is the first open-domain dialogue dataset that defines the fine-grained relationships between speakers with diverse range of time intervals.
3.1 Event Collection
In a single-session dialogue, two speakers engage in conversation around a specific event ignoring any past context. On the other hand, in multi- session dialogue, the context of previous sessions is taken into account and reflected in the conversation of the ongoing session. This ensures coherence and continuous conversational experience in long- term conversations by preserving the context of the entire sessions.
Therefore, when creating multi-session dia- logues, it is important to keep consistent and coherent context throughout episode. To guaran- tee this, we build event graph by linking related events. To be specific, we employ the narratives from SODA (Kim et al.. 2022a),² which one of the large-scale dialogue datasets, and use them as the events (i.e., one narrative corresponds to an event). Then, we connect each event to another based on their relevance and build them into a graph as follows.
Data Distillation. Data collection is one of the most challenging problems in training AT mod- cls. This is due to copyright and privacy issues, as well as the high cost of hiring humans to generate high-quality data. Since the large language mod- els emerged, researchers have been trying to solve the data collection problem by using them. Zheng et al. (2023) uses GPT-J (Wang and Komatsuzaki, 2021) to generate an emotional dataset, AugESC. Through an augmentation framework using GPT- 3 (Brown et al., 2020), Kim et al. (2022b) created ProsocialDialog, dataset that teaches conversa- tional agents respond to problematic content based on social norms. Kim al. (2022a) uses In- structGPT (Ouyang et al., 2022) to create dialogues from narratives. Xu et al. (2023) demonstrates cre- ating a single-session dataset using ChatGPT (Ope- nAI, 2022). These studies suggest that building datasets using LLMs can save time and cost, and also enable the creation of high-quality datasets that comparable
Event Pairing. We use natural language infer- ence (NLI) as the method for linking two related events since it is one of the most reliable ways to model relationships between sentences. An event pair is classified entailment, neutral, contra- diction, depending on whether they are related or not. We compute the relationship between all pos- sible event pairs and retain only ones that have the entailment relationship. We employ the pre-trained BERT-base model (Devlin et al., 2019) and fine- tune it on the SNLI (Bowman et al., 2015) corpus.
Event Graph Building. Since a graph is an effec- tive structure for modeling relationships between nodes, we conceptualize events as nodes and con- nect the entire event pairs using edges. To prevent temporal contradiction between events, we use a directed graph by which the order of premises and hypotheses is specified. From the graph, we tract all possible event sequences with a length of 5, then remove ones if they have more than 3 events in common, leaving only one of them in the list.
writtew ones (Gilardi al., 2023). Therefore, we also leverage LLMS efficiently build the large-scale multi-session dataset, CONVERSATION CHRONICLES.
3 CONVERSATION CHRONICLES
3.2 Chronological Dynamics
In this study, we introduce a new high-quality long- term multi-session conversation dataset, called CONVERSATION CHRONICLES. Our dataset con- sists of IM multi-session dialogues, comprising a total of 200K episodes, each of which consists of 5 sessions. We construct our dataset using the following process.
CONVERSATION CHRONICLES integrates diverse temporal contexts and fine-grained speaker relation- ships in -esti-n conversations implement chronological dynamics. Unlike single session one,
3 Althooghh we use the SODA's narrative in this study, any event descriptions could also be used.

---

MSC
CONVERSATION CHRONICLES
Event List
Event Graph
Time Interval
(Xu 2022a)
lew hours
3,497
159,975
lew days
3,510
159,928
few weeks
160,670
few months
160,050
couple of years
159.377
Table I: Statistics of the time interval between sessions of MSC and our dataset. We aggregate 1-7 hours as a few hours and 1-7 days as a few days in MSC.
Dialogue
Episode
Relationship
Count
Ratio
Clussmates
66,090
33.05%
Neighbors
49,521
24.76%
Co-workers
28,856
14.43%
Relationshpp
Interval
Mentee and Mentor
16,035
8.02%
Husband and Wife
13,486
6.74%
Figure 2: The overall data collection process of CON- VERSATION CHRONICLES.
Patient and Doctor
6,980
3,49%
Purent Child
6,514
3.26%
Student and Teacher
5,018
2.51%
connected to the dialogue context (i.c., events), it would not be appropriate to assign them randomly. Therefore, we pre-define 10 relationships that are typically found in daily lives and assign them by querying ChatGPT. To be specifie, we provide all events in an episode with the list of 10 rela- tionships, then ask ChatGPT to select the most appropriate relationship for the events. Please see Table 2 for the frequency of the relationships and Appendix A for the prompts used.
Employee and Boss
4,811
2.41%
Athlete und Coach
2,689
1.34%
Total
200,000
Table 2: Statistics of the relationship between speakers in CONVERSATION CHRONICLES.
a multi-session conversation considers previous ses- sions, having time interval between each consec- utive session pair. While previous studies have employed the time interval between sessions, the interval typically ranges from few hours to several days, only allowing for a relatively short-term con- versational context. Also, there has been no prior effort to apply the relationships between speakers to conversational interactions, thus limiting the va- ricty and leading monotonous interactions.
3.3 Conversation Episode Generation
LLMs are reported to be able to produce diverse and high-quality data samples that are comparable to those written by humans (Gilardi et al., 2023). Recent works have also reported the use of LLMs to collect dialogues (Kim et al., 2022a; Xu et al., 2023). Thus, we leverage LLMs to generate dia- logues. Specifically, we collect episodes through ChatGPT (OpenAI, 2022) by designing a series of sophisticated prompts. One prompt for session consists of an event, a time interval, and a speaker relationship as the conditions of the current dia- logue, while also containing the full context (events and time intervals of previous sessions). Please re- fer to Appendix A for examples of the full prompts.
Time Interval. To address the short-term inter- val limitation and allow for longer dynamics, we define longer chronological context ranging from few hours few years: "a few hours later", "a few days later", "a few weeks later", "a few months later", and "a couple of years later". We randomly pick one and assign it as a time interval for a con- secutive session pair. We employ approximate time representations (i.e., 'a few' or 'a couple of") rather than a numerical time amount (e.g., '3 days') since we find that minute differences in time units have little effect the context. Please refer Table see the comparison.
Using the prompts, we construct large-scale multi-session conversation dataset, CONVERSA- TION CHRONICLES. By integrating the aforemen- tioned ingredients (events, time intervals, and rela- tionships), it implements chronological dynamics making the multi-session conversation setting more diverse. Please see Figure 2 for the overall process of the data collection.
Speaker Relationship. We define a fine- fine-grained speaker relationship for each episode to give in- teractional dynamics to a dialogue. Relationships between speakers are one of the crucial elements of dialogue since determine the contents that they are speaking about. Since relationships are closely
We collect a total of 200K episodes each of which has 5 sessions, resulting in IM dialogues.

---

Dutasets
Language
of Sessions
of Episodes
of Turns
Avg. Turns per Session
Avg. Words per Turn
MSC (Irain, up sessions)
English
12K
4K
161K
13.5
MSC (train, sessions)
English
4K
1K
53K
13.3
CareCallaza
Korean
7.7K
160K
20.9
4,93
CONVERSATION CHRONICLES
English
IM
200K
11.7M
11.7
18.03
Table 3:
Comparison of ours with other multi- session datascts, Statisties for MSC and CareCallmem are taken from their papers. As we can see, our CONvERSATION CHRONICLES has the largest scale. On the other hand, the average of turns per session is smaller than other datasets, but the average of words per turn much higher.
Please see Table 3 for more statistics of our CON- VERSATION CHRONICLL'S dataset. As the statistics show, we build significantly larger multi-session conversation set than the others. Please see Ap- pendix H for the full dialogue episodes.
Metrics
Avg
Std
Consistency
4.41
Coherence
4.04
1.06
Time interval
4.46
0.77
Relationship 4.40
Overall
4.33
3.4 Quality
Table 4: Human evaluation on the quality of CONVER- SATION CHRONICLES.
Automatic Filtering. Data generated by an LLM does not always guarantee uniform quality. It may include unnecessary information or deviate from the given format. To ensure the consistent quality of our dataset, implement automatic process to filter out such cases (please detailed process in Appendix B). Furthermore, our dataset might have the potential to contain toxic data. Thus, we use Moderation (Markov al., 2023) to remove the harmful data.
4:71
4.51
13.65
15.65
Corseterry
Coheranoe
intrval
Oreral
HSC
Metrics
Figure 3: Comparative evaluation of MSC and CON- VERSATION CHRONICLES (CC).
Human Evaluation. We conduct human evalua- tions verify the quality of CONVERSATION CHRONICLES (see Section 5 for the evaluation de- tails). We sample 5K episodes and ask evaluators to rate each of them based on four criteria (co- herence, consistency, time interval. and relation- ship). Table 4 shows the quality of CONVERSA- TION CHRONICLES,, with an average score of 4.34 out 5 which is quite high considering 5 indicates 'perfect'.
marization module provides the context of the pre- vious sessions by concisely describing the chrono- logical events. The dialogue generation module produces the next response reflecting chronological dynamics presented by the summarization module.
4.1 Chronological Summary
Multi-session conversations must take into account the chronological connectivity between previous and current sessions, and appropriately reflect changes in event states over time. The best so- lution to soundly incorporate this information in the dialogues model would be put the entire conversation history of previous sessions as text. However, it is not computationally efficient to maintain such a system. To address this ineffi- ciency, we propose a summarization module that depicts each of the past dialogue sessions while minimizing information loss.
We also conduct a comparison with MSC (Xu et al., 2022a), one of the representative multi- session conversation datasets, based on the same criteria as the previous evaluation, excluding the re- lationship since the MSC does not have it. We ran- domly sample 0.5K dialogue episodes from each dataset for comparison. As shown in Figure 3, our dataset has higher scores across all metrics, mean- ing our CONVERSATION CHRONICLES has such high quality.
To collect training data for this summarization module, randomly sample 100K sessions (i.e., 20K episodes) from CONVERSATION CHRONI- CLES and use ChatGPT generate summary for each session. We employ TS-base (Raffel et al., 2020), and use 80K from the generated summaries
4 REBOT
We propose novel multi-session dialogue model, REBOT (REmember ChatBOT). REBOT consists two parts: the chronological summarization mod- ule and the dialogue generation module. The sum-

---

Session N-1
Summary
Relationhipp
Summary
Time Interval
Response Generator
Summary
Session
Response
Figure 4: The overall architecture REBOT. consists summarization module and generation module. yellow box indicates the relationship. and the pink box indicates the time interval between dialogue sessions. The summarization module summarizes the previous sessions as input to the generation module.
for training (keeping 20K for val/test splits). The module takes session dialogue as input and gener- ates a chronological summary as output.
5.2 Human Evaluation
Evaluating the quality of open-domain conversa- tions is considered challenging. The reference- based evaluation metries (such as BLEU (Papineni et al., 2002), ROUGE (Lin, 2004), etc.) might not be suitable for use as evaluation metries for open-domain dialogues, for which a wide variety of generations could be considered as proper re- sponses (Liu et al., 2016). Therefore, human eval- uation is desirable to faithfully verify the quality of dialogues on various aspects (such as coherence, contradiction, engagement, etc.). As such, in this study, rely extensive human evaluation for examining the quality of our dataset, CONVERSA- TION CHRONICLES, and dialogues generated by our REBOT.
4.2 Dialogue Generation
To generate utterances in an ongoing session, the dialogue generation module should consider the dialogue history (i.e., previous session summaries), the relationship between speakers, and the time elapsed from the last session. We use a sequence- to- to-sequence architecture, i.e., BART-large (Lewis et al., 2020), to effectively accommodate all the components to be considered during the genera- tion process. Formally, the conditional probability for generating the next response is P(crr, f, h). where is the next utterance, is the relationship, / is the time interval, is the summary, and h the current dialogue context (i.e, previously generated utterances). The input format to the module looks
5.3 Dataset Quality Evaluation
CONVERSATION CHRONICLES implements the chronological dynamics multi-session conver- sation environment. To ensure that our dataset faith- fully reflects the elements (events, time intervals, and relationships) in the dialogues, randomly sample 5K episodes for evaluation, then conduct human evaluation by asking the evaluators rate the dialogues based on 'Coherence', 'Consistency', 'Time interval', and 'Relationship'. Please see Ap- pendix for the detailed definition those criteria.
like: <relationship>> <ty> SN-1 <user>> 11 <bot>> Ci <user>>..
<bot>> Cu-
The overall model architecture shown in Fig- ure 4. REBOT trained on CONVERSATION CHRON-
ICLES can seamlessly generate multi-session dia- logues considering chronological events and long- term dynamics only with 630M parameters.
5 Experiments
5.1 Implementation and Training Details
5.4 Comparison to Other Datasets
We also perform a human evaluation for compari- son with an existing multi-session dataset. Since just a few multi-session datasets have been intro- duced and MSC (Xu et al., 2022a) is the only case
We split the datuset into 160K for train, 20K for val- idation, and 20K for test, out of 200K episodes (IM dialogue sessions). We use AdamW (Loshchilov and Hutter, 2019) as the optimizer and cross- entropy loss as the training objective for all genera- tion tasks. Please see Appendix for more details.
"We hired 41 evaluators from professional evaluation agency and 5 in-house evaluators for the evaluation.

---

Metrics
Avg
Engagingness
4.78
0.54
4.05
Humanness
4.74
0.63
255
Memorability
4.14
0.79
Overall
4.55
Table 5: Human evaluation for the quality of dialogue episodes generated by REBOT.
2:10
for fair comparison due to its session length and language, we choose MSC for the comparison.
Figure 5: Comparative evaluation results of MSC 2.7B and REBOT.
We extract all possible episodes with 5 sessions from MSC (validation 0.5K, test 0.5K, total IK), then randomly sample 0.5K episodes for this com- parison. We also randomly sample 0.5K episodes from CONVERSATION CHRONICLES, excluding those previously extracted for the aforementioned quality evaluation. We use the same metrics for this comparison except for 'Relationship' since there is relationship between speakers in MSC. For more reliable comparison, we perform consensus evaluation. A single episode is rated by three hu- man evaluators, and then we average their scores and take it as the final score of the episode.
els by having conversation with those models on their own (at least turns), considering the persona (in the case of MSC 2.7B) or event summary (in the case of REBoT) of the previous session. For this comparison, the evaluators conduct 50 live chats with each model. Please refer to Appendix for more details about human evaluations.
Results
In this section, we present the evaluations of our REBoT's performance in different experiment se- tups (please see Section 3.4 for the quality of our dataset, CONVERSATION CHRONICLES).
5.5 Model Performance Evaluation
Summarization Performance. We randomly sample 3K generated summaries from the summa- rization module (IK from each of the second, third, and fourth sessions) and ask evaluators judge whether the generated summary fully describes the interaction between speakers in the dialogue.
Dialogue Generation. Table shows the evalu- ation of multi-sessin dialogues generated by RE- Bor. The quite high scores across all the metrics imply that each generated episode is considered natural and engaging like real human conversation. Also, it is rated to have good memory retention with little contradictions from the sessions gener- ated earlier in the dialogue. This corresponds the consistency factor in CONVERSATION CHRON- ICLFES quality evaluation. Please see Appendix G for more detailed statistics by relationship.
Generation Performance. We randomly extract IK of the first sessions (0. 1K session from each of the 10 relationships). Then, we take the relation- ship and the summary of the first session as input to generate the second session, then keep generat- ing the following sessions self-regressiielyy The time interval between sessions is randomly chosen and the dialogue continues in each session until [END] is generated. We ask evaluators evaluate the generated 1K episodes based on 'Engaging- ness', 'Humannes'', and 'Memorability', Please see Appendix for the definition of those criteria.
Interactive Dialogue Generation. We examine the user experience of our model in an interac- tive dialogue setting. Figure 5 shows the com- parison between our model and MSC 2.7B. The better performance REBOT compared MSC 2.7B should be because the characteristies of CON- VERSATION CHRONICLES are well reflected in the model (i.e., coherency, consistency, awareness of time elapsed, and speaker relationship). Especially, through an informal short survey, evaluators report that having conversation based a pre-defined relationship more focused than not having any relationship, proving that the fine-grained relation- ships introduced in our CONVERS ATION CHRONI- CLES work effectively (see the next paragraph for detailed evaluation for the role of 'relationship',
5.6 Comparison to Other Models
We also conduct interactive evaluations and com- pare them with another multi-session dialogue model. Since the only multi-session dialogue model that is publicly available is MSC 2.7B (Xu al., 2022a), we choose it. We ask in-house eval- uators to evaluate with the same criteria as above. Evaluators are asked to rate responses of the mod-

---

Relationship: Husband and Wife
Relationship: Athlete and Coach
USER: think my salary is too low for my work-
USER: Actually, I've been thinking about it since my last injury, but now I'm going to retire as an athlete and live a new life.
load.
REBOT: What do you mean?
USER: I started a new project these days. So I work overtime a lot, but don't think the compen- sation is appropriate.
Thank you. will probably take coaching training for year, and am planning be a full- time coach after that. It would be meaningful to meet each other as a coach later on.
USER:
REBOT: Have you talked to your boss about
USER: No, not yel. Honey, what do you think I should do?
REBOT: That sounds like great plan. wish you all the best in your coaching career, Athlete.
REBOT: Well, you should definitlly talk to your boss and explain your situation. Maybe they can give you raise.
A couple years after
USER: Hey, I finished my training course and finally became full-time coach like you!
Relationship: Employee and Boss
I think my salary is too low for my work- load.
USER:
REBOT: That's great news! knew you had it in
you.
REBOT: understand where you' 're coming from, but we need to be realistic about our budget. Can you give some specific examples of why you feel that way?
USER: Thank you! As became an official coach, I'd like to learn what I need to know as a coach, could you tell me?
USER: As you know, I've been assigned a new project and the workload has doubled. I work a lot of overtime, but think my salary low.
Exactly. As a coach, you need to under- stand your players and their motivations. You need to be able to connect with them deeper level and help them reach their full potential.
REBOT:
REBOT: see your point, but we also have to consider the company's 's financial situation.
USER: Then can you give me bonus if the project ends successfully?
USER: Thank you! will be coach who can raise the potential of the players like you!
REBOT: That's good idea. Let's discuss it fur- ther and see if we come up with a solution that works for both of us.
REBOT: have no doubt that you will. Just re- member stay true yourself and your values coach, and always lead by example.
Table 6: An example of the dialogue flows based on relationships. REBOT generates different responses de- pending on relationships.
Table 7: An example of shifts in relationships. When the relationship is shifted in the last session, REBOT can recognize
This demonstrates that our model effectively learns chronological dynamics from the dataset. Please see Appendix H for examples of the user interac- tions with REBOT and MSC 2.7B.
be a coach, transitioning to work alongside their former coach as pecrs. REBOT (the Coach) can recognize the change in relationship and respond to the conversation accordingly. This shows that our model can handle shifts in relationships due to events and the passage of time, corporates both temporal and relational dynamics.
Speaker Relationship. Table 6 shows an exam- ple of dialogue flow that varies by different re- lationship for the same conversational topic. As we can see, a dialogue beginning with similar context can lead to different interactions depending on speaker relationships. This means that defining a relationship allows conversations to have varying levels of expression such as emotional depth and information exchange. Please see Appendix D for examples of more speaker relationships.
dataset in-
Time Interval. Table 8 shows the effects of time intervals a dialogue episode. The example shows that an event that occurred in a past session is re- called in the following session assuming a given time has elapsed. In particular, we can see that it has memory for past events, even if they are not in the immediately preceding session, and it correctly reflects the accumulation of time intervals.
Additionally, Table shows an example of rela- tionship shift across sessions. The dialogue begins with an initial relationship between 'Athlete and Coach". However, USER (the Athlete) decides to
Chronological Summarization. We conduct hu- man evaluations to check the quality of the gener-

---

Relationship: Student and Teacher
7 Conclusion
We introduce large-scale multi-session conver- sation dataset, CONVERSATION CHRONICLES, which implements chronological dynamics by in- tegrating time interval and speaker relationship in it. To create the multi-scssion conversations, we first build am event graph and then distill a se- ries of dialogues from ChatGPT using well-defined prompts based on the event graph, time interval, and speaker relationship. We verify the quality of our dataset with extensive human evaluations di- verse metries and criteria. We also propose REBOT, multi-session dialogue model, which comprises chronological summarization and dialogue gener- ation modules. The results of human evaluations show our REBOT can generate diverse coherent responses according to different time intervals and speaker relationships with high user engagement without contradiction in long-term conversation setup.
Why don't we go the beach in front of our school?
USER:
few weeks after
USER:
Wow, it's already vacation! had fun this semester.
couple of years after
USER:
can't believe I'm graduating already. Thank you much, teacher.
few hours after
......
USER: What's the most memorable memory you had with us?
REBOT: Oh, that's a tough one. I think the most memorable memory was when we all went to the beach together.
Limitations
USER: Yes, it was fantastic. It's really old mem- ory.
We focused on developing an interactive dialogue model that reflects time intervals and relationships. However, the research was conducted with lim- ited number of specific time intervals and speaker relationships This limitation could potentially limit the generalizability of the research findings. In the future, we plan to expand the research to include more time intervals and speaker relation- ships.
REBOT: Speaking of old memories, do you member when you went on vacation few years ago?
Table An example of dialogue between REBOT and time intervals. This example shows the model can recall past session events considering the time intervals.
user over multiple
In addition, the choice of LLMs can significantly affect the type of dialogue generated. In other words, using different LLMs could lead to different results and types of dialogues, even when using the same framework. Therefore, we plan to con- sider configurations that mix different LLMs as a valuable resource for generating different types of dialogue and content.
ated summaries. The average score of the total of 3K samples is 4.3 out of 5, indicating the summa- rization module works well enough support the generation module by providing important context from previous sessions. Please see Appendix E for examples of chronological summary.
Ethics Statement
Ablation Study. We perform ablation experi- ments to ascertain the significance of incorporat- ing both time and relationship information in our model. When the model is trained devoid of time interval data, it cxhibits trend toward producing responses with generic time information. In the ab- sence of relationship information during the train- ing phase, the model fails to uphold a consistent relationship context with the user. For illustrative examples, please refer to Appendix F.
Despite our best efforts, potentially harmful content may be included in the data. Although our model is trained on toxic-filtered dataset, it may generate responses that users do not want. In addition, the responses generated by our model might be applicable in the real world. For example, medical advice given by a model could not be appropriate in a real medical situation. We recommend using our model for research purposes, or with care

---

real-world applications. Additionally, we commu- nicated with the evaluation agency to check that the annotators were being compensated fairly.
of wikipedia: Knowledge-powert conversational agents. In International Conference on Learning Representations.
Fabrizio Gilardi, Meysam Alizadeh, and Maël Kubli. 2023. Chatgpt outperforms wookers for text- annotation tasks. arXiv preprint urXiv:2303.15056
Acknowledgements
We thank the reviewers for their valuable feed- back. We thank Scokhyun An and Jacho Oh for their helpful discussion. This work was supported by Institute of Information & communications Technology Planning & (TT) grant funded by the Korea governnent(SSTTyNN 0-01336, Artificial Intelligence Graduate School Program(UNISTT) and the 2022 Research Fund (1.220140.01) of UNIST(UIsan National Institute of Science & Technology).
Google. 2023. Google ai updates: Bard and new ai features in search.
Hyunwoo Kim, Jack Hessel, Liwei Jiang, Ximing Lu. Youngjae Yu, Pei Zhou, Ronan Le Bras, Malihe Alikhani, Gunhee Kim, Maarten Sap. et al. 2022a. Soda: Million-scale dialogue distillation with so- cial commonsense contextualization. arXiv preprint arXiv:2212. 10465.
Hyunwoo Kim, Youngjae Yu, Liwei Jiang, Ximing Lu, Daniel Khashabi, Gunhee Kim, Yejin Choi, and Maarten Sap. 2022b. ProsocialDialog: prosocial backbone for conversational agents, In Proceedings of the 2022 Conference Empirical Methods in Nat- ural Language Processing, pages 4005 4029, Abu Dhabi, United Arab Emirates. Association for Com- putational Linguistics.
References
Sanghwan Bac, Donghyun Kwak. Soyoung Kang. Min Young Lee, Sungdong Kim, Yuin Jeong, Hyeri Kim, Sang-Woo Lee, Woomyoung Park, and Nako Sung. 2022. Keep me updated! memory manage- ment in long-term conversations. In Findings of the Association for Computational Linguistics: EMNLP 2022, pages 3769-3787, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics.
Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Veselin Stoyanov, and Luke Zettlemoyer. 2020. BART: Denoising sequence-to-sequene pre-training for natural language generation, translation, and com- prehension. In Proceedings of the 58th Annual Meet- ing of the Association for Computational Linguistics, pages 7871-7880 Online. Association for Computa- tional I. Linguistics.
Jason Baumgartner, Savvas Zannettou, Brian Keegan, Megan Squire. and Jeremy Blackburn. 2020. The pushshift reddit dataset. In Proceedings of the inter- national AAA/ conference on web and social media, volume 14, pages 839.
Yanran Li. Hui Su. Xiaoyu Shen, Wenjie Li. Ziqiang Cao, and Shuzi Niu. 2017. DailyDialog: A manually labelled multi-turn dialogue dataset. In Proceedings of the Eighth International Joint Conference on Nat- ural Language Processing ( (Volume 1: Long Papers). pages 986-995, Taipei, Taiwan. Asian Federation of Natural Language Processing.
Samuel R. Bowman, Gabor Angeli, Christopher Potts, and Christopher D. Manning. 2015, A large anno- tated corpus for learning natural language inference. In Proceedings of the 2015 Conference on Empiri- cal Methods in Natural Language Processing, pages 632-642, Lisbon. Portugal. Association for Compu- tational Linguistics.
Chin- Yenn-Yew!la Lin. 2004, Rouge: A package for automatic evaluation of summaries. In Text summarization branches out, pages 74-81.
Tom Brown Benjamin Mann, Nick Ryder, Melanie Subbiah. Jared D Kaplan. Prafulla Dhariwal, Arvind Neclakantan, Pranav Shyam, Girish Sustry, Amanda Askell, et al. 2020. Language models are few-shot learners. Advances in neural information processing systems, 33: 1877-1999..
Chia-Was Liu, Ryan Lowe, Tulian Serban, Mike Nose- worthy. Laurent Charlin, and Joelle Pineau. 2016. How NOT to evaluate your dialogue system: An empirical study of unsupervised evaluation metrics for dialogue response generation. In Proceedings of the 2016 Conference on Empirical Methods in Natu- ral Language Processing, pages 2122-2132, Austin, Texas, Association for Computational Linguistics.
Jacob Devlin, Ming- Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language under- standing. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguisties: Human Language Tech- nologies, Volume 1 (Long and Short Papers), pages 4171 4171-4186, Minneapolis, Minnesota. Association for Computational Linguistics.
Ilya Loshchilov and Frank Hutter. 2019. Decoupled weight decay regularization. ICLR.
Todor Markov, Chong Zhang, Sandhini Agarwal, Tyna
Eloundou, Teddy Lee, Steven Adler, Angela Jiang, and Lilian Weng. 2023. A holistic approach to unde- sired content detection in the real world. AAAI.
Emily Dinan, Stephen Roller, Kurt Shuster, Angela Fan, Michael Auli, and Jason Weston. 2019. Wizard

---

Alexander Miller, Will Feng, Dhruv Batra, Antoine Bor- des, Adam Fisch, Jiasen Lu, Devi Parikh, and Jason Weston. 2017. PIrlAI: A dialog research software platform. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pages 79-84, Copenhagen, Denmark. Association for Computational Linguis- tics.
Eric Michael Smith, Mary Williamson, Kurt Shuster, Jason Weston, and Y-Lan Boureau. 2020. Can you put it all together: Evaluating conversational agents' ability to blend skills. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 2021-2030, Online, Association for Computational Linguistics.
Rohan Taori, Ishaan Guirajani, Tianyi Zhang, Yann Dubois. Xuechen Li, Carlos Guestrin, Percy Liang, and Tatsunori B. Hashimoto. 2023. Alpaca: A strong. replicable instruction-foolloinng model. https: 01//////////////////////
OpenAI. 2022. Introducing ChatGPT. https:// openai. com/blog/chatgpt
Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, John Schulman, Jacob Hilton, Fraser Kelton, Luke Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul F Christiano, Jan Leike, and Ryan Lowe. 2022. Training language models to follow instructions with human feedback. In Advances in Neural Information Processing Systems, volume 35, pages 27730-27744. Curran Associates, Inc.
crfm. stanford.
Ben Wang and Aran Komatsuzaki. 2021. GPT-J- 6B: A Billion Parameter Autoregressive Lan- guage Model. https: www.ttt.......cgeeeeeee com/kingoflolz/ besh-trranfformer--i
Thomas Wolf. Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pier- ric Cistac, Tim Rault, Rémi Louf, Morgan Funtowicz, et al. 2020. Transformers: State-of-the-an natural language processing. In Proceedings of the 2020 con- ference empirical methods in natural language processing: system demonstrations, pages 38 45.
Kishore Papineni, Salim Roukos, Todd Ward, and Wei- Jing Zhu. 2002. Bleu: method for automatic evalu- ation of machine translation. In Pruceedings of the 40th annual meeting of the Association for Computa- tional Linguistics, pages 311-318.
Chien-Sheng Wu, Andrea Madotto, Zhaojiang Lin, Peng Xu, and Pascale Fung. 2020. Getting to know you: User attribute extraction from dialogues. In Proceed- ings of the Twelfth Language Resources and Evalua- rion Conference, pages 581-589, Marseille, France. European Language Resources Association.
Colin Raffel, Noam Shazeer, Adam Roberts, Kather- ine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. 2020. Exploring the limits of transfer learning with a unified text-to-tex to-text transformer. Journal of Machine Learning Research, 21(140):1-67.
Canwen Xu, Daya Guo, Nan Duan, and Julian McAuley. 2023. Baize: An open-source chat model with parameter-efficient tuning self-chat data. arXiv preprimt arXiv:2304.01196
Hannah Rashkin, Eric Michael Smith. Margaret Li, and Y-Lan Boureau. 2019. Towards empathetic open- domain conversation models: new benchmark and dataset. In Proceedings of the 57th Annual Meet- ing of the Association for Computational Linguistics. pages 5370-5381, Florence, Italy. Association for Computational Linguistics.
Jing Xu, Arthur Szlam, and Jason Weston. 2022a. Be- yond goldfish memory: Long-term open-domain con- versation. In Proceedings of the 60th Annual Meet- ing of the Association for Computational Linguistics (Volume 1: Long Papers), pages 5180-5197, Dublin, Ireland. Association for Computational Linguistics.
Alan Ritter, Colin Cherry, and William B. Dolan, 2011. Data-driven response generation in social media. In Empirical Methods in Natural Language Processing, pages 583- 593, Edinburgh, Scotland, UK. Association for Com- putational Linguistics.
Proceedings of the 2011 Conference
Xinchao Xu, Zhibin Gou, Wenquan Wu, Zheng- Yu Niu, Hua Wu, Haifeng Wang, and Shihang Wang. 2022b. Long time no see! open-domain conversation with long-term persona memory. In Findings of the As- sociation for Computational Linguistics: ACI. 2022, pages 2639-2650, Dublin, Ireland, Association for Computational Linguistics.
Stephen Roller, Emily Dinan, Naman Goyal, Da Ju, Mary Williamson, Yinhan Liu, Jing Xu, Myle Ott, Eric Michael Smith, Y-Lan Boureau, and Jason We- open-domain
ston. 2021. Recipes for building chatbot. In Proceedings of the 16th Conference of the European Chapter of the Association for Compu- national Linguistics: Main Volume, pages 300-325, Online. Association for Computational Linguistics.
Saizheng Zhang, Emily Dinan, Jack Urbanek, Arthur Szlam, Douwe Kiela, and Jason Weston. 2018. Per- sonalizing dialogue agents: have a dog, do you have pets too? In Proceedings of the 56th Annual Meeting of the Association for Computational Lin- guistics (Volume I: Long Papers), pages 2204-2213, Melbourne, Australia. Association for Computational Linguistics.
Kurt Shuster, Jing Xu, Mojtaba Komeili, Da Ju. Eric Michael Smith, Stephen Roller. Megan Ung. Moya Chen, Kushal Arora, Joshua Lane, et al. 2022. Blenderbot 3: a deployed conversational agent that continually learns to responsibly engage. arXiv preprint arXiv:2208.03:: 03188.
Yizhe Zhang, Siqi Sun, Michel Galley, Yen-Chun Chen, Chris Brockett, Xiang Gao, Jianfeng Gao, Jingjing

---

Liu, and Bill Dolan. 2020. DIALOGPT: Large- scale generative pre- pre-training for conversational response generation. In Proceedings of the 58th Annual Meet- ing of the Association for Computational Linguistics: System Demonstrations, pages 270-278, Online. As- sociation for Computational Linguisties.
in two sentences, \n' in'nnnnn [Cooverrsaiion] | Session Dialogues's BaaloguessnnSuummaas n|[ummmayy]"
Prompts Environments. ChatGPT uses rein- forcement learning from human feedback. We use the "gpt-3.5-turbo-02. turbo-0301" model to ensure repro- ducibility, as responses may be different depending on the version.
Chujie Zheng, Sahand Sabour, Jiaxin Wen, Zheng Zhang, and Minlie Huang. 2023. Augesc: Dialogue augmentation with large language models for emo- tional support conversation.
B Dataset Filtering Process
A Prompts Details
To ensure uniform quality of CONVERSATION CHRONICLES, we filter out the following cases: (1) sessions with more than two speakers: (2) ses- sions with unclear alignment between utterance and speaker; (3) sessions where speakers not included in the pre-defined relationship appear; (4) sessions with unnecessary information such stage direc- tions (e.g., any descriptions of actions or situations). We remove all conversation episodes that include at least one of these cases.
Prompts for Relationship. We use ChatGPT to assign fine-grained speaker relationship to each episode. The prompt is used as follows: "Two people want to have conversation about the topic below. Please choose from the options below the most appropriate relationship between the two speakers in the conversation. Don't recommend other options. You are responding without comment. Also, your an- swer is limited to the options below. below.\n'nnon n (Episode Event Description] } (anoonnnnnnnnnnnnnn. (nOtiin:: n1. Husband and n2 Child and Parent) Co-workers',nee n4. Classmates (n.. Student and Teacher' n6. no. Patient and Doctor),n. Employee and Boss' n8. Athlete and (n. Neigh- bors) n10. Mentee and Mentor"
C Implementation and Training Details
Summarization Module. We employ the pre- trained T5- model consisting of 222M parame- ters for the summarization module. We train with the linear scheduler, batch size 32, 512 for the maximum length of the input sequence, and 128 for the maximum length of the output sequence. Training takes about 6 hours on 8 NVIDIA RTX A6000 GPU devices with maximum of epoch 5 with early stopping.
Prompts for Conversation. We use ChatGPT to collect multi-session dialogues for CON- VERSATION CHRONICLES. The prompt is used as follows: "The following is a next conversation between (Relationship).-nknnTT (Relationship) took turns talking about the below (opics:)(n(Sessiie (non) (ime Intervals Between Session N-1 and N} the last topic, this is the topic (Re- lationship] are talking about today:)n|(eessio N Event Description) |n) (Sppeker A)'s state- ments start with [Speaker A] and (Speaker B)'s statements start with [Speaker B]. (Speaker A) and (Speaker B} talk about today's topic, and if necessary, continue the conversation by linking it the conversation topic of the past. Complete the conversation in exactly that format."
Generation Module. We the pre-trained BART-large model consisting of 406M parame- ters for the generation module. We train with the linear scheduler, a batch size 16, 1024 for the maximum length of the input sequence and 128 for the maximum length of the output sequence. Training takes about 3 days on 8 NVIDIA RTX A6000 GPU devices with maximum of epoch 3 with early stopping.
N-1 Event Descrip-
All pre-trained models used are based on Hug- ging Face Transformers (Wolf al., 2020).
D Dialogue Example for Each Relationship
Our CONVERSATION CHRONICLES and REBOT incorporate fine-grained relationships in multi- session environment. There are 10 relationships in total, and the model response differently depending on the relationship, even for conversations about the same context. Table shows different dialogues
Prompts for Summary. We use ChatGPT to generate chronological event summaries. The prompt is used as follows: "You're a sum- marizer. Choose the most important events from given conversation and summarize them

---

for the same context for each relationship. In the dialogue, the user talks about the difficulty of home- work in various relationships. The session begins with the same utterance for each relationship. but the flow of the dialogue varies depending on the relationship. As we can see in the example, the classmate advises their friends to ask the teacher for help, and the teacher opens supplementary class. Parents, also provide emotional support to the child. In other words, the same context can ob- tain different responses depending on the relation- ship, such as providing advice or giving empathy. This demonstrates the works of our fine-grained relationship.
to the time interval. In contrast, model trained without time information generates more generic time-related responses. This implies that a model trained with time information is better at capturing and incorporating specific time intervals into its responses. Next, Table 23 shows another ablation study example that focuses on the influence of re- lationship modeling. This example indicates that model trained with relationship information pro- duces more contextually consistent responses. contrast, considering the model's response, model trained without relationship information exhibits contextually inconsistent responses. This suggests that models trained with relationship information can better understand the context and generate re- sponses that align with the nature of the relation- ship. Overall, it appears that incorporating time interval and relationship into REBOT can improve its ability to generate contextually appropriate and specific responses.
Also, Table 10 to 19 shows example dialogues for all relationship options. The examples are dia- logues between user and REBOT, and can see that defining relationship works quite well. To our best knowledge, our work the first to integrate relationships into dialogues, and extensive human evaluation shows that the relationship between the speakers helps to achieve high engagement in the conversation.
G Generation Quality per Relationship
Table 24 shows the evaluation scores of generated dialogue by REBOT for each relationship category. As we see, the scores are balanced across all relationships, meaning that REBOT effectively mir- rors relational dynamics for all relationships.
E Chronological Summary Example
Please see Table 20 for a chronological summary example. In the previous session, the coach and the athlete were scheduled start training in the morn- ing. However, after few hours, the coach and the athlete decide to change their training schedule. The chronological summary effectively captures the state changes of these events, detailing the shift in the training schedule from morning to afternoon. Next, in Table 21, husband and wife spend time in parks and restaurants on their wedding anniversary in previous sessions. After few months, they de- cide to have dinner restaurant. The husband offers a meal at the restaurant they went to on their wedding anniversary. Then, they have a conver- sation recalling memories at the restaurant. We can see that the summary accurately reflects theirs reminisces.
H Episode Example
Figure 6 shows a sample of the entire episode CONVERSATION CHRONICLES. Figure shows an example of a live chat between a user and REBOT. Figure 8 shows live chat sample for one session with MSC 2.7B.
I Human Evaluation
We conduct human evaluations by asking the eval- uator to read the full episode and rate the dialogue quality based on the following metrics. The score of each metric ranges from to 5 with 5 meaning perfect for corresponding metric (Section 5.3 and Section 5.4).
F Ablation Study Example
The conversation between two speakers should have a natural flow in terms of event transition.
Coherence:
We incorporate temporal and relational dynamics in REBOT. To verify the impact of these dynam- ies, we conduct ablation experiments. Table 22 shows an ablation study example to assess the impact of time information REBOT. This ex- ample suggests that a model trained with time in- formation can produce responses that are specific
Consistency:
Two speakers should not make contradiction from past sessions.
Time interval: The speakers should make a conversation each session as if the desig- nated time has elapsed since the last session.

---

Relationship:
Two speakers are having conversation with the designated relationship. Throughout the session, the two speakers must maintain this relationship.
Relationship: Classmates
USER: feel like I'm not really understanding the content of my homework these days.
REBOT: Have you tried asking the teacher for help?
We ask evaluators to evaluate model perfor- mance based on following three criteria (Sec- tion 5.5 and Section 5.6).
USER: Not yet. was thinking about asking our teacher for help.
Two speakers should interact to create responses that are not only interest- ing but also cell-immersed in the given con- text of the conversation.
Engagingness:
REBOT: That's good idea. It's always better to ask for help when you need
USER: Cool! I'll ask teacher explain the assign- ment.
Relationship: Student and Teacher
Humanness: Two speakers should have con- versation that demonstrates emotional under- standing (e.g., empathy) and the use of natural language and thought processes that are typi- cal of human beings.
USER feel like I'm not really understanding the content of my homework these days.
REBOT: That's understandable. Sometimes can be difficult to grasp certain concepts. Have you tried breaking it down into smaller parts?
Memorability: If two Speakers recall past events correctly by retaining information from previous sessions. 5
USER: Yes, of course. But I'm worried because it's hard to understand.
REBOT: Well, let's work on it together. Maybe we can come up with a plan to help you understand the material better.
Evaluators conduct their evaluations on the plat- form provided by the agency as shown in Figure 9, Figure 10 and (we use ParlAl (Miller et al.. 2017) to live chat with MSC 2.7B) show the in- house human evaluation screen for interactive dia- logue generation.
Wow, thank you so much. Could you open a supplementary lecture for me?
USER:
REBOT: Sure thing. Let me pull it up for you.
Relationship: Parents and Child
USER: feel like I'm not really understanding the
content of my homework these days.
REBOT: What's 's been giving you trouble?
USER: It's hard to understand the homework
problem. I'm upset about myself.
REBOT: Don't be too hard on yourself. It's okay struggle sometimes. Have you tried asking your teacher for help?
USER: It's still difficult even though got help from my teacher.
REBOT: I understand how you feel, but it's im- portant to remember that it's okay to ask for help when you need it.
USER: Thank for always being a big support to me. It's a great consolation.
REBOT: Of course, I'm always here for you. Re- member, we all have our strengths and weaknesses. It's important to focus on your strengths and use them to your advantage.
Table 9: An example of the difference in dialogue based the relationship. When the event is suggested, REBOT will generate different responses depending on relationship.
'Different from other criteria, this score starts with 3 when there is no contradiction among sessions, and if there are correct recalls evaluators raise the score and vice versa.

---

Relationship: Parent and Child
Relationship: Husband and Wife
Child: You know what day it this Wednesday!
Honey, shall we go to the mart together since it's our holiday?
Wife:
Parent: Yes, it is. Why do ask, kiddo?
Child: Hehe what day is it???
Husband: Sure, that sounds like great idea. What do you want to order?
Parent: It's your birthday! Can you tell me more about it?
Wife: We don't Ion't have tissues or detergents, and think we need to buy lot of things to meal.
Child: I'm already excited! How about go to amusement park? really wanted to go.
Husband: That's good point. Maybe we can make list of what want eat and what need to buy.
Parent: That sounds like a great idea! Which one do you want to go to?
Child: How about Disneyland? I really wanted to go!
Wife: Okay! Then can you write it down? What else we need?
Parent: That's great idea! We haven't 't been to Disneyland in long time.
Husband: Well, we also need to get some drinks for ourselves and some snacks for us to enjoy.
Child: Right. When did we go? I think was when I was really young.
Wife: Great! Shall get going now? I think we
could go for a walk.
Parent: We went a few years ago, but it was lot of fun. We went all the rides and ate so much junk food.
Husband: Sure, let's go. But let's make sure we have everything need before leave.
Wife: Sure, don't 'o think we missed anything we need to buy.
Child: Haha, just like that memory, I hope we can make good memories this time!
Parent: Me too, kiddo. I can't 't wait to spend some
Table 10: An example of dialogue between husband and wife.
quality time with you at the amusement park.
Table 12: An example of dialogue between parent and child.
Relationship: Employee and Boss
Relationship: Co-workers
Boss: How are you doing at work these days?
A: Hey, when report due?
Employee: I'm doing well, thanks for asking. I've been working hard and trying improve my skills. Boss: That's 's relief. You't a very important person in our company.
B: was due yesterday. Did you get chance to work on it?
A: Oh, my God.. was so busy that forgot to submit it.
Employee: Thank you, appreciate that. always try to do my best and contribute to the success of the company.
B: That's not good. You need to take responsibility for your work.
A: What should I do? I should tell the boss, right? B: Yes, you should. It's better to be honest and face the consequences than to try and cover it up. A: Thanks for the advice. Then I'll talk to the boss and finish writing it within today.
Boss: Especially, thanks to the project you orga- nized, our company was able to grow a lot.
Employee: Yes, that project was lot of work, but it was worth it in the end. I' glad it paid off.
Boss: pay you a little bonus this month. Please work hard in the future.
B: That's good plan. It's important to prioritize your work and manage your time effectively.
Employee: Wow, thank you so much! That means a lot to me. won't let you down.
Table II: example of dialogue between co-worker A and co- worker B.
Table 13: An example of dialogue between employee and boss.

---

Relationship: Classmates
Relationship: Athlete and Coach
A: Hey, don't t you think we have too much home- work these days?
Athlete: Coach, when is our upcoming game? It's next Saturday, Are you ready for it? Oh my God, can my injury recover by then?
Coach:
B: I don't t know, it just seems like there's always something to do.
Athlete:
A: Really? I'm worried because I think have more homework these days. Is it because the diffi- culty level went up?
Coach: Yes, you should be fully healed by then. You've been working hard on your recovery.
Athlete: Thank you. think it would be good to start rehabilitation training from today, what do you think? Is it too much?
B: I don't I think
I mean, I've noticed that of the assignments are harder than others, but that doesn't it mean we have more homework.
Coach: No, that's good idea. We start with some light exercises and then move on to more intense ones.
A: see. was just trying ask you for advice because I've been having a hard time with my homework.
Athlete: Okay. How adjust the training intensity specifically?
B: Sure, what's been giving you trouble?
A: I have lot of worries because I have to spend more time than thought. there a good way? B: Well, have you tried making schedule to-do list? That might help you manage your time better.
Coach: You can start with some low-intensity exercises and gradually increase the intensity as you get more comfortable with it. We can also adjust the duration of the exercises to make sure you don't 't overdo it.
A: you write to-do list? Then should try it too?
Athlete: All right. Let's start with low-level aero- bic exercise.
B: Yes, do. It helps me stay organized and on track with my assignments.
Coach: Sure, that's good place to start. Re- member warm up properly before starting the exercises.
A: Good! So I'll try it from today!
B: Great! Let me know how works out for you.
Table An example dialogue between classmate and classmate B.
Table 16: An example of dialogue between athlete and coach.
Relationship: Mentee and Mentor
Relationship: Patient and Doctor
Mentor: Mentee, I'm glad that you seem be active these days.
Doctor: What kind of symptoms did you visit?
Patient: had headache, sore throat, and a fever.
Mentee: Yes, am. I feel like it's important stay active and healthy.
Doctor: Oh, man, that must have been tough. How
long have you had symptoms?
Mentor: That's right. What kind of efforts are you making these days?
Patient: It started few days ago.
Doctor: Did you take any medicine?
Mentee: Well, I've been going for runs in the morning and doing some strength training in the afternoon.
Patient: Yeah, took some over-the-counter stuff, but didn't seem to help much
Mentor: 1 think it's really good. Why don't 't you read book, too? It will be of great help to your psychological stability.
Doctor: see. Then I'll prescribe some medicine for you. Try it for about three days and if the symptoms don't improve, a simple examination will be needed.
Mentee: That's is good idea. I haven't 't read book in while. Do you have any recommendations?
Patient: Okay, thank you.
Mentor: Shall go to the bookstore in front of us and choose a book? want to buy you book!
You get prescription from the nurse in the lobby. Need any other advice regarding symptoms?
Doctor:
Mentee: Sure, that sounds great. Thank you, Men-
Patient:
No, that's all for now. Thank you. doctor.
Table 15: An example of dialogue between mentee and mentor.
Table 17: An example of dialogue between patient and doctor.

---

Relationship: Student and Teacher
Student: Teacher!
I have a concern about my
career.
Teacher: What's your mind?
I'm not sure if should do what like or what I'm good at.
Student:
Teacher: That's tough decision to make. Have you thought about what you enjoy doing?
Student: enjoy playing baseball! I'm so happy whenever play baseball.
Relationship: Athlete and Coach
Teacher: That's ' great! Have you considered pur- suing career in baseball?
Past session summary: Athlete and Coach dis- cuss their plans for speed and agility training to- gether. They plan start with morning jog and strength training in the afternoon.
Student: Yes, I've been thinking about it, but I'm afraid baseball player because not good it.
A few hours after
Teacher: Don't worry about that, Student. You don't 't have to be good at baseball to pursue a career in it. You can start by playing for fun and improv- ing your skills.
Coach, when are we supposed to start training?
Athlete:
We're supposed to start tomorrow morn- ing.
Coach:
Table 18: An example of dialogue between student and teacher.
Athlete:
think we need reschedule the training talked about before. forgot the special theory class schedule.
Coach: That's okay, Athlete. We can always reschedule for another day.
Relationship: Neighbors
A: How you these days? It's been really long time since saw you.
Athlete: don't think we need to change the date. There are lectures only in the morning, so how about starting in the afternoon?
B: I'm doing well, thanks for asking. How about you?
Coach: Okay, let's do that then.
A: I'm going to organize the lawn in the yard today. Would you like to join us?
Current session summary: The athlete thinks they need to reschedule their training due to for- getting the special theory class schedule. They suggest starting their training in the afternoon in- stead of changing the morning schedule due to lectures only occurring in the morning.
B: Sure, I'd love to help out. It's been a while since I've done any yard work.
A: Okay! Then shall we organize my lawn and your lawn together?
B: That sounds like a great idea. By the way, have you noticed any changes in the neighborhood lately?
Table 20: An example of a chronological event summary. event
The summary reflects the state change the previous session. This allows the model to capture the flow of events throughout the episode.
A: No, not at all. What it?
B: Well, I heard that there's going to be restaurant opening up down the street. was think- ing of checking it out this weekend.
A: Wow, that's fantastic. can't believe there's restaurant in my town! Can you tell me after you try it? I'm going, too!
tell you all about it. Maybe can even go together sometime.
B: Of course, I'll be
Table 19: An example of dialogue between neighbor A and neighbor B.

---

Relationship: Husband and Wife
USER: Honey, where would be a good place for our honeymoon?
Relationship:
Husband and Wife
Past session summary: The married couple cel- ebrates their wedding anniversary by going for a walk in the park and having dinner at nearby restaurant. They plan to spend the rest of the day together.
ReBOT: Hmm, I'm not sure. Where are you thinking?
USER: I'm thinking of Hawaii as option. What do you think?
REBOT: Hawaii sounds amazing! I've always wanted to go there.
A few months after
USER: Me too, honey. It's our honeymoon in few months. Shall we check the tickets and accommodation?
Husband:
Honey, what shall have for dinner?
Hmm, how about we try that new Italian restaurant that just opened up?
Wife:
REBOT: Yes, let's do that right away. I can't wait to spend some quality time with you in paradise.
one. I was thinking Husband: Aha, there's about going to the restaurant went to on our wedding anniversary.
Case 1- Trained with time information (given time information is "A few months after")
Wife: Oh, that's great idea! It's been while since we' ve been there.
How long did we prepare for this trip to Hawai??
USER:
Husband: The course meals there were really good, right?
REBOT: We've been planning it for months now. wanted to make sure everything was perfect for our special day.
Wife: Yes, they were delicious. still remember pasta dish we had.
Case 2 Trained without time information
Current session summary: The husband sug- gested going restaurant went on his wed- ding anniversary. The course meals there delicious and the wife recalls fond memories of the pasta dish they had.
USER: How long did we prepare this trip to Hawai??
We' ve been looking at flights, hotels, and activities a long time.
REBOT:
Table 21: An example of a chronological event summary. The summary reflects the speakers' memories about past event well.
ablation study example about temporal dynamics. This example shows trained with time in- formation model responses with specific time interval, however trained without time information responses with generic time interval.
Table 22:

---

With Relationship: Patient and Doctor
USER: My stomach hurts much and have a fever since yesterday, so much fever that I'm having hard time.
REBOT: I'm sorry to hear that. How long have you been experiencing these symptoms?
USER: suddenly started hurting last night. Can you prescribe suitable medicine?
REBOT:
Yes, can prescribe you some antacids help with the pain and fever. Have you taken any tver-the-counter medication?
Without Relationship
USER: My stomach hurts so much and have fever since yesterday, much fever that I'm having hard time.
REBOT: Oh no, that sounds terrible. Have you doctor yet?
USER: No, I've never seen a doctor. Can you
prescribe me suitable medicine?
REBOT: Of course, I can prescribe you some antacids to help with the pain and fever. But you should also try to rest and drink plenty fluids.
Table 23: An ablation study example about relational dynamics. This example shows without relationship model generates an inconsistency response.
Relationship
Engagingness Humanness Memorability
Classmates
4.75
4.76
4.12
Neighbors
4.81
Co-workers
Mentee and Mentor
4.79
4.64
Husband and Wife
4.84
4.74
Patient and Doctor
4.70
4.58
4.03
Purent and Child
4.74
4.71
4.12
Student and Teacher
4.73
4.62
4.15
Employee and Boss
4.96
4,70
4.07
Athlete and Couch
4.87
4.73
4.40
Overall
4.78
4.74
4.14
Table 24: Per relationship statistics of human evaluation result for the quality of dialogue episodes generated by REBOT.

---

Relationship: Neighbors
Session
rewe?) wee
A few weeks after
Fre kittery
What? No way! How much did you
Session
Can you bediee
nehr what fools Bio someone comprste fall for and then ast duro from
Wiw: fra's aerachny Congansds
Thank you so rusht Io
So vinut are you golnng ts do with but monoy?
addualyy win.
Uh nc. Wito would you Evcn ask
Vieil the fird thingg I'm gring io do pay dests, altr itst abous
twen thiaiing abou lawey kenos Luouarss crudl but really expermment comrecnels foeling lings can be
socation and inpple inineding some ofll..
That scunds Bin good plan. Hiws you reld anyoon elon yer?
That skay Ne phorr Playing with acrwone's in smotiion (kc messed up.
Yeah: Iold my Sainlly ang a fow Exerpore happy for me..
Ikrows conti
lbet they aeal You'e so lucey. bor whit we were sutling
And beakdee iDe Fin going to cciuaiiy fuust anyonn.
New dream
You orceed hutt me whthh you do- streped can only imm- yould hears
your own buainssa if you wunt to.
Thet's Wilninng lotwe reully focis tho my dheamm
Look, (nnow whatl did the
hagde your Just ronerr. aeser and theee wieely.
skars was wroing this
complately llweeet
wers, will Thanto for with reel
And and you bring ng somoting rp from works ago reyway??
cortoo when your And non you wart use scrnons for grint your patlern Negghoonu..
few weeks after
have patten, youlic
rwwe playe gamnes with some- one before.
Session
F'n so ombonnssed about feppered leat night
That's not the same thing and loase i. you thoough with this you're going searacee
What heppowod??
riort, Lot's kigee oven brought uo Hay. do you want Ini wookordd? df
coolont tnan- onymon just dedittyd thom
chellnged OWn stais?
Weet: know sesunde creoyy
stways beiing in my wis And Iisped over dram los many
A couple of years after
could just ohem hied
Session 5
sngy the momentt the not live ni
Hnnow Hinow
Hay Negttbrr puana
do froel kool
Now
luncersben We whee mo- to cam cown andithirs thruugh your adtioe Ind.
That's Neigher liow you keam E7
time.
Isaatedd t2kk na a diss- ea. montfe aga, and fow
betlor the Nutur.
Thut' roally proresanvo ftane you
Hox, romcensrr whon you wanl the all that money?
Yeah, I chowed my friend the gom and they really li-
Yor'r right dediiiey try do
That's owesoree You know rt- misds mhe of that time weet out your boal leaanel
on anating par of morpage.
Ohiy That was of had pution the jackate?
bought o row car, and eveni look. a rioo vacatin Hovai tyy dedition
Yoat, calrty fintt Spoaring of what you destroyed the sta is ansar??
O A few hours after
Yeah, that wno bit aerbormes. ing limee just Pradereed with tino.
Session
Risalt: weire on boSt 50
Lunderrinnt you know, thought playing jose mm
No novol been to Enisland belon, vo bartiwil now whal
haid
it's smoll Hland, but has some
Bey Stene
Yoar, kroow. I waz bad place buck sten. I'm rsally sorry considering
To us newd to verry aboutt any-
ting while we''n out here?
It's skay Negthoor Everyoee makes mikeaess But werting the things that toppered wir
Not really long swer
our life sccote and play sale. The only westher
sure to check belone we hert.
That' a roldd. I'm auto be greal day.
creams come still cankt hellem Vacc 51 milloon
Definidyy It's popookil out bai abour
Willl I'n you, Neightor And now with your hea move you can way oven greato"
and howe dealdoyyd the stars.
have beens
coginggwl in tha??
Fvt boen doing much bettor re-
cool and thange
bofore caackingi key. And being
"Tai's gyoll to heur it's nn impos-
relec.
and to alo care muntal
rodey, saglor- ing brnuntill stoot refi
read ii
Figure 6: A full episode example from CONVERSATION CHRONICLES.

---

Relationship: Husband Wife
Session
Good mominng Hore: What tic today's aroaliiss?
Habend
Session
Good mon ig dear, are tacce.
Wiff
Yos, theyll love inl rioo
got oway from the City for Oit
But worted about the kids
Hanband
Hendand
know our wodding annierssary in morth Do you want to seything?
We con make surs to bring picnty of socer and exsemnen Pluse when ther..
Vos. do thinking wewlordd the beachh
Yeah we're gonma have to makis sure we have ensuggh for the goona
Hasband
a great idea. Hos about Fricay in teo soedoo??
Husbbad
Dluy: have a good day work.
Wille
ary spaeclic beach mind?
Unssert medyy thonggt atoout much, Viral about
Huntard
Wife
porfect
fors swin in the ocea and relox the sand
A couple of years after
Yean, that soordd te great, bur themky ztll plenty pick bouncee idaaa of
Hasterd
Session
Wife
Suro, [wll
honey
Hobend
remember when wn werl Hawell for our Inneprrzas??
Can you guees??
Haabard
Absokzadty lartzzdo eopadially
Wite
From not suw
Wide
ligmac soa besuchid
I'm roal hhpppy hand, Feed Bos been reunented.
and we has au reuch fon reptoring together
You
servd
ings
Wide
Me loo. cannt was to sparnt soena
these dayo?
Hknband
sualty you
They'r daing will They'ss grow- ingig
Tove work you for breakchad honey
Siunband
hokine
Wide
Youre welcoree Hove a gozd dit
Yeats Farmenberr whan ve look the cine in the desert?
Handuaa
Povee good day
Wife
coudd forgat? Thai was such fun His.
How mush bine has passeed since trp?
Huderaa
A few months after
Pc boen couple years
Session
Hight, we ssverfiizon adée traval since then about for frrs whill, de you Irik??
Guiing the beach anni- versary jas an much furt and cllcosing lo
Wifee
logroe, beoudilal
plack I'm sougltd we gat spend koow quality thiw trgether
have mudde.
devikdon could
Wiffe
That sounts lise groat Mooa Where co you wont go!
how ling has boen dince we
Lot' Beach We
Honern
Wife
been rontha now and taiing con the lods tnat tawant haa much for any-
Witte
Ther asunce perfect Leth diart
How boon since Lust wint Ven Rwacc
Hendera
That's right, why not take Nols on tip this secetion?
Wiff
beer lew rronthee
When thinc
should go?
belleve difkeeet salmaa adla younng sgga will help mi grow.
A few days after
Wifee
ico. thek it's in-
portent for kils leem aboutt difereer
Session
wopa
What are you talking thour lids hovling acciderss?
Hedere
Wife
Iobx?
you
Wife
corry tenking iioo whon you your head on kitciinn lobl.
morth or two. That should give enough tine plan and sove
Kide new ii of bully thene days, unsill thay??
Haaberd
Oley,,
Hrunband
Yeen the can be Imrpor und uvare of their sumound
Wifl Sounce conn't wolitig
vost platning sur neat sdveeturm loguthe:
You. trewed to seadh theme to be
fully sward surouncing rigs.
malk sure acuuade whenl do commhiiing thet crocsss the inn.
A few weeks after
Trarfi inporteve to sut good soartpli frums.
Session
Babs, Iiv get someo bad nowk my vaceloo has been denics
Humbard
Thanks for bolisving in me: luao ginw up
Haabend
with
Wifee
Chisc, to thet'e bembiiel was yoa'' bookngg fonward laking break
Wiffe
Freesonioor
reoar a lesk monthe ago Theil one each beaut expetence.
Jiunband
with this kids this timo? Weo bwachs
Why don 1 go beach
thrt Inwerntionall into with the kids, no how ahoud trip
great
Wide
That's idea. We allll tave some quality ame kogelher.
reigit
great be spend suail Eine together os faenty
I'n prepated se for this trp.
Siunband
Wifle
Dan't sooy alboui 8l We can always
beas1?
the
beer couple
Can you bei lee Exr been as long? meahh how happarted with the thio
bad Venioe Beadd??
boon moners, should
Eigns be a groot opportiity
hrarrt bean keng, why don't
Humbard
Widee
That adu line groat ideal Vi sxpiors desart und spand line
Goeat Good
Witte
Good rayet
Wiith our kids sas.
Figure 7: full episode example from a live chat with REBOT.

---

Model's Persona
User's Persona
have finance degree. live in Washington. like BMW. work the Financial sector the Bank. like to go hiking. would like to hike the Grand Canyon. enjoy being side when it is warm. I use hiking boot
plan to go hiking at Grand Canyon during summer. am civil engineer. like to construct unique buildings. enjoy being outside. need hiking boots. am older. like Corvetles and BMWs.
1 hour after
you your when you go hiking?
User
MSC
do! have lot of places to that are not close to my house. I'm get there plare!
Aha, see. Is there place you often?
User
MSC
like go to the beach. love the water! Lalso like to hike in the mountains.
know you hiking. Do you happen to remember that said needed new hiking?
User
MSC
do remember!! am exciled to go! I will bring my carl Do you
Can you recommend the hiking boots you wear? don't have but you know, have favorite brand.
User
MSC
wear timberland boots. They are them! What brand is your favorite?
User
This time, I'm going to try the brand you recommended.
MSC
That's great ideal I hope you them! I'll bring some water to drink.
You don't give Why do you bring water all of sudden?
User
MSC
prepared.1 don.t want to be situation where
Figure 8: An example of live chat with MSC 2.7B.

---

Brodent nasire
Growen/ Tha
Setweral dusk Sumwar
Figure Human evaluation page for evaluators.
Coach, mikk
Well, Athlets theikyouhe improved ott since we fiss started worbin togther
expert you gle exces engennent
your
Iom grug vaar that, Abhiew Ioeive ast cork
thi point ere any manne buxt should locss more ov!
Figure 10: Live chat page with REBOT.
oader suceann reseea USA Oo butl
Figure 11: Live chat page with MSC 2.7B.


Reviews:
[{"review_id": "8xerRLxzba", "reviewer": "Reviewer_fauZ", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 4, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your valuable feedback on our paper. Please find our responses below.\n\n**(1) Evaluation of relationship.**\n\nWe verified the fine-grained relationships in our dataset using extensive human evaluation. The results, as presented in Table 4, indicate high scores, suggesting that the dataset accurately reflects the speaker relationships. Our dialogue model, ReBot, was trained on this dataset. Consequently, we anticipate that ReBot can recognize and internalize these speaker relationships. To validate ReBot's capability in this regard, we conducted another human evaluation. Here, evaluators were informed of the speaker relationships and then tasked with assessing the quality of the dialogues generated by ReBot based on various criteria. These results are detailed in Table 5 and Figure 5. Given that the evaluators were cognizant of the relationships, the high scores from this evaluation reaffirm that ReBot's generated dialogues effectively mirror the intended speaker relationships. For further clarity, we've included dialogue examples for each relationship in the Appendix (Table 9 to 18, showcasing how these relationships influence dialogue generation).\n\n**(2) Model performance without relationship and time interval.**\n\nThank you for pointing that out. We conducted an ablation experiment to assess the model's performance when trained with and without relationship information. The results showed that, when trained without relationship context, the model does not maintain a consistent relationship with a user. In contrast, when trained with relationship information, the model demonstrates a smooth and consistent conversation flow.\n\nPlease consider this example:\n\n> **With Relationship** (doctor and patient):  \nUSER: My stomach hurts so much and I have a fever since yesterday, so much fever that I'm having a hard time.  \nReBot: I’m sorry to hear that. **How long have you been experiencing these symptoms?**  \nUSER: It suddenly started hurting last night. Can you prescribe me a suitable medicine?  \nReBot: Yes, I can prescribe you some antacids to help with the pain and fever. Have you taken any over-the-counter medication?  \n\n> **Without Relationship**:  \nUSER: My stomach hurts so much and I have a fever since yesterday, so much fever that I'm having a hard time.  \nReBot: Oh no, that sounds terrible. **Have you seen a doctor yet?**  \nUSER: No, I've never seen a doctor. Can you prescribe me a suitable medicine?  \nReBot: **Of course, I can prescribe you some antacids to help with the pain and fever. But you should also try to rest and drink plenty of fluids.**  \n\nAs evident from the 'Without Relationship' dialogue, the model’s responses, particularly **\"Have you seen a doctor yet?\"** and **\"Of course, I can prescribe …\"**, are inconsistent regarding its role.\n\nAdditionally, we also conducted an ablation study to assess the model's performance when trained with and without time information.\n\nPlease consider this example:\n\n> USER: Honey, where would be a good place for our honeymoon?  \nReBot: Hmm, I'm not sure. Where are you thinking?  \nUSER: I'm thinking of Hawaii as an option. What do you think?  \nReBot: Hawaii sounds amazing! I've always wanted to go there.  \nUSER: Me too, honey. It's our honeymoon in a few months. Shall we check the tickets and accommodation?  \nReBot: Yes, let's do that right away. I can't wait to spend some quality time with you in paradise.  \n…..  \n\n> **Trained with time information** (and given a time info. **'A few months after'** when generating response):  \n…..  \nUSER: How long did we prepare for this trip to Hawaii?  \nReBot: **We've been planning it for months now**. I wanted to make sure everything was perfect for our special day.  \n\n> **Trained without time information:**  \n…..  \nUSER: How long did we prepare for this trip to Hawaii?  \nReBot: We've been looking at flights, hotels, and activities **for a long time**.  \n\nAs shown in the “without time information” response, the model generates generic time information (**\"for a long time”**) instead of a specific time amount.  \n\nThis time, we provide time interval to the model trained without time information. Here are the comparisons.  \n\n> USER: Mom! Let's go to an amusement park this weekend!  \nReBot: That sounds like a great idea! Which one do you want to go to?  \nUSER: How about Disneyland?  \nReBot: Oh, I've always wanted to go to Disneyland! Let's do it!  \n…..  \n\n> **Trained with time information** (Time interval: **'A few hours after'**):  \nUSER: When did we talk about amusement parks?  \nReBot: **We talked about it earlier today.** Do you remember?  \n\n> **Trained without time information** (Time interval: **'A few hours after'**):  \nUSER: When did we talk about amusement parks?  \nReBot: **We talked about it a few weeks ago** when I asked you if you wanted to go to one with me.  \n\nAs shown in the response, the model responds with incorrect time information (**“a few weeks ago”**).\n\nWe will update the paper with these ablation experiment results.\n\n**(3) Shifts in relationships.**\n\nThank you for the suggestion. Our primary objective in this study is to determine whether we can develop a dialogue model that consistently yet dynamically establishes relationships between speakers over a series of dialogue sessions. Accordingly, our model is also equipped to handle shifts in relationships due to events and the passage of time, as our dataset incorporates both temporal and relational dynamics. Below is an example of a model-user interaction:\n\n> **[Relationship: Athlete and Coach]**  \nUSER: Actually, I've been thinking about it since my last injury, but now I'm going to retire as an athlete and live a new life.  \n.....  \nUSER: Thank you. I will probably take coaching training for a year, and I am planning to be a full-time coach after that. It would be meaningful to meet each other as a coach later on.  \nReBot: That sounds like a great plan. I wish you all the best in your coaching career, Athlete.  \n.....  \n**[A couple of years after]**  \nUSER: Hey, I finished my training course and finally became a full-time coach like you!  \nReBot: That's great news! I knew you had it in you.  \nUSER: **Thank you! As I became an official coach, I'd like to learn what I need to know as a coach, could you tell me?**  \n…..  \nReBot: **Exactly. As a coach, you need to understand your players and their motivations. You need to be able to connect with them on a deeper level and help them reach their full potential.**  \nUSER: Thank you! I will be a coach who can raise the potential of the players like you!  \nReBot: **I have no doubt that you will. Just remember to stay true to yourself and your values as a coach, and always lead by example.**\n\nAs illustrated in the example, the initial relationship was that of an 'Athlete and Coach'. Athlete decided to be a coach, transitioning to work alongside their former coach as peers. The model can recognize the change in relationship and respond to the conversation accordingly.\n\nWe will update the paper with this dynamic relationship shift analysis.\n\n**(4) Comparison with MSC 2.7B.**\n\nTechnically, the model architectures of ReBot and MSC 2.7B differ. Unlike MSC 2.7B, ReBot is designed to take both the speaker relationship and time interval as inputs. This makes a direct comparison challenging. However, our rationale for selecting MSC 2.7B as a benchmark is its distinction as the only publicly available model designed for multi-session conversations. If MSC 2.7B were modified to accept speaker relationship and time interval inputs and then trained on our dataset, it might achieve performance comparable to ReBot. However, it is difficult to consider the dataset and model architecture separately.\n\nThus, we believe that our contributions—developing a large-scale dataset incorporating speaker relationships and time intervals, and a model design that uniquely leverages these factors—collectively enable such a natural and engaging conversational experience.\n\n**(5) About Figure 1.**\n\nThank you for pointing that out. We should have mentioned Figure 1 at line 084 in Section 1, but we missed it. We will update this in the paper.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. This paper is well-written and easy to follow.\n\n\n2. The authors have constructed multi-session dialogues based on 10 pre-defined relationships between speakers, which presents a novel and intriguing approach.\n\n3. An extensive human evaluation was undertaken to gauge the quality of both the dataset and the generated dialogues. Consequently, the Conversation Chronicles dataset and the ReBot outperformed the MSC dataset and the MSC 2.7 model, respectively.", "reasons_to_reject": "1. There seems to be a gap in empirical experimentation. Given that each multi-session dialogue is mapped to one of the 10 predefined relationships, a detailed evaluation of dialogues generated for each specific relationship would enhance the empirical robustness of the study.\n\n\n2. How might one measure the impact of speaker relationships on the multi-session conversation dataset? Specifically, if the dataset did not factor in speaker relationships, would there be a noticeable performance drop in ReBot or other long-term conversation models?\n\n\n3. Do the authors operate under the assumption that these relationships remain static throughout multi-session conversations? I'm curious as to why potential shifts in relationships weren't considered. For instance, in a \"Co-workers\" scenario, if one or both individuals retire (or quit), they no longer share the same professional bond. In my opinion, in real-life contexts, relationships often evolve, especially over extended periods, such as several years.\n\n\n4. In the comparison between ReBot and MSC 2.7B, do the authors consider this comparison to be fair? There appears to be a significant discrepancy in the number of sessions between the two. Could the enhanced performance of ReBot be attributed to the more extensive training data it had access to?", "questions_to_authors": "", "paper_topic_and_main_contributions": "This paper presents a novel multi-session dialogue dataset called Conversation Chronicles. Furthermore, the authors have incorporated various time intervals and fine-grained speaker relationships to facilitate long-term conversation modeling. The dataset's quality is superior, boasting a significantly higher number of episodes compared to previous multi-session conversation datasets.", "excitement": 4, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Typos_Grammar_Style_and_Presentation_Improvements": "Although Figure 1 is clear, This is never mentioned in this paper."}, "extra_text": {}}, {"review_id": "1WF91iYl1X", "reviewer": "Reviewer_Eydv", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 3, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your valuable feedback on our paper. Please find our responses below.\n\n**(1) Data generation.**\n\nThe primary focus of our study is the quality of the generated dialogues rather than the data generation process itself. While we explored many options before finalizing the process, we did not present all the options in the paper (we added the full prompt in the appendix for reference though) since previous research has already shown that well-crafted prompts can effectively distill knowledge from LLMs. Instead, our attention has been directed more toward evaluating the quality of generated dialogues through comprehensive human evaluations. That said, we will add more prompts that we tried but did not work best in the paper for the benefit of the research community. Thanks for the suggestion.\n\n**(2) Commonsense.**\n\nWe would like to clarify that our argument is not that our dialogue dataset contains common sense. Nonetheless, if we interpret ‘common sense’ in a broad context (i.e., general knowledge), we believe that the dialogues in our dataset have been verified to have enough common sense to hold natural conversation through extensive human evaluation on various criteria; without such common sense, the dialogues would likely receive lower scores (Table 4). And, as shown in Figure 3, our dataset can be considered comparable with human-generated data. When trained on the dataset, our dialogue model exhibits high performance on evaluation metrics such as ‘Engagingness’ and ‘Humanness’. This suggests that the model contains a sufficient amount of general common sense for conversation, as shown in Table 5.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. The paper addressed an important problem in dialogue system and would be of interest to EMNLP audience\n2. The paper provided a new machine-generated corpus that is provably better than human-generated corpus in certain criteria", "reasons_to_reject": "1. The paper contains modeling work only at a bare minimum. The paper would be stronger if it positions itself as a resource paper, and includes more analysis on data generation. For example, it would be interesting to see the impact of prompt engineering on the generated corpus.\n2. In the post-processing of the LLM-generated corpus, the human curation part seems weak. There is no common sense filtering, nor did the authors evaluate the corpus based on common sense. So the gap between this corpus and human-generated data is unclear.\n  ", "questions_to_authors": "", "paper_topic_and_main_contributions": "The paper addressed the problem of multi-session dialogue generation. It provides a new LLM-generated corpus of multi-session chats and shows they are of better quality than the previous human-generated data. The paper then proposed a new model to generate responses and summarize chats.", "excitement": 3, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "8ihd4ge5Sr", "reviewer": "Reviewer_xS68", "summary": "", "questions": "1. As I mentioned above, s there any mechanism of quality control or verification in LLM (i.e. ChatGPT) outputs?", "limitations": "", "rating": 4, "confidence": 3, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your valuable feedback on our paper. Please find our responses below.\n\n**(1) Ensuring quality of data from LLMs.**\n\nAfter generating data using the LLMs, we implemented an automatic filtering process. This process eliminates outputs that do not satisfy specific criteria, such as those containing harmful content and corner cases (appearance of unrelated speakers, stage directions, etc.). We will include the automatic methods we used for data filtering in the paper.\n\nHowever, relying solely on automated methods for quality verification has its limitations. To complement these methods, we conducted extensive human evaluations—these evaluations encompassed around six thousand episodes, a scale far surpassing that of previous studies which typically involve several hundred episodes—based on various criteria, including consistency, coherence, time interval, and relationship.\n\nWhile we might not be able to eliminate all noisy data, our extensive human evaluation yielded high scores, as shown in Table 4 and Figure 3 (ours vs. human-generated dataset). This strengthens our confidence in the dataset's quality. Thus, through a combination of automated filtering and large-scale human evaluations, we believe our dataset meets high-quality standards.\n\n**(2) Human evaluation result for MSC 2.7B.**\n\nThe human evaluation results comparing ReBot and MSC 2.7B are reported in Figure 5 in the ‘Interactive Dialogue Generation’ paragraph of Section 6. We have explained all experiments and evaluation details in Section 5. Following that, Section 6 presents all the experimental and evaluation results (the only exception is the dataset evaluation results being presented in Section 3.4). We will make necessary update to ensure these details stand out for easier navigation and understanding.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "The main reasons to accept:\n\n1. There is little work done on multi-session conversation history in open-domain conversion. The dataset proposed in this work could be very helpful for further studies. \n2. Dataset is carefully collected in large-scale and it is interesting to see another case where LLM can help reduce the cost of dataset collection. \n3. Paper is well written and presented.", "reasons_to_reject": "The main reasons to reject:\n\nSome details are missing:\n1. Despite the fact that LLM is powerful at generation, we should still be careful about the actual outputs. Is there any mechanism of quality control or verification in LLM (i.e. ChatGPT) outputs?\n2. Not clear about the model performance. Although the section 5.6 mentions that MSC 2.7B is used for comparison, there there is no human evaluation reported for MSC 2.7B.", "questions_to_authors": "1. As I mentioned above, s there any mechanism of quality control or verification in LLM (i.e. ChatGPT) outputs?", "paper_topic_and_main_contributions": "This work explores the direction in open-domain conversation where we have multi-session conversation history and various timestamps (from hours to years). To help support consuming conversation history from previous sessions, the authors collected a large-scale dataset, CONVERSATION CHRONICLES, that contains multi-session conversations from parties with various relationships and time intervals between sessions. The authors also proposed a model that shows good human evaluation result.", "excitement": 4, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper presents a significant contribution to the field of dialogue systems by introducing a novel approach to constructing multi-session dialogues and providing a machine-generated corpus. The resources and insights offered in this paper can be expected to facilitate future studies in this area.
Decision: Accept-Main