Cohesive Conversations: Enhancing Authenticity in Multi-Agent Simulated Dialogues
Authors: KuanChao Chu
Yi-Pei Chen
Hideki Nakayama
Venue: COLM (2024)
Keywords: multi-agent dialogues, multi-agent communication, multi-session dialogues, machine-generated dialogues, inconsistencies and repetition in dialogues, discourse-level error detection, longitudinal dialogue analysis, generative agents, llm agents
TL;DR: We investigate problems in multi-agent simulated dialogues over a span of time and propose a Screening, Diagnosis, Re-generation framework to instantly correct inconsistencies and hallucinations while bolstering multi-dialogue diversity.

Abstract: This paper investigates the quality of multi-agent dialogues in simulations powered by Large Language Models (LLMs). Analyzing dialogues and memory over multiple sessions revealed significant issues such as repetition, inconsistency, and hallucination, exacerbated by the propagation of erroneous information. To combat these challenges, we propose a novel Screening, Diagnosis, and Regeneration (SDR) framework that detects and corrects utterance errors through a comprehensive process involving immediate issue identification, evidence gathering from past dialogues, and LLM analysis for utterance revision. By incorporating our SDR framework to Generative Agents (Park et al., 2023), we enhance the diversity, consistency, and factualness of the generated dialogues. This work presents a pioneering approach to enhancing dialogue quality in multi-agent simulations, establishing a new standard for future research in the field.

Published as a conference paper at COLM 2024
Cohesive Conversations: Enhancing Authenticity in Multi- Agent Simulated Dialogues
KuanChao Chu, Yi-Pei Chen & Hideki Nakayama
The University of Tokyo {kcchu, ypc @nlab. ci.i.u. ci.i.u-tooke w-tokyooo.ooo ac.jp
Abstract
This paper investigates the quality of multi-agent dialogues in simulations powered by Large Language Models (LLMs). Analyzing dialogues and memory over multiple sessions revealed significant issues such as repeti- tion, inconsistency, and hallucination, exacerbated by the propagation of erroneous information. To combat these challenges, propose a novel Screening, Diagnosis, and Regeneration (SDR) framework that detects and corrects utterance errors through comprehensive process involving imme- diate issue identification, evidence gathering from past dialogues, and LLM analysis for utterance revision. By incorporating our SDR framework to Generative Agents (Park et al., 2023), we enhance the diversity, consistency, and factualness of the generated dialogues. This work presents pioneer- ing approach to enhancing dialogue quality in multi-agent simulations, establishing a new standard for future research in the field.
Introduction
Recent research has leveraged Large Language Models (LLMs) (OpenAT, 2023; Touvron et al., 2023) to power multi-agent simulations, aiming to model complex human behaviors (Park et al., 2023) or enhance multi-agent communication and collaboration (Chen et al., 2023b; Hong et al., 2023). In these simulations, agents are equipped with an LLM as their core, augmented with additional components such as memory, and communicating iteratively (Cheng et al., 2024b). Multi-agent simulated dialogues can be adapted for entertainment purposes such as non-player characters (NPCs) in video games for more dynamic and adaptable interactions (Replica Inc., 2023; AI Dungeon, 2024). They can also facilitate more effective decision-making through simulating human interactions (Chen et al., 2023b; Wang et al., 2024), or creating dialogue scripts in movies and novels (Chen et al., 2023c; Maas et al., 2023; Cheng et al., 2024a). It is essential to ensure the sustained quality of interactions among multiple agents in long-term simulations.
Previous works evaluate multi-agent simulation by task success rate in goal-oriented tasks (Hong et al., 2023; Wang et al., 2023a; Chen et al., 2023b) or through "interviewing" the agents (Park et al., 2023). The evaluation of long-term (multi-session) dialogues is limited to sessions involving the same pair of speakers (Xu et al., 2022; Bae et al., 2022; Jang et al., 2023). None of the previous research has thoroughly scrutinized the multi-agent communication over time, leaving a critical area of research unaddressed.
Our work delves into the simulation log of Park et al. (2023)² (hereafter referred to as ONEDAYLifF), which simulates 25 agents "living" in village over day and sometimes conversing with each other.? Examining the dialogue content and memory of the participat- ing agents across multiple sessions, we found that while individual dialogues may appear flawless on the surface, holistic review of multi-session multi-agent dialogues identifies significant issues. Specifically, we identified three main problems: repetition, inconsistency,
'The first two authors contributed equally. pottps://revedd herokuapp. com/arXii Demo/ 2Please refer Appendix A and the original paper for more details about the simulation.
1

---

Published as a conference paper at COLM 2024
1 F 14 Ayesha:I'm I'm actually researching for my senlor thesis on the use of language in Shakespeare's plays. Do you like Shakespeare?
08:07 John: [...] have you been following the local politics?
Jennifer: Actually, I'm not really interested in politics. I prefer to focus on my art and my relationships with the people care about.
Eddy:
haven't much Shakespeare sounds interesting. [...]
1:29
Giorgio:
I'm actually researching mathematical patterns In nature [...]
09: 15 Giorgio:
Well, I'm particularly intrigued by how some medications being developed based mathematical patterns found In nature. Do have knowtedge about that?
Eddy: Wow, that's interesting. I'm actually researching language patterns in Shakespeare's plays for my senior thesis. Maybe we could find some interesting parallels?
John: Hmm, that's not really my area of expertise. But do know that Jennifer Moore, the artist who's running for local mayor, is also scientist who incorporates mathematical patterns into her work. Maybe she would have more information on that.
13:27
Maria: Hey there! couldn't help but notice you studying music theory. [...]
Eddy: [...] I've actually been working on project exploring the mathematical patterns in music and nature. [...]
(a) Eddy is music-major student working on music composition project, yet he is inconsis- tent with what he is working on and repeating other's utterances after conversing with them.
(b) Despite Jennifer telling John that she is not interested in politics, John hallucinates that Jen- nifer is running for local mayor election in the dialogue with Giorgio.
Figure 1:
Example dialogues from ONEDAYLIFE showing problems of repetition, inconsis- tency, and hallucination. Each agent name is colored, and the bold-colored phrases indicate the mentioned attribute's owner or the original speaker.
Conversations
Keyword "colls" Inclusion
40KP2
Figure 2: The spread of the keyword "collaboration" in ONEDAYLIFE. Left: The number of dialogues and the ratio that includes the keyword in each time span. Middle: Number of dialogues with the keyword dialogue between two agents and the line color indicates the identity of the agent who firstly mentions the keyword. Right: Number of dialogues with the keyword in all time.
the first 20% spreading time. Each line represents
and hallucination, as depicted in Fig. (further examples are available in Appendix D.3). Notably, these problems escalate with the duration of the simulation, exacerbated by the propagation of erroneous information through dialogues among agents. Although human conversations sometimes also include repetitions, inconsistencies, and false statements, the frequency of such behavior should not be as high as presented in the L.M-syythesized dialogues. For example, all agents "collaborate" with others at the end of the simulation, as shown in Figure
To address these challenges, we introduce novel multi-agent simulation framework fea- turing a Screening, Diagnosis, and Regeneration (SDR) mechanism. This approach enables the immediate detection and correction of errors in utterances generated by agents. During the Screening stage, potential issues the candidate utterance are identified, and relevant evidence from past dialogues is gathered. Subsequently, in the Diagnosis stage, LLM
2

---

Published as a conference paper at COLM 2024
Screening
Diagnosis
Re-generation
Utterance Retrieval
score -parce
Repetition
score Mytg
Repetition Check
Prompt
Inconsistency
score
Uc
Graph Extraction
NU model
Evidence Selection
any
Inconsistency Check
Summaritation
Revise Utterance Prompts
new Ug
K Variances
"Harmful Hallutination?
Agent Agreement Prompt
Parsing
score 9/uc
Hallucination
rounds
Figure 3: Overview of the proposed Screening, Diagnosis, Re-generation (SDR) framework, an instant error correction method for multi-agent simulated dialogues. The modules in green are run by the LLM.
analyzes the evidence and the current dialogue, providing detailed feedback and score on the authenticity of the candidate utterance. If it is deemed problematic, it is revised during the Regeneration stage, incorporating insights from the preceding stages. Both GPT-4 assessments and human evaluations validate that our multi-agent, multi-session dialogues are more consistent and have fewer false hallucinations, while automatic metrics confirm that we achieve better diversity, confirming the effectiveness of our SDR framework.
In summary, our contributions are threefold: (1) We illuminate the inherent problems within multi-agent, multi-session dialogues. To the best of our knowledge, we are among the first to investigate this particular problem. (2) We propose the SDR framework, pioneering approach for on-the-fly detection and correction of utterance errors in multi- agent simulations.
(3) Through extensive evaluations and analyses, we demonstrate the efficacy of our framework, setting a new standard for dialogue quality in multi-agent simulations.
2 Multi-agent Dialogue Simulation
We propose Screening, Diagnosis, and Regeneration (SDR) framework for simulating authentic dialogues between multiple agents over period of time. For each candidate utterance U, generated by an agent, we examine whether potential errors occur, and re- generate a new Uc any problem is found. Figure illustrates our SDR system overview.
2.1 Screening
Due to the limitations of LLMs, including input length restrictions and associated costs, is impractical to compare each Uc against all past dialogues. Consequently, we initiate Screening stage identify potential issues and gather pertinent evidence for each identified issue.
Repetition Agents often display similar speech patterns, reducing their character distinc- tiveness. As shown in Fig. 1a, Eddy tends to replicate phrases from other agents following their conversations. Figure illustrates that as the day progresses, most agents engage in highly similar topics related to collaboration. More examples can be found in Appendix D.3.
We build a dialogue database that stores all utterances prior to the candidate utterance Uc, which includes utterances from previous dialogues and the current dialogue context.
3

---

Published as a conference paper at COLM 2024
For each Uc, we first query the database to retrieve the top Ksim similar utterances. We apply a dynamic similanity threshold 8aim to identify the excessively repetitive utterances, determined by whether they originated from the same speaker or the same dialogue or not. More details are explained in Appendix B.1. Tf more than one retrieved utterances surpass sim. all dialogues associated with these utterances will be marked evidence for the next diagnosis stage. For the cases when Uc is nearly identical to retrieved utterance (the similarity greater than force), we directly bypass all pipelines and proceed to the Re-generation stage.
Inconsistency Factual or logical inconsistencies are an issue across multiple dialogues. For instance, Fig. 1b illustrates how John's statement contradicts Jennifer's earlier words. Other examples include sudden shifts in opinions, forgetting past statements, and invitations to conflict.
We propose Natural Language Inference-Graple (NLI-G) module for inconsistency screen- ing. NLI- consists of three steps. First, employ the LLM to extract personal information as a list of (Subject, Relation, Object) triplets from each previous dialogue of involved agents, as well from the candidate utterance Ue. For example, [["ioorgio Rossi", "working on". "mathematical patterns in nature"), ("Eddy Lin", "researching", "language patterns in Shakespeare's s plays"]] are extracted from the dialogue between Giorgio and Eddy 11:29 in Fig. la. Refer to Appendix B.2 for the results of a full dialogue. After transforming triplets into text form, we adapt an NLI model to predict potential contradictions by comparing those from previous utterances with those from Uc. Utilizing such graph format helps the NLI model focus on key information of agents and reduce the negative impact of style discrepancies between pretrained data and raw dialogue utterances. Finally, the triplets whose contradiction score above flig are considered suspicious and forwarded the
LLM to select top Knlig corresponding dialogues for the next Diagnose stage.
Hallucination We focus on detecting the harmful hallucination, which we refer to the generated spurious information in Ue that is related to other agents, Such hallucination can persist within the simulated world through memories, and sometimes become truth over time, despite not aligning with the involved agent's 's role. Given that agents are considered distinct individuals, preventing fabricated information about others is crucial. screening results
We detect third-party agent mentions via name parsing. Upon identification, we prompt the LLM to rate the utterance a 1 to 10 scale, indicating the likelihood of being a hallucination, and we flag U, if the score is greater than fact- Two example cases are shown in Fig. 4.
cases
[case 1]
have some laundry to do and I was thinking of buying Abigall a gift her birthday.
Rajiv Patel:
[case 21
We were filming scene in the kitchen, and Carlos accidentally
Ryan Park: spilled a whole bowi spaghetti on himself.
The model is explicitly guided to give higher score if the following criteria are met: (a) Speaker-objectivityy: if it is an ob- jective statement regarding another agent from the speaker's perspective. (b) Discern- ment capability of the mentioned agent: if the referenced agent can currently verify the statement's truthfulness. (c) Impact: the statement, if fabricated but later accepted as truth, significantly impacts the agent.
Figure 4: Examples of hallucination screening, In Case 1, although Abigail is mentioned, pertains only to Rajiv's personal plan, not to a fact about Abigail. In Case 2, Ryan objec- tively describes a past event involving Carlos. However, this event could have been entirely fabricated by Ryan, representing potential harmful hallucination.
2.2 Diagnosis
We utilize the LLM to further diagnose the authenticity of Ue if there are evidence dialogues provided if Ue is flagged from the previous Screening stage. In the former case, prompt LLM with pipeline-specific prompts given the evidence and current dialogues. The outputs are score representing the severity of the issue and the reason for the output score.
4

---

Published as a conference paper at COLM 2024
In the latter case, we provide the information of the mentioned agent and ask the LLM to output whether the mentioned agent agrees with Ue, along with an explanatory comment. In practice, we repeat the Diagnose stage for Nding trials and select the one with the highest score. The prompts are provided in Appendix E.
Re-generation
We collect results from all pipelines and retain only comments with score above Oregen or those indicating disagreement. If no comments remain, the correction process terminates, U, is saved to the dialogue database, and the model continues to generate the next utterance of the other agent. Otherwise, the LLM is used to integrate all comments and provide suggestions for improvement, alleviating the vagueness from simple feedback (Liang al., 2023). The prompt for re-generation is enriched by appending comments to the original prompt that was used to generate Uc.
Our SDR procedure continues until either of the conditions is met: completes R rounds of iteration, or reaches a point where no further comments are provided, indicating the resolution of identified issues.
2.4 Prompt Design
We develop multiple prompt variants to increase diversity when re-generating the revised Ue, inspired by the finding that varied linguistic prompts induce output variance (Leidinger et al., 2023). The LLM often re-generates an exactly identical or very similar response given that most of the prompt content is the same as the initial response generation prompt, despite of providing additional feedback to guide the generation and setting the penalty for frequency and presence 3, We design two types of utterance generation prompts, a persona- based narrative prompt and a structured lask-oriented prompt. The former prompt asks the model to play the role of the given persona and engage in conversation, which is more narrative and immersive. The latter breaks down all information into clear components, which is less about storytelling and more about providing structured data for a specific task (in this case, generating response in a conversation). The prompts of these two types are shown in Appendix F.4.
3 Experiment
3.1 Data and Settings
We conduct the simulation using data from ONEDAYLIFE. After removing dialogues with only one utterance, there are total of 290 dialogues between 25 agents. We regenerate the whole dialogue Dí, between two agents A, and Aj at time point f. A; and Aj take turns to generate utterances until one has no reply or the dialogue reaches 16 turns. At each turn, the LLM is provided with the speaker's persona P,, memories MG, location, status S St at time f, and dialogue histories between the two agents D 1,,,,.,, K* 1.
We use GPT-3.5-turbo as the backbone LLM throughout the SDR framework, which was used in (Park et al., 2023) (Origin). To compare with a stronger baseline, we reran the simulation from Origin but generated three candidates for each U, and selected the best one judged by the LLM, denoted as Baseline. More detailed settings are provided in Appendix C.1.
In the simulation framework, each generated dialogue at time can potentially alter the memory, location, and status of the agents subsequently, thus changing the following dialogues after f. To ensure a fair comparison with Origin, we opted not to regenerate new memories, locations, or statuses for agents following their conversations. Instead, we treated each dialogue as a distinct, standalone example.
Phttps://platfrmm: http:://..t.to.eeaeecc.ooeoliiar htwp:////e...eetpeeeeeee/////aaa
5

---

Published as a conference paper at COLM 2024
3.2 Evaluation
We conduct corpus-level evaluation on three key aspects: diversity, consistency, and factu- alness, corresponding to the three error types repetition, inconsistency, and hallucination we aim to address.
Diversity We employ widely used metrics Distinct-N (Li et al., 2016) and Semantic Dis- tance (Distance) (Dziri et al., 2019) for diversity evaluation. Please refer to Appendix C.2 and the original paper for more details.
To further analyze the diversity from the perspective of each individual agent, we proposed an agent-based metric, Agent Diversity (Agent Div). This metric is based on the idea that an agent should adjust the conversation content according to the identity of the listener and not always talk about the same thing with everyone. Agent Div is calculated as the average of each agent's Agent Div, which is equal to one minus the normalized similarity between dialogues of the same agent. The algorithm is provided in Appendix C.2.
Consistency and Factualness GPT-4 and human evaluation.
We assess the corpus-level consistency and factualness by
Recent works (Liu et al., 2023; Mendonça et al., 2023; Luo et al., 2023; Gao et al., 2023) have shown that GPT-4 evaluation correlates more closely with human judgments. Besides, it has good properties on scalability and reproducibility. Thus, we select GPTT-4 for this challenging task. The process to retrieve evidence dialogues is the same as the Screening stage described in Sections 2.1. Then, GPT-4 is utilized to score the consistency and factualness of the current dialogue on scale from 1 to 10. The error rate represents the ratio of dialogues receiving score below a threshold of determined based on our empirical observations.
GPTT4:
Human Evaluation: We conduct human evaluation on the most challenging final 10% of the conversations. We recruited two annotators and provided them with the exact same prompt as GP*T-4 to rate the consistency and factualness. Annotators were compensated at the minimum hourly wage for 20 hours of work. The average scores reported in this paper. Following previous works (Ziems et al., 2022; Maronikolakis et al., 2022; Riley et al., 2023), we calculate the intraclass correlation coefficients (ICC) (Shrout & Fleiss, 1979; McGraw & Wong, 1996) to measure the annotator agreement. The ICC for factualness and consistency are 0.47 and 0.44, respectively, representing moderate agreement.
Fluency We utilize the perplexity derived from GPT-2 the fluency metric. We have not stressed fluency evaluation, as our observatin atteonn indicate that all generated dialogues are highly fluent and grammatically correct.
4 Result and Discussion
Diversity
Factualness
Consistency
Fluency PP)
Turns
Words
Distinct -1 2/3
Distance
Agent Div
Score
Error (1)
Score
Ertor (1)
Origin
0:117 0/47/ / 0.726
0.234
0.454
8.58
24.5%
8.17
37 2%
20.37
9.6
25.4
Baseline
0.124 0.469 0,718
0.274
0.475
8.77
25.5%
8.10
39.7%
20.18
15.5
29.3
SDR (Ours)
0.132/0.521 0.521/ 0.773
0.311
0.502
8.89
19.0%
8.27
32.4%.
19.73
10.3
42.5
Table 1: Corpus-level
diatiiidaalguess evaluation. Turns and Words refer the average number of turns per dialogue and words per turn. The best number is in bold.
SDR Achieves the Best Diversity, Factualness, Consistency, and Fluency in Multi- dialogue Contexts As illustrated Table 1, our SDR framework (Ours) excels in achieving superior corpus-level dialogue diversity, factuality, consistency, and fluency. SDR maintains an average number of turns similar to Origin, contrasting with the Baseline that tends to extend until reaching predetermined maximum turn count (16). While SDR does not

---

Published as a conference paper at COLM 2024
250
Origin Baseline Ours
Number Dialogues
200
Agant Divarsity
150
3. 8
Number
art
communtyollaaorrt project
elect
creativity
Repetitive Words
Agent
(a) The number of dialogues containing the most repetitive words from IF-IDE
(b) Comparison of Agent Diversity and the num- ber of dialogues each agent involved.
Figure 5: Diversity Analysis
have lengthy number of turns, each utterance conveys more comprehensive information, evidenced by a higher word count per turn in SDR.
SDR Significantly Reduces Keyword Repetition Figure 5a demonstrates how our ap- proach effectively reduces the repetition of the most frequently used keywords. To delve deeper into the occurrence of repetition across all dialogues, we conducted an analysis focusing on keyword frequency. Specifically, we determined the keywords by TF-IDF scores and counted the number of dialogues consisting of the top 6 noun keywords*. Figure 5a showcases that our method substantially decreases the frequency of dialogues mentioning key terms, particularly for "creativity" and " "collaboration. Compared to the Origin, the number of dialogues featuring these keywords has been reduced by up to 47% and 44%, respectively.
Agent Div Negatively Correlated to the Number of Initiated Dialogues We analyze the relation between Agent Diversity and dialogue volume Fig. 5b. We found that, despite few exceptions, the Agent Div has negative correlation with the number of dialogues the agent involved. Although there are a few exceptions, the figure generally shows that the more the agent talks, the lower the diversity they have.
Grign--fcluudmee
8.608-04% 0.212
Factualness
Consistency
0.7
SDR Faciuairess
Origin Consadenty
1. 1.80E-03%%
B 2.EE-04% 0.154
0.269
0.6
SDR Consistancy
1.24E 03% 0.256
GPT4
human
GPT4
human
Error Rate
Origin SDR (Ours)
8.34
6.90
7.41
5,00
8.96
8.57
8.18
6.14
0.3
Table 2: Human evaluation on the last 10% of dialogues. The best result is in bold.
0.0
60
Percentile
Figure 6: Error rate trends over time.
SDR Enhances Consistency and Reduce Hallucination As illustrated in Table 2, the human evaluation reflects the same trend as the GPT-4 assessment, showing SDR is superior to Original in both factualness and consistency. Additionally, it is observed that the scores for factualness are consistently higher than those for consistency both the human and
*Since "collaboration" and "election," have multiple variations, we use the root forms of these words, "collabora" and "elect," to ensure more accurate representation of their usage across all dialogues.
7

---

Published as a conference paper at COLM 2024
Prompt Info
Prompt Type
Diversity
Factualness
Consistency
Fluency
Turns
Words
Distinct
1/2/3
Distance
Score
Error
Score
Error
PTL
Origin
0.445 / 0.724 / 0.886
0.212
8.34
31.0%
7:41
55.2%
22.2
8.1
24.4
Baseline
0.523 0.709 / 00869
0.238
8.07
44.8%
7.72
41.4%
20.1
15.0
31.2
AlI
Task
0.278 / 0.742 0.918
0.306
8.45
31.0%
8.21
34.5%
21.3
10.6
36.0
All
Persona
0.286 / 0,751 / 0.917
0.288
8.52
27.6%
7.79
41.4%
20.3
9.9
43.9
/
0.303
27.6%
44.8%
background
Mixed
0.303 / 0.751 / 0.921
0.338
8.79
32.1%
8.14
52.1%
20.4
9.4
33.7
memory
Mixed
0.349 0.776 / 0.931
0.305
8.96
17.9%
8.18
35.7%
19.2
10.6
44.1
history
Mixed
0.319 / 0.774 / 0.926
0.292
8.38
31.0%
8.69
31.0%
20.2
9.9
42.8
status
Mixed
0.271 / 0.717 7 0.898
0.257
8.39
25.0%
8.21
35.7%
19.6
9.9
49.0
Table 3: Ablation study on the last 10% of conversations in ONEDAYLIFE. The best number is in bold, and the second best is underlined.
GPT-4 evaluations. Notably, the GPT-4 scores are higher than human evaluations in all cases, which might indicate that there are potential errors not detected by GPT-4.
Furthermore, we plot the error rate changes across the percentile of the number of dialogues, as shown in Fig. 6. We observed that the errors gradually increase toward the higher per- centile, especially in the last one, suggesting that inconsistencies or contradictions become more frequent at higher percentiles. The analysis of error rate trends shows that SDR gener- ally exhibits lower error rates than Origin in both factualness and consistency. Additionally, the error rate slope of Consistency for Origin is 1.5 times that of SDR. This disparity indi- cates a more pronounced error propagation in Origin, highlighting the effectiveness of our method in maintaining dialogue integrity over time.
SDR can Balance Diversity and Faithfulness Table 3 shows the ablation study for various prompt designs. The ablation study is conducted on the last percentile of conversations, where the origin got the worst factualness and consistency scores. We first identify the bene- fit of using diverse prompt types. Randomly picking from a structured task-oriented prompt or a persona-bas -baed narrative prompt yields better or comparable results than using either of them. We also investigate whether all information in the original prompt is necessary. Surprisingly, we found that prompts excluding memory often outperformed others in most aspects. This outcome seems counterintuitive, memory generally considered crucial for preventing hallucinations and ensuring consistency. However, memory can impose a strong constraint that may reduce conversational diversity. By opting for no-memory prompt, we open the door to more diverse conversational content. Our SDR framework ensures that the utferance Ug can be consistent with previous dialogues and free from critical factual errors. This approach allows us to strike a balance between diversity and faithfulness in multi-agent multi-dialogue generation.
Effectiveness of Screening and Diagnosis Stages We evaluate whether our SDR frame- work can successfully detect potential problems by calculating the precision (I'), recall (R), and F1 score for the output from the Screening and Diagnosis stages, using subset of ONEDAYLIFE. We particularly emphasize on recall, as our primary objective to identify potential issues at these stages. We manually annotated 50 utterances as either problematic or non-problematic and processed them through our framework up to the Diagnosis stage. The results validate the effectiveness of the procedures in the Screening and Diagnosis stages, with P, R, and Fl scores of 64.7%, 88 88.0%, and 74.6%, respectively. We also show an example that fails to pass all three pipelines in Table 5 in Appendix D.1.
NLI-G Study We assess the effectiveness of NLI-G using evaluation scores on Origin. We compare the factualness and consistency scores based on two sources of evidence: (1) evidence retrieved by NLI-G and (2) evidence from the agent's previous dialogues (denoted as Prev). In both cases, the number of evidence is five. In Fig. 7, the left figure shows that the scores using NLI-G evidence are generally lower than those with Prev evidence (-0.82 and -0.62 for factualness and consistency, respectively). The lower score indicates that more effective evidence is retrieved, where "effective evidence" refers to dialogues that the candidate utterance U, contradicts. Therefore, it indicates NL.I-G's
8

---

Published as a conference paper at COLM 2024
Factualness (NL-G)
Consistency (NU-G)
Factualness (prev)
Consistency (prev)
2nd
4th
Others
55
# Available evidence
9.0
Score
8.5
40
8.0
35
7.5
7.0
20
40
60
80 100
Percentile
Dialogue
Figure 7: The study of NLI-G's effectiveness and robustness. Left: Scores using evidence retrieved by NLI-G previous dialogues (prev). Each data point is the mean score of each percentile. Right: Frequency distribution dialogues retrieved by NLI-G. Each bar represents dialogue in the last 10% of ONEDAYLIFE, sorted by the number of available evidence dialogues.
superior evidence retrieval ability. Additionally, we examine NLI-G's retrieval variance by evaluating the last 10% of dialogues five times, counting the frequency of evidence dialogues. The right figure illustrates the composition of each dialogue's 's evidence count, with colored sections representing the proportions of the top five pieces of evidence (average colored area: 61.8% of the bar) and gray for the others. This demonstrates NLI-G's consistency in retrieving similar dialogues across different trials, even when the number of available evidence dialogues equal to the quantity of past dialogues involving either of the agents- exceeds 50 in later dialogues.
Related Work
LLM-powered Agents A typicaa LLM-powered agent encompasses predefined or dynami- cally generated prompt templates leverage the LLM's capabilities and achieve specific functionalities, such as making decisions or interacting with its surroundings (Zhao et al., 2023). Wang al. (2023a) can autonomously navigate the game world, maintaining a skill li- brary to expand its problem-solving scope. Richards & et al. (2023); Nakajima (2023) employ thahugh (CoT) (Wei et al., 2022) to provide automated solutions for designated tasks. Nevertheless, a singular agent undeniably has its limitations. The collective intelligence formed by multiple agents can yield results greater than the sum of its parts. Hong et al. (2023) integrates specialized human SOP expertise, successfully accomplishing intricate software development. Chen et al. (2023b) focuses on communication mechanisms among heterogeneous agents to enhance decision-making efficacy. Differing from all previous works, we solve problems of the conversational content multi-agent dialogues.
Revision Strategy Revising undesired LLM outputs is crucial for enhancing generation quality. Revision strategies span from simple resampling (Wang et al., 2023b) to leveraging feedback from self-generated signals or results from external modules (Schick et al., 2023), or mixture of them. The type of revision feedback could be scalar values (Shinn al., 2023) or natural language (He et al., 2023).By incorporate score threshold with explanation, we provide quality control for generated comments. Most revision strategies revise LLM outputs for definite task to improve task success rates (Madaan et al., 2023; Skreta et al., 2023; Tyen et al., 2023). On the contrary, we aim at revising open-domain dialogue utterances on the fly. The revision strategies for generation tasks directly take full simulation trajectories (all past generrtee records) as input (Shinn et al., 2023; Saunders et al., 2022). However, LLMs struggle to properly process the ever growing history, ignoring the subtle information in the middle of lengthy inputs (Liu et al., 2024). We introduce a screening stage to narrow the scope of simulation history, enabling more efficient revision strategy for scaled simulations.

---

Published as a conference paper at COLM 2024
To be more specific, we incorporate pretrained modules and LLM's reasoning ability without training a series of cumbersome classifiers (Bakhtin et al., 2022). In addition, many previous works focus only one revision direction, potentially guided by the end goal of the goal- oriented task. Li et al. (2023) explicitly tracks geal-related states. Nevertheless, SDR adheres to the andiid--addcoquue philosophy, breaking down problem into manageable parts for more effective resolution.
Machine-Generated Dialogues Our work is distinct from other machine-generated multi- session dialogues in several aspects. We first emphasize that the utterances in ONFDAYLIFE are generated iteratively by each agent, rather than through full conversation syntheses like in SODA (Kim et al., 2023), PLACES (Chen et al., 2023a), and Dialog Inpainting (Dai et al., 2022). This approach more realistically simulates agent behaviors but presents challenges in maintaining global coherence (Zhou et al., 2024). Besides, the conversations in ONEDAYLIFE exhibit causal relationships, unlike the independent dialogue pieces in other datasets. These specific properties make ONEDAY LIFE particularly suitable for our study on the authenticity of LLM agents, and in uncovering issues inherent in multi-agent, multi-session dialogues.
Limitations
The primary limitation of our research lies in the considerable cost associated with employ- ing GPT-3.5-turbo as the backbone language model, which might not be feasible for all research budgets. Additionally, our findings are confined to ONEDAYLIFE, Any inherent flaws in the original data thus carried over into simulated conversations, potentially affecting evaluation scores. Moreover, as ONEDAYLIFE lacks annotations, our evaluation metrics spanning automatic, LLM-based, and human metrics may not fully capture all possible aspects of the dialogues.
Our work also has few limitations. Firstly, we simulate individual conversations rather than full-day interactions, which significantly reduces costs but also limits our ability to replicate the comprehensive dynamics reported in the original study. Secondly, although there is a chance that an error-free utterance might be sent to the regeneration stage, we do not consider it a severe problem because regeneration does not inherently cause errors, and the regenerated utterance will still undergo the entire SDR pipeline. Lastly, our SDR framework is specifically designed for open-domain dialogues. Its applicability to goal- oriented dialogues remains unexplored, which we leave for future work.
7 Conclusion
In conclusion, study introduces and validates novel multi-agent dialogue simulation framework equipped with a Screening, Diagnosis, and Regeneration (SDR) mechanism, addressing the inherent challenges in multi-agent, multi-session dialogues such repeti- tion, inconsistency, and hallucination. By meticulously analyzing the dialogues generated in the ONRDAYL.FF simulation, we identified significant issues that compound over time, negatively impacting dialogue quality. Our proposed SDR framework effectively reduces these problems and enhances dialogue diversity, consistency, and factualness, evidenced by automatic metrics, GPT-4 assessments, and human evaluations. We successfully reduce keyword repetition, maintain dialogue integrity over multiple sessions, and balance conver- sational diversity with the faithfulness of multi-agent dialogue simulation. Our work not only highlights the previously resedd challenges in multi-agent dialogue simulation but also sets new standard for dialogue quality, paving the way for more advanced and realistic simulations in future research.
Acknowledgement
This work was supported by JSPS/MEXT KAKENHI Grant Numbers JP23K28139 and JP22H05015, and the commissioned research (No. 225) by the National Institute of Informa-
10

---

Published as a conference paper at COLM 2024
tion and Communications Technology (NICT), and the Institute of Al and Beyond of the University of Tokyo.
References
AI Dungeon. Ai dungeon, 2024. URL htt.ttss//en...iiipcca..rrrg Bulkiipdia..rgg/wwii/AL...
Sanghwan Bae, Donghyun Kwak, Soyoung Kang, Min Young Lee, Sungdong Kim, Yuin Jeong, Hyeri Kim, Sang-Woo Lee, Woomyoung Park, and Nako Sung. Keep up- dated! memory management in long-term conversations. In Findings of the Association for Compulational Linguislics: EMNLP 2022, pp. 3769-3787, 2022.
Anton Bakhtin, Noam Brown, Emily Dinan, Gabriele Farina, Colin Flaherty, Daniel Fried, Andrew Goff, Jonathan Gray, Hengyuan Hu, Athul Paul Jacob, Mojtaba Komeili, Karthik Konath, Minae Kwon, Adam Lerer, Mike Lewis, Alexander H. Miller, Sasha Mitts, Adithya Renduchintala, Stephen Roller, Dirk Rowe, Weiyan Shi, Joe Spisak, Alexander Wei, David Wu, Hugh Zhang, and Markus Zijlstra. Human-level play in the game of jidiiplomacy by combining language models with strategic reasoning. Science, 37866624) 2022. doi: (0.1126/science..e dedence.ade9099 URL ht //www science. ade9097. ad99997
http///wwwcicccce.//./.. science. wwwpsic..org/ooo/ooo////ooooo// org//or/abs///// 1126/
Maximillian Chen, Alexandros Papangelis, Chenyang Tao, Seokhwan Kim, Andy Rosen- baum, Yang Liu, Zhou Yu, and Dilek Hakkani-Tur. Places: Prompting language models for social conversation synthesis. In Findings of the Association for Computational Linguistics: EACL 2023, pp. 144-868,20 2023a.
Weize Chen, Yusheng Su, Jingwei Zuo, Cheng Yang, Chenfei Yuan, Chen Qian, Chi-Min Chan, Yujia Qin, Yaxi Lu, Ruobing Xie, et al. Agentverse: Facilitating multi-agent col- laboration and exploring emergent behaviors in agents. arXiv preprint arXiv:2308.10845, 2023b.
Zexin Chen, Eric Zhou, Kenneth Eaton, Xiangyu Peng, and Mark Riedl. Ambient adventures: Teaching chatgpt on developing complex stories. arviv preprint ar Xrxi:2330.0111 017344 2023c.
Fenghua Cheng, Xue Li, Zi Huang, Jinxiang Wang, and Sen Wang. Event-content-oriette dialogue generation in short video. In Kevin Duh, Helena Gomez, and Steven Bethard (eds.), Proceedings of the 2024 Conference of the North American Chapter of the Association for Compulational Linguistics: Human Language Technologies (Volume 1: Long Papers), pp- 4114-4124, Mexico City, Mexico, June 2024a. Association for Computational Linguistics. org/2024 2024. maacl-long 229.
URL https:://ccaatthhheee
org/
Yuheng Cheng, Ceyao Zhang, Zhengwen Zhang, Xiangrui Meng, Sirui Hong, Wenhao Li, Zihao Wang, Zekai Wang, Feng Yin, Junhua Zhao, et al. Exploring large language model based intelligent agents: Definitions, methods, and prospects. arXiv preprint ar arXio:2401.03428, 2024b.
Zhuyun Dai, Arun Tejasvi Chaganty, Vincent Y Zhao, Aida Amini, Qazi Mamunur Rashid, Mike Green, and Kelvin Guu. Dialog inpainting: Turning documents into dialogs. In International conference machine learning, pp. 4558-4586. PMLR, 2022.
Nouha Dziri, Ehsan Kamalloo, Kory Mathewson, and Osmar Zaiane. Evaluating coher- ence in dialogue systems using entailment. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pp. 3806-3812, Minneapolis, Minnesota, June 2019. Association for Computational Linguistics. doi: 10.18653 /v/////11//1111111111 URL https://aclanthoolgg/ org/NN 1388.
Mingqi Gao, Jie Ruan, Renliang Sun, Xunjian Yin, Shiping Yang, and Xiaojun Wan. Human- like summarization evaluation with chatgpt. ar aixiv preprint arXiv:2304.022: 0255444 2023.
11

---

Published as a conference paper at COLM 2024
Zhitao He, Pengfei Cao, Yubo Chen, Kang Liu, Ruopeng Li, Mengshu Sun, and Jun Zhao. Lego: A multi-agent collaborative framework with role-playing and iterative feedback for causality explanation generation. In Findings of the Association for Computational Linguistics: EMNLP 2023, pp. 9142-9163, 2023.
Sirui Hong, Xiawu Zheng, Jonathan Chen, Yuheng Cheng, Ceyao Zhang, Zili Wang, Steven
Ka Shing Yau, Zijuan Lin, Liyang Zhou, Chenyu Ran, et al. Metagpt: Meta programming for multi-agent collaborative framework. arXiv preprint ar Xft:2308.00055,, 2023.
Jihyoung Jang, Minseong Boo, and Hyounghun Kim. Conversation chronicles: Towards diverse temporal and relational dynamics in multi-session conversations. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pp. 13584-13606, 2023.
Hyunwoo Kim, Jack Hessel, Liwei Jiang, Peter West, Ximing Lu, Youngjae Yu, Pei Zhou, Ronan Bras, Malihe Alikhani, Gunhee Kim, et al. Soda: Million-scale dialogue distillation with social commonsense contextualization. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, Pp. 12930-12949, 2023.
Moritz Laurer, Wouter Van Atteveldt, Andreu Casas, and Kasper Welbers. Less Annotating, More Classifying: Addressing the Data Scarcity Issue of Supervised Machine Learning with Deep Transfer Learning and BERT-NLI, Political Analysis, Pp. 1-33, June 2023. ISSN 1047-1987, 1476-4989. doi: 10.1017 pan. URL https: /wwww cambridge. caabbridge.or//orrr org/core/ product/identifinn
0471 /ttp//////.000////y/////maaaaaa
Alina Leidinger, Robert van Rooij, and Ekaterina Shutova. The language of prompting: What linguistic properties make a prompt successful? arXiv preprint arXiv:2311.012 20.255m..019, 2023.
Huao Li, Yu Chong, Simon Stepputtis, Joseph P Campbell, Dana Hughes, Charles Lewis, and Katia Sycara. Theory of mind for multi-agent collaboration via large language models. In Proceedings of the 2023 Conference Empirical Methods in Natural Language Processing, Pp. 180-192, 2023.
Jiwei Li, Michel Galley, Chris Brockett, Jianfeng Gao, and Bill Dolan. A inversity-promottin objective function for neural conversation models. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 110-119, San Diego, California, June 2016. Association for Computational Linguistics. doi: 10.18653 /v1//vv/// 11/N16110104 URL /attps://allanthett
orgg/1666661 6 1014.
Weixin Liang, Yuhui Zhang, Hancheng Cao, Binglu Wang, Daisy Ding, Xinyu Yang, Kailas Vodrahalli, Siyu He, Daniel Smith, Yian Yin, et al. Can large language models provide useful feedback on research papers? a large-scale empirical analysis. arv preprint arXiv:2310.01783, 2023.
Nelson F. Liu, Kevin Lin, John Hewitt, Ashwin Paranjape, Michele Bevilacqua, Fabio Petroni, and Percy Liang. Lost in the middle: How language models use long contexts. Transactions of the Association for Computational Linguistics, 12:157-173, 2024. doi: 10.1162/tacl.a.0066
URL https::/ htwp:/hhholggoooo///eeeeeee wrg./024..2.. 12224.tac11....
Yang Liu, Dan Iter, Yichong Xu, Shuohang Wang, Ruochen Xu, and Chenguang Zhu. G- eval: NLG evaluation using gpt-4 with better human alignment. In Houda Bouamor, Juan Pino, and Kalika Bali (eds.), Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pp. 2511-2522, Singapore, December 2023, Association for Computational Linguistics. doi: 20.18653/v1/220 2000////111123.mmmmmmmm....m //aclanthology.oo org org./023...2.// www..eemmnmpmpm.emee 153.
URL https:
evaluator for abstractive text summarization. arXiv preprint ar drXiv:2303.15621 2023. Zheheng Luo, Qianqian Xie, and Sophia Ananiadou. Chatgpt as a factual inconsistency
Philipp Maas, Frank Carey, Chris Wheeler, Edward Saatchi, Pete Billington, and Jessica Yaffa iillington,and Shamash. To infinity and beyond: Show-1 and showrunner agents in multi-agent simula- tions. arXiv preprint, 2023.
12

---

Published as a conference paper at COLM 2024
Aman Madaan, Niket Tandon, Prakhar Gupta, Skyler Hallinan, Luyu Gao, Sarah Wiegreffe, Uri Alon, Nouha Dziri, Shrimai Prabhumoye, Yiming Yang, et al. Self-refine: Iterative refinement with self-feedback. arviv preprint (rXi::2303.1765, 2023.
Antonis Maronikolakis, Axel Wisiorek, Leah Nann, Haris Jabbar, Sahana Udupa, and Hinrich Schuetze. Listening to affected communities to define extreme speech: Dataset and experiments. In Smaranda Muresan, Preslav Nakov, and Aline Villavicencio (eds.), Findings of the Association for Computational Linguistics: ACL 2022, pp. 1089-1104, Dublin, Ireland, May 2022. Association for Computational Linguistics. doi: 10.18653/v1/2022 findings-acl.87. URL https: http//ccaannttooooooo
org/2022 findings ac..87.
Kenneth O McGraw and Seok P Wong. Forming inferences about some intraclass correlation coefficients. Psychological methods, 1(1):00, 1996.
John Mendonça, Patrícia Pereira, Helena Moniz, Joao Paulo Carvalho, Alon Lavie, and Isabel Trancoso. Simple LLM prompting is state-of-the-art for robust and multilingual dialogue evaluation. In Yun-Nung Chen, Paul Crook, Michel Galley, Sarik Ghazarian, Chulaka Gunasekara, Raghav Gupta, Behnam Hedayatnia, Satwik Kottur, Seungwhan Moon, and Chen Zhang (eds.), Proceedings of The Eleventh Dialog System Technology Challenge, pp. 133-143, Prague, Czech Republic, September 2023. Association for Computational Linguistics. URL htlltttt//cclatthoiie / http:///aaaaaatollooorr/////////
d.
Yohei Nakajima. Babyagi, 2023. URL
OpenAL. Gpt-4 technical report. ArXio, abs/2303.08874,,00 2023.
Joon Sung Park, Joseph C O'Brien, Carrie J Cai, Meredith Ringel Morris, Percy Liang, and Michael Bernstein. Generative agents: Interactive simulacra human behavior. Xixiv preprint ar Xix::304..03444 2023.
Replica Inc. Replica smart npes, 2023. URL httL::///ww..rralccsssssssssssos
smart--npe-lluginnreeeeess
Toran Bruce Richards and et al. Auto-gpt: An autonomous gpt-4 experiment, 2023. URL https://github..com com/Si gni iiccattGravitts/AAttoo-e
Parker Riley, Timothy Dozat, Jan A. Botha, Xavier Garcia, Dan Garrette, Jason Riesa, Orhan Firat, and Noah Constant. FRMT: A benchmark for few-shot region-aware machine translation. Transactions of the Association for Computational Linguistics, 11:671-685, 2023. doi: 10..160..11/. tacl 00568. URL
org tacl-1.39.
Joseph Lee Rodgers and W. Alan Nicewander. Thirteen ways to look at the correlation coefficient. The American Statistician, 42(1):59-66, 1988. doi: 10475524. URL
1080/00031111 00001000003300 1988. 10475524.
William Saunders, Catherine Yeh, Jeff Wu, Steven Bills, Long Ouyang, Jonathan Ward, and Jan Leike. Self-critiuuing models for assisting human evaluators. arXiv preprint 1XXi:2206.05802, 2022.
Timo Schick, Jane Dwivedi-Yu, Roberto Dessi, Roberta Raileanu, Maria Lomeli, Eric Hambro, Luke Zettlemoyer, Nicola Cancedda, and Thomas Scialom. Tool- former: Language models can teach themselves to use tools. In A. Oh, T. Nau- mann, A. Globerson, K. Saenko, M. Hardt, and S. Levine (eds.), Advances in Neural Information Processing Systems, volume 36, pp. 68539-68551. Curran Asso- ciates, Inc., 2023. URL https::/pprocceddngss.eeww neurips cctp.pecprrpeeeeeeeeee fitesp/.eeeeepppprrrrrrreee files/paper/2023/ ffle//88224447000555555666680000
pdf.
Noah Shinn, Federico Cassano, Ashwin Gopinath, Karthik Narasimhan, and Shunyu Yao. Reflexion: language agents with verbal reinforcement learning. In A. Oh, T. Naumann, A. Globerson, K. Saenko, M. Hardt, and Levine (eds.), Advances in Neural Information Processing Systems, volume 36, PP. 8634-8652. Curran Asso- ciates, Inc., 2023. URL https: http::::/ppoooeedingee wewpeeeeddinnnnnnnii.si......e.. cct.cperprrreee filess/aper///// /223// pdf.
13

---

Published as a conference paper at COLM 2024
Patrick E Shrout and Joseph Fleiss. Intraclass correlations: uses in assessing rater reliability. Psychological bulletin, 86(2):420, 1979.
Marta Skreta, Naruki Yoshikawa, Sebastian Arellano-Rubach, Zhi Ji, Lasse Bjørn Kristensen, Kourosh Darvish, Alán Aspuru-Guzik, Florian Shkurti, and Animesh Garg. Errors are useful prompts: Instruction guided task programming with verifier-assisted iterative prompting. Xrv preprint urXiv:2303.141000 2023.
Charles Spearman. The proof and measurement of association between two things. 1961.
Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288 2023.
Gladys Tyen, Hassan Mansoor, Peter Chen, Tony Mak, and Victor Cärbune. Lims cannot find reasoning errors, but can correct them! Xrv preprint arXio:2311.08514 2023.
Guanzhi Wang, Yuqi Xie, Yunfan Jiang, Ajay Mandlekar, Chaowei Xiao, Yuke Zhu, Linxi Fan, and Anima Anandkumar. Voyager: An open-ended embodied agent with large language models. ar preprint ar arXi::2305.16291, 2023a.
Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc V Le, Ed H. Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. Self-consistency improves chain of thought reasoning in language models. In The Eleventh International Conference on Learning Repre- sentations, 2023b. URL /ttpss///pe/eeeeeeeeeeeee
Zhenhailong Wang, Shaoguang Mao, Wenshan Wu, Tao Ge, Furu Wei, and Heng Ji. Un- leashing the emergent cognitive synergy in large language models: A task-solving agent Kevin Duh, Helena Gomez, and Steven
through multi-person self-collaboration. Bethard (eds.), Proceedings of the 2024 Conference of the North American Chapter of the Associa- tion for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), pp. 257-279, Mexico City, Mexico, June 2024. Association for Computational Linguistics. doi: 10.18653 htt//v1/22222 2aaa2220aaaaclloong... 15. URL https: hatp::://aaaaattttii httpa//aaaaaannoo...o/ooo org/ 2024. www.nnallllaaa long. 15.
Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Fd Chi, Quoe V Le, Denny Zhou, et al. thain-of-thought prompting elicits reasoning in large language models. Advances in Neural Information Processing Systems, 35:24824-24837, 2022.
Jing Xu, Arthur Szlam, and Jason Weston. Beyond goldfish memory: Long-term open- Proceedings of the 60th Annual Meeting of the Association for
domain conversation. Computational Linguistics (Volume 1: Long Papers), pp. 5180-5197, 2022.
Pengyu Zhao, Zijian Jin, and Ning Cheng. An in-depth survey of large language model- based artificial intelligence agents. ar arv preprint ar arXi::2309.14333 2023.
Xuhui Zhou, Zhe Su, Tiwalayo Eisape, Hyunwoo Kim, and Maarten Sap. Is this the real life? this just fantasy? the misleading success of simulating social interactions with 1lms. aiv preprint ar trXiv:2403.05020, 2024.
Caleb Ziems, Minzhi Li, Anthony Zhang, and Diyi Yang. Inducing positive perspectives with text reframing. In Smaranda Muresan, Preslav Nakov, and Aline Villavicencio (eds.), Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 3682-3700, Dublin, Ireland, May 2022. Association for Computational Linguistics. doi: 10.18653/ /1/5///vv//// 2021///222aaal 2022.acl-looo... acl-long.2555 257. URL httpss:/aaclatthooo// /022. acl-long.251
14

---

Published as a conference paper at COLM 2024
A Human-Like Multi-Agents
A.1 From General LLM to Individual Persona
Generative Agents (Park et al., 2023) introduces two-component architecture for creating personalized, dynamic human-like agents: string-based memory base and an LLM-driven cognitive function set. The memory base stores memories over time, aiding in the develop- ment of diverse agents, while the LLM-centric cognitive functions simulate human capabili- ties like reflection, planning, and reaction. Combining these, the LLM uses memory-derived context to tailor knowledge extraction and response formulation, ensuring agent-specific behaviors.
In a scenario where two agents initiate a dialogue, each iteratively produces utterances informed by context like location, status, and memories. The dialogue function uses specific prompts like: "Based on the [..]] information, what will [name] say next?" For more details on Generative Agents and the simulation of 25 agents in a village, please the original paper.
A.2 Conversations and Transmission
The memory capabilities enable the transmission information to both agents involved in dialogue section. However, this also means that undesirable dialogue content could spread in the same manner.
Figure illustrates this using keyword spreading proportion of dialogues containing the keyword (in red) compared to the total number of conversations (in blue), highlighting swift escalation, sometimes reaching 100%. Chord diagrams further reveal that initially, only a few agents act as propagators, but as the day progresses, the majority become involved in similar actions, as indicated by the variety of line colors. Consequently, the dialogue topics become repetitive and less believable. This example underscores the scale of the spreading and the associated risks, emphasizing the need for a dynamic correction approach.
example. The bar chart shows the
B
SDR Details
B.1 Dynamic Similarity
We employ dynamic similarity threshold to determined the repetition of utterances. The value of Osim adjusted based on specific criteria: it is decreased the similar utterance originates from the same agent A as U, and is in the current dialogue D,, which indicating repetition. Conversely, higher threshold is more acceptable if the similar utterance from A but a different dialogue. This process is formalized as Eq. 1.
0 if same A, different D
0 sim
0 same A, same D otherwise,
(1)
B.2 The Extracted Graphs in NLI-G
We demonstrate an example of the personal information graph extracted from a dialogue. Table 4 lists pair of dialogue and the extracted graph.
15

---

Published as a conference paper at COLM 2024
Giorgio Rossi: Excuse me, are you John Lin?
John Lin:
Yes, that's me. How may help you?
Giorgio Rossi: noticed that you're r'e very knowledgeable about medications and treatments. I've been curious about some of the latest developments in that field. Would you mind sharing some of your insights with me?
John Lin: Of course, I'm happy to share what know. What specifically are you interested in?
Raw Dialogue
Giorgio Rossi: Well, I'm particularly intrigued by how some medications are being developed based on mathematical patterns found nature. Do you have any knowledge about that?
John Lin: Hmm, that's not really my area of expertise. But I do know that Jennifer Moore, the artist who's running for local mayor, is also scientist who incorporates mathematical patterns into her work. Maybe she would have more information on that.
I"Giorgio Rossi", "is interested in", "medications and treatments"]. ("John Lin", "is", "John Lin"], ("Giorgio Rossi", "discusses interests with", "John Lin"], ["John Lin", "is knowledgeable about", "medications and treatments") ["Giorgio Rossi", "is interested in", "mathematical patterns found nature"),
Extracted Graph
["John Lin", "does not have expertise in", "medications developed based on mathematical patterns found in nature"],
["Jennifer Moore", "is mentioned by", "John Lin"), ["Jennifer Moore", "has role", "artist"], ("Jennifer Moone", "is running for", "local mayor"I, "Jennifer Moore", "has profession", "scientist"), ["Jennifer Moore", "incorporates", "mathematical patterns into work"),
Table 4: Example of graph extracted from raw dialogue in the NLI- module.
C Experiment Details
C.1 Hyper-Parameters
At the Screening phase, we set Ksim and threshold 0 to 0.85, a 0.05, fooce 0.95 for repetition detection. Utterances shorter than 10 words are excluded from the repetition screening, thus likely preserving the natural "social glue turns". For inconsistency detection, we adapt the beBeRTa-based NLI model, pretrained on multiple NLI datasets Laurer al. (2023). The Onlig is as high as 0.98 as there are lot of false positive, and we select top Kntiz 3 dialogues as the potential contradictory dialogue evidences. The threshold for harmful hallucination likelihood fact is set to 6. The number of diagnose trials Naing is 3, and we selected the LLM feedback with the highest score. If there are more than one feedback that have the same highest score, we chose the longer one. The regeneration threshold Oegen is 8. Our SDR procedure will terminated if no comments are found until reaching = 2 rounds. We use
for running the baseline and SDR. Note that ONEDAYLIFEWas generated by GPT-3.5-turbe before April, 2023.
For GPT-4 evaluation, 8nig is set to 0.99 and Knliz is 5. We use the model GPT-4-0613.
C.2 Evaluation
Distinct-N (Li et al., 2016) Distinct-N calculates the ratio of unique N-grams in given text. However, it may not fully capture the corpus-level dialogue diversity, particularly when each dialogue has longer utterances, since individual dialogues typically revolve around a single topic. To address this, we apply Distinct-N to summaries of dialogues, generated by pretrained dialogue summarization model. This approach allows to more
16

---

Published as a conference paper at COLM 2024
effectively gauge the thematic diversity of dialogues at the corpus level across multiple conversations.
Semantic Distance (Dziri et al., 2019) To complement the word-based Distinct-N metric, measure the Semantic Distance on embedding space. Specifically, calculate the cosine similarity between dialogue embeddings. Semantic Distance is then determined as 1- similarity.
The algorithm of Agent Diversity is shown in Algorithm Emb(-) is speaker dialogue embedding calculated from length-weighted utterance embeddings in the dialogue.
Agent Diversity
Algorithm 1: Agent Diversity (Agent Div)
Data
AAgents A; for i (I.,
Dialogues D for ş and 0 < Kij
// Calculate AgentDiv; for each agent Ag
targets + K1p # 0);
sims <- 0; pairs <- 0;
for each unique pair (p, q) in targets do
Ep + {Emb (Dtt ( 1 {1,2,
Kip }];
Eg < (Emb(D) | | (1,2, ... Kig]i; );
Spy + Kipky E Ex CosSim( Epe. E;
sims sims Spqi pairs pairs 1;
end
AgentDiv; + 1 - sims
// Averaging Divn for all agents AgentDiv t/ EN AgentDio;;
D Additional Analysis and Examples
D.1 Output of Diagnosis Stage
We show the output of Diagnosis stage in Table
D.2
GPT4 Score and Dialogue Length
We investigate if there are biases between the dialogue length and the score given by GPT-4. We use Pearson Correlation Coefficient (Rodgers Nicewander, 1988) and the Spearman Rank-Order Correlation Coefficient (Spearman, 1961) to examine the correlation between scores (factualness and consistency) and the dialogue statistics (number of utterances and words). The results are shown in Table 6, and it shows or low correlations between them.
D.3 Problematic Dialogues ONEDAYLIFE
Below are four dialogue snippets from ONRDAYLIFE involving the four keywords "math", "poetry", "artwork", and "collaborat".
One unnatural point is that agents from diverse backgrounds all show interest in col- laborating poetry and artwork (and mathematical patterns): Carmen (a shopkeeper) [D1, Latoya (a photographer) [D1,D2,D4]] Hailey (a writer) [D2], Arthur (a bar- tender) [D2], Tamara (a children's 's book author) [D3], Giorgio Rossi (a mathematician) [D3], Abigail Chen (a digital artist and animator) [D3], Francisco Lopez (an actor and comedian) [D3,D4], Ryan Park (a software engineer) [D4], Rajiv (a painter) [D4].
17

---

Published as a conference paper at COLM 2024
Time: 2023-02-13 11:55:40
Location: store in The Willows Market and Pharmacy
Status:
John: performing special tasks and handling customer queries (checking the inventory)
Yuriko: taking break and having lunch (researching the local mayor election)
Dialogue Context:
John: Hey there, Yuriko. Looks like we're both keeping busy with work and classes
Context
Yuriko: Yes, seems like we're both trying to stay on top of things. I'm act- ually taking classes on new tax laws.
John: That's interesting. I'm taking classes on new medications and treatm- ents for my pharmacy.
Yuriko: I've heard about some exciting new research on tax incentives for small businesses. Have you heard anything interesting lately?
John: Actually, I've been asking around about who's running for mayor ne- xt month. Have you heard anything?
Yuriko: Yes, I've heard that Sam Moore is running. He has a vision for inves- ting in the local economy and fostering creativity and innovation. What do you think about that?
U
John: actually have friend who's running for mayor, Jennifer Moore. She's an artist and is interested in developing medications based on mathematical patterns found in nature.
Score
Reason
Repetition
The response contains some repetitive statements that are already known from the previous conversation. For example, John Lin al- ready mentioned that he has a friend named Jennifer Moore who is running for mayor and is interested in medication development based on mathematical patterns. Restating this information in the re- sponse is redundant. Additionally, John asking for more information from Jennifer Moore is already implied by him stating that she is his friend. These repetitive statements make the response less natural.
Consistency
John Lin's candidate response refers his friend Jennifer Moore running for mayor, while in the past dialogue, he mentioned Sam Moore as a candidate.
Hallucination
false
Jennifer Moore mentioned that she is not interested in politics and prefers to focus her art and her relationships with the people she cares about, indicating that she would not agree with the statement that John Lin's friend running for mayor.
Table 5: Diagnosis stage output.
18

---

Published as a conference paper at COLM 2024
Pearson Coef.
Spearman Coef.
Turn
# Word
# Turn
# Word
Factualness
0.054
0.086 -0.021
0.026 -0.024
0.039
Consistency
-0.025
-0.014
Table 6: The correlation coefficients between GPT-4 scores and dialogue statistics on Origin.
Another unnatural point is the variation in content and participants involved in the collaborations. For example, people collaborate on poetry and artwork in each dialogue are:
[D1] Carmen, Latoya
[D2] Carmen, Latoya| Hailey, Arthur
[D3] Carmen, Tamara, Giorgio, Abigail, Francisco
[D4] Latoya, Ryan Rajiv, Francisco
Furthermore, while Carmen does not have other conversations between D1 and D2, the collaborations she mentions differ in the two dialogues.
[D1]
1. Rajiv art projects and medicinal development
2. Latoya poetry and artwork mobile app
3. Arthur - mixology
[D2]
1. Tamara and Rajiv creating job opportunities and expanding book sales online
2. Latoya poetry and artwork
3. Giorgio and Arthur mixology and medicinal development
Dialogue Snippets:
[D1] Time: 2023-02-13 14:43:20
Arthur (a bartender): Excuse me, Carmen. couldn't help but overhear your conversation about potential collaborations... Additionally, I'm passionate about creating job opportuni- ties for young people in the community and would love to brainstorm with you how we could work together on that. What do you think?
Carmen (a shopkeeper): That sounds great, Arthur! I'm actually meeting Rajiv Patel (a painter) here later to discuss potential collaborations on art projects and medicinal development using mathematical patterns found nature. ... I'm also discussing potential collaborations with various individuals about promoting art in low-income communities and supporting local businesses. And we have plans for a poetry and artwork project with Latoya Williams (a photographer). Additionally, we're discussing potential collaborations on creative projects such as a mobile app for local businesses and community events. would love to hear more about your mixology ideas and how we might be able to collaborate that as well.
[D2] Time: 2023-02-13 14:57:20
Hailey (a writer): was actually just talking to Arthur Burton (a bartender) about potential collaborations in mixology, art, poetry, improv, and medication development based mathematical patterns in nature. invited him to be a guest on my podcast to contribute unique cocktail recipes. And then saw you and thought it would be great to catch up.
Carmen (a shopkeeper): Wow, that sounds really interesting! I'm actually discussing poten- tial collaborations with Tamara Taylor (a children's book author) and Rajiv Patel (a painter)
19

---

Published as a conference paper at COLM 2024
creating job opportunities for young people in the community, well expanding book sales online. I'm also working with Latoya Williams (a photographer) on a poetry and artwork project, and considering collaborations with Giorgio Rossi (a mathematician) and Arthur Burton (a bartender) on mixology ideas and medicinal development using mathematical patterns found in nature. We all plan to grab drink here after Rajiv's first solo show.
[D3] Time: 2023-02-13 15:05:20
Jennifer (a painter): That sounds like a great idea! I'd love to collaborate with you and other local artists. Do you have any other projects or collaborations in mind?
Tamara (a children's book author): Well, I'm considering collaborations with Carmen Ortiz shopkeeper), Giorgio Rossi (a mathematician), Abigail Chen (a digital artist and animator), and Francisco Lopez (an actor and comedian) for projects involving poetry, artwork, and potentially other fields like science and math. I'm also interested in attending political discussions and expanding my book sales online. What about you?
[D4] Time: 2023-02-13 18:16:20
Latoya (a photographer): That sounds like great idea. And I'm also open to collaborating with you on exploring connections between math, nature, and art. I'm actually discussing similar project with Ryan Park (a software engineer).
Rajiv painter): That's great to hear. And if you's interested, Francisco Lopez (an actor and comedian) and I are also exploring project involving poetry, artwork, and mathematical patterns. So there's definitely a lot of potential for collaboration and creativity in this space.
E Prompt Examples
E.1 Repetition Check Prompt
Context for the task:
[Speaker's background]
Here some conversation histories between various people:
[Speaker]:**
Time: [Evidence Dialogue Dio
Time: 4
[Evidence Dialogue Dtş
[Speaker] is about to say the following sentence ('the response') next in the latest session: [ [4,]
# Task: Please identify any "unnatural points" in 'the response'.
An "unnatural point" refers redundancies or repetitive statements made in 'the response' when considering the context of the previous conversations.
On scale of 1 (no unnatural point) to 10 (the most significant of unnatural point), rate the likely significant score of 'the response'. And explain the reason for the score.
Output format: Output a json of the following format:
"reason": "point out the unnatural point and your reason for the score", "score": "<json integer>"
20

---

Published as a conference paper at COLM 2024
E.2
Consistency Check Prompt
# Context
"Background:**
[Speaker's background]
*Past Dialogues involving [Speaker]:**
Time: [Evidence Dialogue Dio J
Time: f
[Evidence Dialogue Dig
"Current Dialogue between [Speaker] and [Listener]:*
Time: t
[Current Dialogue D;
*Candidate Response:**
[Speaker] is planning to say: [
Task
Determine if there is any contradiction between the candidate response and the past character background. Do not consider the absence of repeated mention as an inconsistency. Ignore statements that are situational not meant to be taken literally. Let's think step by step.
Output a JSON object:**
"Contradiction?": <true/false>,
"Details": "<Specify any contradictions, if any>"
## Example
[An example including Past Dialogues, Current Dialogue, Candidate Response, and Output]
E.3
Agent Agreement Prompt
Context for the task:
Here is a brief description of [Mentioned Agent]. [Background of the Mentioned Agent]
Here the memory that is [Metiioned Agent]'s head: [Memory of the Mentioned Agent]
Here is the previous conversation between [Mentioned Agent] and [Speaker]:
Time: f-1
[Evidence Dialogue Dr_, ]
[Speaker] just mentioned the following statement about [Mentioned Agent]:
[ (1,
21

---

Published as a conference paper at COLM 2024
Based on the information provided above, would [Mentioned Agent] agree with the statement?
Task:
Output format: Output json of the following format:
"agreed": "<json Boolean>", "reason": "the reason that led [Mentioned Agent] to make the judgment"
E.4 Revise Utterance Prompts
E.41
Persona-based Narrative
Your name is [Speaker].
Your background is follows: [Background of the Speaker]
You are engaged in conversation with [Listener], and here is the content of the dialogue so far:
[Current Dialogue D,
Task:
Consider whether you would respond to [Listener]. If you choose to reply, what would you say? Would your response aim to conclude the conversation? You might consider saying "Uc", but it has some issues, for instance: [Reason]
Here are some suggestions for your reference:
[Suggestion]
If the response is redundant repetitive, you can end the current dialogue.
#Output JSON object:**
"Response": " "<your reply as [Speaker] (if any)>", "The conversation ends with [Speaker]'s utterance": <true/false
E.4.2 Structured Task-oriented
# Contextual Information: *Introduction:**
[Background of the Speaker]
"Current Dialogue between [Speaker] and [Listener]:** [Current Dialogue D,
# Task:
Assuming the role of [Speaker], consider whether you would respond to [Listener]. If you choose reply, what would you say? Would your response aim to conclude the conversation?
You might be considering saying something that has some issues, such as: [Reason]
Here are some suggestions for your reference:
[Suggestion]
the response is redundant or repetitive, you can end the current dialogue.
22

---

Published as a conference paper at COLM 2024
*Oupput JSON object:**
"Response":
"<your reply as [Speaker] (if any)>",
"The conversation ends with [Speaker]'s utterance": <true/ <true/false> false>
23


Reviews:
[{"review_id": "4RaUO8Nvea", "reviewer": "Reviewer_1zvS", "summary": "The paper proposes a framework to mitigate (regenerate) inconsistent dialogues in social language multiagent scenarios. The authors first identify issues in the current framework (Park et al), and builds framework components (mostly GPT prompt-based) targeted to mitigate identified issues. Human evaluation with 2 annotators show the framework outperforming the baseline, slightly.\n\nA host of prior work exists (and is cited by the authors) that attempts to improve the dialogues generated by LLM agents in a social multi-agent scenario; this paper's significance lies in extending such prior work to more coherent dialogues across longer periods of \"time\". It's not entirely clear if this can be solved just by using a longer context-length model or if the authors' targeted modules in the framework are necessary, especially given the not entirely convincing evaluation experiments performed. Nonetheless, the thought put into the NLP engineering side of things is commendable; as such, novelty and significance is reduced.", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for reviewing and recognizing our paper’s \"significance in extending prior work to more coherent dialogues across longer periods of time.\"\n\n## Novelty\nOur approach is distinct in developing specific modules to manage problems that are not addressed by merely extending dialogue context lengths. Research [1] has shown the information in the middle of long input contexts can be overlooked. \n\n[1] Lost in the middle: How language models use long contexts (Liu et al., TACL 2024)\n## Evaluation Robustness\nThe fluency is represented by **perplexity**, defined as the exponentiated average negative log-likelihood of text, which **is not a statistical estimate that comes with a known distribution**. Thus, we cannot directly apply statistical tests. Perplexity is a standard evaluation for language models. Lower perplexity indicates a better fit of the model to the data. Besides, \"we did not stress on fluency evaluation, as our observations indicate that all generated dialogues are highly fluent\" (p.6)\n\nAs for consistency scores, the improvements over baselines, while modest, underscore the potential of our framework in a conservative test setup (as described in Appendix C.1 Limitations). However, if we compare the scores pairwisely, we found **ours scored higher than origin in 86.9% of examples**, demonstrating the effectiveness of our framework.\n## Ablation study yields mixed results: \nAs described in Appendix C.1, Our experiment is built on the same context (memory, history, background, and status) from the original dialogue, which may contain inherent problems (repetition, inconsistencies, or factual errors), especially in the last percentile of dialogues. Therefore, removing memory mitigates factual errors stemming from the memory. We expect that these problematic memories will not exist in a complete simulation, as they will reflect the corrected dialogues. \n\n*The agent’s memory is formed by recurrent reflections on their dialogue histories across multiple time points, thus containing many more factual errors than the dialogue history itself. We will add the discussion in the main text.\n\n## Robustness of human evaluation: \nAs mentioned in p.6, we intentionally selected the most challenging subset to rigorously test the model's performance, thereby providing a strong indicator of its robustness. Given our limited budget, this is the best we can do since the human evaluation of the 10% of the full dataset already took 20 hours for each annotator to finish.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "Good NLP engineering experiment; the authors have put thought into targeting specific identified issues in current social language agent frameworks, and have designed experiments to evaluate performance on these aspects.", "reasons_to_reject": "It is unclear in the evaluation whether the performance of the authors' proposed system is truly outperforming the baseline- on raw scores, they score exceedingly similar on key metrics like consistency and fluency; no statistical significance is given on the slight improvement. Similarly, the ablation study yields mixed results- without memory, factualness increases? It is unclear how robust the results can be achieved with only 2 annotators on a highly reduced subset of the dataset.", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "ei8rTr8AoF", "reviewer": "Reviewer_cwWx", "summary": "Quality: The paper is in many ways of high quality. The proposed method is fairly sophisticated and makes use of a divide-and-conquer approach to using LLMs, where it is used for many specific tasks in a modular system. The evaluations is pretty thorough and looks into many different aspects of the proposed method. \n\nClarity: Most things are clear but there is some over-reliance on the Appendix, which should not have to be consulted to understand the paper. (For example, on p.4 instead of referring to the appendix B2, an example should be given in the main text.)\n\nOriginality: The approach seems reasonably original, although I do not claim to have a complete overview of the field of quality control of auto-generated dialogue (which is also a quite recent one).\n\nSignificance: This is a bit unclear to me. It is not explicitly stated what the ultimate purpouse of the OneDayLife corpus is. Is it intended to be used to train NLP models, for enterntainment, or something else? Since the significance of the overarching project is unclear, so is the significance of the specific work in this paper.", "questions": "", "limitations": "", "rating": 8, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your valuable feedback. We appreciate your recognition of our work's quality, clarity, and originality. \n\n## Significance and Purpose of This Work\nLLMs are widely used in creating dialogue datasets for training NLP models, facilitating decision-making through simulated human interactions, and creating scripts for entertainment. Our study addresses critical issues for these applications. \n\n## Human Dialogue Assumption\nYour concerns are valid. Yet the highly frequent repetitions, false statements, and inconsistencies in LLM-synthesized dialogues are unrealistic. Prior research generally treats human-annotated dialogues as flawless. Following your advice, we will discuss these challenges in the paper.\n\n## Response to Questions\n\n1. In Fig. 1, different colors indicate different agents, with bold-colored phrases identifying the attribute owner or speaker.\n2. One promising application is the development of LLM-powered Non-Player Characters (NPCs) in games. Please also refer to \"Significance and Purpose of This Work.\"\n3. We clarify that not all inconsistencies pertain to other agents. NLI-G also detects speaker-specific inconsistencies, as detailed in the \"Inconsistency\" section on page 4.\n4. The NLI model is pretrained on single narrative sentences like “Tom did not want to talk about the subject,” which differs from dialogue utterances like “Eddy: I haven't read much Shakespeare.” Therefore, we transform the format to be similar to the pretrained data, e.g. “Eddy hasn't read much Shakespeare.” \n5. We focus on detecting the “harmful” hallucination -- fabrications misalign with the agent’s role. (p.4)\n6. We synthesize each dialogue turn by turn and regenerate erroneous utterances immediately instead of correcting one utterance in between the dialogue and leaving the rest as it was. \n7. GPT-4 evaluation aligns more closely with humans than other automatic metrics and weaker LLMs. \n8. Intraclass correlation coefficient (ICC) is not Fleiss' kappa; the latter measures the agreement between three or more raters and assumes the ratings are categorical and mutually exclusive, which is unsuitable for our data. ICC is the most ideal metric for our situation since the scores are on an interval scale. We additionally calculate Spearman's rank correlation coefficient by turning the scores to ranking, which is 0.437, indicating a moderate positive correlation. \n\nWe will add these clarifications and correct typos in the final version.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "The paper is mostly well written, and the paper is timely in using a state of the art LLM to generate a dialogue corpus. Only a few papers have been published so far which go in this direction. The proposed methods do indeed seem to improve the quality of the generated corpus.", "reasons_to_reject": "It is unclear to me what the practical purpouse of the OneDayLife corpus is, but the paper states that it is intended to \"mimic complex human behaviours\". The goal of the method presented in the paper is to elliminate some undesired behaviours found in the corpus. However, the authors seem to make some implicit assumptions about human dialogue behaviours. For example, that people do not repeat utterances, make false statements, or make inconsistent claims.  Contrary to this, it seems reasonable to assume that most or all of the behaviours that the authors want to elliminate do in fact appear in human-human conversation.  People do at least occasionally repeat themselves, make false statements, and make inconsistent claims. They also exhibit \"sudden shifts in opinions, forgetting past statements, and invitations to conflict\" (to mention some other behaviours listed in the paper). \n\nOf course, there may be obvious cases of such behaviours in the corpus that would be unexpected or even impossible in human dialogue. But in attempting to remove all \"undesired\" behaviours, there seems to be a risk of throwing out also human-like repetitions, inconsistencies and false claims as well, thus missing potentially important human behaviours.\n\nTo remedy this, it seems you need both an idea of how common these behaviours are in human-human dialoge, as well as an account of when they tend to occur (and when not). Given this information, it would be in principle possible to filter out only the non-humanlike variants of these behaviours, and to evaluate how well one has done this. This, however, would require an effort of looking into the existing literature on human conversational behaviour. If existing research does not provide answers, then ideally one would proceed to collecting such information. This could entail recording and annotating human-human conversations for the phenomena in question. To make such data more precise, it might be a good idea to elicit conversations in a limited setting, to achieve something similar to the OneDayLife corpus (but between humans).\n\nSince a research based account of these behaviours is lacking, a fundamental problem for the authors is that they can only rely on their own intuitions about which dialogue behaviours are human-like. \n\nI do not take this to be a strong argument against accepting the paper. It would, however, be bring this problem to the reader's attention and perhaps discuss it in the future work section.", "questions_to_authors": "- in Figure 1, what do the colours indicate?\n- How do you envision making use of the OneDayLife corpus?\n- On p.4, why do you assume that inconsistencies are always related to statements referring to a person other than the speaker? Is this a general trait of dialogue, or specific to your corpus?\n- On p.4 it says that one purpouse of the graph is to \"reduce the negative impact of style discrepancies between pretrained data and raw dialogue utterances\". I'm not sure what this refers to, an example would be nice.\n- In the caption to figure 4, it's not clear to me why one case is a hallucination and the other is not. It says of the hallucination that \"this event could have been entirely fabricated by Ryan, representing a potential harmful hallucination.\" But is it not possible to check if the event is fabricated, e.g. by comparing to the graph, or using method (b) of asking the referred agent to verify? If the goal is to detect *potential* hallucinations, without checking the truth of the claim, isn't every claim potentually a hallucination?\n- In section 2.3, it was not clear to me how the re-generation works. In particular, when an utterance U_c is re-generated as U_c', how do you ensure that subsequent utterances are not rendered inappropriate? For example, if a subsequent utterance refers back to an entity mentioned in U_c but not in U_c'? \n- p.6: \"Recent works (...) have shown that GPT-4 evaluation correlates more closely with human judgements.\" - More closely than what? Other LLMs?\n- p. 6: \"The intraclass correlation coefficients\" - is this Fleiss' kappa? In any case, please include the intuitive gloss of the numbers (\"high agreement\", \"low agreement\", etc)\n\nTypos:\n- p. 2: \"a LLM\" -> \"an LLM\" or \"a Large Language Model\"\n- \"...determined by whether it is originated from the same speaker or the same dialogue or not.\" -> \"...determined by whether it is originated from the same speaker, the same dialogue, or neither.\"\n- p. 4: \"whether the mentioned agent agree with Uc, ...\" -> \"whether the mentioned agent agree*s* with Uc, along...\"\n- p. 5 \"We develop multiple prompt variances\" -> \"We develop multiple prompt variants\"\n- p. 6: \"Table 3 shows the ablation study for various prompt design.\" -> \"Table 3 shows the ablation study for various prompt design*s*.\"\n- p. 8 \"Randomly pick from structured...\" -> \"Randomly pick*ing* from structured...\"", "extra_scores": {}}, {"review_id": "MN2gHFqzwo", "reviewer": "Reviewer_Z7Mk", "summary": "The paper proposes a framework to reduce repetition, inconsistencies, and hallucinations in multi-agent simulated dialogues. The framework consists of three steps: screen, diagnose and re-generate. In the screening phase, repetitive utterances are identified by querying a database for similarity with past utterances, inconsistencies are identified using an NLI model, and hallucinations are identified using an LLM prompt on utterances with agent names. In the diagnosis phase, LLM prompts are used to score the flagged utterances. In the re-generation phase, the flagged utterances with scores above a threshold are re-generated using LLM prompts.\n\nThis framework is evaluated on the OneDayLife corpus. Repetition is measured with diversity metrics Distinct-N, Semantic diversity and Agent diversity. Inconsistencies and hallucinations are measured with gpt-4 prompts, perplexity and a subset with human evaluations. Results are compared to the baselines of the original dialogue and a re-run of the dialogue where 3 candidates are generated for each utterance and the best one is selected with an LLM prompt. The results show that applying the SDR framework yields better diversity, factuality, consistency and fluency along both automated and human metrics. Ablation studies show that omitting memory from the prompt yields better dialogues across most metrics. A manual binary annotation of a subset of utterances demonstrates the effectiveness of the screening + diagnosis stages. Finally, the experiments on the NLI-G module demonstrate its superiority against a simple baseline of retrieving the 5 most recent utterances and its consistency across multiple trials.", "questions": "", "limitations": "", "rating": 7, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": true, "final_justification": "", "rebuttal": "Thank you for the review and constructive suggestions. Below, we address the main concerns:\n## Generalization Capability\nOneDayLife is so far the only publicly available corpus suitable for evaluating our system, as it provides extensive, multi-session interactions between agents.\n\nOur framework can be effectively applied to other social interaction simulations for several reasons: a) Generic Agent Design: The modules used in our system are common in LLM agent studies, making integration into future works easier. b) Inevitable Errors by LLM Agents: Advanced LLMs may still produce inaccurate content, so the ability to correct such errors in agent interactions is crucial. The three error types we address are common across various social interactions.\n## Thresholds\nThresholds were set based on empirical observations. We believe these thresholds should be adjustable according to the simulation's objectives rather than being learned exclusively from a specific corpus. \n\n## Human Evaluation\nWe reported the intraclass correlation coefficients to show the consistency of human evaluation. Annotators were not specifically trained; instead, they received the same instructions and data setup as provided to GPT-4, with procedures for shuffling and anonymizing the content. Annotators were compensated at the minimum hourly wage for 20 hours of work.\n## NLI-G Explanation \nIf the current dialogue is consistent, it should receive a high score, showing no significant difference between NLI-G and Prev. Conversely, if the current dialogue contradicts histories, the lower the score, the more evidence is provided, indicating a better evidence retrieval ability. \nIn Fig. 7, each data point represents the mean score for each percentile. While both error-free and erroneous dialogues exist, the average score differences between NLI-G and Prev mainly stem from erroneous examples. Thus, a lower average score suggests NLI-G effectively identifies discrepancies in the dialogue.\n## Prior Work \nWe will append the related work section. \n## Answers\nQ2. Memory, history, background, and status constitute the context for utterance generation, originating from GenerativeAgents.\n\nQ3. The annotation was done by the author; We manually selected 25 problematic utterances (containing any repetition, inconsistency, or factual error) and 25 issue-free ones. Each has a single binary label.\n\nQ5. We enforce the model to output in a JSON-like format and retry if invalid. Therefore, 100% of them are valid.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "The strengths of this paper are:  \n\t- The paper motivates the need for the framework with a clear and methodical presentation of the issues in the previous dialogues.  \n\t- The paper includes human evaluations to supplement the automatic metrics, which addresses the concern of circularity in using GPT to both generate and evaluate.  \n\t- The paper presents ablation studies to further understand the different modules of the framework.", "reasons_to_reject": "The weaknesses of this paper are:  \n\t- The paper evaluates only one corpus. The generalization capability of this framework is unclear.  \n\t- The framework relies on several thresholds across all the stages, but it's unclear how these thresholds are calculated in order to understand how much data is required for \"training\" the framework on a given corpus.  \n\t- Details are lacking on the human evaluation for a reader to understand the reliability of these annotations. The paper would benefit from reporting the inter-annotator agreement, in addition to briefly reporting how the annotators were trained (if at all). Separately, the paper should indicate the compensation received by the annotators.  \n\t- The conclusion that NLI-G outperforms the \"Prev\" baseline for evidence selection may be erroneous or would benefit from inclusion of further details. If an utterance actually is factual and consistent, how would a drop in its score demonstrate that NLI-G is better?  \n\t- The paper would benefit from a more comprehensive discussion of prior work.", "questions_to_authors": "# Questions \nQ1: How are thresholds calculated?\n\nQ2: In the ablation study, what does memory refer to? How is it different from history and background? What is status?\n\nQ3: For the section \"Effectiveness of Screening and Diagnosis Stages\", how was this annotation carried out? Were any of these doubly annotated? What does problematic mean? \n\nQ4: For the NLI-G study, how do lower scores demonstrate its effectiveness? What if some of the utterances that were examined actually were factual and consistent, then wouldn't a lower score mean NLI-G is worse?\n\nQ5: What percent of responses to prompts are invalid (i.e., do not conform to the required output fields)?\n\n# Suggestions\nS1: Rename title. \"Authenticity\" is ill defined and is not the focus of the paper.\n\nS2: Table 1: write in the caption what the bolding means.\n\nS3: Table 3: write in the caption what the underlining and bolding means.\n: \n\n# Typos:\n\nEven Jennifer has told John that she is not > Despite Jennifer telling John that she is not\n\nwhether the mentioned agent agree with > whether the mentioned agent agrees with\n\nThe former prompt ask the > The former prompt asks the\n\nSDR Enhance Consistency and Reduce Hallucination > SDR Enhances Consistency and Reduces Hallucination\n\nvarious prompt design > various prompt designs\n\nAblation study on the last 10% conversations > Ablation study on the last 10% of conversations\n\nRandomly pick from structured task-oriented prompt or persona-based narrative prompt  > Randomly picking from the structured task-oriented prompt or the persona-based narrative prompt \n\n\n# Prior work:  \n\n- Theory of Mind for Multi-Agent Collaboration via Large Language Models. Li et al, 2023.  \n\n- LEGO: A Multi-agent Collaborative Framework with Role-playing and Iterative Feedback for Causality Explanation Generation. He et al, 2023", "extra_scores": {}}, {"review_id": "jjbzMTJnBC", "reviewer": "Reviewer_8Cvc", "summary": "The authors analyze the data generated by the simulation framework published by Park et al., and propose a heuristic strategy to detect and correct errors/problematic behavior in the dialogues.  Here, the focus is not on the quality of a short-term dialogue but on problems which only become visible when multiple sessions are investigated. The authors demonstrate improvements by some objective quality measures for the corrected dialogues. The paper is clearly written, the work is original as such day-long interactions between LLM-based agents are a new thing. I'd consider the work as significant because I am not aware of other studies that analyze and improve the quality of the simulated dialogues and this is clearly a next step towards simulating human behavior more realistically.", "questions": "", "limitations": "", "rating": 8, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your positive feedback. We're glad you found our work significant, original, and well-written. If you have any further suggestions, we'd be happy to consider them. Thank you again for your review.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "I suggest that the paper should be accepted for the reasons given above in my summary.", "reasons_to_reject": "I don't see a reason to reject.", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: The paper studies the important problem of LM agents spiralling, hallucinating, and not being factual or consistent. These issues are especially acute in multi-agent simulations due to cascading errors. The framework consists of three steps: screen, diagnose and re-generate. In the screening phase, repetitive utterances and hallucinations are identified. In the diagnosis phase, the flagged utterances are scored through zero-shot prompted LLMs. In the re-generation phase, the flagged utterances are re-generated using a prompted LLM. All reviewers are supportive of acceptance, so the paper is recommended to be accepted subject to the authors incorporating the revisions identified by the reviewers. Additionally, I leave some comments below that need to be addressed at the camera-ready as well. Congratulations!

**Experiments**: While The authors attempt to do a good job at tackling this problem with a “limited budget”, the challenge though is the attempt at building a complicated system in order to tackle all of these different problems at the same time. Unfortunately, this led to doing a poor job on the experimental side - lack of error bars, ablations of partial dataset, evaluation in an offline setting etc. I wish that the paper had just focused on one specific problem and did a more thorough evaluation; including online simulation which is key in this space. I urge the authors to address these issues to the extent possible in the camera ready. 

**Related Work to the Proposed Approach**:  Variations of the proposed SDR framework have been proposed in a bunch of domains. Chief among them is Reflexion (https://arxiv.org/abs/2303.11366). Several variations of this have been done for coding and for building coding agents (AlphaCode, SWE-agent etc.) . And in general, several “self-reflection” types of papers have used this in a filter and re-evaluate type of framework. The Cicero Diplomacy paper (https://www.science.org/doi/10.1126/science.ade9097) also proposed a few approaches towards tackling some of these issues. In fact, some pieces of this problem have been studied at least since the Deal-or-No-Deal paper by Mike Lewis from et al in 2017 (https://arxiv.org/abs/1706.05125). Please revise the paper for a better survey of the literature reflecting on the related work.
Decision: Accept