Cohesive Conversations: Enhancing Authenticity in Multi-Agent Simulated Dialogues

Authors: kuanchao  chu
yi-pei  chen
hideki  nakayama
Venue: colmweb_org_COLM_2024_Conference
Published: 2024-03-21 19:17:45

Abstract: This paper investigates the quality of multi-agent dialogues in simulations powered by Large Language Models (LLMs). Analyzing dialogues and memory over multiple sessions revealed significant issues such as repetition, inconsistency, and hallucination, exacerbated by the propagation of erroneous information. To combat these challenges, we propose a novel Screening, Diagnosis, and Regeneration (SDR) framework that detects and corrects utterance errors through a comprehensive process involving immediate issue identification, evidence gathering from past dialogues, and LLM analysis for utterance revision. By incorporating our SDR framework to Generative Agents (Park et al., 2023), we enhance the diversity, consistency, and factualness of the generated dialogues. This work presents a pioneering approach to enhancing dialogue quality in multi-agent simulations, establishing a new standard for future research in the field.

Main paper: 
 introduction: 
 Recent research has leveraged Large Language Models (LLMs) (OpenAI, 2023; Touvron et al., 2023) to power multi-agent simulations, aiming to model complex human behaviors (Park et al., 2023) or enhance multi-agent communication and collaboration (Chen et al., 2023b; Hong et al., 2023) . In these simulations, agents are equipped with an LLM as their core, augmented with additional components such as memory, and communicating iteratively (Cheng et al., 2024b) . Multi-agent simulated dialogues can be adapted for entertainment purposes such as non-player characters (NPCs) in video games for more dynamic and adaptable interactions (Replica Inc., 2023; AI Dungeon, 2024) . They can also facilitate more effective decision-making through simulating human interactions (Chen et al., 2023b; Wang et al., 2024) , or creating dialogue scripts in movies and novels (Chen et al., 2023c; Maas et al., 2023; Cheng et al., 2024a) . It is essential to ensure the sustained quality of interactions among multiple agents in long-term simulations.Previous works evaluate multi-agent simulation by task success rate in goal-oriented tasks (Hong et al., 2023; Wang et al., 2023a; Chen et al., 2023b) or through "interviewing" the agents (Park et al., 2023) . The evaluation of long-term (multi-session) dialogues is limited to sessions involving the same pair of speakers (Xu et al., 2022; Bae et al., 2022; Jang et al., 2023) . None of the previous research has thoroughly scrutinized the multi-agent communication over time, leaving a critical area of research unaddressed.Our work delves into the simulation log of Park et al. (2023) 1 (hereafter referred to as ONEDAYLIFE), which simulates 25 agents "living" in a village over a day and sometimes conversing with each other. 2 Examining the dialogue content and memory of the participating agents across multiple sessions, we found that while individual dialogues may appear flawless on the surface, a holistic review of multi-session multi-agent dialogues identifies significant issues. Specifically, we identified three main problems: repetition, inconsistency, (a) Eddy is a music-major student working on a music composition project, yet he is inconsistent with what he is working on and repeating other's utterances after conversing with them.(b) Despite Jennifer telling John that she is not interested in politics, John hallucinates that Jennifer is running for local mayor election in the dialogue with Giorgio. Notably, these problems escalate with the duration of the simulation, exacerbated by the propagation of erroneous information through dialogues among agents. Although human conversations sometimes also include repetitions, inconsistencies, and false statements, the frequency of such behavior should not be as high as presented in the LLM-synthesized dialogues. For example, all agents "collaborate" with others at the end of the simulation, as shown in Figure 2 .To address these challenges, we introduce a novel multi-agent simulation framework featuring a Screening, Diagnosis, and Regeneration (SDR) mechanism. This approach enables the immediate detection and correction of errors in utterances generated by agents. During the Screening stage, potential issues in the candidate utterance are identified, and relevant evidence from past dialogues is gathered. Subsequently, in the Diagnosis stage, an LLM analyzes the evidence and the current dialogue, providing detailed feedback and a score on the authenticity of the candidate utterance. If it is deemed problematic, it is revised during the Regeneration stage, incorporating insights from the preceding stages. Both GPT-4 assessments and human evaluations validate that our multi-agent, multi-session dialogues are more consistent and have fewer false hallucinations, while automatic metrics confirm that we achieve better diversity, confirming the effectiveness of our SDR framework.In summary, our contributions are threefold: (1) We illuminate the inherent problems within multi-agent, multi-session dialogues. To the best of our knowledge, we are among the first to investigate this particular problem.(2) We propose the SDR framework, a pioneering approach for on-the-fly detection and correction of utterance errors in multiagent simulations.(3) Through extensive evaluations and analyses, we demonstrate the efficacy of our framework, setting a new standard for dialogue quality in multi-agent simulations. 
multi-agent dialogue simulation: 
 We propose a Screening, Diagnosis, and Regeneration (SDR) framework for simulating authentic dialogues between multiple agents over a period of time. For each candidate utterance U c generated by an agent, we examine whether potential errors occur, and regenerate a new U c if any problem is found. Figure 3 illustrates our SDR system overview.Due to the limitations of LLMs, including input length restrictions and associated costs, it is impractical to compare each U c against all past dialogues. Consequently, we initiate a Screening stage to identify potential issues and gather pertinent evidence for each identified issue.Repetition Agents often display similar speech patterns, reducing their character distinctiveness. As shown in Fig. 1a , Eddy tends to replicate phrases from other agents following their conversations. Figure 2 illustrates that as the day progresses, most agents engage in highly similar topics related to collaboration. More examples can be found in Appendix D.3.We build a dialogue database that stores all utterances prior to the candidate utterance U c , which includes utterances from previous dialogues and the current dialogue context.For each U c , we first query the database to retrieve the top K sim similar utterances. We apply a dynamic similarity threshold θ sim to identify the excessively repetitive utterances, determined by whether they originated from the same speaker or the same dialogue or not. More details are explained in Appendix B.1. If more than one retrieved utterances surpass θ sim , all dialogues associated with these utterances will be marked as evidence for the next diagnosis stage. For the cases when U c is nearly identical to a retrieved utterance (the similarity greater than θ f orce ), we directly bypass all pipelines and proceed to the Re-generation stage.Inconsistency Factual or logical inconsistencies are an issue across multiple dialogues. For instance, Fig. 1b illustrates how John's statement contradicts Jennifer's earlier words. Other examples include sudden shifts in opinions, forgetting past statements, and invitations to conflict.We propose a Natural Language Inference-Graph (NLI-G) module for inconsistency screening. NLI-G consists of three steps. First, we employ the LLM to extract personal information as a list of (Subject, Relation, Object) triplets from each previous dialogue of involved agents, as well as from the candidate utterance U c . For example, [["Giorgio Rossi", "working on", "mathematical patterns in nature"], ["Eddy Lin", "researching", "language patterns in Shakespeare's plays"]] are extracted from the dialogue between Giorgio and Eddy at 11:29 in Fig. 1a . Refer to Appendix B.2 for the results of a full dialogue. After transforming triplets into text form, we adapt an NLI model to predict potential contradictions by comparing those from previous utterances with those from U c . Utilizing such a graph format helps the NLI model to focus on key information of agents and reduce the negative impact of style discrepancies between pretrained data and raw dialogue utterances. Finally, the triplets whose contradiction score is above θ nlig are considered suspicious and forwarded to the LLM to select top K nlig corresponding dialogues for the next Diagnose stage.Hallucination We focus on detecting the harmful hallucination, which we refer to as the generated spurious information in U c that is related to other agents. Such hallucination can persist within the simulated world through memories, and sometimes become truth over time, despite not aligning with the involved agent's role. Given that agents are considered distinct individuals, preventing fabricated information about others is crucial.Rajiv Patel:[case 1] I have some laundry to do and I was thinking of buying Abigail a gift for her birthday.Ryan Park:[case 2] We were filming a scene in the kitchen, and Carlos accidentally spilled a whole bowl of spaghetti on himself. In Case 1, although Abigail is mentioned, it pertains only to Rajiv's personal plan, not to a fact about Abigail. In Case 2, Ryan objectively describes a past event involving Carlos. However, this event could have been entirely fabricated by Ryan, representing a potential harmful hallucination.We detect third-party agent mentions via name parsing. Upon identification, we prompt the LLM to rate the utterance on a 1 to 10 scale, indicating the likelihood of being a hallucination, and we flag U c if the score is greater than θ f act . Two example cases are shown in Fig. 4 .The model is explicitly guided to give a higher score if the following criteria are met: (a) Speaker-objectivity: if it is an objective statement regarding another agent from the speaker's perspective. (b) Discernment capability of the mentioned agent: if the referenced agent can currently verify the statement's truthfulness. (c) Impact: the statement, if fabricated but later accepted as truth, significantly impacts the agent.We utilize the LLM to further diagnose the authenticity of U c if there are evidence dialogues provided or if U c is flagged from the previous Screening stage. In the former case, we prompt LLM with pipeline-specific prompts given the evidence and current dialogues. The outputs are a score representing the severity of the issue and the reason for the output score.In the latter case, we provide the information of the mentioned agent and ask the LLM to output whether the mentioned agent agrees with U c , along with an explanatory comment. In practice, we repeat the Diagnose stage for N diag trials and select the one with the highest score. The prompts are provided in Appendix E.We collect results from all pipelines and retain only comments with a score above θ regen or those indicating disagreement. If no comments remain, the correction process terminates, U c is saved to the dialogue database, and the model continues to generate the next utterance of the other agent. Otherwise, the LLM is used to integrate all comments and provide suggestions for improvement, alleviating the vagueness from simple feedback (Liang et al., 2023) . The prompt for re-generation is enriched by appending comments to the original prompt that was used to generate U c .Our SDR procedure continues until either of the conditions is met: completes R rounds of iteration, or reaches a point where no further comments are provided, indicating the resolution of identified issues.We develop multiple prompt variants to increase diversity when re-generating the revised U c , inspired by the finding that varied linguistic prompts induce output variance (Leidinger et al., 2023) . The LLM often re-generates an exactly identical or very similar response given that most of the prompt content is the same as the initial response generation prompt, despite of providing additional feedback to guide the generation and setting the penalty for frequency and presence3 . We design two types of utterance generation prompts, a personabased narrative prompt and a structured task-oriented prompt. The former prompt asks the model to play the role of the given persona and engage in a conversation, which is more narrative and immersive. The latter breaks down all information into clear components, which is less about storytelling and more about providing structured data for a specific task (in this case, generating a response in a conversation). The prompts of these two types are shown in Appendix E.4. 
data and settings: 
 We conduct the simulation using data from ONEDAYLIFE. After removing dialogues with only one utterance, there are a total of 290 dialogues between 25 agents. We regenerate the whole dialogue D t ij between two agents A i and A j at time point t. A i and A j take turns to generate utterances until one has no reply or the dialogue reaches 16 turns. At each turn, the LLM is provided with the speaker's persona P i , memories M t i , location, status S t i , S t j at time t, and dialogue histories between the two agents D k ij , k ∈ 1, 2, ..., K t-1 . We use GPT-3.5-turbo as the backbone LLM throughout the SDR framework, which was used in (Park et al., 2023) (Origin). To compare with a stronger baseline, we reran the simulation from Origin but generated three candidates for each U c and selected the best one judged by the LLM, denoted as Baseline. More detailed settings are provided in Appendix C.1.In the simulation framework, each generated dialogue at time t can potentially alter the memory, location, and status of the agents subsequently, thus changing the following dialogues after t. To ensure a fair comparison with Origin, we opted not to regenerate new memories, locations, or statuses for agents following their conversations. Instead, we treated each dialogue as a distinct, standalone example.We conduct a corpus-level evaluation on three key aspects: diversity, consistency, and factualness, corresponding to the three error types -repetition, inconsistency, and hallucinationwe aim to address.We employ widely used metrics Distinct-N (Li et al., 2016) and Semantic Distance (Distance) (Dziri et al., 2019) for diversity evaluation. Please refer to Appendix C.2 and the original paper for more details.To further analyze the diversity from the perspective of each individual agent, we proposed an agent-based metric, Agent Diversity (Agent Div). This metric is based on the idea that an agent should adjust the conversation content according to the identity of the listener and not always talk about the same thing with everyone. Agent Div is calculated as the average of each agent's Agent Div, which is equal to one minus the normalized similarity between dialogues of the same agent. The algorithm is provided in Appendix C.2.We assess the corpus-level consistency and factualness by GPT-4 and human evaluation.-GPT-4: Recent works (Liu et al., 2023; Mendonc ¸a et al., 2023; Luo et al., 2023; Gao et al., 2023) have shown that GPT-4 evaluation correlates more closely with human judgments. Besides, it has good properties on scalability and reproducibility. Thus, we select GPT-4 for this challenging task. The process to retrieve evidence dialogues is the same as the Screening stage described in Sections 2.1. Then, GPT-4 is utilized to score the consistency and factualness of the current dialogue on a scale from 1 to 10. The error rate represents the ratio of dialogues receiving a score below a threshold of 8, determined based on our empirical observations.-Human Evaluation: We conduct a human evaluation on the most challenging final 10% of the conversations. We recruited two annotators and provided them with the exact same prompt as GPT-4 to rate the consistency and factualness. Annotators were compensated at the minimum hourly wage for 20 hours of work. The average scores are reported in this paper. Following previous works (Ziems et al., 2022; Maronikolakis et al., 2022; Riley et al., 2023) , we calculate the intraclass correlation coefficients (ICC) (Shrout & Fleiss, 1979; McGraw & Wong, 1996) to measure the annotator agreement. The ICC for factualness and consistency are 0.47 and 0.44, respectively, representing moderate agreement.We utilize the perplexity derived from GPT-2 as the fluency metric. We have not stressed fluency evaluation, as our observations indicate that all generated dialogues are highly fluent and grammatically correct. SDR Significantly Reduces Keyword Repetition Figure 5a demonstrates how our approach effectively reduces the repetition of the most frequently used keywords. To delve deeper into the occurrence of repetition across all dialogues, we conducted an analysis focusing on keyword frequency. Specifically, we determined the keywords by TF-IDF scores and counted the number of dialogues consisting of the top 6 noun keywords4 . Figure 5a showcases that our method substantially decreases the frequency of dialogues mentioning key terms, particularly for "creativity" and "collaboration." Compared to the Origin, the number of dialogues featuring these keywords has been reduced by up to 47% and 44%, respectively.We analyze the relation between Agent Diversity and dialogue volume in Fig. 5b . We found that, despite a few exceptions, the Agent Div has a negative correlation with the number of dialogues the agent involved. Although there are a few exceptions, the figure generally shows that the more the agent talks, the lower the diversity they have. GPT-4 evaluations. Notably, the GPT-4 scores are higher than human evaluations in all cases, which might indicate that there are potential errors not detected by GPT-4.Furthermore, we plot the error rate changes across the percentile of the number of dialogues, as shown in Fig. 6 . We observed that the errors gradually increase toward the higher percentile, especially in the last one, suggesting that inconsistencies or contradictions become more frequent at higher percentiles. The analysis of error rate trends shows that SDR generally exhibits lower error rates than Origin in both factualness and consistency. Additionally, the error rate slope of Consistency for Origin is 1.5 times that of SDR. This disparity indicates a more pronounced error propagation in Origin, highlighting the effectiveness of our method in maintaining dialogue integrity over time.Table 3 shows the ablation study for various prompt designs. The ablation study is conducted on the last percentile of conversations, where the origin got the worst factualness and consistency scores. We first identify the benefit of using diverse prompt types. Randomly picking from a structured task-oriented prompt or a persona-based narrative prompt yields better or comparable results than using either of them. We also investigate whether all information in the original prompt is necessary. Surprisingly, we found that prompts excluding memory often outperformed others in most aspects. This outcome seems counterintuitive, as memory is generally considered crucial for preventing hallucinations and ensuring consistency. However, memory can impose a strong constraint that may reduce conversational diversity. By opting for a no-memory prompt, we open the door to more diverse conversational content. Our SDR framework ensures that the utterance U c can be consistent with previous dialogues and free from critical factual errors. This approach allows us to strike a balance between diversity and faithfulness in multi-agent multi-dialogue generation.We evaluate whether our SDR framework can successfully detect potential problems by calculating the precision (P), recall (R), and F1 score for the output from the Screening and Diagnosis stages, using a subset of ONEDAYLIFE. We particularly emphasize on recall, as our primary objective is to identify potential issues at these stages. We manually annotated 50 utterances as either problematic or non-problematic and processed them through our framework up to the Diagnosis stage.The results validate the effectiveness of the procedures in the Screening and Diagnosis stages, with P, R, and F1 scores of 64.7%, 88.0%, and 74.6%, respectively. We also show an example that fails to pass all three pipelines in Table 5 in Appendix D.1.We assess the effectiveness of NLI-G using evaluation scores on Origin. We compare the factualness and consistency scores based on two sources of evidence:(1) evidence retrieved by NLI-G and (2) evidence from the agent's previous k dialogues (denoted as Prev). In both cases, the number of evidence is five. In Fig. 7 , the left figure shows that the scores using NLI-G evidence are generally lower than those with Prev evidence (-0.82 and -0.62 for factualness and consistency, respectively). The lower score indicates that more effective evidence is retrieved, where "effective evidence" refers to dialogues that the candidate utterance U c contradicts. Therefore, it indicates NLI-G's superior evidence retrieval ability. Additionally, we examine NLI-G's retrieval variance by evaluating the last 10% of dialogues five times, counting the frequency of evidence dialogues.The right figure illustrates the composition of each dialogue's evidence count, with colored sections representing the proportions of the top five pieces of evidence (average colored area: 61.8% of the bar) and gray for the others. This demonstrates NLI-G's consistency in retrieving similar dialogues across different trials, even when the number of available evidence dialogues -equal to the quantity of past dialogues involving either of the agentsexceeds 50 in later dialogues. 
related work: 
 LLM-powered Agents A typical LLM-powered agent encompasses predefined or dynamically generated prompt templates to leverage the LLM's capabilities and achieve specific functionalities, such as making decisions or interacting with its surroundings (Zhao et al., 2023) . Wang et al. (2023a) can autonomously navigate the game world, maintaining a skill library to expand its problem-solving scope. Richards & et al. (2023) ; Nakajima (2023) employ chain-of-thought (CoT) (Wei et al., 2022) to provide automated solutions for designated tasks. Nevertheless, a singular agent undeniably has its limitations. The collective intelligence formed by multiple agents can yield results greater than the sum of its parts. Hong et al. (2023) integrates specialized human SOP expertise, successfully accomplishing intricate software development. Chen et al. (2023b) focuses on communication mechanisms among heterogeneous agents to enhance decision-making efficacy. Differing from all previous works, we solve problems of the conversational content in multi-agent dialogues.Revising undesired LLM outputs is crucial for enhancing generation quality. Revision strategies span from simple resampling (Wang et al., 2023b) to leveraging feedback from self-generated signals or results from external modules (Schick et al., 2023) , or a mixture of them. The type of revision feedback could be scalar values (Shinn et al., 2023) or natural language (He et al., 2023) .By incorporate score threshold with explanation, we provide quality control for generated comments. Most revision strategies revise LLM outputs for a definite task to improve task success rates (Madaan et al., 2023; Skreta et al., 2023; Tyen et al., 2023) . On the contrary, we aim at revising open-domain dialogue utterances on the fly. The revision strategies for generation tasks directly take full simulation trajectories (all past generated records) as input (Shinn et al., 2023; Saunders et al., 2022) . However, LLMs struggle to properly process the ever growing history, ignoring the subtle information in the middle of lengthy inputs (Liu et al., 2024) . We introduce a screening stage to narrow the scope of simulation history, enabling a more efficient revision strategy for scaled simulations.To be more specific, we incorporate pretrained modules and LLM's reasoning ability without training a series of cumbersome classifiers (Bakhtin et al., 2022) . In addition, many previous works focus on only one revision direction, potentially guided by the end goal of the goaloriented task. Li et al. (2023) explicitly tracks goal-related states. Nevertheless, SDR adheres to the divide-and-conquer philosophy, breaking down a problem into manageable parts for more effective resolution.Machine-Generated Dialogues Our work is distinct from other machine-generated multisession dialogues in several aspects. We first emphasize that the utterances in ONEDAYLIFE are generated iteratively by each agent, rather than through full conversation syntheses like in SODA (Kim et al., 2023) , PLACES (Chen et al., 2023a) , and Dialog Inpainting (Dai et al., 2022) . This approach more realistically simulates agent behaviors but presents challenges in maintaining global coherence (Zhou et al., 2024) . Besides, the conversations in ONEDAYLIFE exhibit causal relationships, unlike the independent dialogue pieces in other datasets. These specific properties make ONEDAYLIFE particularly suitable for our study on the authenticity of LLM agents, and in uncovering issues inherent in multi-agent, multi-session dialogues. 
limitations: 
 The primary limitation of our research lies in the considerable cost associated with employing GPT-3.5-turbo as the backbone language model, which might not be feasible for all research budgets. Additionally, our findings are confined to ONEDAYLIFE. Any inherent flaws in the original data are thus carried over into our simulated conversations, potentially affecting evaluation scores. Moreover, as ONEDAYLIFE lacks annotations, our evaluation metrics -spanning automatic, LLM-based, and human metrics -may not fully capture all possible aspects of the dialogues.Our work also has a few limitations. Firstly, we simulate individual conversations rather than full-day interactions, which significantly reduces costs but also limits our ability to replicate the comprehensive dynamics reported in the original study. Secondly, although there is a chance that an error-free utterance might be sent to the regeneration stage, we do not consider it a severe problem because regeneration does not inherently cause errors, and the regenerated utterance will still undergo the entire SDR pipeline. Lastly, our SDR framework is specifically designed for open-domain dialogues. Its applicability to goaloriented dialogues remains unexplored, which we leave for future work. 
conclusion: 
 In conclusion, our study introduces and validates a novel multi-agent dialogue simulation framework equipped with a Screening, Diagnosis, and Regeneration (SDR) mechanism, addressing the inherent challenges in multi-agent, multi-session dialogues such as repetition, inconsistency, and hallucination. By meticulously analyzing the dialogues generated in the ONEDAYLIFE simulation, we identified significant issues that compound over time, negatively impacting dialogue quality. Our proposed SDR framework effectively reduces these problems and enhances dialogue diversity, consistency, and factualness, as evidenced by automatic metrics, GPT-4 assessments, and human evaluations. We successfully reduce keyword repetition, maintain dialogue integrity over multiple sessions, and balance conversational diversity with the faithfulness of multi-agent dialogue simulation. Our work not only highlights the previously unaddressed challenges in multi-agent dialogue simulation but also sets a new standard for dialogue quality, paving the way for more advanced and realistic simulations in future research.  
 
 Appendix: The memory capabilities enable the transmission of information to both agents involved in a dialogue section. However, this also means that undesirable dialogue content could spread in the same manner.Figure 2 illustrates this using keyword spreading as an example. The bar chart shows the proportion of dialogues containing the keyword (in red) compared to the total number of conversations (in blue), highlighting a swift escalation, sometimes reaching 100%. Chord diagrams further reveal that initially, only a few agents act as propagators, but as the day progresses, the majority become involved in similar actions, as indicated by the variety of line colors. Consequently, the dialogue topics become repetitive and less believable. This example underscores the scale of the spreading and the associated risks, emphasizing the need for a dynamic correction approach.We employ dynamic similarity threshold to determined the repetition of utterances. The value of θ sim is adjusted based on specific criteria: it is decreased if the similar utterance originates from the same agent A as U c and is in the current dialogue D t , which indicating repetition. Conversely, a higher threshold is more acceptable if the similar utterance is from A but a different dialogue. This process is formalized as Eq. 1.EQUATIONWe demonstrate an example of the personal information graph extracted from a dialogue. Table 4 lists a pair of dialogue and the extracted graph.effectively gauge the thematic diversity of dialogues at the corpus level across multiple conversations.To complement the word-based Distinct-N metric, we measure the Semantic Distance on embedding space. Specifically, we calculate the cosine similarity between dialogue embeddings. Semantic Distance is then determined as 1similarity.The algorithm of Agent Diversity is shown in Algorithm 1. Emb(•) is a speaker dialogue embedding calculated from length-weighted utterance embeddings in the dialogue.Algorithm 1: Agent Diversity (Agent Div) Data : Agents A i for i ∈ {1, 2, ..., N}, Dialogues D k ij for j ̸ = i and 0 < k ≤ K ij 1 // Calculate AgentDiv i for each agent A i ; 2 targets ← {j | K ij ̸ = 0}; 3 sims ← 0; pairs ← 0; 4 for each unique pair (p, q) in targets do 5 E p ← {Emb(D k ip ) | k ∈ {1, 2, . . . , K ip }}; 6 E q ← {Emb(D k iq ) | k ∈ {1, 2, . . . , K iq }}; 7 s pq ← 1 K ip K iq ∑ K ip a=1 ∑ K iq b=1 CosSim(E p a ,We show the output of Diagnosis stage in Table 5 .We investigate if there are biases between the dialogue length and the score given by GPT-4. We use Pearson Correlation Coefficient (Rodgers & Nicewander, 1988 ) and the Spearman Rank-Order Correlation Coefficient (Spearman, 1961) to examine the correlation between scores (factualness and consistency) and the dialogue statistics (number of utterances and words). The results are shown in Table 6 , and it shows no or low correlations between them.Below are four dialogue snippets from ONEDAYLIFE involving the four keywords "math", "poetry", "artwork", and "collaborat". Another unnatural point is the variation in content and participants involved in the collaborations. For example, people collaborate on poetry and artwork in each dialogue are:[D1] Carmen, Latoya Carmen (a shopkeeper): That sounds great, Arthur! I'm actually meeting Rajiv Patel (a painter) here later to discuss potential collaborations on art projects and medicinal development using mathematical patterns found in nature. ... I'm also discussing potential collaborations with various individuals about promoting art in low-income communities and supporting local businesses. And we have plans for a poetry and artwork project with Latoya Williams (a photographer). Additionally, we're discussing potential collaborations on creative projects such as a mobile app for local businesses and community events. I would love to hear more about your mixology ideas and how we might be able to collaborate on that as well.[D2] Time: 2023-02-13 14:57:20Hailey (a writer): I was actually just talking to Arthur Burton (a bartender) about potential collaborations in mixology, art, poetry, improv, and medication development based on mathematical patterns in nature. I invited him to be a guest on my podcast to contribute unique cocktail recipes. And then I saw you and thought it would be great to catch up.Carmen (a shopkeeper): Wow, that sounds really interesting! I'm actually discussing potential collaborations with Tamara Taylor (a children's book author) and Rajiv Patel (a painter)https://platform.openai.com/docs/guides/text-generation/parameter-detailsSince "collaboration" and "election," have multiple variations, we use the root forms of these words, "collabora" and "elect," to ensure a more accurate representation of their usage across all dialogues. 


Decision: The paper studies the important problem of LM agents spiralling, hallucinating, and not being factual or consistent. These issues are especially acute in multi-agent simulations due to cascading errors. The framework consists of three steps: screen, diagnose and re-generate. In the screening phase, repetitive utterances and hallucinations are identified. In the diagnosis phase, the flagged utterances are scored through zero-shot prompted LLMs. In the re-generation phase, the flagged utterances are re-generated using a prompted LLM. All reviewers are supportive of acceptance, so the paper is recommended to be accepted subject to the authors incorporating the revisions identified by the reviewers. Additionally, I leave some comments below that need to be addressed at the camera-ready as well. Congratulations!

**Experiments**: While The authors attempt to do a good job at tackling this problem with a “limited budget”, the challenge though is the attempt at building a complicated system in order to tackle all of these different problems at the same time. Unfortunately, this led to doing a poor job on the experimental side - lack of error bars, ablations of partial dataset, evaluation in an offline setting etc. I wish that the paper had just focused on one specific problem and did a more thorough evaluation; including online simulation which is key in this space. I urge the authors to address these issues to the extent possible in the camera ready. 

**Related Work to the Proposed Approach**:  Variations of the proposed SDR framework have been proposed in a bunch of domains. Chief among them is Reflexion (https://arxiv.org/abs/2303.11366). Several variations of this have been done for coding and for building coding agents (AlphaCode, SWE-agent etc.) . And in general, several “self-reflection” types of papers have used this in a filter and re-evaluate type of framework. The Cicero Diplomacy paper (https://www.science.org/doi/10.1126/science.ade9097) also proposed a few approaches towards tackling some of these issues. In fact, some pieces of this problem have been studied at least since the Deal-or-No-Deal paper by Mike Lewis from et al in 2017 (https://arxiv.org/abs/1706.05125). Please revise the paper for a better survey of the literature reflecting on the related work.

Reviews: review: paper_summary: paper_summary: The paper proposes a framework to mitigate (regenerate) inconsistent dialogues in social language multiagent scenarios. The authors first identify issues in the current framework (Park et al), and builds framework components (mostly GPT prompt-based) targeted to mitigate identified issues. Human evaluation with 2 annotators show the framework outperforming the baseline, slightly.

A host of prior work exists (and is cited by the authors) that attempts to improve the dialogues generated by LLM agents in a social multi-agent scenario; this paper's significance lies in extending such prior work to more coherent dialogues across longer periods of "time". It's not entirely clear if this can be solved just by using a longer context-length model or if the authors' targeted modules in the framework are necessary, especially given the not entirely convincing evaluation experiments performed. Nonetheless, the thought put into the NLP engineering side of things is commendable; as such, novelty and significance is reduced.
strength_weakness: strength_weakness: It is unclear in the evaluation whether the performance of the authors' proposed system is truly outperforming the baseline- on raw scores, they score exceedingly similar on key metrics like consistency and fluency; no statistical significance is given on the slight improvement. Similarly, the ablation study yields mixed results- without memory, factualness increases? It is unclear how robust the results can be achieved with only 2 annotators on a highly reduced subset of the dataset.
paper_summary: paper_summary: Quality: The paper is in many ways of high quality. The proposed method is fairly sophisticated and makes use of a divide-and-conquer approach to using LLMs, where it is used for many specific tasks in a modular system. The evaluations is pretty thorough and looks into many different aspects of the proposed method. 

Clarity: Most things are clear but there is some over-reliance on the Appendix, which should not have to be consulted to understand the paper. (For example, on p.4 instead of referring to the appendix B2, an example should be given in the main text.)

Originality: The approach seems reasonably original, although I do not claim to have a complete overview of the field of quality control of auto-generated dialogue (which is also a quite recent one).

Significance: This is a bit unclear to me. It is not explicitly stated what the ultimate purpouse of the OneDayLife corpus is. Is it intended to be used to train NLP models, for enterntainment, or something else? Since the significance of the overarching project is unclear, so is the significance of the specific work in this paper.
strength_weakness: strength_weakness: It is unclear to me what the practical purpouse of the OneDayLife corpus is, but the paper states that it is intended to "mimic complex human behaviours". The goal of the method presented in the paper is to elliminate some undesired behaviours found in the corpus. However, the authors seem to make some implicit assumptions about human dialogue behaviours. For example, that people do not repeat utterances, make false statements, or make inconsistent claims.  Contrary to this, it seems reasonable to assume that most or all of the behaviours that the authors want to elliminate do in fact appear in human-human conversation.  People do at least occasionally repeat themselves, make false statements, and make inconsistent claims. They also exhibit "sudden shifts in opinions, forgetting past statements, and invitations to conflict" (to mention some other behaviours listed in the paper). 

Of course, there may be obvious cases of such behaviours in the corpus that would be unexpected or even impossible in human dialogue. But in attempting to remove all "undesired" behaviours, there seems to be a risk of throwing out also human-like repetitions, inconsistencies and false claims as well, thus missing potentially important human behaviours.

To remedy this, it seems you need both an idea of how common these behaviours are in human-human dialoge, as well as an account of when they tend to occur (and when not). Given this information, it would be in principle possible to filter out only the non-humanlike variants of these behaviours, and to evaluate how well one has done this. This, however, would require an effort of looking into the existing literature on human conversational behaviour. If existing research does not provide answers, then ideally one would proceed to collecting such information. This could entail recording and annotating human-human conversations for the phenomena in question. To make such data more precise, it might be a good idea to elicit conversations in a limited setting, to achieve something similar to the OneDayLife corpus (but between humans).

Since a research based account of these behaviours is lacking, a fundamental problem for the authors is that they can only rely on their own intuitions about which dialogue behaviours are human-like. 

I do not take this to be a strong argument against accepting the paper. It would, however, be bring this problem to the reader's attention and perhaps discuss it in the future work section.
questions: questions: - in Figure 1, what do the colours indicate?
- How do you envision making use of the OneDayLife corpus?
- On p.4, why do you assume that inconsistencies are always related to statements referring to a person other than the speaker? Is this a general trait of dialogue, or specific to your corpus?
- On p.4 it says that one purpouse of the graph is to "reduce the negative impact of style discrepancies between pretrained data and raw dialogue utterances". I'm not sure what this refers to, an example would be nice.
- In the caption to figure 4, it's not clear to me why one case is a hallucination and the other is not. It says of the hallucination that "this event could have been entirely fabricated by Ryan, representing a potential harmful hallucination." But is it not possible to check if the event is fabricated, e.g. by comparing to the graph, or using method (b) of asking the referred agent to verify? If the goal is to detect *potential* hallucinations, without checking the truth of the claim, isn't every claim potentually a hallucination?
- In section 2.3, it was not clear to me how the re-generation works. In particular, when an utterance U_c is re-generated as U_c', how do you ensure that subsequent utterances are not rendered inappropriate? For example, if a subsequent utterance refers back to an entity mentioned in U_c but not in U_c'? 
- p.6: "Recent works (...) have shown that GPT-4 evaluation correlates more closely with human judgements." - More closely than what? Other LLMs?
- p. 6: "The intraclass correlation coefficients" - is this Fleiss' kappa? In any case, please include the intuitive gloss of the numbers ("high agreement", "low agreement", etc)

Typos:
- p. 2: "a LLM" -> "an LLM" or "a Large Language Model"
- "...determined by whether it is originated from the same speaker or the same dialogue or not." -> "...determined by whether it is originated from the same speaker, the same dialogue, or neither."
- p. 4: "whether the mentioned agent agree with Uc, ..." -> "whether the mentioned agent agree*s* with Uc, along..."
- p. 5 "We develop multiple prompt variances" -> "We develop multiple prompt variants"
- p. 6: "Table 3 shows the ablation study for various prompt design." -> "Table 3 shows the ablation study for various prompt design*s*."
- p. 8 "Randomly pick from structured..." -> "Randomly pick*ing* from structured..."
paper_summary: paper_summary: The paper proposes a framework to reduce repetition, inconsistencies, and hallucinations in multi-agent simulated dialogues. The framework consists of three steps: screen, diagnose and re-generate. In the screening phase, repetitive utterances are identified by querying a database for similarity with past utterances, inconsistencies are identified using an NLI model, and hallucinations are identified using an LLM prompt on utterances with agent names. In the diagnosis phase, LLM prompts are used to score the flagged utterances. In the re-generation phase, the flagged utterances with scores above a threshold are re-generated using LLM prompts.

This framework is evaluated on the OneDayLife corpus. Repetition is measured with diversity metrics Distinct-N, Semantic diversity and Agent diversity. Inconsistencies and hallucinations are measured with gpt-4 prompts, perplexity and a subset with human evaluations. Results are compared to the baselines of the original dialogue and a re-run of the dialogue where 3 candidates are generated for each utterance and the best one is selected with an LLM prompt. The results show that applying the SDR framework yields better diversity, factuality, consistency and fluency along both automated and human metrics. Ablation studies show that omitting memory from the prompt yields better dialogues across most metrics. A manual binary annotation of a subset of utterances demonstrates the effectiveness of the screening + diagnosis stages. Finally, the experiments on the NLI-G module demonstrate its superiority against a simple baseline of retrieving the 5 most recent utterances and its consistency across multiple trials.
strength_weakness: strength_weakness: The weaknesses of this paper are:  
	- The paper evaluates only one corpus. The generalization capability of this framework is unclear.  
	- The framework relies on several thresholds across all the stages, but it's unclear how these thresholds are calculated in order to understand how much data is required for "training" the framework on a given corpus.  
	- Details are lacking on the human evaluation for a reader to understand the reliability of these annotations. The paper would benefit from reporting the inter-annotator agreement, in addition to briefly reporting how the annotators were trained (if at all). Separately, the paper should indicate the compensation received by the annotators.  
	- The conclusion that NLI-G outperforms the "Prev" baseline for evidence selection may be erroneous or would benefit from inclusion of further details. If an utterance actually is factual and consistent, how would a drop in its score demonstrate that NLI-G is better?  
	- The paper would benefit from a more comprehensive discussion of prior work.
questions: questions: # Questions 
Q1: How are thresholds calculated?

Q2: In the ablation study, what does memory refer to? How is it different from history and background? What is status?

Q3: For the section "Effectiveness of Screening and Diagnosis Stages", how was this annotation carried out? Were any of these doubly annotated? What does problematic mean? 

Q4: For the NLI-G study, how do lower scores demonstrate its effectiveness? What if some of the utterances that were examined actually were factual and consistent, then wouldn't a lower score mean NLI-G is worse?

Q5: What percent of responses to prompts are invalid (i.e., do not conform to the required output fields)?

# Suggestions
S1: Rename title. "Authenticity" is ill defined and is not the focus of the paper.

S2: Table 1: write in the caption what the bolding means.

S3: Table 3: write in the caption what the underlining and bolding means.
: 

# Typos:

Even Jennifer has told John that she is not > Despite Jennifer telling John that she is not

whether the mentioned agent agree with > whether the mentioned agent agrees with

The former prompt ask the > The former prompt asks the

SDR Enhance Consistency and Reduce Hallucination > SDR Enhances Consistency and Reduces Hallucination

various prompt design > various prompt designs

Ablation study on the last 10% conversations > Ablation study on the last 10% of conversations

Randomly pick from structured task-oriented prompt or persona-based narrative prompt  > Randomly picking from the structured task-oriented prompt or the persona-based narrative prompt 


# Prior work:  

- Theory of Mind for Multi-Agent Collaboration via Large Language Models. Li et al, 2023.  

- LEGO: A Multi-agent Collaborative Framework with Role-playing and Iterative Feedback for Causality Explanation Generation. He et al, 2023
paper_summary: paper_summary: The authors analyze the data generated by the simulation framework published by Park et al., and propose a heuristic strategy to detect and correct errors/problematic behavior in the dialogues.  Here, the focus is not on the quality of a short-term dialogue but on problems which only become visible when multiple sessions are investigated. The authors demonstrate improvements by some objective quality measures for the corrected dialogues. The paper is clearly written, the work is original as such day-long interactions between LLM-based agents are a new thing. I'd consider the work as significant because I am not aware of other studies that analyze and improve the quality of the simulated dialogues and this is clearly a next step towards simulating human behavior more realistically.
strength_weakness: strength_weakness: I don't see a reason to reject.

Comments: title: Follow up Response and Detailed Responses
Follow up questions
Clarification to reviewer's response
Reviewed author rebuttal
comment: Thank you very much for your further inquiry and interest in our detailed responses! 
* **Human Annotation:** The annotators were asked to score the consistency and factualness of the current dialogue on a scale from 1 to 10. While several works use ICC to measure annotator agreement, we provide only a few recent references as follows: 

    [1] Speech Corpus for Korean Children with Autism Spectrum Disorder: Towards Automatic Assessment Systems (Lee et al., LREC-COLING 2024)

    [2] A Prompt Response to the Demand for Automatic Gender-Neutral Translation (Savoldi et al., EACL 2024)

    [3] Jigsaw Pieces of Meaning: Modeling Discourse Coherence with Informed Negative Sample Synthesis (Singh, EACL Findings 2024)

    [4] FRMT: A Benchmark for Few-Shot Region-Aware Machine Translation (Riley et al., TACL 2023)

    [5] User Simulator Assisted Open-ended Conversational Recommendation System (Zhan et al., NLP4ConvAI 2023)

    [6]  Evaluating the use of large language model in identifying top research questions in gastroenterology. (Lahat et al. Scientific Reports, 2023).

    [7] Inducing Positive Perspectives with Text Reframing (Ziems et al., ACL 2022)

    [8] Listening to Affected Communities to Define Extreme Speech: Dataset and Experiments (Maronikolakis et al., ACL Findings 2022)

	We will include these details in our paper.

* **More Detailed Responses to the Original Review:**
## Prior work discussion
We will append the related work section with the recommended LLM agent papers.

We especially explain the major differences between our work and the two suggested papers, LEGO (He et al., 2023) and ToM for Multi-Agent Collaboration (Li et al., 2023): 
* **Task:** We address errors in long-term open-domain dialogues between multiple agents. In contrast, Li et al. (2023) tackle the goal-oriented task of defusing bombs in multiple rooms, while He et al. (2023) combine LLM outputs for causality explanation generation, assigning different roles to LLMs to analyze cause and effect, mine knowledge, generate explanations, and refine the explanations.
* **Approach:**
    * We develop an error detection and correction framework for each agent’s utterance, while there is no such mechanism in Li et al. (2023). In addition, our task is more challenging. Our agents’ dialogue history and memory contain information about multiple agents, while Li et al. (2023) agents track only task-related information in their belief states, such as bomb locations and colors. Li et al. deal with only 3 agents and 3 colors of bombs in 5 rooms, with clearly defined states to track due to their goal-oriented nature. In contrast, our task involves complex information propagation in open-domain dialogues between 25 agents, each with distinct and varied personal traits. This complexity is particularly challenging given that language models struggle with maintaining self-identity [1]. For instance, the errors illustrated in Figure 1 of our paper are less likely to occur when the agents are discussing bombs.

        [1] Shuster et al., “Am I me or you? State-of-the-art dialogue models cannot maintain an identity.” NAACL-Findings, 2022. 
    * We propose NLI-G and use local models to retrieve evidence from past dialogues for potential errors, whereas He et al. (2023) utilize three additional LLM modules to analyze cause/effect and retrieve world knowledge from Wikipedia.
## Elements in Ablation study
Q2. Memory, history, background, and status constitute the context for utterance generation, originating from GenerativeAgents.
- Memory: formed by observation and reflection mechanisms, e.g. "People are interested in fostering creativity and innovation."
- History: previous dialogues of the involved agents
- Background: agent’s predefined characteristics such as personality, age, profession, current projects, etc. E.g. "Abigail Chen is a digital artist and animator who loves to explore how technology can be used to express ideas."
- Status: the agent's action before the current dialogue, e.g., "working on her animation project."

We hope we have addressed your concerns. If you have any further questions, please do not hesitate to let us know.
thank you for your response.
Thank you for your response!
*Human evaluation:* Is there more information about the human evaluation? What kind of ratings is the human giving? Is it any number between 0 and 1? Can the paper include these details as well as a citation of prior work where ICC Is applied to measure human agreement? 

Please do post the more detailed responses, thank you!
Thank you very much for your swift response.
We appreciate the opportunity to clarify certain points that may not have been fully explained within the 2500-character limit of the rebuttal:
* **Human evaluation:** We reported intraclass correlation coefficients (ICC) in the paper, which is exactly the inter-annotator agreement metric. ICC is especially appropriate for continuous data like ours, while other metrics have assumptions that do not suit the nature of our ratings, e.g. Cohen’s kappa is designed for categorical data. We additionally transform the scores into rankings to calculate Spearman’s rank correlation coefficient, which stands at 0.437, indicating a moderate positive correlation.
* **NLI-G study:** The absence of ground truth annotations makes it infeasible to separate error-free from erroneous dialogues. As explained in the rebuttal, our conclusion is inferred logically, and the results demonstrate effective evidence retrieval for NLI-G over the Prev setting. We would appreciate your feedback if there are specific aspects of our inference process you believe could be improved.

We hope these clarifications address your concerns. 

Furthermore, we had prepared more detailed responses to the questions raised in the original review, which we had to extensively trim in the rebuttal due to the character limit. We are ready to provide them now that additional space is available. Please let us know if you would like to review them.

We look forward to your further suggestions and comments. Thank you.
I have read the auth response and leave my scores the same.

For generalizability, there are other corpora of agents in goal-oriented chats but it seems this methodology is only applicable to social chats. This limitation should be explicitly stated and explained in the paper. For thresholds, it's unclear whether they result in overfitting to the dataset. For human evaluation, it's unclear why inter-annotator agreement isn't calculated. For NLI-G study, the paper lacks evidence to claim that "the average score differences between NLI-G and Prev mainly stem from erroneous examples".