MAMM -Refine: A Recipe for Improving Faithfulness in Generation with Multi-Agent Collaboration (2025)

Abstract: Multi-agent collaboration among models has shown promise in reasoning tasks but is underexplored in long-form generation tasks like summarization and question-answering. We extend multi-agent multi-model reasoning to generation, specifically to improving faithfulness through refinement, i.e., revising model-generated outputs to remove factual inconsistencies. We investigate how iterative collaboration among multiple instances and types of large language models (LLMs) enhances subtasks in the refinement process, such as error detection, critiquing unfaithful sentences, and making corrections based on critiques. We design intrinsic evaluations for each subtask, with our findings indicating that both multi-agent (multiple instances) and multi-model (diverse LLM types) approaches benefit error detection and critiquing. Additionally, reframing critiquing and refinement as reranking rather than generation tasks improves multi-agent performance. We consolidate these insights into a final “recipe” called **M**ulti-**A**gent **M**ulti-**M**odel **Refine**ment (MAMM-Refine), where multi-agent and multi-model collaboration significantly boosts performance on three summarization datasets as well as on long-form question answering, demonstrating the effectiveness and generalizability of our recipe. Our code is publicly available.

## introduction large language models (llms) have achieved remarkable performance in natural language generation but still suffer from hallucinations and a lack of faithfulness @xcite @xcite @xcite , where the generated content is inconsistent with the input source or the world. to address this problem, many studies have developed post-hoc selfrefinement techniques @xcite @xcite @xcite . however, these techniques have been found to be less effective without external feedback, as models require external information to identify errors @xcite . one promising avenue for extending models beyond their inherent capabilities is multi-agent debate @xcite @xcite , where multiple llms improve their answers over the course of a debate or discussion. the agents can be multiple instances of the same model or different models (i.e. multi-model). while several approaches focus on improving generation faithfulness through refinement, e.g. by breaking down the refinement process into finegrained subtasks @xcite , past work has used a single instance of the same model for each of these subtasks, without multi-agent collaboration. different models, due to their diverse training data and methods, often exhibit different hallucination patterns @xcite @xcite . therefore, adopting a multi-model, multi-agent framework could help systems achieve higher faithfulness by allowing models to revise their solutions based on diverse answers obtained through collaboration. in such collaborative settings, different models' hallucinations might cancel out. however, several challenges remain before the promise of multi-agent and multi-model approaches can be realized in generation tasks: first, multi-agent frameworks have shown great promise for reasoning tasks @xcite @xcite where the final answers are generally from a closed set and easily verified, leading to easy stopping criteria and enabling voting across agents. applying multi-agent collaboration to generative tasks such as summary refinement -where final answers are long and difficult to verify -is less straightforward. additionally, due to the complexity and multitude of the design choices in generation and refinement tasks, it is not clear which components would benefit from a multi-agent framework. as verified by our empirical results, naively applying multi-agent reasoning to all subtasks might unnecessarily increase cost and could even hurt performance, as agents may lead each other down incorrect paths. to extend multi-agent approaches to long-form generation, we focus on the task of improving faithfulness through refinement, as it is backed by extensive literature on evaluation metrics and generation strategies. as illustrated in figure 1 , we conduct a comprehensive analysis to determine which refinement subtasks benefit most from a multi-agent pipeline. focusing on the three-subtask approach from @xcite , a state-of-the-art refinement strategy, we consider detect, critique, and refine subtasks, which can be recombined into different pipelines (e.g., detect-refine, critique-refine, refine only). we apply multi-agent collaboration to these subtasks, framing critique and refine with two approaches: a discriminative method (rerank) that selects the best option among multiple candidates, and gen-erate, which updates the answer freely. our research addresses three core questions: (1) which refinement subtasks benefit from a multi-agent approach? (2) which subtasks benefit from a multi-model approach? (3) for which task type (generate or rerank) is the multi-agent approach most effective? to answer these, we perform an extensive intrinsic analysis to find the optimal setting for each subtask, creating a "recipe", multi-agent multi-model refinement (mamm-refine), that combines the best configurations. using tofueval @xcite , a dataset with humanannotated sentence-level faithfulness judgments and critiques, we design intrinsic evaluation tasks for each of the three subtasks, detect, critique, and refine, as illustrated in figure 2 . our findings show that multi-agent approaches generally outperform single-model baselines, with multi-model variants offering further gains, and for critique and refine, multi-agent methods provide consistent gains with rerank but not with generate. next, after determining the best configuration for each subtask from the intrinsic evaluations, we evaluate end-to-end performance on three summarization datasets with mamm-refine, mediasum @xcite , meetingbank @xcite , and ultrachat @xcite , comparing our approach with other refinement baselines. across different refinement pipelines that use various combinations of subtasks, we show that selecting the best components from our intrinsic analysis gives us a generalizable recipe for improved refinement that holds across generation tasks and datasets, with gains on all three summarization tasks. we further show that our recipe generalizes to long-form question answering, improving faithfulness in a non-summarization domain. ## method we begin refinement with a model-generated output y and optionally an input context x (e.g., a summarization document or question-answering context @xcite ). using a refinement prompt and model m r , we transform y into a refined output y r . we first outline common refinement pipelines and their corresponding subtasks, and next illustrate how each adapts to our multiagent setting for generative tasks: direct refinement. a single task directly prompts the refinement model to improve the summary based on the document: y @xmath0, y ). ## results we report the intrinsic and extrinsic results, and examine how mamm-refine generalizes to longform question-answering. we provide additional discussions and show improvement from additional agents and how multi-agent performance changes after each round of discussion in appendix b. ## related work multi-agent systems with llms. a large body of research focuses on multi-agent systems for reasoning tasks @xcite @xcite @xcite @xcite , where multiple llms engage in debates or discussions. recent studies have also proposed multi-agent systems for llm evaluation, where agents either undergo a peer review process, obtaining a win rate by ranking each other @xcite , or engage in debates to determine the better llm response @xcite . to address hallucination, @xcite propose using a multi-agent system to identify knowledge gaps between llms. the success of this paradigm hinges on the fact that reasoning tasks typically have welldefined solutions. in contrast, multi-agent systems for generation tasks largely focus on enhancing creativity through role-playing @xcite @xcite , where evaluation metrics are less established. to the best of our knowledge, we are the first to propose a multiagent long-form generation in the context of improving faithfulness on summarization and longform question-answering. refinement. refinement has gained significant focus, including leveraging human feedback (saunders, 2023) and automatic feedback through selfrefinement from the same model @xcite @xcite , other trained models @xcite @xcite @xcite , or external tools @xcite @xcite . for improving faithfulness of summarization, many post-processing approaches @xcite @xcite focus on training such refinement model, or using human-annotated numeric scores as feedback @xcite @xcite . more recently, efforts have concentrated on using llms to directly refine generations, such as by utilizing fine-grained feedback from a faithfulness detector at the level of atomic, non-decomposable facts @xcite , or employing a two-step @xcite or three-step @xcite refinement approach. our work is complementary to past refinement and multi-llm work, as we measure the effect of multi-agent approaches across the components of the refinement pipeline. by testing mm and ma settings, we create a generalizable refinement recipe across generation tasks. ## conclusion we carefully curate components for incorporating multi-agent collaboration into generation, improving generation faithfulness through refinement. through intrinsic evaluations, we find that employing multiple agents, particularly multiple models, benefits discriminative tasks like detect and rerank. we then show how to apply rerank to critique and refine. in extrinsic evaluations, we find that the best variation for each component improves several refinement methods, and our final recipe shows gains on three summarization benchmarks and transfers to long-form questionanswering tasks, showing its generalizability.