DERA : Enhancing Large Language Model Completions with Dialog-Enabled Resolving Agents (2024)

Abstract: Large language models (LLMs) have emerged as valuable tools for many natural language understanding tasks. In safety-critical applications such as healthcare, the utility of these models is governed by their ability to generate factually accurate and complete outputs. In this work, we present dialog-enabled resolving agents (DERA). DERA is a paradigm made possible by the increased conversational abilities of LLMs. It provides a simple, interpretable forum for models to communicate feedback and iteratively improve output. We frame our dialog as a discussion between two agent types – a Researcher, who processes information and identifies crucial problem components, and a Decider, who has the autonomy to integrate the Researcher’s information and makes judgments on the final output.We test DERA against three clinically-focused tasks, with GPT-4 serving as our LLM. DERA shows significant improvement over the base GPT-4 performance in both human expert preference evaluations and quantitative metrics for medical conversation summarization and care plan generation. In a new finding, we also show that GPT-4’s performance (70%) on an open-ended version of the MedQA question-answering (QA) dataset (Jin 2021; USMLE) is well above the passing level (60%), with DERA showing similar performance. We will release the open-ended MedQA dataset.

## introduction large language models (llms; brown et al. (2020); lewis et al. (2020)) are deep-learning models trained to predict natural language text conditioned on an input. these models have led to advances in natural language performance far beyond traditional language modeling tasks, including on few-shot learning (brown et al., 2020) and multimodal tasks (driess et al., 2023). within llm-powered methods use natural language instructions called prompts. these instruction sets often include a task definition, rules the predictions must follow, and few-shot examples of the task input and output (reynolds and mcdonell, 2021; brown et al., 2020). the ability of generative language models to create output based on natural language instructions (or prompts) removes the need for task-specific training (min et al., 2022) and allows non-experts to build upon this technology. while many tasks can be formulated as a single prompt, later work has shown that breaking down single tasks into sub-tasks (called chaining) has benefits in terms of task performance and interpretability (wu et al., 2022). chain-of-thought (cot) (wei et al., 2022) is one example of a chaining strategy in which the model is prompted to think through a problem as an expert might approach it, leading to improvements in some tasks all of these chaining approaches attempt to coerce the correct generation from a base language model. however, one fundamental limitation of this strategy is that they are usually sequential and manually engineered for every task. even with this complexity, chained approaches struggle with generating factually accurate text and often can include hallucinations and omissions (maynez et al., 2020; dziri et al., 2022; berezin and batura, 2022). this poses a significant hurdle when applying them to real-world scenarios, especially in the clinical domain. the increasingly robust and realistic conversational capabilities of llms (openai, 2023; pal et al., 2022) leads us to ask -can reformulating language tasks as conversations between llm agents improve generative output? we present a framework, dera (dialog-enabled resolving agents), for improving performance on natural language tasks using agents tasked with refining task output through dialog. we pair an agent that generates the initial task output with one that can guide the other by suggesting areas of focus in each round of the conversation. dera is a task-agnostic framework that refines text generation issues such as hallucinations and omissions. the dialogue medium adds interpretability to the process and allows the generation to be refined holistically. we propose that scoping each agent in the dialog to a specific role will better enable them to focus on discrete portions of the task and ensure their partner agent stays aligned with the overall goal. our paper makes the following contributions: ## dera: overview dera is a general chat framework that leverages dialog-capable agents to collaboratively work through a task (figure 1 ). we focus on agent setups that work to probe knowledge sources, whether internal or external (from text, documents, etc.). we propose that pairing an information-focused agent with a decision-maker agent will lead to a higherquality output. furthermore, this approach allows for dera to alternate between processing knowledge and acting upon information, as opposed to doing them concurrently. first, we propose the use of a researcher agent, shown in orange in figure 1 . the goal of a researcher agent is to review pieces of informationwhich can be internal to an llm or external -and make suggestions on what is likely to be crucial in solving the problem. as we do not have a definitive source of what is and is not relevant, we rely on an llm's ability to identify relevancy in light of the current task. we do not treat this agent as the definitive source of truth. rather, we task it with being helpful and constructive during the dialog. second, we propose the use of a decider agent, shown in green in figure 1 . in addition to starting the conversation as shown in the left part of the figure, this agent is tasked with responding to the information provided by the researcher agent, and deciding whether to integrate that information into the task output. this allows an llm to make discrete decisions in reaction to the information highlighted by the researcher. at no point, however, does the decider defer to the researcher. this agent is ultimately responsible for the final decision. while it is tasked with reviewing all information highlighted by researcher, it does not have to use any of that information. the specific directives of each agent can vary for different tasks. for question answering, the researcher is tasked with pulling information from the question, using the internal knowledge of an llm alone. for summarization, the researcher has access to external texts which contain the full patient encounter. conversely, the edits to the text generation task are made incrementally by the decider in the summarization task, while they are made more discretely in the question-answering task. in some settings, agents take a hybrid role, each having access to different information and jointly making decisions. overall, the goal remains the same -that this approach allows for informa-tion to be processed in a role-defined and iterative manner, producing better quality output. we use gpt-4 (openai, 2023) as the llm for this paper, but we propose that this approach can generalize to other llms @xcite . we apply dera to three natural language generation tasks. the first, medical conversation summarization ( §3), probes the ability of dera to create a summary of a doctor-patient chat. this requires the ability to identify and rewrite medicallyrelevant information in a concise format. the second, care plan generation ( §4), tests whether dera can generate doctor-facing suggestions for potential actions to address patient concerns. this requires similar abilities, with the added challenge of knowing the appropriate next steps for a variety of medical conditions. finally, medical questionanswering ( §5) tests the ability of dera to generate a wide variety of medical knowledge in a short format. ## medical conversation summarization overview the task of medical conversation summarization is to encapsulate a patient-doctor conversation (enarvi et al., 2020; joshi et al., 2020; zhang et al., 2021; chintagunta et al., 2021). doctors use these summaries for downstream tasks such as clinical decision-making, and hence it is important that the generated summaries are both factually accurate (no hallucinations) and complete (no omissions). we focus on summarizing patient-doctor chats into six independent sections: demographics and social determinants of health, medical intent, pertinent positives, pertinent negatives, pertinent unknowns, and medical history. this structured format requires the model to summarize the chat while placing each piece of information in the appropriate section. ## care plan generation we also analyze the performance of dera on the task of generating a care management plan. this care plan contains suggestions that are meant to be physician-facing -that is, we generate suggestions that a physician would be required to approve of and then communicate to a patient. our care plans contain five sections: medications, referrals, tests, lifestyle, and supportive care. dera setup as in the medical conversation summarization task, the goal of dera is to improve the quality of the generated care plan by suggesting more appropriate home care for the patient, recommending additional lab tests, or otherwise better aligning the generated summary. the dera setup is the same as the medical conversation summarization task with care plan-specific prompts. the decider starts with an initial care plan. the researcher is prompted (prompt 10) to converse with the decider (prompt 9). finally, the decider generates the final care plan (prompt 11). by combining the initial care plan with the content of the 'scratchpad' accumulated during the conversation. we run dera on the care plan generation task using gpt-4 with the settings mentioned in ## open-ended medical question answering we also investigate the use of dera for shortform medical reasoning. a commonly used dataset for this task is medqa in all previous work for this dataset, the primary focus was on the multiple-choice question format which has limited applicability in the real world. if these models are to support doctors in decisionmaking, they need to operate without any options provided. to mimic this setting, we extend the medqa dataset to be open-ended to evaluate the model in a more realistic and harder setting. in an open-ended form, the model must generate the correct answer free-form and not choose from a given bank of options. we also evaluate a set of continuing education questions from the new england journal of medicine (nejm), again in an openended setting. a method that can perform at a high level on this task requires several attributes. first, it must be able to recall a large set of knowledge across multiple domains of medicine. second, it must be able to reason over long questions, which will likely include both irrelevant and crucial facts needed to arrive at the solution. experimental setup for our dera setup, we include multiple prompts for the decider agent, including one that generates a distribution of answers based on a self-consistency approach, one that discusses the question with the researcher, and one that answers the question given the question and chat. we also formulate the researcher agent with a single prompt. we report results on two question-answering datasets that were rewritten as open-ended questions using gpt-4. we include further details in appendix section a.4 about both the dera setup and datasets. to measure the relatedness between generated answers and the gold standard answer, we use a gpt-4 prompt (prompt 19). similarly, we use a separate prompt to make a binary exact match determination (prompt 20). finally, we evaluate the generated and gold answer similarity using bertscore (zhang et ## discussion and conclusion we introduce dera, a framework improving llm generations. this approach reduces the need for an llm to produce a high-fidelity generation in one or two passes. we find that using llm-powered agent dialog is an effective forum to improve output. we use two types of agents -researcher, tasked with reviewing and selecting information, and decider, tasked with integrating that information into the final output. however, we propose that this approach can generalize to other agent setups as dictated by the task. as found in sections 3 and 4, we find dera im-proves the quality of the generated text in a variety of metrics. importantly, this reduces the number of hallucinations and omissions in the resulting text. this finding is important given the ability of llms to generate text that is fluent but potentially prone to errors, especially with gpt-4. the ability of dera to identify and correct these hallucinations and omissions is critical when applying these models to real-world scenarios. a key feature is that the same llm can be harnessed in both roles. we did not find similar improvements in the question-answering task. as discussed in section 5, dera produced little to no improvement over a gpt-4 baseline. we suggest this is due to several factors, including the requirement to generate a single, granular answer. dera often adds information to an answer, which is not helpful for short text generation. these findings, paired with those discussed above, suggest this method is well-suited for longer-generation tasks. the chat-based format of dera allows for increased interpretability when auditing the results. even though llms such as gpt-4 may achieve high performance in zero-shot or one-shot settings, generating long-form explanations does not provide a granular forum for understanding resulting generations. conversely, the chat-based format allows for discussions that are granular and could be verified by an end user for mistakes. we believe these insights are applicable to other domains and tasks given the plug-and-play nature of dera. the dera setup could be altered to include human input in the discussion. alternatively, different problems may dictate the inclusion of different types of agents. overall, we believe that while llm-based tools are critical in increasing the quality of natural language performance, research is required to ensure they are consistent and auditable. finally, we reiterate the need for further research in automated metrics for evaluating llm output. human-led qualitative evaluations can provide important insights, but it remains a challenge to measure improvement given the limited tools currently available.