Chatbot To Help Patients Understand Their Health (2025)

Abstract: Patients must possess the knowledge necessary to actively participate in their care. To this end, we developed NoteAid-Chatbot, a conversational AI designed to help patients better understand their health through a novel framework of learning as conversation. We introduce a new learning paradigm that leverages a multi-agent large language model (LLM) and reinforcement learning (RL) framework—without relying on costly human-generated training data. Specifically, NoteAid-Chatbot was built on a lightweight 3-billion-parameter LLaMA 3.2 model using a two-stage training approach: initial supervised fine-tuning on conversational data synthetically generated using medical conversation strategies, followed by RL with rewards derived from patient understanding assessments in simulated hospital discharge scenarios. Our evaluation, which includes comprehensive human-aligned assessments and case studies, demonstrates that NoteAid-Chatbot exhibits key emergent behaviors critical for patient education—such as clarity, relevance, and structured dialogue—even though it received no explicit supervision for these attributes. Our results show that even simple Proximal Policy Optimization (PPO)-based reward modeling can successfully train lightweight, domain-specific chatbots to handle multi-turn interactions, incorporate diverse educational strategies, and meet nuanced communication objectives. Our Turing test demonstrates that NoteAid-Chatbot surpasses non-expert human. Although our current focus is on healthcare, the framework we present illustrates the feasibility and promise of applying low-cost, PPO-based RL to realistic, open-ended conversational domains—broadening the applicability of RL-based alignment methods.

## introduction patients' access to their electronic health record (ehr) notes, called opennotes @xcite , represents a personalized communication * equal contribution † co-corresponding authors channel. opennotes has been shown to enhance disease understanding @xcite , patient-provider communication @xcite , medication safety @xcite , self-managed care @xcite , and health outcomes @xcite . a patient's ability to comprehend ehrs is related to his/her level of health literacy, which is defined by the institute of medicine as "the degree to which individuals have the capacity to obtain, process, and understand basic information and services needed to make appropriate decisions regarding their health." (institute of medicine (us) committee on health literacy, 2004) thirty-six percent of american adults have limited health literacy @xcite and have shown difficulty in comprehending ehrs @xcite @xcite @xcite . limited health literacy has been identified as one of the major barriers to patient portal use @xcite @xcite . therefore, we developed noteaid-chatbot, a novel multi-component artificial intelligence (ai) system that helps patients comprehend their ehr notes. communication is the central process of education @xcite in learning as conversation @xcite , a patient does not read an ehr note, but gains information and knowledge through conversation with a chatbot that reads the note. one challenge for developing noteaid-chatbot is the lack of training data, making the development cost prohibitive. therefore, inspired by @xcite , we developed a novel training paradigm that is based on a multi-agent framework where we leverage state-of-the-art large lanthe noteaid-chatbot training pipeline. we first construct a two datasets: 1) gold dataset that consists real-world ehr notes and questionnaires annotated by experts, 2) silver dataset which is synthetic dataset (ehr notes, conversation records, questionnaires) generated using six medical content criteria and medical conversation strategy. we apply supervised fine-tuning on this conversation dataset to build a baseline chatbot model. leveraging the silver dataset, we align the chatbot via reinforcement learning (ppo), where the chatbot noteaid interacts with the patient agent (gpt-4o-mini) and receives verifiable reward signals based on the patient's performance on the comprehension test. this two-stage alignment enables emergent instructional behaviors in slms. (right: evaluation) we evaluate noteaid-chatbot with the gold comprehension dataset and conduct general evaluation and a turing test. above illustrates the generation evaluation based on the simulation with a virtual patient simulated from gold and silver dataset. we evaluated medical content generation and medical conversational strategies of our model. below illustrates the noteaid-chatbot in turing test. the noteaid-chatbot poses questions derived from a patient's discharge note to improve their understanding through interactive question answering. at the end of the session, the patient completes an exam assessing comprehension, which serves as the measurable learning outcome. ## related work large language models in healthcare: llms such as gpt-4 and chatgpt have shown promising performance in various healthcare tasks, particularly in answering medical questions with high accuracy and readability @xcite @xcite @xcite @xcite . domain-specific adaptations like google's med-palm @xcite have further improved safety and factual consistency by aligning llms with curated medical knowledge. for instance, med-palm significantly reduced hallucinated or harmful responses and was rated more helpful by clinical professionals @xcite . however, general-purpose llms still suffer from issues @xcite such as factual errors and lack of personalization, making them unsuitable for patient-facing education tasks without proper alignment @xcite . patient education chatbots and ehr comprehension tools: improving patient understanding of their ehrs has long been a goal of clinical nlp @xcite . early systems such as noteaid @xcite helped patients comprehend ehrs by linking medical jargon @xcite to lay definitions @xcite . more recently, paniniqa @xcite introduced an interactive qa system that automatically generates patient-specific questions from discharge notes and verifies answers to reinforce understanding. their method drew inspiration from dialogic reading @xcite and focused on guiding patients to uncover relationships between medical events via causal or correlational reasoning @xcite . our work shares paniniqa's goal of enhancing post-visit comprehension through interactive conversation. however, there are three key differences. first, paniniqa relies on structured event and relation annotations to control question generation; in contrast, noteaid-chatbot is trained end-to-end using synthetic conversations and reinforcement learning, requiring no expert-annotated supervision during training. second, while paniniqa emphasizes question generation and selection, we frame the problem as a full multi-turn education task, where the chatbot dynamically guides patients through dialogue and reinforces learning based on test outcomes. third, our reward signals are verifiable and outcomebased, derived from simulated comprehension tests, enabling scalable rl-based alignment. conversational learning and medical qa: recent research highlights the importance of learning through conversation, especially for patient education @xcite @xcite . dialogic reading @xcite @xcite shows that guided dialogue can significantly improve knowledge retention in learners. while it is not always feasible to engage human clinicians in repeated one-on-one education sessions, chatbot-based dialogue systems can offer scalable and personalized alternatives @xcite . unlike most clinical qa systems, which focus on fact retrieval or physicianstyle queries @xcite @xcite , our system prioritizes education-oriented dialogue. this includes conversational strategies such as simplification, clarification, and empathy, which are crucial for improving patient comprehension (king and hoppe, 2013). moreover, while prior work has used llms for general and medical question generation @xcite @xcite , these approaches rarely incorporate verifiable outcomes or educational objectives in their evaluation. ## conclusion we present a multi-agent framework for automating the development of domain-specific, lightweight chatbot for patient education using rl-based alignment. the proposed approach utilizes supervised fine-tuning with synthetically generated data, followed by alignment through simple ppo technique. our findings demonstrate that reinforcement learning significantly enhances the overall performance of the chatbot. also, in a turing test evaluation, noteaid-chatbot exhibited performance comparable to that of human educators.