Expertly Informed, Generatively Summarized: A Hybrid RAG Approach to Informed Consent Summarization with Auxiliary Expert Knowledge (2025)

Abstract: The utility of retrieval augmented generation (RAG) systems is actively being explored across a wide range of domains. Reliable generative output is increasingly useful in fields where routine tasks can be streamlined and potentially improved by integrating domain-specific data in addition to individual expert knowledge, such as medical care. To that end, we present a hybrid RAG and GraphRAG user interface system to summarize the key information (KI) section in IRB informed consent documents. KI summaries are a unique task, as generative summarization helps the end user (clinical trial expert) but can pose a risk to the affected user (potential study participants) if inaccurately constructed. Thus, the KI summarization task requires reliable, structured output with input from an expert knowledge source outside of the informed consent document. Reviewed by IRB domain experts and clinical trial PIs, our summarization application produces accurate (70% to 100% varied by accuracy type) and useful summaries (63% of PIs stating summaries were as good as or better than their accepted summaries).

## introduction applied in the medical field, retrieval augmented generation (rag) systems have shown promise in streamlining routine tasks, providing structure for standard medical procedures, and ensuring current information is integrated into decision making @xcite @xcite . however, there is concern surrounding the reliability and trustworthiness of generative output in high-risk, real-world implementations in which incorrect information can lead to severe personal harm. researchers and practitioners have focused on studying and identifying reliable generative artificial intelligence (genai) use cases to optimize a routine procedure for the end user (e.g., medical professional) and minimize potential harm for the affected individual (e.g., patient receiving care). as technical advances in genai, natural language generation, and information retrieval continue, applications of these systems and models become viable tools for real-world implementation. building on the rag pipeline (a user query coupled with an llm and knowledge base), the development of graphrag created a retriever system that uses a knowledge graph to provide context and entity relationships derived from the selected knowledge base @xcite . thus, graphrag systems can be used to extract key information with known feature relationships, providing more transparent outputs linked directly to data sources in the knowledge base. in this study, we design a hybrid rag and graphrag system to optimize summaries of key information in irb informed consent documents. informed consent documents (icds) are provided to individuals who are considering participating in a medical study for new treatment that may affect their health and path to recovery. thus, icds are required and reviewed to ensure that participants are well-informed about their rights as well as the nature, risks, and benefits of the study. the key information portion of the icd serves as a concise summary highlighting the most critical aspects of the study. specifically, it helps potential participants understand the key information to make a well-informed decision about agreeing to participate. it is important that these summaries balance providing comprehensive details with clarity and minimizing technical jargon. we designed and evaluated a pilot key information summary application on four key measurements: (1) factual accuracy, (2) standard of care vs. research differentiation, (3) information weighting, and (4) style and structure. evaluations were performed by three irb subject matter experts to iteratively improve our model design and output over 11 cycles. additionally, eight senior principal investigators assessed the machine-generated summaries for further improvement input. we found that the key challenge in our pilot summarizer was differentiating between standard of care risks versus research risks (i.e., what risks were associated explicitly with participating in the study outside of standard treatment?). to address the risk differentiation challenge we implement temporary auxiliary graphrag (tagrag) for expert question and answering. using search terms associated with the proposed study's disease focus and standard of care medical treatment, we query open-source articles via pubmed central's api and select relevant research publications as input to a graphrag system. the graphrag is instantiated for each summary instance and removed after each summary is generated. our tagrag component enables expert information to support current and relevant standard of care risks associated with the corresponding study's disease focus without maintaining a large knowledge graph of medical research. ## genai in medical research prior research has investigated the utility of genai in the medical field for a wide range of tasks using text and image data. @xcite provide a survey on how models like chatgpt and dall-e can be implemented in medical tasks such personalized patient treatment, healthcare operations and research, and clinical trial optimization. the authors highlight four directions for future research: our work covers each of the four areas highlighted by @xcite , as we designed, developed, and deployed a generative summarization tool that incorporates expert knowledge (enhancing decision making and bridging the knowledge gap), decreases administrative work for clinical experts (streamlining administrative operations), supports effective communication between clinical experts and non-expert study participants (enhanced patient and worker interactions), and provides a user interface for summarizing a document based on the domain-specific implementation (customized/personalized suggestions and a platform for information exchange). ## informed consent key information summary structure an institutional review board (irb) works to regulate human-subject research, ensuring ethical procedures and minimal risk to participants. research institutions are required to receive irb approval prior to engaging with any potential participants or beginning any human-subject experiments. producing an informed consent document is a requirement of the irb approval process and these documents follow a regulated structure and format to maintain consistency across research studies-they are designed to consistently protect participants. a critical component (and requirement) of an informed consent document is the key information section, which explains the details of the study in clear language (with minimal technical jargon) and identifies the potential risks involved as a participant that are distinct from the risks involved from standard of care treatment. specifically, key information summaries are designed to support potential participants in deciding whether or not they would like to be a part of the research study. according to u.s. federal regulations, key information summaries should include the following five elements: (1) a statement acknowledging that the project is research and participation is completely voluntary; with their structured output and routine requirement for researchers, writing key information summaries is a suitable task for experimentation using generative ai. additionally, key information summaries require knowledge of the informed consent document and context-relevant medical expertise for the proposed study.