Towards Conditioning Clinical Text Generation for User Control (2025)

Abstract: Deploying natural language generation systems in clinical settings remains challenging despite advances in Large Language Models (LLMs), which continue to exhibit hallucinations and factual inconsistencies, necessitating human oversight. This paper explores automated dataset augmentation using LLMs as human proxies to condition LLMs for clinician control without increasing cognitive workload. On the BioNLP ACL’24 Discharge Me! Shared Task, we achieve new state-of-the-art results with simpler methods than prior submissions through more efficient training, yielding a 9% relative improvement without augmented training and up to 34% with dataset augmentation. Preliminary human evaluation further supports the effectiveness of our approach, highlighting the potential of augmenting clinical text generation for control to enhance relevance, accuracy, and factual consistency.

## introduction large language models (llms) like openai's gpts @xcite ), google's palm @xcite and gemini @xcite , and lately meta's llama @xcite have shown remarkable versatility across a wide range of applications, including healthcare @xcite . in clinical environments, llms offer potential for automating tasks such as summarizing clinical notes, supporting diagnostic decisions, and streamlining patient communication @xcite @xcite @xcite @xcite . however, deploying ai in clinical settings remains a critical challenge due to the high cost of hallucinations, factual inconsistencies, and misinterpretations @xcite @xcite ; the generation resumes with user-verified content. @xcite . even minor inaccuracies in aigenerated clinical content can lead to severe consequences, such as misdiagnoses, incorrect treatments, or harmful patient outcomes. ethical considerations further complicate this process, calling for clinicians to hold accountability for medical decisions through rigorous oversight @xcite . at the same time, verifying ai-generated content introduces new cognitive burdens, potentially negating the intended efficiency gains of automation. as clinicians already face high cognitive workloads, addressing this paradox is essential to harness ai's potential in clinical settings without increasing risks or workloads. to strike this balance, ai systems must provide clinicians with control and transparency, ensuring outputs align with clinical contexts, communication styles, and guidelines. this paper explores whether augmenting traditional datasets to condition llms for controlled clinical text generation is a viable solution. specifically, we introduce a system that separates stylistic and content-related requirements, breaking down generation into distinct, manageable writing subtasks. this reduces the complexity of content creation and human verification through a separation of concerns, empowering users to impose authoring guidelines and dynamically guide the process while moving away from black-box models that limit clinician involvement. since traditional datasets do not inherently support such user control, we augment them with authoring guidelines and topic segmentation to condition models for style and content control. automated evaluation suggests that our approach significantly enhances relevance, accuracy, and factual consistency, highlighting the potential of such augmentations for clinical text generation. furthermore, we find that traditional instruction-tuning for clinical text generation can be significantly improved through optimized hyperparameter settings, without increasing the compute budget. our key contributions are: new state-of-the-art. we set a new state-of-theart on the bionlp acl'24 shared task 'discharge me!' challenge through efficient training, while being simpler and requiring less training compute. ## related work in recent years, research on llm-based clinical text generation has grown, demonstrating promise in generating discharge summaries @xcite @xcite , brief hospital courses @xcite @xcite , and radiology reports @xcite @xcite . notably, some studies report that physicians often prefer ai-generated clinical texts over manually written ones @xcite . de-spite this progress, most approaches treat clinical text generation as an end-to-end task, limiting user control and intervention. a recent example is the bionlp acl'24 shared task 'discharge me!' @xcite , targeting discharge summary generation. however, the complexity of clinical texts, which often require external sources of information and are subject to individual guidelines and writing styles, makes end-to-end generation less feasible in practice, and points to the need for more flexible generation allowing users to control specific aspects of the output, such as content and style. controlled text generation (ctg) addresses this by allowing users to steer outputs through specific conditions-such as tone, structure, or terminology-while maintaining fluency and relevance @xcite . prior work in this area has explored different control mechanisms, such as structure control @xcite , general style control @xcite , and personal style control @xcite . moreover, recent studies have explored using question-answer (qa) pairs as a blueprint to guide the text generation process. this approach has been shown to reduce hallucinations and improve the factual consistency of generated content @xcite . it is based on the question under discussion (qud) theory @xcite , which states that all utterances in a discourse @xcite ) serve to answer either implicit or explicit questions. building on these insights, we adapt the qud framework for clinical document generation by framing clinical documents as responses to implicit questions arising from their intended purpose. these questions are typically addressed in a structured manner, even when the document appears unstructured. using fine-grained topic segmentation, we aim to uncover this underlying structure by generating headings and quds, with corresponding text segments acting as their answers. this approach aligns topics with specific writing subtasks, simplifying the generation process while preserving the inherent structure of clinical documentation. ## conditioning clinical text generation for user control we explore two strategies to condition large language models (llms) for controlled clinical text generation: (a) topic-level structured generation and (b) authoring guidelines. however, implement-ing these strategies reveal limitations in traditional datasets, particularly in clinical text generation. ## results and discussion in this section, we analyze the impact of our data augmentation strategies on general instructiontuned llms, evaluate the efficiency of our stateof-the-art training approach, and assess how conditioning text generation for user control enhances clinical text generation. we further present human evaluation results, validating the effectiveness of our approach. ## conclusion and future work in this work, we explored strategies for conditioning llms to give clinicians control over both con-tent and style in clinical text generation. using the bionlp acl'24 shared task discharge me! as a case study, we demonstrated that augmenting datasets with authoring guidelines and topic segmentation significantly improves accuracy, relevance, and factual consistency. notably, our findings raise concerns about metrics exhibiting significant sensitivity to stylistic deviations, even when fact-based, warranting further research. our preliminary human evaluation suggests that llms can serve as proxies for expert annotations, enabling dataset augmentation at scale. by introducing a separation of content and style, we extended the traditional clinical text generation paradigm to facilitate the integration of clinical communication and authoring guidelines. since such guidelines are crafted once per task, they offer a low-cost enhancement to clinical text generation without adding cognitive burden. we also establish a new state-of-the-art for conventional clinical text generation on discharge me!, surpassing prior submissions while using fewer parameters and significantly lower computational costs. to support further research and real-world adoption, we disclose our methods, allowing hospitals and clinical institutions to adapt these augmentations to their own data and workflows. while preliminary human evaluation validates the effectiveness of our approach, a systematic study is needed to identify which specific components of authoring guidelines contribute most to downstream performance. future work should also focus on scaling human evaluation, assessing generalization across diverse clinical datasets, and refining llm conditioning techniques to improve adaptability to real-world medical documentation workflows. additionally, user inspired by @xcite , who proposed to replace linear decay with a cosine cyclic decay to increase the duration of higher learning rates, we adopt a learning rate scheduler with a stable phase of 1,000 steps with learning rate α 1 = 1e -4 , a decay phase of 1,800 steps corresponding to 0.25 cosine cycles to reduce the learning rate to α 2 = 5e -6 , and another smoothing decay phase of 200 steps corresponding to the remaining 0.25 cosine cycles to reduce the learning rate to α 3 = 1e -6 . this increases the duration of high learning rates even further.