R ecord T win: Towards Creating Safe Synthetic Clinical Corpora (2025)

Abstract: The scarcity of publicly available clinical corpora hinders developing and applying NLP tools in clinical research. While existing work tackles this issue by utilizing generative models to create high-quality synthetic corpora, their methods require learning from the original in-hospital clinical documents, turning them unfeasible in practice. To address this problem, we introduce RecordTwin, a novel synthetic corpus creation method designed to generate synthetic documents from anonymized clinical entities. In this method, we first extract and anonymize entities from in-hospital documents to ensure the information contained in the synthetic corpus is restricted. Then, we use a large language model to fill the context between anonymized entities. To do so, we use a small, privacy-preserving subset of the original documents to mimic their formatting and writing style. This approach only requires anonymized entities and a small subset of original documents in the generation process, making it more feasible in practice. To evaluate the synthetic corpus created with our method, we conduct a proof-of-concept study using a publicly available clinical database. Our results demonstrate that the synthetic corpus has a utility comparable to the original data and a safety advantage over baselines, highlighting the potential of RecordTwin for privacy-preserving synthetic corpus creation.

## introduction in-hospital clinical documents, such as discharge summaries, contain sensitive patient information that must be anonymized before these corpora can be shared outside the hospital. the scarcity of publicly available clinical corpora, due to the challenges of this anonymization process, significantly hampers the development and application of 1 our code is available at: https://github.com/ seiji-shimizu/recordtwin natural language processing (nlp) tools in clinical research @xcite . conventionally, research on text anonymization focuses on de-identification, using named entity recognition (ner) to detect and then remove, replace, or generalize personally identifiable information @xcite . however, ner models cannot guarantee perfect precision and recall in practice, especially on unseen data, necessitating manual review to ensure anonymity. recent studies have turned to synthetic corpus generation to address this limitation @xcite @xcite . in this approach, generative language models are trained on the original corpus to produce new, natural sounding text. the concept of plausible deniability is central to the privacy guarantees of this method: it is difficult for users to determine whether the information contained in the document comes from the original data or is fabricated. while these synthetic corpora have shown high utility, they still carry the risk of privacy breaches due to unintended memorization. unintended memorization refers to the issue where the generative model memorizes sensitive information from the training data @xcite . this is particularly problematic when rare expressions are involved, as there may be few such cases in the original corpus, leading to the worst-case scenario where documents are exactly generated as they are. even privacy-preserving techniques like differential privacy-based text generation @xcite @xcite are not immune to these risks, as sensitive information can still influence the training process. the problem lies in learning from the documents intended for anonymization. to overcome this challenge, we propose recordtwin, a novel method for creating shareable synthetic clinical corpora by combining two key strategies: entity anonymization: we extract patient information as entities from documents and apply k-anonymization to ensure that the same set of entities appears in at least k records. this mitigates the risk of re-identification and restricts the information contained in the synthetic corpus. context generation: instead of learning from inhospital documents, we generate synthetic clinical documents using a general-domain large language model (llm). by leveraging a small subset of privacy-preserving original documents as examples, we fill the context, including writing style and formatting, between entities, preventing the generation of any sensitive information beyond the entity sequences. fig. 1 illustrates the comparison of generation with (a) learning from the original document, (b) few-shot learning with an un-anonymized entity set, and (c) few-shot learning prompted with an anonymized entity set. in the original text, contextual details like "81-year-old female" and "metastases to the liver" could reveal the patient's identity when combined with the diagnosis of "metastic melanoma". (a) has a risk of generating the combination of all those details via unintended memorization. (b) can mitigate the risk of generating contextual information such as "81-year-old female" by prompting the generative model only with ex-tracted entities. however, the combination of disease names "metastic melanoma" and "metastases to the liver" can lead to the identification of a specific patient when it is rare in the original corpus. on the other hand, (c) the generated document with recordtwin does not contain contextual information (e.g., 81-year-old female) or a disease name combination (e.g., metastases to the liver) that is revealing of the patient's identity. although our pipeline is safer than existing synthetic corpus creation methods by design, challenges remain in maintaining the utility of the generated documents. the synthetic documents, while anonymized, may be degraded from their original counterparts in terms of utility, which could impact downstream tasks like language model pre-training or clinical document classification. therefore, evaluating the utility of these synthetic documents in real-world applications is critical. in this paper, we present a proof-of-concept study to evaluate the utility of our synthetic corpus using mimic-iii @xcite . specifically, we assess its utility across multiple nlp tasks, including named entity recognition and clinical document classification, demonstrating that the performance of models trained on the synthetic corpus is comparable to those trained on original data. ## proposed method: recordtwin this study proposes recordtwin, a new method for synthetic clinical corpus creation aiming to mitigate the risk of revealing patient's personal information. the overview of recordtwin is presented in fig. 2 . we first extract clinical entities from the original documents and apply kanonymization to the set of extracted entities. this ensures that at least k records containing an identical set of entities are included in the synthetic corpus (sect. 2.1). next, anonymized entities and an example document are given to llm to generate synthetic documents (sect. 2.2). the example can be sampled from a small subset of simulated or manually anonymized original documents. in this way, we can simulate the original document in terms of writing style and formatting without learning from the original document itself. ## experiment to demonstrate the effectiveness of recordtwin, we conducted a proof-of-concept study, creating a synthetic corpus from discharge summaries in mimic-iii @xcite with recordtwin (sect. 3.1). we evaluate the utility of the synthetic corpus in pre-training for clinical ner and fine-tuning for document classification ( sect. 3.2 ). the evaluation in pre-training aims to assess the quality of generated context in fewshot learning, while the evaluation in fine-tuning aims to assess whether recordtwin preserves the patient statistics in the original corpus during the k-anonymization. ## related work we summarize the related work of this paper in two groups: (1) a method that removes personal information through ner, and (2) a method that generates synthetic documents with generative models. ## conclusion in this paper, we present recordtwin, a novel method to create a synthetic clinical corpus combining entity anonymization and context generation through few-shot learning. recordtwin is safe for two reasons: (1) it anonymizes the patient statistics using k-anonymization (2) it does not learn from the in-hospital documents intended for anonymization. we conduct a proof-of-concept experiment and evaluate the recordtwin from utility perspectives. the results suggest that the generated corpus has high utility in downstream tasks. we believe this work presents an innovative solution for corpus scarcity in the clinical domain and lays the foundation for creating publicly available synthetic clinical corpora in real-world settings.