Critic–Adviser–Reviser Cyclic Refinement: Towards High-Quality EMR Corpus Generation with LLMs
Authors: Chen Ning
Xien Liu
Chenwei Yan
Xiao Zhang
Xinxin You
Yuxuan Zhou
Xiangling Fu
Ji Wu
Venue: ICLR 2026 Poster (2026)
Keywords: Large Language Model
Synthetic Data Generation
Electronic Medical Record
Primary area: foundation or frontier models, including LLMs

Abstract: Electronic medical records (EMRs) are vital for healthcare research, but their use is limited by privacy concerns. Synthetic EMR generation offers a promising alternative, yet most existing methods merely imitate real records without adhering to rigorous clinical quality principles. To address this, we introduce LLM-CARe, a stage-wise cyclic refinement framework that progressively improves EMR quality through three stages, each targeting a specific granularity: corpus, section and document. At each stage, a Critic, an Adviser, and a Reviser collaborate iteratively to evaluate, provide feedback, and refine the drafts. This structured, multi-stage process produces records that better satisfy clinical quality standards. Experiments show that LLM-CARe significantly enhances EMR quality across all levels compared to strong baselines and yields improved performance on real-world clinical tasks such as diagnosis prediction. Unlike prior work, our method requires no real EMR text for training or prompting, demonstrating the effectiveness of stage-wise, cyclic refinement for generating high-quality, privacy-preserving EMR datasets.

Published as a conference paper at ICLR 2026
CRITCC-ADIIER-REVIIER CYCLIC REFINEMENT: TOWARDS HIGH-QUALITY EMR CORPUS GENERA- TION WITH LLMS
Chen Ning', Xien Liu"1, Chenwei Yan', Xiao Zhang Xinxin You', Yuxuan Zhou', Xiangling Fu', Ji Wu
Department of Electronic Engineering, Tsinghua University
"School of Artificial Intelligence and Data Science,
University of International Business and Economics
'ByteDance
"School of Computer Science, Beijing University of Posts and Telecommunications
"College of AI, Tsinghua University
6 "Beijing National Research Center for Information Science and Technology
ABSTRACT
Electronic medical records (EMRs) are vital for healthcare research, but their use is limited by privacy concerns. Synthetic EMR generation offers a promising alter- native, yet most existing methods merely imitate real records without adhering to rigorous clinical quality principles. To address this, we introduce LLM-CARe, a stage-wise cyclic refinement framework that progressively improves EMR quality through three stages, each targeting specific granularity: corpus, section and doc- ument. At each stage, a Critic, an Adviser, and Reviser collaborate iteratively to evaluate, provide feedback, and refine the drafts. This structured, multi-stage pro- cess produces records that better satisfy clinical quality standards. Experiments show that LLM-CARe significantly enhances EMR quality across all levels com- pared to strong baselines and yields improved performance on real-world clinical tasks such as diagnosis prediction. Unlike prior work, our method requires no real EMR text for training prompting, demonstrating the effectiveness of wise, cyclic refinement for generating high-quality, ewwacy-preserving EMR datasets.
INTRODUCTION
Electronic Medical Records (EMRs) are valuable resource for healthcare research (Ma et al. 2017 Shang et al., 2019 Shen et al. 2025), offering large-scale, clinically grounded insights that reflect real-world medical practice. However, the sensitive nature of patient information poses significant privacy challenges, which severely limit the open sharing and use of real EMRs (Tyengar et al.] 2018 Shah & Khan 2020 Tertulino et al., 2024). To mitigate these concerns, rescarchers have explored synthetic EMR generation methods that am to preserve data utility while protecting patient confidentiality (Yan et al. 2022 Murtaza al. 2023 Yuan et al.) 2024 Lin et al.
2025).
Existing EMR synthesis approaches primarily focus on mimicking real records (Lee 2018 Baowaly
et al. 2018: Yoon et al.) (023). without explicit ensuring clinical soundness. This imitation imitation-basedd based strategy is risky: real EMRs may contain errors (Aerts et al. [202]] Mohamed et al. 2023), which can be inadvertently inherited by synthetic data (Figure [ a)). In practice, EMRs are professional medical documents whose reliability depends satisfying key requirements such as completeness,
consistency, and distribution alignment. Synthetic records that fail to meet such requirements may be less useful-an or even misleading for downstream clinical or research applications.
Recent advances large language models (LLMs) make them a promising tool for EMR generation, due to their text generation ability and rich internal knowledge. However, as shown in Figure a[a,, our preliminary analysis reveals that LLM outputs often exhibit biased distributions such as unre- alistic gender patterns-an tend to produce only the most typical presentations of a disease, lacking
"Corresoonding author.

---

Published as a conference paper at ICLR 2026
Existing Synthetic EMR Generation Methods
method
Task-Speciii Model
generate
Synthelic EMRs
evailaate
Contradictory narrativs
(e.g. GAN, LSTM)
Factual crrors
Issue List Distnbutional
Real-Worid
method
Large Language Model
generate
Synthelic
evailuate
EMRs
EMRs
coverage
(b) Ours: High-Quality EMR Synthesis via LLM-CARe
Quality Checklist
Reference
guide
Distribution
/ Medical correctness
/ Content completenecs
Guideline
Clinical
Advice
generate
Synthetic EMRs
evallato
consislency
Section
Demogrephic typicallty
Corsistancy
/ Knowleagg coverage
Constraint
Document
Naviarr
Figure I: (a) Traditional EMR generation that mimic real EMRs without considering quality often leads to suboptimal outputs. (b) Our proposed LLM-CARe incorporates cyclic refinement based on quality principles, synthesizing high-quality EMRs.
coverage of the diverse and less common clinical scenarios seen in practice. These challenges high- light the need for more structured approaches to harness L.MSs effectively for EMR synthesis.
To bridge this gap, propose LLM-based Critic-Aviier-RRevve
Cyelic Refinement (LLM- CeRe), framework that enhances synthetic EMR quality through progressive refinement across three levels. As illustrated in Figure bb), LLM-CARe incorporates clinical quality principles into the generation process, producing records that align with standards of medical documentation. These requirements are organized into concrete principles of corpus distributional aligmment, section com- pleteness, and document consistency, forming the basis for refinement. Guided by them, LLM- targeting
CARe proceeds through three stages of refinement-corpoee a distinct aspect of EMR quality. Within every stage, a Critic, an Adviser, and a Reviser collaborate in a cyclic loop: the Critic evaluates the drafts, the Adviser provides targeted feedback, and the Reviser updates the records. While the interaction pattern is shared, the role of each agent adapts to the stage: corpus stage aligns the dataset with realistic distributions, section stage enforces sec- tion completeness, and document stage ensures logical consistency within record. This structured process enables systematic enhancement of EMRs from local detail to global corpus characteristics.
section, and document-
To validate the effectiveness of our approach, conduct experiments a reeldwoord EMR dataset containing 192k records across 302 diseases. We assess the intrinsic quality of generated records us- ing strong L.MM judge, complemented by statistical comparisons to real EMRs. Additionally, we evaluate downstream utility by training models on synthetic EMRs and testing on real records across three representative clinical tasks: diagnosis prediction, examination recommendation, and treatment recommendation. Results show that LLM-CARe consistently outperforms baselines in both record quality and task performance. Notably, our method requires no access to real EMR text for training or prompting during generation, fully preserving patient privacy while producing clini- cally meaningful and practically useful EMRs. Our main contributions are summarized as follows:
We propose L.LM-CAR Re, stage-wise multi-agent framework for high-quality EMR syn- thesis that employs cyclic refinement based on clinical quality principles.
LLM-CARe consistently improves EMR quality compared to baselines across multiple levels. Further analysis shows that all three agents play essential and complementary roles.
Without training prompting real EMR text, synthetic data yields superior down- stream task performance compared to baselines, ensuring both utility and privacy.
2 RELATED WORK
There has been growing interest in synthesizing EMRs to address privacy concerns and facilitate secure data sharing. We categorize existing methods into three main paradigms:
GAN-based EMR Generation. Generative adversarial networks (GANs) have been extensively explored for EMR synthesis. Some works generate EMRs from random noise vectors (Choi al.
2

---

Published as a conference paper at ICLR 2026
2017 Baowaly et al.) 2018 Chin-Cheong et al.
2019 Yoon et al. 2023 while other methods
incorporate structured conditions such as diagnosis codes to guide generation process (Rashidian
et al. 2020 Zhang et al., 2020 Guan et al. 2021 Li al. 2023). Although effective at modeling
data distributions, these methods typically ignore the clinical quality of the generated records.
Auto-regressive EMR Generation. Another line of research leverages auto-regressive models to generate EMRs. Recurrent neural networks (RNNs) have been used to model sequential EMR data (Leel 2018; Melamud & Shivade 2019 Mosquera et al.) 2023 Ganguli et al. 2023). and more recently, transformer-based architectures have been introduced to capture long-range dependencies within records (Wang al., 2019] Amii-Nejad et al.) 2020 Theodorou al., 2023) Karami al.j
2024). While these models excel at learning temporal and structural patterns, they generally treat
EMRs as sequences of tokens without mechanisms to ensure clinically meaningful coherence.
LLM-based EMR Generation. With the emergence of large language models (LLMs), recent studies have explored prompting LLMS to synthesize EMRs, either by providing brief clinical de- scriptions by asking the model to emulate real patient records (Litake et al., 20244; Kumichev jet al., 2024 AAbdel-Khale et al.) 2024 Kweon et al. 2024 Lin et al. 2025). While LLMs exhibit strong capabilities, direct generation often results in diverge from realistic corpus-level distributions.
Recent work has also explored diffusion models for EMR synthesis (Yuan al. 2023). but existing methods primarily target numerical representations rather than free-text EMRs, making them less applicable to the textual EMR generation setting studied in this work.
METHOD
In this section, we present LLM-CARe, stage- wise cyclic refinement framework for enhancing the quality of synthetic EMRs. Unlike ordinary free- form text, EMR must satisfy quality requirements to be clinically meaningful. We focus on five principles across three levels: demographic typicality (plausi- ble population patterns), knowledge coverage (suf- ficient breadth of clinical conditions), content com- pleteness (each section includes its essential infor- mation), medical correctness (information is clin- ically valid), and context consistency (sections do not contradict one another). Each principle is further divided into concrete criteria that can be explicitly checked and refined. Figure 2 shows representa- tive examples, with full details in Appendix
Level
Principle
Example of Criteria
Deroggapphic
The gender ralio synthelic data
Typicelity
should algn with reel populetions.
Corpus
Kroeledge Coverage
The syntheli delasel should cover
comprehenssvve symplome
Section
Completenees
Conient
The chief complaint should state the reeson for admissson
Medical
The diaghosie should be voli given
Conrecneess
the palien's gender
Dosument
Context
Symploms in chief complaint and history of present illnes should align.
Consistency
Quality principles across three lev- els with representative criteria.
Figure 2:
To address these requirements, LLM-CARe refines EMRs in three successive stages-corpus, sec- tion, and document. As illustrated in Figure 3 each stage involves the collaboration of three agents in cyclic loop: the Critic, who evaluates drafts against stage-specific objectives; the Adviser, who pinpoints areas for improvement and suggests strategies; and the Reviser, who incorporates feedback to update the drafts. This iterative process progressively improves EMRs from global distributional alignment, to field-level completeness, and finally record-level coherence.
Corpus Level CARe
Violaton
High Quality EMR Corpus
Generator
Alig
mm
Level Quality
Section CARo
Critkc
Cyclic Refinement
Advisor
Krassagg Covaragee
Demographhe: Typianlly
Initall EMR Drafts
Incamppate Suction
Complete
Level Quality Content Comaktennss
Document
fri Hit Lin Lim
b
Actonable Buggeatio
Document Level Quality
Medical Correctness
Inconsisiont
Reviter
Censistent
Context Consitenny
Figure 3: Overview of our proposed LLM-CARe framework for synthesizing high-quality EMRs.
3

---

Published as a conference paper at ICLR 2026
3.1 INITIAL DRAFT GENERATION
The generation process begins with a generator agent Mgenerator, which produces initial EMR drafts based on an input prompt x. This prompt specifies key information such the target primary diag- nosis and required EMR fields (e.g.. chief complaint, history of present illness). For each prompt, the generator samples multiple drafts form a starting draft pool:
D (E E.
E Magnnemtor(()
where D(O) denotes the initial draft set and each E is an EMR instance. These drafts may exhibit omissions, inconsistencies, clinically implausible details, but they provide the foundation for subsequent stage-wise refinement.
At the dataset scale, high-quality synthetic EMRs must preserve realistic and representative distribu- tions. We focus on two aspects: Demographic Typicality, ensuring variables such as age and gen- der reflect real-world patient populations (sce Figure IT for examples), and Knowledge Coverage, ensuring the corpus contains both common and rare clinical conditions (with detailed dimensions listed in Table 9 Maintaining these corpus-level properties is essential, as deviations may introduce demographic bias or insufficient coverage clinically important but less frequent scenarios. These goals addressed through corpus-level agent interactions, where the critic, adviser, and reviser collaborate to align the overall distribution.
Corpus-level Critic. At the corpus stage, the critic focuses dataset-wide properties, capturing how well synthetic EMRs aligns with target distributions. For each attribute Coorpus (c.g., age, gender, or a knowledge dimension), it measures the deviation of the current corpus D(() from the refer- ence distribution Ta derived from aggregate statistics on the training set, including age and gender distributions and the frequencies of clinical concepts of interest (e.g.. symptoms, medications):
corpus
(D(t), Tai Coorpus,
corpus,
Corpus-level Adviser. The adviser interprets the critic's feedback to guide modifications at the dataset level. Based on the deviations, it identifies subset of records s(() would most effectively reduce distributional mismatch, and generates actionable feedback (r()
k c DD(t) whose adjustment
corpus,
((t)
Fl0 corpus,
corpus adviser
(Dit), Coorpus, corpus,k
((()
Corpus-level Reviser. The reviser applies the adviser's instructions to the selected subset s(t) mod- ifying or enriching records to better match the reference distribution:
s((+1)
corpus
(s(() .F(())
corppus,
(4)
3.2 SECTION LEVEL CARE
At the section scale, high-quality EMRs must ensure Content Completeness: each field should contain the essential clinical elements expected for its type. Ensuring section-level completeness crucial, as omissions in key fields can result in records that misrepresent the patient's condition and compromise clinical validity. To operationalize this, we define set of section-specifie criteria derived from clinical guidelines (see Table () and apply cyclic refinement with critic, adviser, and reviser agents to supplement missing information.
Section-level Critic. The critic operates for each single section. For section i.m. in record F:( and criterion Ce derived from clinical guidelines, it determines whether the criterion is met:
(1)
Mucc eritic
(01.m (scm:Cace.kk)
scc.s.k (0,
s.c.6.k
Section-level Adviser. For unmet criteria (6(t) (6.... = 0), the adviser examines the section and designs specific instructions to indicate exactly which clinical elements should be added clarified:
p(t)
sec,i.k =
Mareisee
adviser mi, Csec,,
(6)
4

---

Published as a conference paper at ICLR 2026
Section-level Reviser. Using the adviser's guidance, the reviser updates the section by incorporating the recommended elements while preserving existing content and coherence:
(++1)
M
soc reviser
Filt) ki,m: sec,..kk
Through this cycle, sections are progressively completed and made sufficient for their clinical role.
3.3 DOCUMENT-LEVEL CARE
At the document scale, EMRs must ensure both Medical Correctness -that clinical statements valid given the diagnosis-ant Context Consistency- that information across sections does not conflict. Without enforcing these criteria, even small cross-section inconsistencies or incorrect clinical relations undermine the internal logic of the record to the point where the overall EMR becomes clinically unreliable. To make these requirements concrete, we define detailed criteria for both aspects (see Tables and and Table To enforce them, refine EMRs through document-level agent interactions, where the focus is on coherence across multiple sections.
Document-level Critic. The critic evaluates each record whole, checking constraints across sections for logical and clinical plausibility. For a consistency criterion Cdoc, it outputs a judgment:
(0) d,c
-Mann
(E(2) Cdoc.k)
(t) doc E {00
Document-level Adviser. When inconsistencies flagged dec 0), the adviser generates targeted feedback, often suggesting edits to the less influential section to restore harmony:
(t)
M aulviser
(E," Cdoc,k) Caoo,)))
(9)
doc,s.k
Document-level Reviser, Finally, the reviser integrates this feedback to harmonize the conflicting sections and yield an updated document:
E ((+11)
dac (E!!),
((() (
d,c
(10)
Through this process, records are refined into coherent, consistent, and clinically valid narratives.
3.4 STAGE-WISE ORDERING
At each stage, the critic, adviser. and reviser interact in cycles to refine the drafts according to stage-specific principles. Once the drafts have been improved at the current granularity, they are passed to the next stage, where the agent interaction continues under different focus. The staged order is intentional: modifications at one level can influence others, we proceed from the most flexible to the most stringent stage. Corpus-level refinement is relatively soft, aiming to align distributions without requiring exact matches, and is therefore performed first. While document- level refinement enforces strict logical consistency across sections, where errors could introduce serious contradictions, thus is performed last. By progressing in this order, each stage builds on the previous one while minimizing unintended conflicts. Through this staged refinement, the synthetic EMRs achieve high quality across corpus, section, and document levels.
EXPERIMENTAL SETUP
Dataset To validate the effectiveness of our method, we conduct experiments on a real-world EMR dataset comprising 192k records across 302 diseases. The dataset is carefully de-identified by removing all sensitive patient information. Unlike many prior studies that focus on synthesizing a single field in isolation (e.g., chief complaint), we consider multiple fields that together capture the clinical trajectory from admission to discharge to provide a comprehensive view of each clinical episode. To ensure consistent disease distribution across subsets, perform an 8:2 stratified split based on disease categories, maintaining proportional representation in both the training and test sets. Further details provided in Appendix
5

---

Published as a conference paper at ICLR 2026
Baselines We compare method against two categories of baselines. (1) Non-LLM generative models. LSTM (Lee 2018) and mtGAN (Guan et al. (021) share the same generation setup- both take the disease label as input and are trained from scratch on the training split to produce full EMRs, differing only in model architecture (LSTTM vs. GAN). (2) LLM-based generators. MedSyn (Ku- michev et al. 2024) generates EMRs by prompting training set along with realted symptoms; LLM Direct generates EMRs from instructions without quality control; and Self-Refine (Madaan et al. 2023) performs iterative self-critique that evaluates all quality requirements jointly in each refinement round. None of these LLM based baselines in- volve fine-tuning. All approaches are conditioned on disease labels, and generate the same number of EMRs with same disease distribution as the test set. Details are provided in Appendix C
LLM with real EMR examples from the
Evaluation Settings We employ two types of metrics to evaluate synthetic EMRs:
EMR quality is assessed based on the five principles introduced above. For medical correctness, content completeness, and context consistency, we use an LLM-based judge that evaluates each EMR against our criterion set. For demographic typicality and knowledge coverage, compute statistical similarity between synthetic and real data within each disease- compaainn age and gen- der distributions, and measuring the proportion of clinically relevant entities (symptoms, exams, treatments) covered in synthetic EMRs.
Downstream task performance provides an practical way to evaluate the utility of synthetic data. Following prior work, we train task-specific models on synthetic EMRs and evaluate their performance on real-world test data. This setup reflects a common use case of synthetic data in low- resource scenarios. We consider three rep- resentative tasks of high clinical relevance: diagnosis prediction, examination recom- mendation, and treatment recommenda- tion. These tasks collectively span key as- pects of medical decision-making. Each task is framed a multiple-choice ques- tion, where the model predicts answers based on the chief complaint and history of present illness. Illustrative examples are shown in Figure 4 More details are provided in Appendix D
EMR
Downstream Tasks
Gender Maii Age 11
Otagnasis Predittion
Primary Blagnosis: Carabral interction
the nost losly diegnoals from hillwing optins
Cheel Comp east Saxicnn Haaory of Present lisss The Castitbes ciegresuse
Chiet Complaint
weskress with stamed speech for hours
Examination Recommendation
Bosed the potennss complain Nittry
appropriate
pstentts
Haattoy of Present Ulrax:
Comp aint Sudiion
H sisoory of Present lines The
paraen dioeloped hour
Candidato eonnination:
Apcomriaal Iracound
Treatment Recommendation
Hospital Course: Chienal CT rovalied...
Beaad the patsent' clel compann and hadoay prasant Breens Cleal Comprenn: Sudcim Hudoryy of Preset lisse The
Discharge Instruction. mg
Caesitant bositeente
trackn B.Aapasr Toreiledarry Grospraad
Figure 4: Construction of multiple-choice questions for three downstream tasks from an EMR.
(Yang et al., 2025) as the backbone model for all LLM- methods. For EMR quality evaluation, we adopt Qwen2.5-32B 3---2B-Inntttc,c as larger models tend to provide more reliable judgments. For downstream tasks, we fine-tune Qwen2.5- Inst-Inst on synthetic EMRs and evaluate performance on real test data. Detailed experimental settings are provided in Appendix E
Implementation Details We use Qwen2.5-7BBInnse
EXPERIMENTAL RESULTS AND DISCUSSION
5.1 COMPARISON OF EMR QUALITY
Table summarizes performance across the five quality principles. LLM-CARe consistently out- performs all basclines on every metric, improving both individual-record quality and corpus-level characteristics. Among the baselines, LLM-based baselines generally exceed traditional models on section- and document-level criteria, yet without structured refinement they can underperform on corpus-level dimensions. Self-Refine provides only minor gains over LLM Direct and remains far below our method, indicating that generic all-in-one revision cannot satisfy EMRs' multi-level qual- ity requirements. This shows that simply employing LLMs is not sufficient to guarantee compre- hensive EMR quality. In contrast, LLM-CARe achieves strong and balanced improvements across all levels through principle- guided, stage- wise refinement. To confirm that these findings are not tied to a specific judge, we also evaluate using additional LLMs (Appendix F). Fine- Fine-grained subgroup analyses across demographic and disease- partitions are provided in Figure 12

---

Published as a conference paper at ICLR 2026
Table 1: Quality (%) of generated EMRs across principles, where higher values indicate better performance. (*) denotes standard deviation calculated from 3 runs with different random seeds.
Rely on EMR
Section Level
Document Level
Corpus Level
Type
Method
Content Completeness
Medical Correctness
Context Consistency
Demographic Typicality
Knowledge Coverage
Text
Non-
LSTM
70.8(1.)
65.0ko.4)
21.7(2.3)
93.3(0.6)
70.4<0.4)
LLM
mIGAN
55.8(2.9)
51.8(6.2)
21.4(4.)
93.6(1.4)
76.3(377
McdSyn
84.8(0.3)
95.3(0.8)
91.9(1.1)
(0.1
84.5(588)
L.M--
LIM Direct
(7.1
90.7(0.2)
87.9(0.1)
77.7(0.1)
73.9(0.2)
Based
Self-Refine
78.3(..2)
90.9(0.0)
88.5(0.1)
77.7(0.4)
78.0(2.4)
LLM-CARe(ours)
91.2(0.4)
98.6(0.0)
93.8(0.1)
96.8(1.4)
94.1(0.1)
Contant Comploemes
06
Scotion Level
50
HIT Suraten
General
HC: Evartnation
Trudtert
Mediekknn
Medical Cormeznees
Content Comeadency
0.6
Doc Levvll
0.0
0.0
Cc Sergen
Ceath Syrreeer
Betti Evenenation
De Meboat 20
DG-ftt Syrsper
Krone Coverage
Demogpaphiic Typically
Corpua Level
Symplo
Surgety
0.
MedSyn
Direct Bett-Relic (ours)
Figure 5: Detailed EMR quality evaluation across 3 levels. Abbreviations: CC-Chief Complaint, HPI-History of Present Illness, HC-Hospital Course, DI-Discharge Instructions, Dx- Dx-Diagnosis.
Figure presents fine-grained breakdown of EMR quality across several representative and clin- ically important criteria; complete results are provided in Appendix H Our method achieves the best or competitive performance on the majority of criteria, demonstrating robustness across diverse quality dimensions. Notably. MedSyn underperforms even the LLM Direct on some criteria, such as the completeness of the hospital course. This suggests that real EMRs-user by MedSyn as in-context exemplars-mal contain omissions that propagate into the generated records. These findings further highlight the limitations of purely imitative approaches and emphasize the importance of explicitly modeling and enforcing quality standards during generation.
5.2 COMPARISON OF DOWNSTREAM TASK PERFORMANCE
Table 2: Accuracy (%) of three downstream tasks, where miero and macro averaged across diseases. (*) denotes standard deviation calculated from 3 runs with different random seeds.
Rely on EMR Text
Diagmosis Prediction
Examination
Treatment
Type
Method
Recommendation
Recommendation
Micro
Macro
Micro
Macro
Micro
Macro
Non-
LSTM
>
74.0r2.0)
73. (2.00) 75.7(0.3)
76.4(0.2)
(6.7.06) 50.0k0.7)
LLM
mIGAN
81.9(2.2)
80.9(2.)
72.4(1.5
73.4k1.4
58.6(2.8 52.9(3.0)
MedSyn
81.7(0.0)
81.7(a.)
82.9(0.1)
82.2(a.1)
74.5(a.1)
71.3(0.2)
LLM-
LLM Direct
X
81.8(0.0)
81.8
64,4(0.0)
65.4k0.)
60.9(0.2)
59.0k0.3
Based
Self-Refine
81.9r0.3)
81.80.3)
64.9(0.8)
(5.7
(0.3)
61.3(0.5)
LLM-CARR(ours)
X
82.6(0.3) 82.4(0.)
85.3(0.))
85.2(0.))
(6.9(0.3) (4.1
To evaluate the utility of synthetic EMRs, assess their effectiveness in training models for down- stream tasks. As shown Table LLM-CARe achieves the best performance across all three tasks, without training prompting any real EMR text. In contrast, most baselines rely
7

---

Published as a conference paper at ICLR 2026
real records for model training in-context examples, which may raise privacy concerns (privacy evaluation is shown in [p While downstream tasks do not map directly to quality, the over- all trend is consistent: methods with higher quality also tend to perform better downstream tasks. For example, MedSyn ranks second to ours on both quality and tasks. Notably, baseline methods perform relatively well diagnosis but show larger gaps examination and treatment tasks. We attribute this to their higher coverage of symptom-rclated knowledge but limited representation of clinical concepts related to examinations, procedures, and medications--ey to the latter two tasks, These results highlight the advantage of our structured framework in producing semantically rich and clinically useful synthetic records. Further experiments using an alternative backbone and an external dataset are provided in Appendix D
with subgroup analyses shown in Figure 13
Diagnosis Prediction
Examination Recommendation
Treatment Recommendation
1.0
1.0
1.0
Accuracy
0.8
0.9
0.8
0.8
0.6
0.2
0.8
0.2
0.0
0.5
0.0
LSSTM
MIGAN
MedSyn
LLL Direct
Selt-Refine
LLM-CARe (ours)
Figure 6: Accuracy distribution across diseases of different methods on three downstream tasks.
Figure 6 further illustrates the performance distribution across diseases. The box plots show that our method not only achieves higher average performance but also exhibits narrower variance across dis- eases. This consistency suggests that our approach is broadly effective and robust across a wide range of clinical conditions. In contrast, some baselines display wide performance fluctuations, indicating limited generalization to diverse disease types. These findings underscore the reliability of our method in real-world clinical settings, where robustness across varied discases is critical.
5.3 ANALYSIS OF PERFORMANCE ACROSS STAGES
Figure 7 shows how EMR quality and downstream task performance evolve through the three refine- ment stages. Quality dimensions improve most notably in their corresponding stages (e.g., complete- ness during the section stage). While some dimensions may show temporary fluctuations other stages, the staged Briign-progressnng from softer corpus-level constraints to stricter document- level checks ensures that all dimensions ultimately exceed direct generation by clear margin. For downstream tasks, examination and treatment recommendation benefit most from corpus-level refinement, since they rely on broad and diverse clinical concepts present in the training data. In contrast, diagnosis prediction depends more directly on complete histories and symptom-diagne alignment, thus improves primarily at the section and document stages.
Gsianey
Carpue Quality
1.0
1.0
Seciion Rogee
Deanvent So
BY
Copus Segge
Seatir Stage
Dovurint Souge
07
Corpa Stage
Secióo Stage
Comp
Denographic
Downsrrem
Parfomance Diagnosis Prediction
Exominaoo Recommenatiion
Troctmont Recommondaion
0.
08
0.6
Gopun Stage
Section Sage
Uccament Stage
0.6
Cocoue Stage
Sector Slage
Document Slage
0.6
Corpun Stage
Secaan Stage
Jocument Stage
Average Marro Auarage
Average Average
Marro Average.
Figure 7: Trends on (a) EMR quality and (b) downstream task performance across stages.
5.4 ABLATION STUDY OF MULTI-AGENT COMPONENTS
To assess the contribution of each agent in framework, conduct an ablation study by indi- vidually removing the Critic. Adviser, and Reviser agents. When the Critic is removed, the Adviser generates feedback for all quality criteria, regardless of whether they are already satisfied. Without
8

---

Published as a conference paper at ICLR 2026
the Adviser, the Reviser receives only high-level information about ummet criteria, without action- able suggestions. When the Reviser removed, the system cannot update existing drafts insteed, prompt the Generator to regenerate EMRs using all quality criteria as input.
0.96
0.85
Actureyy
Accuracy
0.80
0.80
0.85
0.75
0.80
Quality we
Downstreem Tesk (Micro)
Downstreem Tesk (Macro) LLM-CARe (ours)
we
Figure 8: Impact of removing each agent on EMR quality and downstream task performance.
Figure 8 shows that removing any of the agents leads to noticeable performance drop in both EMR quality and downstream tasks. The most significant declines oceur when either the Critic or Reviser is ablated, highlighting two key insights: accurate assessment of the current draft is crucial for targeted refinement; and large language models struggle to satisfy all quality criteria in single generation step, underscoring the need for cyelic refinement. Besides, removing the Adviser also results in a performance drop, suggesting that concrete, actionable feedback is more effective than abstract criterion-level input in guiding successful revisions.
5.5 ANALYSIS OF ROBUSTNESS ACROSS LLM BACKBONE
Table compares framework with the LLM Direct baseline across three different
Table 3: EMR quality and downstream performance (%) across LLM backbones, averaged over principles and tasks.
LLM backbones: a general- purpose model
(LLaMA 3.1) (Dubey et al. 2024). a med-
ically pre-trained model (Meditron) (Chen
Backbone
Generation Strategy
EMR Quality
Downstrearn Task
Micro Average
Macro
jet al. 2023), and a reasoning-orienter model (R1) (Guo et al. 2025). Without
Average
any prompt tuning or model-specific adap- tation, our method consistently improves both EMR quality and downstream task performance across all backbones.
Llama3. 1
LLM Direct
49.3
54.9
55.8
-8--Instruct
LLM-CARe
77.5
73.1
71.7
Meditron3
LLM Direct
53.9
53.7
54.8
-8B
LLM-CARe
76.4
73.6
72.4
RI Distill
LLM Direct
55.5
51.3
52.4
Notably, although Meditron is explic- itly trained for medical domains, it still struggles to directly generate high-quality EMRs and gains substantial improvements when in- tegrated into our framework. Similarly, RI does not significantly outperform the general model in direct generation. indicating that internal reasoning alone is insufficient to meet the nuanced requirements of EMR. These findings emphasize the necessity of principle-drice refinement that complements backbone capabilities and cannot be replaced by pretraining reasoning alone.
-Lama-8B
LLM-CARe
80.5
72.8
71.5
5.6 EFFECT INCORPORATING REAL EMR TEXT
To further examine whether LLM-CARe ben- efits from access to real EMR text, we intro- duce variant in which the initial draft genera- tor is given a real EMR as a reference example. As illustrated in Figure 9 the performance of this variant remain highly similar to the orig- inal LLM-CARe. These findings show that the effectiveness of LLM--ARe arises from its structured multi-agent cyclic refinement, which leads to strong performance without relying on real EMR text. Additional comparisons with other variants are provided in Appendix K
EMR Quality
Downetreem Task
Diagnocle
Compratereee
Treatneett (Macro)
Dllggoees (kacroi
Coverage
85
Carrechiaas
Y5
80
Treatwent (MFooo
Examination (Moro)
Demographis Typcoaity
Context
Examination (Kooom)
DILL-CARR
DILLM- CARe with Real EMR taxt
Figure 9: Comparison between LLM-CARe and variant that incorporates real EMR text as an ex- ample during initial draft generation.
5.7 CLINICIAN EVALUATION

---

Published as a conference paper at ICLR 2026
To validate the reliability of
Table 4: Agreement between human clinicians and LLM-based evaluation on EMR quality.
using LLM as a judge, we
conducted a human evaluation study. A total of 200 syn- thetic EMRs were sampled (40 from each of five methods) and independently assessed by four licensed clinicians, who rated completeness, consistency, and correctness for each record. As shown in Table 4. the agreement
Quality Level
Clinician LLM Agreement
Inter- Aiter-Clinician Agreement
Cohen's Kappa
Confidence Interval (95%)
Fleiss's Kappa
Confidence
Interval (95%)
Section
0.866
[0.834, 0.896]
0,975
[0.959, 0.988]
Document
0.797
[0.764, 0.829]
0,932
[0.912, 0.950J
Overall
0.833
[0.809, 0.855]
0.950
[0.937, 0.962]
between clinicians and LLM is consistently high (Cohen's Kappa 0.833 overall, where values ex- coeding 0.8 indicate near-perfect agreement), with tight confidence intervals. Inter-clinician agree- ment is also strong (Fleiss's 's Kappa = 0.950 overall), confirming that the evaluation criteria are well-defined and consistently interpretable by human experts. Together, these results demonstrate that the LLM-based evaluation closely aligns with human judgment, supporting its validity as an efficient proxy for large-scale quality assessment. To ensure the stability of human evaluation, gradually expanded the annotated subset from 100 to 200 EMRs and observed consistently stable agreement levels. Detailed results are reported in Appendix L
5.8 CASE STUDY
Table presents examples of quality is- sues that commonly arise when genera- tion methods lack explicit adherence to quality standards. These cases reveal that without structured quality control, gener- ated EMRs often exhibit missing details, medical inaccuracies, or inconsistencies.
Table 5: Examples of quality issues in synthetic EMRs. Abbreviations: CC-Chief Complaint, HPI-History of Present Illness, HC-Hospital Course, Dx-Diagnosis.
Method
Example
Problem
Dx: Uterine leiomyomas Gender: Male
Males do not have a uterus.
LSTM
HC: Discharged after
No treatments mentioned in HC.
In contrast, Figure 10 demonstrates how LLM-CARe progressively improves draft quality through refinement different levels. This underscores the importance of stage-wise cyclic refinement in producing high-quality EMRs.
mIGAN
feeling stable.
CC: Diarrhea for days.
CC mentions diarrhea, but HPI denies it.
MedSyn
HPI: ... no diarrhea...
Initial EMR Draft
EMR After Section-Level CARe
EMR After Document-Level CARe
Chief Complaine: Cheal pann with dough gh illettry experteniing theel and cough monehe bageet
Cheer Complaint Cheel pain montts eaug 'unnc Illebory and tough monthe
Chiee Complatnt Cheel pain monts, coug? usees for manth and developed
bapon
Quality Chack
Quality Chack
Quality Check
wdidd Completeness: Comptner
X
rrdyte Completeness: Complntt
Completeness: rydute resel
Comptent
Contect Consirtency "hhe creM lime in GSW
Contect Contiitency: "he areel Sime in CHW Compponnt and Heelory Presen lineee stoutc agy..
Contect Considiency "he ddeal Ste In GiW Compoé el and (letory of Presei lines stouic alyn.
Companni and Htatory Presert linnee ahou sagn.
Figure 10: Illustration of quality improvements through LLM-CARe. Revisions are marked in blue.
6 CONCLUSION
In this work, we tackle the limitations of existing EMR synthesis methods which mimic real records without considering quality requirements. To overcome these, we propose LLM-CARe, a stage- wise cyclic refinement framework driven by the collaboration of Critic. Adviser, and Reviser agents. Instead of single-pass generation, LLMM-ARee progressively enhances drafts through three dedicated stages: aligning corpus-level distributions, ensuring section-level completeness, and en- forcing document-level consistency and correctness. Experiments on a large real-world dataset demonstrate that LLM-CARe substantially improves the quality of EMRs across all granularities. Moreover, models trained on the refined synthetic corpus achieve superior performance on vari- downstream tasks, highlighting the practical value of our approach. These results show the effectiveness of LLM-CARe in generating synthetic EMRs that are both high-quality and clinically meaningful, offering reliable and privacy- preserving foundation for healthcare AI development.
10

---

Published as a conference paper at ICLR 2026
ACKNOWLEDGEMENT
This research was supported by Beijing Natural Science Foundation NO. 4252046.
ETHICS STATEMENT
This work adheres to the ICLR Code of Ethics. This work focuses on improving the quality of synthetic EMRs guided by clinical quality principles. The core methodology does not involve train- ing actual EMRs. During evaluation, limited set of test cases was accessed within secure, institutional data environment. These records had been fully de-identified by the hosting healtheare organization and remained within its controlled data management platform. The study did not entail any active data collection from patients or clinicians. All data usage adhered to institutional policies and conducted under the oversight of the relevant data governance framework.
REPRODUCIBILIIY STATEMENT
The collection and preprocessing of the EMR dataset are described Section 4 and Appendix B Experimental settings, model configurations, and evaluation protocols are detailed in Section 4 and Appendix Our code is available at https:://////. /ttt:///githbb com/ Hinp/THUMPP// CIINNLLPLLMMMCOO LLM-CARM
REFERENCES
Sayed Obdel-Khalek, Abeer D. Algami, Ghada Amoudi, Salem Alkhalaf, Fahad Mohammed Al- homayani, and Shankar Kathiresan. Leveraging ai-generated content for synthetic electronic health record generation with deep learning-basse diagnosis model. IEEE Transactions Con- sumer Electronics, pp. 1-1, 2024. doi: 10.
Hannelore Aerts, Dipak Kalra, Carlos Sáez, Juan Manuel Ramirez-Anguitt, Miguel-Angel Mayer, Juan M Gareia-Gomez, Marta Herä-Hernández. Geert Thienpont, and Pascal Coorevits. Quality of hospital electronic health record (ehr) data based on the international consortium for health outcomes measurement (ichom) in heart failure: Pilot data quality assessment study. JMIR Medi- cal Informatics, 98), 2021. ISSN 2291-9694 -9944. doi: httpp:///bbi.ogg/1121////7844880 |RLhttps:
//www. www.caciecceiiccco rect. com www.iinne/aaiiclepp.......e http:///i.//22999999999
Ali Amin-Nejad, Julia Ive, and Sumithra Velupillai. Exploring transformer text generation for medical dataset augmentation. In Nicoletta Calzolari, Frédéric Béchet, Philippe Blache, Khalid Choukri, Christopher Cieri, Thierry Declerck, Sara Goggi, Hitoshi Isahara, Bente Maegaard, Joseph Mariani, Hélène Mazo, Asuncion Moreno, Jan Odijk, and Stelios Piperidis (eds.), Pro- ceedings of the Twelfth Language Resources and Evaluation Conference, pp. 4699-4708, Mar- scille, France, May 2020, European Language Resources Association. ISBN 979-10-95546-34-4 2020 1rec- 578/
URL https:: htt.ps::/aclaathholoo....
Mrinal Kanti Baowaly, Chia-Ching Lin, Chao-Lin Liu, and Kuan-Ta Chen. Synthesizing electronic health records using improved generative adversarial networks. Journal of the American Medical Informatics Associaaion, 26(3):228-244 241, 12 2018, ISSN 1527-974X. doi: 10. 1093//amiia/ocy 142. 142
Zeming Chen, Alejandro Hernández Cano, Angelika Romanou, Antoine Bonnet, Kyle Matoba, Francesco Salvi, Matteo Pagliardini, Simin Fan, Andreas Köpf, Amirkeivan Mohtashami, Alexan- dre Sallinen, Alireza Sakhaeirad, Vinitra Swamy, Igor Krawezuk, Deniz Bayazit, Axel Marmet, Syrielle Montariol, Mary-Anne Hartley, Martin Jaggi, and Antoine Bosselut. Meditron- Scal- ing medical pretraining for large language models, 2023. URL htthhttps:/aaaae wog/ abs/ 231 6079
Kieran Chin-Cheong, Thomas Sutter, and Julia E Vogt. Generation of heterogeneous synthetic electronic health records using gans. In workshop machine learning for health (ML4H) the 33rd conference on neural information processing systems (NeurIPS 2019). ETH Zurich, Institute for Machine Learning, 2019.

---

Published as a conference paper at ICLR 2026
Edward Choi, Siddharth Biswal, Bradley Malin, Jon Duke, Walter F. Stewart, and Jimeng Sun. Generating multi-label discrete patient records using generative adversarial networks. In Fi- nale Doshi- Jim Fackler, David Kale, Rajesh Ranganath, Byron Wallace, and Jenna Wiens (eds.), Proceedings of the 2nd Machine Learning for Healthcare Conference, volume 68 of Proceedings of Machine Learning Research, pp. 286-305. PMLR, 18-19 Aug 2017. URL https::/proceedii. www.innnn.......mmmmmmmme......m mll. wtt.pppeeeee..oiiiiiee
DeepSeek-AL. Deepseck-v3.2-exg Boosting long with deepseek sparse attention, 2025.
Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, al. The llama 3 herd of models. arXiv e-prints, pp. arXiv-2407, 2024.
Reetam Ganguli, Rishik Lad, Alice Lin, and Xiaotian Yu. Novel generative recurrent neural network framework to produce accurate, applicable, and deidentified synthetic medical data for patients with metastatic cancer. JCO Clinical Cancer Informatics, (7)xe2200125, 2023. doi: 10. 100/ CC1.22.00125. URL PMID: 37130342.
doi/ /ttp:///.o/.//////////0001//////
Jiaqi Guan, Runzhe Li, Sheng Yu, and Xuegong Zhang. A method for generating synthetic electronic medical record text. IEEÉ/ACM Transactions on Computational Biology and Bioinformatics, 18 (1):173-182, 2021.
doi: 10.1109/TCBB.2019.2222
Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Peiyi Wang. Qihao Zhu, Runxin Xu, Ruoyu Zhang, Shirong Ma. Xiao Bi, et al. Deepseek-rl incentivizes reasoning llms through reinforce- ment learming. Nature, 645(808 ()3088))))333-:1111 638, 2025.
Arun Iyengar, Ashish Kundu, and George Pallis. Healthcare informatics and privacy. IEEE Internet Computing, 22(2):29-31, 2018. doi: 10.100/0..2211880220002
Hojjat Karami, David Atienza, and Anisoara Paraschiv-loneseu. SynEHRgy: Synthesizing mixed- type structured electronic health records using decoder-only transformers. In GenAl for Health: Potential, Trust and Policy Compliance, 2024. URL htLttttts//ppereee//// net forum?
Id=k4CTvn02xx
Gleb Kumichev, Pavel Blinov, Yulia Kuzkina, Vasily Goncharov, Galina Zubkova, Nikolai Zen- ovkin, Aleksei Goncharov, and Andrey Savchenko. Medsyn: Llm-based synthetic medical text generation framework. In Machine Learning and Knowledge Discovery in Databases. Applied Data Science Track: European Conference, ECMI PKDI) 2024, Vilnius, Lithuania, September 9-13, 2024, Proceedings, Part X, pp. 215-230, Berlin, Heidelberg, 2024. Springer- ISBN 978-3-031--0380-55 doi: 10.1007/978-23 0001:/97788---111781------111111 10C7/978 3- 031 2 24
URL https: /ww..//.i..../ org/10..
Sunjun Kweon, Junu Kim, Jiyoun Kim, Sujeong Im, Eunbyeol Cho, Seongsu Bae, Jungwoo Oh, Gyubok Lee, Jong Hak Moon, Seng Chan You, Seungjin Back, Chang Hoon Han, Yoon Bin Jung, Yohan Jo, and Edward Choi. Publicly shareable clinical large language model built on synthetic clinical notes. In Wei Ku, Andre Martins, and Vivek Srikumar (eds.), Findings of the Association for Computational Linguistics: ACI. 2024, pp. 5148-5168, Bangkok, Thailand, August 2024. Association for Computational Linguistics. doi: 10.18653/w120244 305. URL aattps://aclanthholl/ org /rtp////...////2//////////2///// www.ffiiiiggee aic. Docc..303 3055
http:///i///...i.in--no.a acl.
Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng. Lianmin Zheng, Cody Hao Yu, Joseph E. Gonzalez, Hao Zhang, and lon Stoica. Efficient memory management for large language model serving with pagedattention. In Proceedings of the ACM SIGOPS 29th Symposium on Operating Systems Principles, 2023.
Scott H. Lee. Natural language generation for electronic health records. npj Digital Medicine, (1):63, Nov 2018. ISSN 2398-6352, doi: 10.1008/441744601880070000 URL https: /oo... hTg/100.038/s441
12

---

Published as a conference paper at ICLR 2026
Jin Li, Benjamin J. Cairss, Jingsong Li, and Tingting Zhu. Generating synthetic mixed-type lon- gitudinal electronic health records for artificial intelligent applications. npj Digital Medicine, 6(1):98, May 2023. ISSN 2398-6352. doi:
URL
https:
7/doi org/10. 1038/ 146--17666000 023 00834--
Yihan Lin, Zhirong Yu, and Simon A. Lee. A case study exploring the current landscape of synthetic medical record generation with commercial Ilms. In Xuhai Orson Xu, Edward Choi, Pankhuri Singhal, Walter Gerych, Shengpu Tang, Monica Agrawal, Adarsh Subbaswamy, Elena Sizikova, Jessilyn Dunn, Roxana Daneshjou, Tasmie Sarker, Matthew McDermott, and Irene Chen (eds.), Proceedings of the sixth Conference on Health, Inference, and Learning. volume 287 of Proceedings of Machine Learning Research, pp. 105-129, PMLR, 25-27 Jun 2025. URL pettps:/prroese
m www.sressppe /vnsss/v22111 v287 1ww.88/1in25aahhccmm
Onkar Litake, Brian H Park, Jeffrey L Tully, and Rodney A Gabriel. Constructing synthetic datasets with generative artificial intelligence to train large language models classify acute renal failure from clinical notes. Journal of the American Medical Informatics Association, 4046)1440--1110, 04 2024. ISSN 1527-974X. 1093/ / htt////ama/occa055
doi: 10. 00.10933jamia/ccae0811 URL https: pattps://doi.com http://ooooorg/../
Fenglong Ma, Radha Chitta, Jing Zhou, Quanzeng You, Tong Sun, and Jing Gao. Dipole: Di- agnosis prediction in healthcare via attention-based bidirectional recurrent neural networks. In Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD 17, pp. 1903-1911, New York, NY, USA. 2017. Association for Com- URL https:
puting Machinery. ISBN 978 9781450348874.
doi: 10. 1 7,.114/3/999833309000
//doi.org/10. org/1-0rg/10- 11.00...4555 1141/11599999 0tt//3300799830098888
Aman Madaan, Niket Tandon, Prakhar Gupta, Skyler Hallinan, Luyu Gao. Sarah Wiegreffe, Uni Alon, Nouha Dziri, Shrimai Prabhumoye, Yiming Yang, et al. Self-refine: Iterative refinement with self-feedback. Advances in Neural Information Processing Systems, 36:46534-46594, 2023.
Oren Melamud and Chaitanya Shivade. Towards automatic generation of shareable synthetic clinical notes using neural language models. In Anna Rumshisky, Kirk Roberts, Steven Bethard, and Tristan Naumann (eds.), Pruceedings of the 2nd Clinical Natural Language Processing Workshop, pp. 35-45, Minneapolis, Minnesota, USA, June 2019. Association for Computational Linguistics. doi: 10.18653/v1/W 1905. URL htttps::/aclanthhooog... wttpoooog//o/gooooee
Yahia Mohamed, Xing Song, Tamara M McMahon, Suman Sahil, Meredith Zozus, Zhan Wang, Greater Plains Collaborative, and Lemuel R Waitman. Electronic health record data quality vari- ability across multistate clinical research network. J Clin Transl Sci, 7(1) 7(1):e130, May 2023.
Lucy Mosquera, Khaled El Emam, Lei Ding, Vishal Sharma, Xue Hua Zhang, Samer El Kababji, Chris Carvalho, Brian Hamilton, Dan Palfrey, Linglong Kong, Bei Jiang, and Dean T. Eurich. A method for generating synthetic longitudinal health data. BMC Medical Research Methodology, 23(1):67, Mar 2023. ISSN 1471-2288, doi: 10.11866112874-223-111 //doi.org// 10. 1111/111111111 -21----------------
URL https:
Hajra Murtaza, Musharif Ahmed, Naurin Farooq Khan, Ghulam Murtaza, Saad Zafar, and Am- breen Bano. Synthetic data generation: State of the art in health care domain. Computer
Science Review, 48: 1005460 2023. ISSN 1574-0137.
doi: http://ooi.r///10.0006..ooooo URL https: www. www.ssiincedireec.......mmmo.cce http://///ccceeee//e/rri///////a
2023.100546.
91 6157401372300011
OpenAL. 12t-oss-120b & gpt-oss-20b model card, 2025. URL httts::/arxiv..org/ 2508. 10925
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Köpf, Ed- ward Yang, Zach Vito. Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. Pytorch: An imperative style, nigh-performance deep learning library, 2019. URL.httge rxiv wwtp:///w////////oa/
/t/h 01
13

---

Published as a conference paper at ICLR 2026
Sina Rashidian, Fusheng Wang, Richard Moffitt, Victor Garcia, Anurag Dutt, Wei Chang, Vishwam Pandya, Janos Hajagos, Mary Saltz, and Joel Saltz. Smooth-gam Towards sharp and smooth synthetic ehr data generation. In Martin Michalowski and Robert Moskovitch (eds.), Artificial Intelligence in Medicine, pp. 37-48, Cham, 2020. Springer International Publishing. ISBN 978- 3-030-59137-3.
Shahid Munir Shah and Rizwan Ahmed Khan. Secondary use of electronic health record: Oppor- tunities and challenges. IEEE Access, 8:136977-13665 2020. doi: 10. 2001 1099.
Junyuan Shang, Cao Xiao, Tengfei Ma, Hongyan Li, and Jimeng Sun. Gamenet: Graph augmented memory networks for recommending medication combination. Proceedings of the AAAI Confer- ence on Artificial Intelligence, 33(01):1126-1133 Jul. 2019. doi: 16.1669aaaai.............a..aaia v33101.3001 126.
URL
aaal. org/ index..in.. php/AAAI
AAAI aaraaaiiiccee htt.//AAAA//aattieaeaeeeeee view / 3905
Yun Shen, Jiamin Yu, Jian Zhou, and Gang Hu. Twenty-five years of evolution and hurdles in electronic health records and interoperability in medical research: Comprehensive review. Journal of Medical Internet Research, 27, 2025. ISSN 1438-887
doi: https://doi.org/ htt.://wwcccccacccccooooo///eeea
10.2196/59024. BtLhttps:// www.
http:::ww....eeeaa.e....... www.ssseenedirecc.o com/ http://cennnaarricceeeeee
S 61438887125000377
Rodrigo Tertulino, Nuno Antunes, and Higor Morais. Privacy in electronic health records: a systematic mapping study. Journal of Public Health, 32(3):435 Mar 2024. ISSN
1613-2238.
doi: 10. 10.1077/s1089--000 1007//10333--022-19955
URL https://doi http://o...rrg////// www.oro..... 10.0
5 010389-022-01795-2
Brandon Theodorou, Cao Xiao, and Jimeng Sun. Synthesize high-dimensional longitudinal elec- tronic health records via hierarchical autoregressive language model. Nature Communica- tions, 14(1):5305, Aug 2023. ISSN 2041-1723. doi: 10. 10.1038/44447--02-4400000 10381 00057777-23---0900
URL
httpp:://oi..oo
Zixu Wang, Julia Ive, Sumithra Velupillai, and Lucia Specia. Is artificial data useful for biomedi- cal natural language processing algorithms? In Dina Demner-Fushman, Kevin Bretonnel Cohen, Sophia Ananiadou, and Junichi Tsujii (eds.), Proceedings of the 18th BioNLP Workshop and Shared Task, pp. 240-249. Florence, Italy, August 2019. Association for Computational Linguis- tics. doi: 10. 10.18653/v1// I// 5026. URL httph:tt: htt.ttts///acllctoolloggg org W1 9- -
Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Remi Louf, Morgan Funtowiez, Joe Davison, Sam Shleifer, Patrick Von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mariama Drame, Quentin Lhoest, and Alexander Rush. Transformers: State-of-the-art natural language processing. In Qun Liu and David Schlangen (eds.). Proceedings of the 2020 Confer- ence on Empirical Methods in Natural Language Processing: System Demonstrations, pp. 38- 45, Online, October 2020. Association for Computational Linguistics. doi: 10. 10.18653vv/2022) emnlp-demos.6. URL httpt::////// ac anthology org. 2020 emn w-w.mppddmms...s. 6/
Chao Yan, Yao Yan, Zhiyu Wan, Ziqi Zhang, Larsson Omberg, Justin Guinney, Sean D. Mooney, and Bradley Malin. A multifaceted benchmarking of synthetic electronic health record gener- ation models. Nature Communications, 13(1):7609, Dec 2022. ISSN 2041-1723 doi: 10.1038/ 941467022-35225-11 URL/ttts:://dos org. /080//11/6666 022 35295-1
An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, Huan Lin, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang. Jiaxi Yang, Jingren Zhou, Junyang Lin, Kai Dang, Keming Lu, Keqin Bao. Kexin Yang, Le Yu, Mei Li, Mingfeng Xue, Pei Zhang, Qin Zhu, Rui Men, Runji Lin, Tianhao Li, Tianyi Tang, Tingyu Xia. Xingzhang Ren, Xuancheng Ren, Yang Fan, Yang Su, Yichang Zhang, Yu Wan, Yuqiong Liu, Zeyu Cui, Zhenru Zhang, and Zihan Qiu. Qwen2.5 technical report, 2025. URL https: rx org/ als / 2.115.1115
Jinsung Yoon, Michel Mizrahi, Nahid Farhady Ghalaty, Thomas Jarvinen, Ashwin S. Ravi, Pe- ter Brune, Fanyu Kong, Dave Anderson, George Lee, Arie Meir, Farhana Bandukwala, Elli
14

---

Published as a conference paper at ICLR 2026
Kanal, Sercan Ô. Anı,, and Tomas Pfister. Ehr-safe: generating high-fidelity and privacy- preserving synthetic electronic health records. upj Digital Medicine, 6(1):141, Aug 2023. ISSN 2398-6352. doi: 10. 10.10088/44144--220008888 URL wttpt//ttt////a..oo//11.0..0000. 141776-000 023-00888
Hongyi Yuan, Songchi Zhou, and Sheng Yu. Ehrdiff: Exploring realistic ehr synthesis with diffusion models. arXiv preprint arXiv:2303.05656, 2023.
Hongyi Yuan, Songchi Zhou, and Sheng Yu. EHRDiff Exploring realistic EHR synthesis with diffusion models. Transactions on Machine Learning Research, 2024. ISSN 2835-8856. URL
https::/openrrviiee httpe//wwwwwwwet..r.oeeeeeeeeeee (o
Ziqi Zhang, Chao Yan, Thomas A Lasko, Jimeng Sun, and Bradley A Malin. Synteg: a framework for temporal structured electronic health data simulation. Journal of the American Medical In- formatics Association, 28(3):596-604, 11 2020. ISSN 1527-974X. doi: 10.1093/jamialccaa222 1093/ /ttp
URL
15

---

Published as a conference paper at ICLR 2026
A DETAILS OF EMR QUALITY PRINCIPLES
In this section, we provide a detailed description of each criterion corresponding to the EMR quality principles, along with representative examples.
A.1 CONTENT COMPLETENESS
Table lists the criteria used assess content completeness, which evaluate whether each field contains all essential information. The criteria follow standard clinical documentation conventions and use ordinary clinical terms (e.g.. "major symptoms" refers to the primary complaints driving the visit, as opposed to secondary associated manifestations).
Table 6: Description of quality criteria for content completeness, along with representative exam- ples, Abbreviations: CC-Chief Complaint, HPI-History of Present Illness, HC-Hospital Course, DI-Discharge Instructions.
Criterion
Abbreviation
Positive Example
Negative Example
Chief complaint states reason for admission
CC: Cough for day
CC
CC: Admitted on 2025/05/16
CC: Thyroid nodule
Reason
noted for months
CC: Dizziness accompanied by nausea CC: Poor recent
CC: Fever for 6 days
Chief complaint includes onset time
CC
CC: Chest pain for 1 year,
Onset
worsened over past month
glycemic control
HPI: Sudden-onsct
HPI: Experienced
History of present
HPI
headache week ago
headache over month ago
illness describes acuity of onset
Acuity
HPI: Gradual onset
Developed gait instability recently
unsteady gait for 4 months
HPI: Abdominal pain after
HPI: Sudden right eye
History of present illness mentions
HPI
alcohol intake day ago
vision loss week ago HPI: Cough onset days ago
Cause
HPI: Dizziness for weeks
possible causes
without obvious cause
History of present illness lists major symptoms and onset time
HPI: Vomited 4-5 times
HPT
over the past half day
HPI: Experienced
Symptom
HPI: Poor appetite and
dizziness for days
fatigue over past 2 weeks
HPI: Normal mental
History of present
HPT
status, sleep, appetite, bowel and bladder function; no significant weight change
HPI: Appetite decreased
illness includes all general conditions
General
HC: Chest CT revealed
Hospital course includes auxiliary examinations
Patient underwent further examinations after admission
HC:
HC
pulmonary mass lesion HC: Admission labs
Examination
laboratory examinations
showed CRP: 12.3 mg/L.
HC: Appendectomy
Hospital course includes treatment interventions
HC
under general anesthesia
HC: Given pharmacological therapy
Treatment
HC: Aspirin given
for antiplatelet therapy
DI: Atorvastatin
Discharge instruction includes medication dosage and usage
DI
tablet nightly
DI: Take antibiotics regularly
Medication
Amoxicillin
Ig twice daily
A.2 MEDICAL CORRECTNESS
Table ouutines the criteria for medical correctness, which assess whether the clinical content aligns with the patient's diagnosis.
16

---

Published as a conference paper at ICLR 2026
Description of quality criteria for medical correctness, along with representative exam- plcs. Abbreviations: CC-Chief Complaint, HPI-History of Present Illness, HC-Hospittal Course, DI-Discharge Instructions, Dx- Dx-Diagnosis, PD-Patient Demographics.
Table 7:
Criterion
Abbreviation
Positive Example
Negative Example
Diagnosis
Dx-PD Gender
Dx: Pregnancy
Dx: Pregnancy
matches the patient's 's gender Symptoms in chicf complaint align with diagnosis
Gender: Female
Gender: Male
Dx-CC
Dx: Pneumonia
Dx: Pneumonia
Symptom
CC: Cough for day
CC: Knee pain for days
Dx: Cerebral infarction
Dx: Acute
Symptoms history of present illness align with diagnosis
Dx-HPI Symptom
appendicitis
HPI: Sudden slurred speech day ago
HPI: Sudden blurred
vision weeks ago
Dx: Cerebral
Examinations in hospital course align with diagnosis
Dx: Pneumonia HC: Chest CT indicated pneumonia
infarction
Dx-HC Examination
HC: Abdominal
ultrasound showed
appendiceal thickening
Medications in discharge instructions align with diagnosis
Dx: Type 2 diabetes DI: Metformin (0.5g), one tablet twice daily
Dx: Pneumonia
Dx-DI Medication
DI: Insulin injection
before meals
A.3 CONTEXT CONSISTENCY
Table presents the criteria for context consistency, which evaluate whether information across different EMR sections is logically coherent.
Table 8: Description of quality criteria for context consistency, along with representative examples. Abbreviations: CC-Chief Complaint, HPI-History of Present Illness, HC-Hospital Course.
Criterion
Abbreviation
Positive Example
Negative Example
Symptoms in chief complaint are consistent with those in history of present illness Onset time in chief complaint is consistent with that in history of present illness
CC-HPI Symptom
CC: Cough for I day HPI: wich cough ...
CC: Cough for I day HPI: wichout cough
CC: Chest pain for
CC: Chest pain for month
CC-HPI Onset
1 month
HPI: Chest pain over
HPI: Chest pain
past month
past 2 months
Affected site in history of present illness is consistent with the site of examination or treatment in hospital course
HPI: Left leg pain
HPI: Left leg pain
HPI-HC
after fall
after fall
Site
HC: X-ray showed
HC: X-ray showed a fracture of the right leg.
fracture of the left leg.
A.4 DEMOGRAPHIC TYPICALITY
For demographic typicality. we focus on two key patient attributes: gender and age. We evaluate whether the distributions of these attributes in the synthetic EMRs align with the target distributions. Figure [T illustrates representative examples of gender and age distributions that are aligned with and deviate from the target distribution.
17

---

Published as a conference paper at ICLR 2026
Attribute
Positive Example
Negative Example
Target Distribution
Target Distribution
Gender
Synthetic Distribution
Synthetic Distribution
Target Distribution
Target Distribution
Age
Synthetic Distribution
Synthetic Distribution
Figure 11: Examples of synthetic data distributions that are either consistent or inconsistent with the target distribution.
A.5 KNOWLEDGE COVERAGE
For knowledge coverage, we focus on five key categories of clinical knowledge: symptoms, auxil- iary examinations, laboratory examinations, surgeries, and medications. Table 9 lists representative entities from each category.
Table 9: Knowledge categories and representative entities.
Category
Example
Symptom Auxiliary Examination Laboratory Examination
Cough, Fever, Headache, Nausea, Chest CT, Brain MRI, ECG, Abdominal Ultrasound,
Complete Blood Count, Liver Function Test, C-Reactive Protein, Appendectomy, Tonsillectomy, Cataract Surgery, Cholecystectomy,. Aspirin, Penicillia, Metformin, Atorvastatin,
Surgery
Medication
B DETAILS OF DATASET
B.1 DATASET CONSTRUCTION
We conduct our experiments on a large-scale real-world EMR dataset containing 1.82 million de- identified medical records collected from hospitals. Personally identifiable information (e.g., patient and clinician names, phone numbers, locations) had already been removed by the data provider, ensuring compliance with privacy standards. All experiments were conducted on hospital-comtrollld infrastructure to ensure data security and prevent risk of privacy leakage.
To ensure data quality, we first remove records that are missing critical information, such as patient age or gender, primary diagnosis, any of the four target ficlds: chief complaint, history of present illness, hospital course, and discharge instruction. After this filtering step, 905k records remain.
We then apply length-based filtering criterion to further improve data quality. During inspection, we found that overly short entries often contain placeholders or incomplete content, while exces- sively long entries are more likely to include unintelligible text. Therefore, we retain only records where the chief complaint under 20 words, and each of the other fields falls within the 10 to 1,000 word range. This step yields subset of 710k high-quality records.
18

---

Published as a conference paper at ICLR 2026
Lastly, to ensure the reliability and stability of downstream evaluation, retain only common diseases with sufficient data volume. Specifically, we exclude any diagnosis category with fewer than 500 records. This ensures that each disease has at least 100 samples in the test set after an 80/20 train-test split. Diseases with very few examples can lead to high-variance estimates and lack statistical significance in evaluation. After this final filtering step, obtain 192k EMRs spanning 302 distinct disease categories.
B.2 CONSTRUCTION OF DOWNSTREAM TASKS
For all downstream tasks, the correct options are extracted directly from the EMR. For the diagnosis prediction task, incorrect options are randomly sampled from other diagnoses in the dataset. For the test and treatment prediction tasks, incorrect options are selected to be incompatible with the gold diagnosis: we first exclude all tests or treatments that appear in EMRs with the same diagnosis, and then randomly sample from the remaining pool. Using this approach, from synthetic EMRs, and evaluate model performance on questions built from real EMRs in the held-out test set. The diagnosis prediction task contains exactly question per EMR, yielding 38k samples in the test set. For examination and treatment recommendation, each extracted examination treatment entity forms one question, resulting in 346k and 1 100 test questions respectively.
construct training examples
C BASELINE IMPLEMENTATION DETAILS
For the non-LLM baselines (LSTM and mtGAN), we trained both models from scratch on the train- ing set. Each model receives one-hot disease vector as input and optimized to generate the corresponding real EMR, treated as the supervision target.
For the LLM-based baselines (MedSyn, LLM Direct, and Self-Refine), we used the same backbone model as in LLM-CARe generation, we randomly sample one real EMR of the target disease from the training set as an in-context example, and randomly sample five symptoms associated with the disease as additional cues. The model then generates an EMR single pass conditioned on these inputs. L.LM Direct: We generate EMRs using the same initial-draft prompt template as in LLM-CARe, without applying any refinement or evaluation steps. Reff-Refine: We aggregate all section-level and document-less criteria into single combined instruction. The model first identifies issues based on this unified prompt and then produces a revised EMR accordingly, without multi-level decomposition.
Frren255-7BIInstruee to ensure fair comparison. MedSyn: For each
D EVALUATION DETAILS
For the LLM-based evaluation of EMR quality, prompt the model to assess each generated EMR against the predefined criteria for medical correctness, content completeness, and context consistency. Each criterion is formulated as binary classification thethrr given EMR satisfies the criterion or not. The model outputs a yes/no response for each criterion per EMR, and we compute the final score by averaging over all EMRs.
For demographic typicality, we compare the distribution of demographic attributes in synthetic EMRs to those in the real dataset. For gender, we use the total variation distance (TVD) between the two distributions. For age, which is continuous variable, we compute the Wasserstein dis- tance. Since lower distance values indicate higher similarity, we transform the scores by computing 1 TVD and 1 Wasserstein, respectively, so that higher values consistently reflect better quality across all metrics.
For knowledge coverage, first extract medical entities associated with each diagnosis from the real EMRs. We then measure the proportion these entities that appear in synthetic EMRs with the same diagnosis. To avoid the complexity and potential noise of semantic matching, we use exact string-level matching to compare entity presence.
For the downstream task evaluation, report both macro and micro accuracy. Macro accuracy averages the model performance across all diagnoses by first computing the accuracy within each disease category, then averaging across categories. Micro accuracy, in contrast, computes the overall
19

---

Published as a conference paper at ICLR 2026
accuracy across all samples regardless of diagnosis. This dual evaluation provides comprehensive view of model generalizability across frequent and less frequent disease types.
E
EXPERIMENTAL DETAILS
E. PROMPTS USED FOR EACH AGENT
In this section, we list the prompts used for each agent in our LLM-CARe framework.
Generator
Please generate electronic medical record according to the following require-
ments:
1. The patient's primary diagnosis is: [diag]-
2. Include only the following sections: 'Gender', 'Age', 'Primary Diagnosis',
"Chief Complaint', 'History of Present Illness', 'Hospital Course', and 'Discharge Instructions'.
3. Section-specifie instructions:
The Chief Complaint should briefly describe the reason for admission.
-he History of Present Illness should describe the onset and development of the condition in detail.
-hee Hospital Course should mention the examinations and treatments the patient received.
The Discharge Instructions should specify post-discharge recommendations, such as prescribed medications.
4. Output the result in JSON format with the structure: "Section Name": "Section Content", where each section content is single string.
Section-Level Critic
Below is the [section.name]" section from an electronic medical record: [section]
Please determine whether the above '[section name]' meets the following require-
ment: [requirement].
Respond in the following JSON format:
("Meets Requirement": true/false)
Section-Level Adviser
Below is the "[section_name name]" section from electronic medical record: [section]
This section does meet the following requirement: [requirement]. Please pro- vide a specific revision suggestion based on the section content, explaining how it should be modified meet the requirement.
Respond following JSON format:
("Revision Suggestion": "specific suggestion")
Section-Level Reviser
Below is the "[section name]' section from an electronic medical record with an issue:
[section]
The "[section_name]" section misses essential content. Please revise the record based on the following suggestion: [feedback]
Return the result in JSON format using the pattern "Section Name": "Section Con- tent", and include only the "[section name]" section. The content the section should be single string.
20

---

Published as a conference paper at ICLR 2026
Document-Level Critic
Below is an electronic medical record consisting of multiple sections: [record]
Please evaluate whether the record satisfies the following requirement: [require-
ment]. Identify any conflicts implausible statements across sections. Respond in the following JSON format:
("Meets Requirement": true/false]
Document-Level Adviser
The following record has issues violating the requirement: [requirement]: [record]
Please provide targeted suggestions to resolve the problem. Prioritize changes the sections that minimally disrupt overall coherence.
Respond in the following JSON format:
("Revision Suggestion": "specific suggestion")
Document-Level Reviser
Below is electronic medical record with flagged issues: [record]
Please revise the record according to the following suggestion: [feedback]. Return the updated record in JSON format, including all sections. The content of each section should be single string.
Corpus-Level Agents: For the corpus-level stage, use statistical analysis tools as the critic and adviser rather than LLMs. Therefore, no natural language instructions are required for these agents; their operations are fully automated and operate on dataset-wide distributions. For the corpus-level reviser, each sample in the selected subset is modified individually, using the same type of instruc- tions as the document-lever reviser.
E.2 LLM BACKBONES
We the following pretrained large language models in experiments:
Qwen2.5 Yang et al.
Licensed under the Apache 2.0 License We use the model checkpoints available on Huggingfacd
2025
LLaMA 3.1 (Dubey et We use the model checkpoints available on Huggingfacq
2024
Licensed under the LLaMA 3.1 Community License
Meditron (Chen et al. 2023): Licensed under the LLaMA 3.1 Community License We the model checkpoints available on Huggingface
(Guo et al. 2025 Licensed under the MIT License
We
use the model checkpoints available on Huggingfacd
httpa:///uggiingfcccccooomm
kttpss/ww.laaasccoo//lmaa
8H
21

---

Published as a conference paper at ICLR 2026
E.3 HYPERPARAMETERS
For LLM-CARE and LLM Direct-we adopt the default generation configuration provided with each model checkpoint. For all refinement stages (corpus, section, and document), the Critic, Adviser, and Reviser agents are iterated for two cycles before proceeding to the next stage. We empirically observed that additional iterations beyond two provided negligible improvements in EMR quality and downstream task performance.
For EMR quality evaluation, we use greedy decoding to ensure deterministic outputs. For down- InIstuctt using the Adam optimizer with batch size of
stream tasks, fine-tune 16, a learning rate of 2e-5, and a cosine learning rate scheduler with 5% warmup steps. The model fine-tuned for epochs the synthetic dataset and evaluated on the real test set.
E.4 SOFTWARE
EMR generation with LLMs is conducted using vLLM (Kwon et al. 2023). PyTorch (Paszke et al.) 2019) is used for training and inference non-LLM baselines. Fine-tuning of downstream task models is performed using the Huggingface Transformers library (Wolf et al. 2020). Evaluations are also executed using vLLM.
E.5 COMPUTATIONAL RESOURCES
All exper for EMR quality evaluation-are are conducted on NVIDIA RTX 4090 GPU with 24GB of memory. EMR quality evaluation, which uses 1ane2.5--2B-IInttrue is performed on NVIDIA A100 GPU with 80GB of memory.
F ROBUSTNESS OF LLM-AS-JUDGE EVALUATION
Table 10: Quality scores evaluated by different LLMs.
Content Completeness
Medical Correctness
Context
Evaluation LLM
Method
Consistency
LSTM
70.8
65.0
21.7
mtGAN
55,8
51.8
21.4
Qwen2.5-32B
MedSyn
84.8
95.3
91.9
LLM Direct
77.1
90.7
87.9
LLM-CARe (ours)
91.2
98.6
93.8
LSTM
73.9
88.9
8.9
miGAN
62.8
87.7
13.4
GPT-OSS-20B
MedSyn
79.9
97.3
86.9
LLM Direct
81.1
96.7
70.2
LLM-CARe (ours)
93.4
99.0
91.3
LSTM
69.9
72.5
2.6
miGAN
57.9
64.0
0.7
Deepseek- Reepseek-V3..2-Exx
MedSyn
80.6
97.5
85.0
LLM Direct
79.9
96.8
77.8
LLM-CARe (ours)
91.1
98.5
89.8
To examine whether our intrinsic quality evaluation is robust to the choice of LLM judge, we per- formed an extended cross-model assessment using three independent evaluators: Qwen2.5-32B, GPT-OSS-20B (OpenAl, 2025), and DeepSeek-V3.2-15 --p2-Exp (DeepSeek-Al 2025). Qwen2.5-32B is our original judge; GPT-OSS 20B was included due to its reported strength in medical-language un- derstanding tasks, and DeepSeek V3.2-Exp (API) added large commercial model to further test evaluation generality across architectures and training regimes. Owing to the computational cost of the API, DeepSeek evaluation was conducted 10% sampled data.
22

---

Published as a conference paper at ICLR 2026
All three evaluators provided with the same scoring instructions used in main experiments. As shown in Table [0] the absolute scores vary across models as expected due to differences in calibration and alignment-but the overall ranking pattern remains consistent, with LLM-CARe achieving the top scores across all intrinsic quality dimensions under all judges. The baselines also maintain broadly similar relative positions, with only minor ordering differences between closely performing systems. These results indicate that our evaluation conclusions are not tied to a spe- cific model family and that LLM-CARe demonstrates robust superiority under multiple independent evaluators.
G SUBGROUP ANALYSIS ACROSS DEMOGRAPHIC AND DISEASE FREQUENCY
Gender
Age Group
Disease Frequency
0.9
0.9
0.9
0.8
08
0.6
0.5
0.5
0.5
Female
Male
0:17
19:44
46-84
Top 25%
25%6-60%
50%--75%
Bottom 25%
LLSTM
miGAN
MedByn
LLM Direct
LLM-CARe (ours)
Figure 12: Subgroup analysis of intrinsic EMR quality.
Gender
Age Group
Disease Frequency
0.9
0.9
0.9
0.8
0.7
0.7
0.8
0.6
0.6
Fe
Made
6-17
18-4
46-84
Top
25%--50%
50%-789% Bottom 25%
LSTM
miGAN
MedSyn
LLM Direct
LLM-CARe (ours)
Figure 13: Subgroup analysis of downstream task performance.
To examine whether LLM-CARe amplifies or mitigates subgroup disparities, we conducted strat- ified evaluation across three grouping dimensions: gender, age (0-17, 18-44, 45-64, 65+), and disease-frequency strata. Figure 122 reports intrinsic EMR quality across subgroups, and Figure [3 shows downstream performance under the same partitions. LLM-CARe consistently achieves the highest scores across all subgroups, and the variation across demographic and frequency groups is comparable to or smaller than that of the baselines. These results indicate that L.LM-CARe does not reinforce demographic clinical biases and maintains robustness across diverse subpopulations, supporting its applicability to balanced synthetic EMR corpus construction.
H FULL RESULTS OF EMR QUALITY
Figure 14 provides the complete breakdown of EMR quality across all evaluated criteria, extending the representative results presented in Figure 5 of the main text. These detailed results offer more comprehensive view of how different methods perform with respect to each quality dimension.
I PRIVACY EVALUATION VIA MEMBERSHIP INFERENCE ATTACK
To assess whether synthetic EMRs generated by differ- ent methods inadvertently reveal information from real patient records, we conduct a membership inference at- tack (MIA)-2 standard privacy evaluation technique that
Table 11: Membership inference attack accuracy (closer to 0.50 is better).
Method
Attack Accuracy
LSTM mIGAN
0.499
0.504
23
MedSyn
0.533
LLM Direct
0.500
(ours)
0.504

---

Published as a conference paper at ICLR 2026
Content Completaness
Section
0.0
Cause
Symptom
General
Examnetion
Medication
Medical Correctness
Context Consistancy
Document Level 0.0
Gender
Symptom
Exomination
Onszt
Knowledge Coverage
Demographic Typicality
Corpus
0.9
Symptom
Laboratery
Surgery
Examinaion
MeddSn Drect
(ours)
Figure 14: Fine-Grainee EMR quality evaluation across three level of criteria. Abbreviations of EMR sections: CC-Chief Complaint, HPI-History Present Illness, HC-Hospital Course, DI- Discharge Instructions, Dx-Diagnosis.
tests whether attacker can distinguish training exam- ples from non-members. An accuracy of 0.50 corre- sponds random guessing; values close to 0.50 therefore indicate stronger privacy, as the attacker cannot reliably infer membership.
Table 11 reports the attack accuracy across all methods. LLM-CARe and LLM Direct both achieve values near 0.50, as expected, since neither method accesses real EMR text during generation. LSTM and mIGAN also remain close to 0.50. Although these models are trained on real EMRs, their limited model capacity-combined with the complexity of multi-section EMRs used in our reduces their ability to memorize full clinical notes. In contrast, MedSyn, which directly uses real EMRs in-context exemplars, exhibits higher attack accuracy, indicating elevated privacy risk due to closer exposure to specific real records.
These results show that LLM-CARe introduces no observable privacy risk and that its generation process remains indistinguishable from non-member data, consistent with the fact that it operates without using any real EMR text.
ADDITIONAL VALIDATION OF DOWNSTREAM TASK RELIABILITY
To further examine the robustness of downstream task results, we conducted two complementary evaluations.
Table 12: Downstream task performance using Llama-3.2- In-nstrutttt as the backbone model.
Diagnosis Prediction
Examination Recommendation
Treatment Recommendation
Method
Micro
Macro
Micro
Macro
Micro
Maero
LSTM
68.5
67.1
74.7
75.4
55.6
48.6
meGAN
78.8
77.7
72.8
73.6
58.9
53.1
MedSyn
77.2
77.8
82.4
81.7
72.0
68.7
LLM Direct
78.9
79.0
61.9
62.8
59.4
57.0
LLM-CARe (ours)
80.1
79.8
83.3
83.2
74.7
71.8
First, we replaced the backbone model used for downstream classifiers and retrained all methods using Llama-3.2-11 3.B--BB-BBBBBB. IB-nnstruct. As reported in Table 12 the relative ordering of all methods remained stable, with LLM-CARe achieving the highest accuracy on diagnosis prediction, examination rec- ommendation, and treatment recommendation. This consistency under different model architecture
24

---

Published as a conference paper at ICLR 2026
indicates that performance gains do not depend properties of the Qwen family, but instead arise from the proposed multi-level refinement framework.
Table 13: Diagnosis prediction accuracy MIMIC-IV-Note dataset.
Method
Micro Average Macro Average
LSTM mIGAN
0.587
0,552
0.589
0.555
MedSyn
0,584
0,571
LLM Direct
0,589
0.532
LLM-CARe (ours)
0.606
0.590
Second, to test whether models benefit from superficial alignment with the textual format of the generation prompts. we constructed an additional diagnosis-predicte task using MIMIC-IV-Note. We selected only records corresponding to the same discase categories but originating from a dif- ferent clinical institution and written in a distinctly different narrative style. All models trained on synthetic corpora exhibit lower accuracy due to the distributional and stylistic shift, yet LLM-CARe again achieves the best performance (Table [33), demonstrating that the downstream advantages not tied to prompt-format similarity but to improved clinical quality the synthesized EMRs.
K
ROBUSTNESS PROMPTING STYLES AND INITIAL DRAFT QUALITY
Table 14: Intrinsic EMR quality under prompt rephrasing and improved initial drafts.
Section Level
Document Level
Corpus Level
Method
Content Completeness
Medical Correctness
Context Consistency
Demographic Typicality
Knowledge
Coverage
LLM-CARe (ours)
91.2
98.6
93.8
96.8
94.1
+Rephrased Prompts
94.1
98.0
93.8
96.8
95.1
+Better Initial Draft
93.3
98.6
94.0
96.8
96.2
Table 15: Downstream prediction performance (micro/macro accuracy) under prompt and draft per- turbations.
Diagnosis Prediction
Examination Recommendation
Treatment Recommendation
Method
Miero
Macro
Micro
Macro
Miero
Macro
LLM-CARe (ours)
82.6
82.4
85.3
85.2
76.9
74.1
Rephrased Prompts
81.1
80.7
83.4
83.4
75.3
73.0
+ +Better Initial Draft
82.6
81.8
84.7
84.4
76.8
73.9
We evaluated the stability of LLM-CARe under variations in prompting style and initial draft quality. First, all agent prompts were substantially rephrased using GPT-5 while preserving only high-level intent. Second, to test sensitivity to draft quality, replaced the LLMM-Direct drafts with higher- quality MedSyn drafts. Tables 14 and 15 report intrinsic quality and downstream task performance.
Across both perturbations, numerical differences are small and the overall ranking of methods re- mains unchanged. This robustness reflects the structure of LLM-CARe: quality requirements are single criterion at time, making
decomposed into explicit criteria, and each agent operates the refinement cycle insensitive to prompt wording. Likewise, the staged refinement progressively corrects deficiencies from multiple dimensions, reducing dependence the initial draft. These properties together ensure that moderate changes in prompt phrasing or draft quality do not materi- ally affect the final refined EMRs.
25

---

Published as a conference paper at ICLR 2026
L DETAILS OF CLINICIAN EVALUATION
L.1 RELIABILITY OF CLINICIAN EVALUATION
Table 16: Agreement between clinicians and LLM- evaluation across different sample sizes.
Clinician-LLMM Agreement
Inter-Clinician Agreement
# Samples
Cohen's s
Confidence
Fleiss's Kappa
Confidence
Kappa
Interval (95%)
Interval (95%)
100
0.837
[0.804, 0.868]
0.947
[0.928, 0.964]
150
0.842
[0.816, 0.868)
0.948
[0.933, 0.962]
200
0.833
[0.809, 0.855]
0.950
[0.937, 0.962]
To ensure that clinician evaluation reflects a broad range of clinical scenarios, all clinician-labeled samples were obtained through stratified sampling. Diseases were grouped by frequency, and each group contributed proportionally to the evaluation set. The sampling was likewise balanced across generation methods, gender, and age groups, ensuring that the annotated EMRs covered both com- mon and less-common situations and represented diverse demographic and clinical patterns.
To examine the stability of human evaluation, we increased the evaluation size from 100 to 150 and then to 200 EMRs. As shown in Table [6] clinician Minician-LLMM agreement (Cohen's 's Kappa) remains con- sistently above 0.8 across all three subset sizes, and inter-clinician agreement (Fleiss's Kappa) stays above 0.9. Confidence intervals narrow as sample size increases, but the agreement values them- selves remain highly similar, indicating that both human annotations and LLM-- evaluations are stable and reliable.
L.2 EXAMPLES OF DISAGREEMENT BETWEEN LLM AND CLINICIANS
We present examples of disagreement observed in our human evaluation.
LLM-Clinician Disagreement
Criterion: Discharge instructions should specify both the medication name and its dosage/usage.
Discharge Instruction: Continue oral antibiotic therapy...
LLM: Correct
Clinician: Wrong
In such cases, the LLM treats non- non-specific statements (e.g., "continue oral antibiotic therapy") as sufficiently informative, whereas clinicians regard them incomplete because essential details such as drug name and dosage are missing. This reflects the LLM's more permissive interpretation of medication-related content.
Inter-Clinician Disagreement
Criterion: The history of present illness should describe general conditions (diet, sleep, mental status, bowel/urination, weight change). History of Present Illness: The patient developed dizziness and fatigue days ago... Appetite and sleep are normal; bowel and urination regular; no weight change.
Clinician A: Correct
Clinician B: Wrong
Here, clinicians differ in how they interpret the requirement to document mental status. Some con- sider dizziness and fatigue implicitly convey reduced alertness, whereas others expect explicit statement. Such differences reflect variation in documentation style rather than clinical competence and represent common source of mild disagreement in narrative EMR review.
26

---

Published as a conference paper at ICLR 2026
L.3 EVALUATION SCHEMA
To verify the reliability of our evaluation, we asked licensed clinicians to assess the quality of syn- thetic EMRs. Clinicians were instructed read each synthetic record carefully and then answer sev- eral yes/no questions regarding completeness, consistency, and correctness. The questions were designed to simple binary judgments ensure reproducibility. The detailed labeling instructions are as follows:
Please review the synthetic EMR text shown below.
Synthetic EMR:
Gender: Male
Age: 45 years old
Primary Diagnosis: Pneumonia
Chief Complaint: Fever and cough for days
History of Present Illness: Patient developed fever three days ago, accompanied by cough and mild chest pain...
Based the above synthetic EMR, please answer the following questions. For each question, mark your judgment in the blank: Yes if the requirement is satis- fied, No otherwise. Completeness
Does the history present illness include major symptoms? Answer: (Yes/No)
Consistency
Are the symptoms in the chief complaint consistent with those in the history of present illness? Answer: - (Yes/No
Correctness
the patient's sex valid given the diagnosis? Answer:
(Yes/to /No)
Confidence interval estimation: To quantify agreement, we report Cohen's Kappa and Fleiss's Kappa with 95% confidence intervals. The intervals were computed using non-parametric boot- strap procedure with 10,000 resamples, which provides uncertainty estimates without assuming nor- mality of the statistics.
M COMPUTATIONAL COST
On single RTX 4090 GPU, producing 38k synthetic EMRs takes approximately 13 hours with direct generation, whereas LLM-CARe requires about 36 hours, which includes initial draft generation and all three refinement stages. Although the framework performs three levels of Critic-Adviser-Reviise interactions, the actual overhead is moderated by two factors. First, the outputs of these agents much shorter than full EMRs, making each refinement step relatively lightweight. Second, a draft already satisfies the criteria at a given stage, it bypasses subsequent agents, avoiding unnecessary iterations. Moreover, the computational cost scales linearly with the number of EMRs, and the cyclic refinement process can be parallelized across disease categories, making the framework feasible for scaling to larger corpora.
N USE OF LLMS
In preparing this manuscript, used LLM solely assistive tool for text refinement, including grammar correction, and language polishing. The research ideas, experimental design, implementa- tion, and analysis were entirely conceived and executed by the authors.
27


Reviews:
[{"review_id": "MIDZbjstpr", "reviewer": "Reviewer_omri", "summary": "This paper proposes LLM-CARe, a  Critic–Adviser–Reviser cyclic refinement framework for synthetic EMR generation. The method refines drafts progressively at corpus, section, and document levels. Each level targets specific quality principles such as distributional alignment, completeness, and consistency. Experiments show that LLM-CARe outperforms baseline GAN- and LLM-based EMR generators on both intrinsic quality metrics (using LLM-as-judge) and downstream clinical tasks (diagnosis/exam/treatment prediction). The paper emphasizes that its method requires no real EMR text, thereby ensuring privacy.", "questions": "* How many iterations per stage were run, and what is the stopping criterion?\n\n* Does the system risk “mode collapse,” where cyclic feedback converges to repetitive templates?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "* The motivation is clear and well grounded in real-world needs. The paper is clearly written and polished throughout.\n\n* The Critic–Adviser–Reviser framework is intuitive and well explained. The three-stage design (corpus, section, document) aligns nicely with EMR structure and gives the approach a coherent logic.\n\n* Experiments are extensive and consistently favorable. The method improves both intrinsic quality metrics and downstream task accuracy, with convincing ablations and backbone comparisons.\n\n* The inclusion of a clinician study strengthens credibility.", "weaknesses": "* The novelty is limited. The multi-agent cyclic refinement closely follows prior self-reflection or debate-style frameworks (e.g. Self-Refine [1]). The contribution is mostly in domain adaptation rather than algorithmic innovation.\n\n* The major intrinsic metrics rely on LLM-based judgments (Qwen-32B), which risks circularity—since the same model family is used for generation and evaluation. Although clinician validation is reported, the sample size (n = 100) is small and does not fully calibrate the quantitative scores in Table 1.\n\n* No comparison to recent diffusion-based EMR generators (e.g., EHRDiff [2])\n\n* No discussion of computational cost or typical failure patterns is provided, leaving scalability and robustness uncertain.\n\n-------\nReferences\n\n[1] Madaan, A., Tandon, N., Gupta, P., Hallinan, S., Gao, L., Wiegreffe, S., ... & Clark, P. (2023). Self-refine: Iterative refinement with self-feedback. Advances in Neural Information Processing Systems, 36, 46534-46594.\n\n[2] Yuan, H., Zhou, S., & Yu, S. EHRDiff: Exploring Realistic EHR Synthesis with Diffusion Models. Transactions on Machine Learning Research.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "twa5SMKuH9", "reviewer": "Reviewer_CMTW", "summary": "The paper presents LLM-CARe, a large-language-model framework for generating synthetic electronic medical records (EMRs) without using real patient data. It employs three agents—a Critic, Adviser, and Reviser—that work in a cyclic refinement loop to iteratively improve data quality. The process operates in three stages: corpus-level alignment with real-world demographic and diagnostic distributions, section-level completion of clinical fields, and document-level consistency of medical logic. Experiments on a large de-identified EMR dataset show that LLM-CARe surpasses traditional GAN- and LLM-based baselines such as MedSyn in both intrinsic quality metrics and downstream predictive tasks, including diagnosis, examination, and treatment recommendation", "questions": "Q1. Why three separate agents? Could a single unified agent with self-critique and iterative prompting achieve similar results with lower computational cost?\n\nQ2. Why does the proposed EMR quality not align with downstream task performance. For instance, why does MedSyn, which scores poorly on section-level quality metrics, still achieve competitive results in downstream prediction tasks?\n\nQ3. The document-stage refinement shows limited incremental benefit in Figure 6, raising concerns about its necessity and efficiency. Why is the document-level stage retained when its contribution appears minimal?", "limitations": "", "rating": 4, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "S1. The authors build an appealing three-agent loop (Critic – Adviser – Reviser) and wrap it in a staged pipeline.\n\nS2. The authors perform extensive evaluations, including both intrinsic “LLM-as-a-judge” metrics and downstream predictive tasks, and even conduct a clinician study to validate alignment between automatic and human assessments. \n\nS3. The experiments are thorough and cover several baselines across different LLM backbones, providing a thorough empirical picture", "weaknesses": "W1. The claimed “clinical quality principles” are human-defined checklists, not learned constraints; therefore the method is rule-driven text refinement, not genuine reasoning or data synthesis.\n\nW2. The multi-agent design seems overcomplicated for its marginal gains, with no comparison to a single self-refining LLM.\n\nW3. The approach relies on static, non-temporal EMR data, which limits realism and generalizability to real clinical settings.\n\nW4. The definition of “content completeness” assumes that every section should be fully filled, which may contradict real-world medical documentation patterns where incompleteness reflects diagnostic uncertainty or missing data. Have the authors tested whether enforcing strict completeness might produce clinically implausible or overly templated records?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "PLqFwUBkES", "reviewer": "Reviewer_UAQV", "summary": "This paper introduces LLM-CARe, a novel framework for generating high-quality synthetic electronic medical records using large language models. The approach employs a three-stage cyclic refinement process with specialized agents (Critic, Adviser, Reviser) that progressively improve EMR quality at corpus, section, and document levels without requiring access to real patient data during generation.", "questions": "1. Please clarify whether the corpus-level reference distributions used for alignment are derived from real EMRs, public statistics, or synthetic approximations. If real EMRs were used, specify exactly which aggregate statistics (e.g., age, gender, disease frequency) were extracted.\n2. Report per-criterion Cohen’s Kappa values between clinician and LLM judgments, as well as inter-clinician Kappas.\nInclude confusion matrices for each quality dimension on the clinician-labeled samples.\n3. Why was exact string matching chosen instead of concept-level or semantic matching (e.g., UMLS normalization or fuzzy embedding similarity)?\n4. Provide comparative results using a semantic approach or at least an analysis quantifying the under-count due to exact matching.\n5. Supply (a) the distributions of age, gender, and disease before and after each filtering stage, and (b) an analysis of how the ≥ 500-record disease cutoff affects downstream generalization to less-common conditions.\n6. Provide paired statistical significance tests comparing LLM-CARe with each baseline on primary metrics. Include per-disease performance tables or aggregates grouped by disease frequency.\n7. Consider evaluating with an independent LLM judge (from a different vendor), expanding clinician evaluation beyond 100 EMRs, and including edge or rare cases.\n8. Provide subgroup analyses (e.g., by gender, age bracket, or ethnicity if available) for both EMR quality metrics and downstream task outcomes.", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "1.  LLM-CARe aims to generate realistic EMRs without relying on real patient records, addressing a critical privacy barrier in healthcare AI. This approach could enable large-scale data creation for medical model training without exposing sensitive information.\n2. The paper conducts detailed ablation studies isolating the Critic, Adviser, and Reviser components, as well as corpus-, section-, and document-level refinements. These experiments clearly demonstrate that each module contributes meaningfully to the overall EMR quality improvements, validating the necessity of the multi-agent architecture.\n3. Evaluation is multi-layered—combining automatic LLM-based judging, human clinician assessment, and downstream task performance\n4. The framework is tested across several LLM backbones (Qwen 2, LLaMA 3.1, Meditron, R1) and maintains performance advantages on all of them, indicating robustness to model architecture and training data differences.\n5. The paper includes detailed prompts, backbone configurations, and ablation setups, signaling commitment to transparent and reproducible research", "weaknesses": "1. The paper’s structure is fragmented: key definitions, evaluation criteria, and implementation details are scattered across main text and appendices. This hampers readability and makes it difficult to follow the end-to-end methodology. \n2. The five quality dimensions (content completeness, medical correctness, context consistency, demographic typicality, knowledge coverage) are not well defined in the main text. Critical terms — e.g., “major” vs “minor” symptoms — are undefined.Additionally, the paper provides insufficient detail about baseline methods (particularly MedSyn, described in only one sentence )\n3. The introduction claims to address “insufficient coverage of less typical clinical cases,” yet the dataset construction retains only common diagnoses (≥500 records) and explicitly excludes rare conditions. This directly contradicts the motivation presented by the authors and undermines the paper’s central claim.\n4. The clinician validation is small (100 records, 4 clinicians) and the paper reports aggregate clinician–LLM agreement without per-criterion reliability (e.g., per-criterion Cohen’s κ, confusion matrices). Given the paper relies on an LLM judge, stronger human validation and per-criterion calibration are needed.\n5. The corpus-level alignment step matches synthetic data to a reference distribution. The paper never specifies whether 𝑇\\_𝑑  is computed from public statistics, synthetic approximations, or the authors’ real EMR dataset. If it is derived from real EMRs, it contradicts their claim that the generation “requires no real EMRs”. The source and exact construction of this distribution must be stated explicitly.\n6. Aligning synthetic data to a reference distribution derived from real data risks reproducing existing demographic or clinical biases. The paper lacks subgroup/fairness analyses to evaluate whether corpus-level alignment amplifies or mitigates such biases.\n7. The experiments predominantly use a single model family (Qwen) for generation, judging, and downstream evaluation, which risks correlated errors and vendor-specific bias. This open the possibility that performance gains may be partially due to vendor/model-specific bias rather than general methodological benefit.\n8. Missing critical details such as exact sample sizes for each downstream task.\n9. The authors provide no statistics on EMR length, number of sections, or content depth, nor do they compare these properties between synthetic and real EMRs. Without such analysis, claims of realism and practical utility are unsupported.\n10. Medical correctness is assessed via surface-level alignment checks rather than deeper clinical plausibility. The evaluation does not test whether records reflect realistic multi-disease interactions or plausible event chronology.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "rN8xXaNjjo", "reviewer": "Reviewer_3q4A", "summary": "This paper proposes LLM-CARe (Critic–Adviser–Reviser Cyclic Refinement), a multi-agent, stage-wise framework for generating high-quality synthetic electronic medical records (EMRs) using large language models without access to real data. The framework operates at three granular levels—corpus, section, and document—each involving cyclic collaboration among three agents: a Critic that evaluates, an Adviser that provides feedback, and a Reviser that updates drafts. The stages sequentially enforce corpus-level distributional realism, section-level content completeness, and document-level logical and clinical consistency. Evaluations on a de-identified dataset of 192k EMRs across 302 disease categories demonstrate that LLM-CARe produces synthetic data of significantly higher intrinsic quality and downstream utility than GAN-based, autoregressive, and direct LLM baselines. Notably, synthetic corpora produced by LLM-CARe enable superior model performance on diagnosis, examination, and treatment prediction tasks, all without using any real EMRs for prompting or fine-tuning", "questions": "- How do you control for potential overfitting between the quality evaluator (LLM judge) and the generator, given that both are LLMs possibly from the same family?\n- Can you quantify the computational cost of one full cyclic refinement iteration (critic–adviser–reviser triplet) and estimate scalability for large corpora?\n- How stable is the framework under different prompting styles or initial draft qualities? Does the cyclic loop always converge, or can it oscillate or degrade?\n- Given that human evaluation was performed on 100 samples, can you report inter-annotator disagreement cases or examples where LLM and clinicians diverged significantly? I think this would make for some nice analysis in the appendix in general to showcase divergences in thinking between specialists and AI", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 4, "strengths": "- The paper articulates a pressing problem and approaches it with a structured, multi-agent refinement paradigm inspired by human editorial cycles. This Critic–Adviser–Reviser decomposition is conceptually elegant and operationally grounded.\n- The division into corpus-, section-, and document-level quality control provides a principled way to address EMR quality at increasing semantic granularity. This explicit separation reflects a solid understanding of how clinical documentation structure affects utility and consistency.\n- Extensive experiments compare LLM-CARe to autoregressive (LSTM), GAN-based (mtGAN), and LLM-based (MedSyn, LLM Direct) baselines. LLM-CARe consistently achieves top scores across all five quality dimensions (completeness, correctness, consistency, demographic typicality, and knowledge coverage) and outperforms on downstream tasks\n- The framework improves EMR quality across several backbone models (LLaMA 3.1, Meditron, R1-Distill), showing that the refinement process generalizes beyond a single model or domain setting.\n- The ablation study isolates each agent’s contribution, empirically confirming that cyclic interaction—not one-shot prompting—is essential to achieve quality improvements.", "weaknesses": "- While baselines like MedSyn and mtGAN rely on real EMRs for conditioning or training, LLM-CARe is evaluated without them, creating a methodological imbalance. A fair comparison should include a variant of LLM-CARe that also conditions on limited real data to isolate the contribution of cyclic refinement rather than data access differences.\n- Please include a recent citation found related to LLM based synthetic EMR generation at a recent ML for health conference (https://proceedings.mlr.press/v287/lin25a.html)\n- The paper asserts privacy preservation without providing empirical checks for memorization or data leakage—critical for synthetic data generation claims.\n- Because the evaluation tasks (diagnosis, examination, treatment) are derived from the same structure as the generation prompts, there may be an inductive bias favoring models trained under similar textual formats. Try following a clinical task derived from MEDS or something unrelated (https://openreview.net/forum?id=IsHy2ebjIG)\n- The multi-agent cyclic process is likely computationally heavy, but the paper does not report inference or generation latency, making it difficult to assess the feasibility of applying LLM-CARe to million-record corpora.\n-", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)