Fine-grained List-wise Alignment for Generative Medication Recommendation
Authors: Chenxiao Fan
Chongming Gao
Wentao Shi
Yaxin Gong
Zhao Zihao
Fuli Feng
Venue: NeurIPS 2025 spotlight ()
Keywords: Medication Recommendation
Large Language Models
Reinforcement Learning
Electronic Health Record
Reward Shaping
Primary area: applications

Abstract: Accurate and safe medication recommendations are critical for effective clinical decision-making, especially in multimorbidity cases. However, existing systems rely on point-wise prediction paradigms that overlook synergistic drug effects and potential adverse drug-drug interactions (DDIs).  We propose FLAME, a fine-grained list-wise alignment framework for large language models (LLMs), enabling drug-by-drug generation of drug lists.  FLAME formulates recommendation as a sequential decision process, where each step adds or removes a single drug.  To provide fine-grained learning signals, we devise step-wise Group Relative Policy Optimization (GRPO) with potential-based reward shaping, which explicitly models DDIs and optimizes the contribution of each drug to the overall prescription. Furthermore, FLAME enhances patient modeling by integrating structured clinical knowledge and collaborative information into the representation space of LLMs. Experiments on benchmark datasets demonstrate that FLAME achieves state-of-the-art performance, delivering superior accuracy, controllable safety–accuracy trade-offs, and strong generalization across diverse clinical scenarios. Our code is available at https://github.com/cxfann/Flame.

Fine-grained List-wise Alignment for Generative Medication Recommendation
Chenxiao Fan' Chongming Gao"
*Wentao Shi
Yaxin Gong' Zihao Zhao' Fuli Feng!*
Unniversity of Science and Technology of China
(simonfan, shiwentao123, gyx2022, 22h1998) @mail ustu. edu. chongminggao@ustc. edu. cn, fulifeng930gmail. com
Abstract
Accurate and safe medication recommendations are critical for effective clinical decision-makinn making, especially in multimorbidity cases. However, existing systems rely on point-wise prediction paradigms that overlook synergistic drug effects and potential adverse drug-drug interactions (DDIs). We propose FLAME, a fine- grained list alignment framework for large language models (LLMs), enabling drug generation of drug lists. FLAME formulates recommendation a sequential decision process, where each step adds or removes single drug. To provide fine-grained learning signals, we devise step-wise Group Relative Policy Optimization (GRPO) with potential-based reward shaping, which explicitly models DDIs and optimizes the contribution each drug to the overall prescription. Furthermore, FLAME enhances patient modeling by integrating structured clinical knowledge and collaborative information into the representation space of LLMs. Experiments on benchmark datasets demonstrate that FLAME achieves state-of- the-art performance, delivering superior accuracy, controllable safety-accurte trade-offs, and strong generalization across diverse clinical scenarios. Our code is available at https: / //g thub. com/ com/cxf cxffan/////occcmmmmmccooc Fmme..
Introduction
Accurate and safe medication recommendation essential for clinical decision Decision-making, especially in complex cases involving multimorbidity []] In these scenarios, clinicians must consider not only the therapeutic effects of individual drugs, but also their potential interactions and cumulative safety risks. Recent advances in AI have led to the development of automated medication recommendation systems
yet their effectiveness remains limited in practice.
Traditional approaches typically adopt a point-wise prediction paradigm, where each drug is evaluated independently based on structured patient data such as diagnoses, procedures, and drug codes 66] 7 While longitudinal models ( attempt to capture patient history. they are still constrained by their reliance on discrete labels and lack the capacity to represent unstructured clinical context. More recently, large language models (LLMs) have emerged as promising solution, due to their strong language understanding and ability to incorporate free-text information such as clinical notes [O] However, most M---esed methods still follow the point-wise drug set by aggregating drugs with high predicted scores.
wise formulation, constructing the final
This formulation introduces a fundamental limitation: it overlooks the synergistic effects and safety constraints that exist between drugs. In practice, effective prescriptions must balance therapeutic efficacy with the risk of adverse drug-drug interactions (DDIs) [1. Point-wise models, by design,
"Coresponning author
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Outcome- based advantages
Evaluate each step
List-wise advartages
Step-wise advantages
-0.38
mg
mg
0.38
mi
0:38
-0.7
-.3
-2.
-0.38
m
+0.40
m
mg
-0.7
00.8
+0.40
my
mg
+1.35
my
mg
m4
-0,3
0022
0.8
0,7
35
my
ml
m75
-1,37
m)
mg
+0.7
-0.2
0.8
-1.37
m1
ma
Some advantage for all tokens?
Step-wise signals, finer advantages!
Figure 1: Contrasting advantage computation in GRPO and wise GRPO. (a) Outcome- advantages in GRPO assign a uniform reward to all drugs in completion. (b) Step-wise GRPO treats the generation as a sequence of medication decisions, where each m, denotes distinct drug and simultaneously defines one decision step. Each step endowed with a potential function, whose change reflects the incremental quality of adding that drug ("Evaluate each step"), and these potential- based signals are aggregated with the list-level outcome reward to yield the final step-wise advantages, enabling finer-grained credit assignment.
cannot account for this interplay. To address this, we argue that medication recommendation should be viewed list-wise decision-making problem, where the goal is to generate coherent set of drugs that jointly optimize accuracy and safety.
Existing list-wise methods in NLP often rely on reinforcement learning (RL) techniques to align model outputs with desired properties. One promising approach Group Relative Policy Optimization (GRPO) [2). which updates model preferences based relative advantages among group of candidate outputs. Yet standard GRPO operates at the sequence level (Fig. (a)) assigning a single scalar reward to the entire output- making it difficult to assign credit individual actions.
To this end, we propose FLAME (Fine- Frine-grained List-wise Alignment for generative Medication #Ecommendation), an L.MM-based framework that formulates drug generation as a drug-by-drug decision process. At its core is novel extension of GRPO step-wise TRROO-wiie models the generation as sequence of state transitions. Each step adds or removes drug. and reward shaping is applied via a potential function to provide token-level feedback (Fig. (b)) throughout the generation process. This structure enables FLAME to learn nuanced and controllable decision policies.
To support comprehensive patient modeling, FLAME incorporates multi-source medical knowledge into the LLM via hybrid representations. In addition to the LLM's 's natural language inputs, we inject structured clinical features (e.g., codes, embeddings from prior models) into the token space. enabling the model to capture both textual context and collaborative signals. We build on Llama3. 1-Aloe-Beta- 8B a domain-specifie L.M with enriched medical knowledge.
We evaluate FLAME on benchmark datasets including MIMIC-II 14, MIMIC-IV [15), and eICU (6). Experimental results show that FLAME achieves sfate-of-the-arr accuracy while main- taining controllable safety trade-off'. Moreover, FLAME exhibits strong generalization across time and institutions, validating its adaptability real-world clinical scenarios.
Our contributions are summarized as follows:
We propose FLAME, an LLM-based list-wise medication recommendation framework that gener- ates prescriptions drug trug-by-drrug bby-drg, explicitly modeling drug interactions and integrating multi-source medical knowledge via hybrid representations.
We introduce step-wise GRPO, which models medication recommendation as sequential state transition process and leverages potential-based reward shaping to provide fine fine-grained feedback for each deve-level decision, enabling controllable and clinically safer recommendations.
Extensive experiments on benchmark datasets demonstrate that FLAME achieves state-of-the-an performance accuracy, safety- acceurcy trade-offs, and cross-dataset generalization, validating its effectiveness in diverse clinical settings.
2 Related Work
In this section, we provide a brief review of medication recommender systems and LLM fine-tuning methods, highlighting their respective strengths and limitations.

---

2.1 Medication Recommendation
Existing medication recommendation methods can be broadly categorized into instance-based and longitudinal approaches, based on how patient information is modeled.
Instance- based methods, such LEAP [6, treat the medication recommendation task as multi- instance multi-label learning problem. These approaches typically rely on structured features extracted from a single patient visit. In contrast, longitudinal methods incorporate temporal information from patients' hospitalization bistories to model long-term disease progression and treatment trajectories. For example, GameNet 77) utilizes hospitalization records and graph augmented memory module to model DDIs. SafeDrug 3 enhances this by using dual molecular graph encoders to capture drug structural information. COGNet [4] introduces copy-or-pree predict mechanism for medication recommendation from historical data, while MoleRec [ focuses on the relationship between health status and molecular substructures. RAREMed [5] uses pectrain-finetune framework for structured representation extraction, and NLA-MMR 17 applies a multimodal alignment framework to jointly
learn from patient and drug views.
Recently, the development of LLMs has brought new possibilities for medication recommendation. Instead of relying solely structured codes, these approaches leverage natural language to represent patient conditions more comprehensively. For example, LAMO [[O] integrates structured diagnoses and procedures with unstructured textual descriptions of patients, enabling LLMs to model clinical context in a semantically rich manner and generate more personalized drug suggestions.
Many existing methods, including recent LLM-based approaches, still follow the point-wise prediction paradigm [8
assigning independent scores or binary labels to individual drugs while ignoring inter-drug dependencies. In contrast, we formulate medication recommendation as a list- wise decision process and introduce step-wise GRPO for fine- fine-grained reward modeling, integrating medical knowledge, collaborative signals, and LLM-based semantic understanding.
19 20
2.2 LLM Fine-Time Fine-Tuning Approaches
Fine-tuning LLMs for complex decision tasks has evolved from supervised fine-tuning (SFT) to preference-based reinforcement learning approaches such as reinforcement learning-based fine- tuning (RLHF). While RLHF methods like Proximal Policy Optimization (PPO) (21] offer online adaptability, they incur high overhead due to their multi-component design. Recent advances such as Direct Preference Optimization (DPO) [22) reduce this complexity by leveraging static preference data, but struggle with dynamic optimization.
GRPO [12 further improves training efficiency and policy stability by introducing group-wise relative preference comparisons, achieving strong performance in structured reasoning tasks. However, GRPO remains outcome-based lee--badddassinninnn rewards only to complete outputs- -thu failing to capture step- level quality variations within structured decision processes.
Several recent efforts have sought to improve GRPO. DAPO (23) introduces token-level policy- gradient losses mitigate length bias, ensuring each token contributes equally; however, still adopts a list-wise ad advantage formulation, lacking localized reward assignment. StepGRPO [24] decomposes responses into sequential actions and supplements outcome-based evaluation with step- level accuracy and validity rewards. Yet, it continues to aggregate advantages at the list level and often depends on ground- -round-truth process data. While these works enhance signal fidelity and stability, they do not resolve the fine-grained reward allocation problem we address.
On the contrary, step-wise GRPO is a fine-grained alignment method that decomposes generation into decision steps and applies a potential-basee reward mechanism to guide each sub-decision. This enables I.LMs to better learn the internal logic of complex tasks such as medication recommendation.
3 Preliminary
This section formulates the medication recommendation problem through three components: elec- tronic health record, DDI graph, and the optimization goal. We then introduce Group Relative Policy Optimization, reinforcement learning framework that models outcome- based ad advantages.
3

---

3.1 Medication Recommendation
Electronic Health Records (EHRs). Each patient's EHR
251 contains both structured and unstruc-
tured clinical information and is represented as sequence of multivariate visits. For patient
with V historical visits, the EHR is denoted as X (3) [x
X2 ())
x Each visit x x (2)
consists of the following components: X = [(( ny M. where r.)) E {0, 13121 is multi-hot vector encoding structured features from the set F (e.g., demographics, diagnoses, and
procedures), and ( is the corresponding unstructured clinical note in natural language. The set MD) denotes the medications prescribed during the y-th visit. For simplicity, we omit the patient index j when is clear from context.
Drug-Dre Drug Interaction (DDI) Graph. The DDI graph models harmful pairwise interactions between medications and is represented as symmetric binary adjacency matrix D E {0, 1MMMi.MMM where Di- indicates a known adverse interaction between medications m, and my-
Task Definition. Given a patient's s historical visit records Xy- as well as the structured fields and unstructured note of the current visit (fy, ny), the objective is to generate an output oy, representing the recommended medication set My, such that: (1) it closely approximates the ground-truth prescription Mar; and (2) it minimizes the risk of harmful drug-drug interactions as defined by the DDI graph D.
3.2 Group Relative Policy Optimization (GRPO)
GRPO 12 is reinforcement learning algorithm that improves policies via group-wise reward normalization. Given a prompt q ~ P(Q) from the task distribution, GRPO samples a group of G candidate outputs ) from the current policy Tewa Each output is assigned a scalar reward, and the updated policy Ty is optimized to favor relatively better responses within the group.
The training objective (omitting clipping for brevity) is:
Toe (ol (l. Ar.. - BDxL [
(cRPo(0) Egg(or>
(I)
T
Toass (O (o,
where 3 controls the KL divergence regularization toward a reference policy Tref-
The token-level advantage A is uniformly derived from the normalized reward of the full response:
Å F; mean(r)
std(r)
where r; is the scalar reward for the i-th output, r = [r....., rel is the reward vector for the group.
Although GRPO effectively promotes group-wise preference learning, its outcome-level reward is applied uniformly across all tokens, ignoring variation token-level quality. This coarse credit assignment hinders fine-grained learning and limits optimization efficiency.
4 Method: FLAME
We first introduce step-wise GRPO, the core of FLAME's fine-grained alignment. We then describe the optimization process, followed by the two-stage recommendation framework. Finally, we present the multi-source knowledge fusion strategy.
4.1 Step-wise GRPO
Output Segmentation. To enable fine-grained credit assignment, we decompose the LLM output into multiple decision steps [26]. each corresponding a semantically coherent token span (e.g.. single medication). Given a generated sequence o, = =[o], of o. [oil] we segment it into N, steps:
N;
(n) where o(") 10 ...., 0 o.
0,
n=1

---

0
O
22 (0
12
10
(2)
4
Aip Au Ai Ass Au Asa Aie Ass
Ma
0
3
Step(()
1
1
2
2
Ma
M3
40 (2)
(3)
0
m
m2
m
m
m2
ma
m22
(a) Output segmentation
(b) State transition
(c) Fine-grained reward
Figure 2: (a) Output is segmented by medication names into decision steps. (b) Each step is viewed as a state, with potentials ş(n) derived from comparisons with ground truth MGT. (c) Potential differences are combined with outcome rewards to provide fine-grained training signals A.
Each o; denotes the token span the n-th step. The process is illustrated in (Fig. 2
This segmentation allows us to reinterpret generation as Markov Decision Process, where each step forms state-action pair. This framing enables localized reward signals that better align step-wise actions with the global output quality.
Potential-based Reward Shaping. We extend the standard GRPO advantage (Eq. with shaping term that captures quality changes between consecutive steps:
Å, = F + A (astep(t),step(3) 1)
(4)
Here, step(t) maps token to its step index, and F(::)) measures step-wise potential difference. a weighting coefficient. To preserve policy invariance [27), we define as the difference of real-valued potential function jo:
F(n, n - 1) = ((() [0(nn -1)
(5)
with = 1, assuming equal importance across steps.
Final Advantage. The final advantage used for policy optimization becomes:
Å. F + X (y((tee(()) y (step(()) - 1))
(6)
This formulation delivers dense training signals aligned with intermediate decision quality encouraging token-level improvements toward globally effective prescriptions.
28
4.2 Fine-grained Alignment for Medication Recommendation
We apply step-wise GRPO to our list-wise medication recommendation task by treating each decision step as a span representing a single medication. We define:
State s, the patient profile and current medication set Mn,
Action a, adding or removing medication m.-
The state transition follows:
Mn U [m,). addddng, Mr (ma). removing.
Mn+1
(7)
To evaluate the quality of intermediate states, we define a step-level potential function (stepp(t)
incorporating: (1) Correctness: Jaccard similarity to the ground-truth set Mørs (2) Adherence: constraint violation with respect to the candidate set Mc (all unique medications in the processed
training data), and (3) Safety: DDI-based risk from the interaction graph D. The potential function given by:
(tstep(t)) Jaw.aaaaaaaM,aaaaa Mor) a DDI(M,, D) - RefusallRate(MM,MM (8)
where:
Mn MMT
1mm Mn & McH
Jaccard(M,, MoT)
RefusalRate(M,,,
MM, McT
MMal
[(m; m,) my m4, m, E Mn, i <.j. D(me, 9lma,my]= m,y= =1
DDI(M,, D)
(M
5

---

Drug-level filtering
List-level refinement
Accuracy Safety
Addd
Yes. No. Yes.
No.
AMremove
Drug condidates M
Mp
Recommended list My
Patient info.
π
cls
π
Tist
MAW
Figure 3: Illustration of the two-stage recommendation framework.
This potential function ((ste((()) allows computing local advantage via step-wise differences (Eqq. / 6 assigning credit based on incremental improvement in the medication set M,. The overall list-level reward is defined as the net potential change from the initial to the final step:
R₂ = ((.)) (0))
(9)
Theorem 4.1 (Optimal Policy Equivalence under Reward Reshaping). Let the token-level reward for each token in the generated output o; be defined under the following two schemes:
Ru, ift is the terminal token,
(I) Outcome-based reward: Ti, (0
(10)
otherwise
(2) Step-wise shaped reward: rhe ((step(()) <pstte(()) - -1)
(11)
Then, both reward formulations yield the same optimal policy.
The proof is provided in Appendix A
Using the potential-shaped advantage Åi, in Eq. (6). we optimize the policy via step-wise GRPO, while retaining the same objective form standard GRPO (Eq. TT)
4.3 Two-stage Recommendation Framework
We adopt a cascaded framework to generate an accurate and safe medication set Ma for a patient's w-th visit. The process consists of stages: drug-level filtering and list-level refinement. The former is implemented via a binary classification model, and the latter through policy fine-tuned with step-wise GRPO, illustrated in Fig.
Drug-level Classifier (Tas). We implement Tels as an LLM-based binary classifier, obtained by SFT the Llama3. 1-Allo-Betta [13 base model, that evaluates the relevance of each candidate drug m E M, given the patient input æ. The model returns a Yes/No decision, producing personalized subset:
Mp = {m E Me Teis (2, m) = Yes)
(12)
This step establishes personalized relevance by filtering drugs based on individual compatibility prior to joint reasoning.
List-wise Policy (mis).. To support global optimization, we introduce a list-wise policy Tia that performs instruction-condttioneed edits over MM. Given patient input æ, medication list M, and an instruction (Add Drug or Remove Drug), the model predicts modification set:
AM (a, M, Instruction).
(13)
We initialize Tis with Tas, leveraging its drug-patient matching ability, and further adapt it to instruction-driven editing via supervised fine-tuning and step-wise GRPO (see Section 4.1
Inference. At inference, we first apply Tets to obtain Mp using Eq. 12 then perform list-level edits:
AMadd min Mp, Add Drug), AM -iis (I, (r, Remove Drug).
Remove
(14)
The final recommendation is obtained by applying the edits:
M, (Mp AMaa)) A
(15)
This two-stage procedure combines individualized assessment with global reasoning for more con- trollable and context-aware recommendations, Implementation details, including prompt templates and training setup, are provided in Appendix

---

The patient a 66-year-alee old male with a history of breast cancer recest ascites allergic to penicillin
Unstructured
The patient is
EHR
Patient
Acute Kidn Hypertension
Semantic labels
V Umbilical Hernia Repair
Acute Kidney Injury
Dictionary table
Structured codes
<Diag-67> 67>
Ding 112
Hypertension
<Pro- -33-
<Pro-886>
LLM takenize & embedding layer
External Collaborative Model
Projector
Patient representation
Figure 4: Overview of patient representation construction.
4.4 Multi-source Knowledge Fusion
Traditional methods often encode structured inputs (e.g.. diagnoses, procedures) into embeddings and integrate external knowledge via architectural modules such graph encoders (3 7l However, LLMs operate primarily on unstructured text, making the incorporation structured signals non-trivial.
We propose a fusion mechanism that injects collaborative signals from structured sources into LLMs 29 30), as illustrated Fig. For each structured entity, we construct two complementary representations: a textual embedding exex E RR from natural language descriptions, and a collab- orative embedding eolllb IR'd' from a domain-specift encoder. The collaborative embedding is projected into the LLM space via learnable linear map P RRd - IR, yielding:
e fused Concat( e P(esollab)
(16)
This fusion enriches LLM inputs with high-level domain signals. We use four types of collaborative
embeddings: (1) Patient-level: from RAREMed [5], a pretrained model encoding clinical context
via a transformer. (2) Diagnosis-level and (3) Procedure-level: from MICRON [31. capturing
co-occurrence patterns from EHRs. (4) Medication-level: from Mole-BERT 32 modeling drug
substructure-funette relationships via molecular graphs.
Moreover, we adopt fusion of structured embeddings with textual semantics. This hybrid representation not only enhances the modelto capture fine-ge me--graiedd clinical signals but also benefits from the comprehensive medical knowledge encoded in the LLM, together forming our multi-source knowledge fusion framework.
[13], medical LLM, as our backbone, enabling the
5 Experiments
We begin by detailing the experimental setup. We then evaluate FLAME against baselines across three dimensions: accuracy, ablation study assesses the contribution of each component our framework.
trade-offs, and cross-dataset generalization. Finally, an
5.1 Experimental Settings
Datasets. We use real-world EHR datasets: MIMIC-III 14 for training and evaluation, and MIMIC- IV (15) and eICU [16] for generalization testing. DDI relations are obtained from TWOSIDES [I]. Following prior works, MIMIC-III is split into soiiiing/alidattinnn//
sets (4:1:1).
Implementation. All methods are implemented in PyTorch and trained on NVIDIA A100 GPUs.
Details on preprocessing, hyperparameters, and prompts are in Appendices B and C
Baselines. We compare FLAME with: (1) Traditional methods: LEAP [6. GAMENet M. Safe-
Drug 31 COGNet 4 MICRON BT. MoleRec (8) NLA-MMR 17]. RAREMed 5 (2) LLM-
based method: LAMO [T0]
Metries. Correctness evaluated by Jaccard similarity and F1 score; safety by DDI rate. Definitions and calculation details are in Appendix C
7

---

5.2 Overall Performance Comparison
We evaluate FLAME from three perspectives: correctness, safety controllability, and generaliz- ability. EHR datasets contain inherent DDI risks (e.g.. 13.69% in MIMIC-III) meaning data fitting does not guarantee safe prescriptions. Minimizing DDIs often compromises accuracy, making the correctness-safety trade-off key metric for practical utility.
Unlike prior works reporting single performance point, we explicitly decouple correctness and safety in evaluation, enabling a clearer assessment of a model's reasoning ability and controllability under varying safety constraints. To assess generalization, further test on MIMIC-IV and eICU, examining performance under distribution shifts beyond the training domain.
Correctness Comparison. Table I reports correctness results MIMIC-III Instance-basee models (e.g.. LEAP) perform poorly due to a lack of historical context modeling. Longitudinal models like SafeDrug and GAMENet improve performance by leveraging temporal data, while MoleRec further enhances accuracy by incorporating molecular substructure information. RAREMed achieves stronger results with transformer-based clinical representations. The LLMM-based LAMO benefits from unstructured clinical notes but lacks structured co-occurrence knowledge, limiting its potential. Our proposed FLAME outperforms all baselines in Jaccard and F1 scores by formulating medica- tion recommendation as a list-wise decision problem and fusing multi-source clinical knowledge.
Safety Control Comparison. We evaluate con- trollable safety by adjusting the penalty param- eter a in Eq. g generating recommendations under varying DDI constraints. Fig. (aa) shows Jaccard-DDi trade-off curves. Among models supporting safety control, FLAME consistently achieves better accuracy-safety balance than baselines. Notably, methods like LAMO are excluded as they lack safety adjustment mecha- nisms. These results validate the effectiveness of step-wise GRPO in enabling controllable, clin- ically safer recommendations.
Table I: Correctness comparison on MIMIC-III #Med indicates the average number of prescribed medications per patient (ground truth: 23.43).
Model
Jaccard
F1
#Med
LEAP
0.3073
0,4610
19.69
SafeDrug
0.3556
0.5135
22.94
GAMENet
0.3851
0.5422
20.02
COGNet
0.3876
0.5458
28.45
MICRON
0.3843
0.5419
20.49
NLA-MMR
0.3867
0.5453
25.54
MoleRec
0.4081
0.5677
24.73
Generalization Comparison. We assess gener- alization through temporal and external valida- tions. For temporal validation, models trained on MIMIC-III are evaluated on MIMIC-IV, which covers newer time periods and ICD-10 codes. Fig. 5)b shows that LLM-based models (FLAME, LAMO) degrade less over time compared to structured-data baselines, highlighting the robustness of language-based representations to cod- ing shifts. FLAME further surpasses LAMO due its integration of multi-source knowledge and step-wise reasoning.
RAREMed
0.4174
0.5776
23.03
LAMO
0.4701
0.6294
23.66
FLAME
0.4836
0.6408
22.24
For external validation, evaluate eICU, multi-center dataset with distributional and coding differences. As shown in Fig. [(c), LLLM-hased methods again outperform structured-input base- lines. FLAME achieves superior -uttof-distribution performance, demonstrating the advantage of combining list-wise decision modeling with collaborative knowledge fusion.
0.50
DA4
laccadd
Jaccard
Jaccard
0.20
0.40
0.2
0.15
0.35
RareMed -FI- MotcRe:
0.1
0.30
0.10
0.0
2016
0.00
FLAME
LAMO
RaroMadi
MuleRac
DDI (a) MIMIC-III
MIMIC-IV Time Period (b) MIMIC-IV
(c) cCCU
Figure 5: Comparison with strong basclines. (a) aacutyy trade-off on MIMIC-III (b)
Temporal generalization from MIMIC-III to MIMIC-IV. (c) External generalization to eCCU.
8

---

reward
Table 2: Ablation study. "w/o" denotes "without".
0.060
Model Variants
Jaccard
F1
0.054
(a) w/o Tels
0.3473
0.5024
0.048
(b) w/o Tiist
0.4785
0.6354
(c) w/o SFT in Tiitt
0.4810
0.6382
0.042
(d) w/o step-wise GRPO in Wist
0.4789
0.6367
5000
10000
reward_ss std
(c) with standard GRPO
0.4813
0.6385
(1 w/o structured codes
0.4535
0.6122
0.036
g w/o unstructured notes
0.4104
0.5692
0.030
(h) w/o glexi
0.3930
0,5509
w/o goollab
0.4448
0.6080
0.024
(i with ecullab
0.4653
0.6231
0.018
5000
10000
(k) FLAME-LLLMA2
0.4591
0.6168
(I) FLAME-LLaMA3
0.4774
0.6344
GRPO
Step-wise GRPO
FLAME
0.4836
0.6408
Figure 6: Training curves of GRPO and step-wise GRPO.
5.3
Ablation Study
We conduct ablations to assess the impact of each component in FLAME, including the list-wise decision model, step-wise GRPO, and multi-source knowledge fusion.
List-wise Decision Model. We first examine the roles of Tels and Tist- Table 2 (a)-(b) shows that removing Tets significantly harms performance, highlighting the importance of individualized filtering, Excluding Tist also degrades results, indicating that list-wise modeling is essential for capturing decision structures beyond point-wise analysis.
We further evaluate the two-stage training of Tia- Skipping SFT limits task-form alignment, while omitting step-wise GRPO weakens preference learning (Table (c)- These results confirm the necessity of both SFT and step-wise preference alignment.
Step-wise GRPO. Replacing step-wise GRPO with standard GRPO (Table 2 (e)) results in clear performance drops, demonstrating that outcome-only rewards insufficient. Fig. shows that step-wise GRPO provides denser feedback, accelerating learning and improving stability.
Multi-source Knowledge Fusion. Integrating structured codes and unstructured clinical notes significantly boosts performance; removing either degrades results (Table 2 (f)(g)). To analyze embedding strategies,
replace collaborative emheddings with random vectors écollab, While random embeddings offer slight gains over text-only inputs, they lack meaningful domain knowledge, resulting in notable gap from FLAME (Table 2 (h)-
LLM Backbone. Replacing the models (LLaMA2 and LLaMA3) leads to noticeable performance drops (Table 2 (k)-(1), This highlights the critical role domain-specific medical knowledge embedded Llama3. 1amma..-AAoe. 8B for accurate medication recommendation.
backbone with general large language
5.4 In-Depth Analysis
To further understand the empirical behaviors of FLAME, conduct in-depth analyses the necessity of the list-wise refinement policy Tiss and the effectiveness step-wise GRPO. These analyses complement the quantitative results and provide qualitative evidence for our design choices.
Role of the List-wise Policy mis- Table bb), removing Tis results in a relatively small numerical drop (Jaccard 0.4836 0.4785), which may appear marginal at first glance. However, this is expected, since Tha acts as a lightweight refiemment module rather than a primary predictor. the MIMIC-III test set, Tets outputs lists of average length 20.83, while Tis performs only ~2.03

---

add/remove edits per patient on average. These small-scale edits are crucial: they primarily resolve subtle drug-drug dependencies and redundancies that point-wise classifiers cannot capture.
controllability. Tels is optimized to mimic ground truth prescriptions, producing DDI rates (0.1336) close to the real data distribution (0. (3.13.9). While effective for data fitting, this limits its ability to adapt to stricter safety constraints required in real-world clinical practice. In contrast, Tistt incorporates the safety penalty (Eq. B), enabling explicit and controllable adjustment of safety levels. (2) Modeling drug dependencies. Since Tels predicts each drug independently. it cannot capture co- prescription patterns or redundancies. For instance, Piperacillin (ID 130) is broad-spectrum 3-lactam antibiotic overlapping in indication with Cefepime (ID 92), Meropenem (ID 45), and Ampicillin (ID 1 113). In such cases, Rels may recommend redundant combinations or omit suitable alternatives. By performing list-level reasoning, Tisa corrects these issues effectively: across the test set, it edited 28 Viperacilinncontaaiiin lists with an 89% correctness rate, demonstrating its capacity for meaningful refinement beyond point-wise learning.
We identify two major henefits of Tis-- (1) Safery-accura
Table 3: Examples of Tis refinements. IDs 45, 92, 113, and 130 denote antibiotics with overlapping effects. Correct, over- peer-predicted, and missing drugs are colored green, red and blue, respectively. Drugs in parentheses indicate those absent from Mp but present in the ground truth list.
Case ID
d Output Mn
Tist Refinement
266
30, 76, 113, 137, 6, 74, 130, (14, 104, 114),
| add: 14, 1 114; remove: 130
887
2, 71, 99, 8, 22, 130, (34, 48, 57, 80, 92, 126). ... add: 34, 48, 57, 92; remove: 130
As shown in Table +] Tiist rectifies redundant and missing predictions by exploiting list-wise signals. In Case 266, Tels simultancously predicts IDs 1 113 and 130, which share overlapping effects, while omitting several relevant items. Tiss removes the redundant 130 and adds the missing entries, thereby enhancing the precision of the generated medication list. In Case 887, the point-wise model again over- over-predicts 130 and misses 92, two drugs with overlapping indications, while Tiat corrects both errors through list-level reasoning. These examples demonstrate that even lightweight list-wise edits can yield clinically coherent refinements by leveraging inter-drug dependencies absent in point-wise prediction, confirming the essential role of Tis beyond its modest aggregate numerical gain.
Effectiveness of Step-wise GRPO. Step-wise GRPO is the optimization backbone for This- While the absolute improvement over standard GRPO in Table([ee is modest, the relative effect is substantial: list-wise refinement's 's advantage over point-wise classification increases from 0.0028 (standard GRPO) 0.0051 (step-wise GRPO), an 82.1% relative gain. Additionally, Fig. 6 shows faster reward convergence and more stable training dynamics under step-wise GRPO, owing to its denser intermediate feedback rather than reliance on terminal rewards an important trait for long edit sequences such as medication list refinement.
6 Conclusion
We presented FLAME, an LLM-based list-wise medication recommendation framework that formu- lates prescription generation as a sequential drug-by-drug decision process. By introducing step-wise GRPO with potential-han -hased reward shaping, FLAME enables fine-grained credit assignment, effec- tively capturing drug interactions and enforcing safety constraints such as DDIs. Through hybrid representations that fuse structured clinical data, collaborative signals, and unstructured textual infor- mation, FLAME enhances the LLM's capacity for accurate and clinically grounded recommendations. Extensive experiments demonstrate FLAME's s superior accuracy, controllable safety- accuracy trade- offs, and strong generalization across temporal and institutional shifts. These results highlight the potential of fine-grained wist alignment for building reliable, adaptable clinical decision support systems. future work, we plan to further improve FLAME's generalization in diverse and evolving healthcare environments, advancing towards scalable and equitable Al-driven solutions for real- world clinical practice. In addition, we will explore the integration of physician decisions with FLAME to build human-in in lhop medication recommendation system.
10

---

Acknowledgments
This work is supported by the National Natural Science Foundation of China (62277437..624044700 the University Synergy Innovation Program of Anhui Province (GXXT-2023-077), Anhui Provincial Natural Science Foundation (2408085QF1 189), and the advanced computing resources provided by the Supercomputing Center of the USTC.
References
[1] Z. Ali, Y. Huang, Ullah, J. Feng, Deng, N. Thierry, A. Khan, U. Jan, X. Shen, W. Rui, et al., "Deep learning for medication recommendation: a systematic survey." Data Intelligence, vol. 5, no. 2. pp. 303-354, 2023.
[2] H. Sun, S. Xie, S. Li, Y. Chen, J.-R. Wen, and R. Yan, "Debiased, longitudinal and coordinated drug recommendation through multi-visit clinic records," Advances in Neural Information Processing Systems, vol. 35, pp. 27837-27849, 2022.
[3] C. Yang, C. Xiao, F. Ma, L.. Glass, and J. Sun, "Safedrug: Dual molecular graph encoders for recommending effective and safe drug combinations," in 30th International Joint Conference on Artificial Intelligence, IJCAI 2021, pp. 3735-3741, International Joint Conferences Artificial Intelligence, 2021.
[4] R. Wu, Z. Qiu, Jiang, G. Qi. and X. Wu, "Conditional generation net for medication recom- mendation," in Proceedings of the ACM web conference 2022, pp. 935-945, 2022.
[5] Z. Zhao, Y. Jing, F. Feng, J. Wu, C. Gao, and X. He, "Leave no patient behind: Enhancing medication recommendation for rare disease patients," in Proceedings the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 533-542, 2024.
[6] Y. Zhang. R. Chen, J. Tang, W. F. Stewart, and Sun, "Leap: learning to prescribe effective and safe treatment combinations for multimorbidity." in proceedings of the 23rd ACM SIGKDD international conference knowledge Discovery and data Mining, pp. 1315-1324, 2017.
[7] J. Shang, C. Xiao, T. Ma, H. Li, and J. Sun, "Gamenet: Graph augmented memory networks for recommending medication combination." in proceedings of the AAAI Conference on Artificial
Intelligence, vol. 33, pp. 1126 1 1133, 2019.
[8] N. Yang, K. Zeng, Q. Wu, and Yan, "Molerec: Combinatorial drug recommendation with substructure-awwad molecular representation learning," in Proceedings of the ACM web confer- ence 2023, pp. 4075-4085, 2023.
[9] K. Singhal, S. Azizi, T. Tu, S. S. Mahdavi, J. Wei, H. W. Chung, N. Scales, A. Tanwani, H. Cole-Lewis, S. Pfohl, el al., "Large language models encode clinical knowledge," Nature, vol. 620, no. 7972, pp. 172-180, 2023.
[10] Z. Zhao, C. Fan, C. Gao, F. Feng, and X. He, "Addressing overprescribing challenges: Fine-tuning large language models for medication recommendation tasks." arXiv preprint arXiv:2503.00 03687, 2025.
[11] P. Tatonetti, P. Ye, R. Daneshjou, and R. B. Altman, "Data-driven prediction of drug effects and interactions." " Science Iranslational medicine, vol. 4, no. 125, pp. 125ra31-125raá1, 2012.
[12] Z. Shao, P. Wang, Q Zhu, R. Xu, J. Song, X. Bi, H. Zhang, M. Zhang, Y. Li, Wu, et al., "Deepseekmath: Pushing the limits of mathematical reasoning in open language models," arXiv preprint arXiv:2402.088 033000 2024.
[13] A. K. Gururajan, E. Lopez-Cuena, J. Bayarri-Planas, A. Tormos, D. Hinjos, P. Bernabeu-Perez. A. Arias-Duart, P. A. Martin-Torre L. Prcelay-Ganzabal M. Gonzalez Mallo, et al., "Aloe: A family of fine-tuned open healthcare llms," arXiv preprint arXiv:2405. 01886, 2024.
[14] A. E. Johnson, T. J. Pollard, L. Shen, L.-w. H. Lehman, M. Feng, M. Ghassemi, B. Moody, Szolovits, L. Anthony Celi, and R. G. Mark, "Mimic-iii, a freely accessible critical care database," Scientific data, vol. 3, no. 1, pp. 1-9, 2016.
[15] A. E. Johnson, L. Bulgarelli, L. Shen, A. Gayles, A. Shammout, S. Horng, T. Pollard, S. Hao. B. Moody, B. Gow, et al., "Mimic-iv, a freely accessible electronic health record dataset," Scientific data, vol. 10, no. 1, p. 2023.

---

[16] T. J. Pollard, A. E. Johnson, J. D. Raffa, L. A. Celi, R. G. Mark, and O. Badawi, "The eicu collaborative research database, freely available multi-center database for critical care research," Scientific data, vol. 5, no. 1, pp. 1-13, 2018.
[17] J. Tan, Y. Rong. K. Zhao, T. Bian, T. Xu, J. Huang, H. Cheng, and H. Meng, "Natural language-assisted multi-modal medication recommendation," in Proceedings of the 33rd ACM International Conference on Information and Knowledge Management, pp. 2200-2209, 2024.
[18] C. Gao, R. Chen, S. Yuan, K. Huang. Y. Yu, and X. He, "Sprec: Self-play to debias Imsed recommendation." in Proceedings of the ACM on Web Conference 2025, pp. 5075-5084, 2025.
[19] C. Gao, S. Wang, S. Li, J. Chen, X. He, W. Lei, B. Li, Y. Zhang, and P. Jiang, "Cirs: Burst- ing filter bubbles by counterfactual interactive recommender system," ACM Transactions on Information Systems, vol. 42, no. 1, pp. 2023.
[20] S. Lin, C. Gao, J. Chen, S. Zhou, B. Hu, Y. Feng. C. Chen, and C. Wang, "How do recommenda- tion models amplify popularity bias? an analysis from the spectral perspective." " in Proceedings of the Eighteenth ACM International Conference on Web Search and Data Mining, WSDM '25, p. 659-668, 2025.
[21] J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov, "Proximal policy optimization algorithms," arXiv preprint (70,006777..3377, 2017.
[22] R. Rafailov, A. Sharma, E. Mitchell, C. D. Manning. S. Ermon, and C. Finn, "Direct preference optimization: Your language model is secretly reward model," Advances in Neural Information Processing Systems, vol. 36, pp. 53721- 2023.
[23] Q. Yu, Z. Zhang, R. Zhu, Yuan, X. Zuo, Yue, W. Dai, T. Fan, G. Liu, Liu, et al., "Dapo: An open-source Ilm reinforcement learning system at scale," arXiv preprint arXiv:2503. 14476, 2025.
[24] Zhang, J. Huang, H. Yao, S. Liu, X. Zhang, S. Lu, and D. Tao, "RI-vl: Learning to reason with multimodal large language models via step-wise group relative policy optimization," arXiv preprint arXiv:2503.1 12937, 2025.
[25] M. R. Cowie, J. I. Blomster, L. H. Curtis, S. Duclaux, I. Ford, F. Fritz, S. Goldman, S. Janmo- hamed, J. Kreuzer, M. Leenay, et al., "Electronic health records to facilitate clinical research,"
Clinical Research in Cardiology, vol. 106, pp. 1-9, 1-9, 2017.
[26] C. Gao, M. Gao, C. Fan, S. Yuan, W. Shi, and X. He. "Process-supervieeed I1m recommenders via flow-guided tuning." " in Proceedings of the 48th International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 1934-1943, 2025.
[27] A. Y. Ng. D. Harada, and S. Russell, "Policy invariance under reward transformations: Theory and application to reward shaping." " in JCML, pp. 278 278-287, Morgan Kaufmann, 1999.
[28] R. Rafailov, J. Hejna, R. Park, and C. Finn, "From to q": Your language model is secretly
q q-function," arXiv preprint arXiv:2404. 12358, 2024.
[29] Y. Zhang, F. Feng, Zhang, K. Bao, Q. Wang, and X. He, "Collm: Integrating collaborative embeddings into large language models for recommendation," IEEE Transactions on Knowledge and Data Engineering, 2025.
[30] Liao, S. Li, Yang. Wu, Y. Yuan, X. Wang, and X. He, "Llara: Large language- recommendation assistant," in Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 1785-1795, 2024.
[31] C. Yang, C. Xiao, L. Glass, and Sun, "Change matters: Medication change prediction with recurrent residual networks," arXiv preprint arXiv: 2XX::2/05.00 105.01876, 2021.
[32] J. Xia, C. Zhao, B. Hu, Z. Gao, C. Tan, Y. Liu, S. Li, and S. Z. Li, "Mole-bert: Rethinking pre-training graph neural networks for molecules," 2023.
12

---

A Theoretical Analysis
Proof. We prove the theorem using the reward shaping framework introduced by [27] which shows that augmenting reward function with a potential shaping term does not alter optimal policy.
Lemma A.1 (Ng al., 1999). Let rst,, ae) and (st, 0i) be two reward functtins. If there exists potential function (s) such that:
(st, a+) r(st, ax) (8(4+)) ((8)),
(17)
where y is the discount then the optimal policies under and r' are identical.
Now, consider the outcome-based reward:
if is terminal,
Jerminal
0, otherwise.
We define the shaped step-wise reward as:
step r ie
(sstep(()) - ((tte((t) - 1).
Let us define potential function ((1) = (p(ttp(()) Then the step-wise reward can be rewritten as:
ri (tt) +(t 1).
Summing over the episode:
T
Σ
stp Tie
4(T) - = (().)) (00),
assuming step(T) = J, Since (0)) constant across all trajectories, does not affect the opti-
mization. Thus, both reward schemes only differr by constant shift and a potential-base shaping term.
By the lemma, such transformation does not change the optimal policy.
Conclusion: The outcome- based terminal reward and the shaped step-wise reward are equivalent in terms of the policies they induce.
B Dataset Details
B.1 Data Preprocessing
We follow the data preprocessing procedures adopted in prior work [10] using structured and unstructured EHR data from MIMIC-III (14] database. For structured clinical information, we extract diagnosis and procedure codes and retrieve their corresponding textual descriptions from the ICD dictionary tables. Since the original long titles in the dictionary can be overly verbose and difficult for LLMs to process, we utilize GPT-4o to compress them into concise titles while preserving the essential semantic content. This compression strategy follows the same approach as adopted in LAMO [00 to ensure semantic integrity and model compatibility.
For medication data, we map National Drug Codes to DrugBank IDs and obtain associated text descriptions from corresponding lookup tables. To incorporate unstructured information, we follow the previous pipeline and extract segments from clinical notes, including History of Present Illness, Past Medical History. Allergies, and Medications on Admission, using GPT-4o as a parser to construct patient condition descriptions. A representative example of unstructured text extracted from clinical notes is shown below:
13

---

Example of unstructured extracted from clinical notes
History of Present Illness: The patient is 66-year-ol rear-old female with history of breast cancer and recent ascites and pelvic mass. She presented with worsening abdominal discomfort, poor oral intake, and dyspnea.
Past Medical History: The patient has a history of breast cancer, hypertension, hypothyroidism, tubal ligation, and previous metacarpal fracture.
Allergies: [codeine].
Medications on Admission: [Lisinopril, Effexor, Levoxyl, Tamoxifen, Fosamax].
B.2 Dataset Statisties
After preprocessing the structured clinical codes (diagnoses, procedures, and medications), we obtain
the dataset statisties summarized Table
Table 4: Statistics of processed data in MIMIC-II
Items
MIMIC-III
of visits / # of patients
14207 6226
dis. / /prod. / med. space size
1676 / 511 / 151
avg. max # of visits
2.28
avg. /mx # of dis. per visit
13.59 /
avg. / max # of pro. per visit
4.23 27
avg. / max # of med, per visit
23.36/77
C Experimental Setup
C.1 Training Details and Hyperparameters
We conduct all experiments NVIDIA A A000-SXM4-80GB GPUs, with Python 3. 3.10 and PyTorch 2. During all training stages, the model quantized using bf16 and LoRA, and optimized using the adamw_ torchtttrchhhohh optimizer.
For the SFT of drug-level classifier Tass we use Aaaeaa--Aooo--B--Appe as the base model. Four projectors are randomly initialized: pat projector, diag projector, pro pro_projector and med_ projecco,,, each consisting of two-layer MLP with GELU activation. The learning rate is set to 5e-4, with batch size of 128 and one epoch.
For the SFT of list-wise policy miss, the model and projector weights from the previous step (Tras) used as initialization. The learning rate remains 5e-4, with a batch size of 64 and one epoch.
When performing step-wise GRPO Tisst initialize the model and projector weights from the previous SFT step. The projector parameter 'r 'r_grad' is set to False We use the Unsloth framework and vLLM 0.7.3 for acceleration. The learning rate is set to le-5, with batch size of 16, 'num generateenggs's set to 8, and one epoch. The hyperparameter a chosen from the set [0, 2, 5, 10, 20, 30, 40, 50] to adapt different DDI requirements, while / is set 0.5, and À is set to 5.
C.2 Baseline Methods and Evaluation Metrics
We provide comprehensive overview of baseline models that have been widely used for the medication recommendation task:
LEAP 6 formulates the medication prediction task using only the current visit and models
label dependencies via multi-instance multi-label learning.
GAMENet 7 integrates EHR and DDI graphs via graph-augmented memory networks to ensure both accuracy and safety.
14

---

SafeDrug [
employs dual molecular encoders to learn molecular representations and improve safe prescription decisions.
COGNet leverages a copy-or-predles mechanism to incorporate historical prescriptions into current decision making.
MICRON 31 introduces a recurrent residual learning model that treats medication change dynamics as the primary learning target.
aware attention mechanism that models drug substruc- ture-patient condition interactions at fine-graines level.
MoleRee proposes a substructure-aves -warre
RAREMed [ utilizes Transforrer-basee encoder derive comprehensive patient representations and mitigate fairness issues recommendation.
NLA-MMR designs a multi-modal alignment framework to jointly encode patient and medication representations across modalities.
LAMO 10 builds LLaMA-2 and incorporates unstructured clinical notes better capture the patient's 's condition, leveraging the semantic understanding capabilities of LLMs for accurate medication recommendation.
We adopt Jaccard Score and Score evaluate the prediction accuracy, and DDI rate assess the safety of recommended medication sets. Given the model prediction M, and the ground-truth set Mar, we define:
Jaccard Score:
MM n Marl
J.
(18)
MM M.r
Fl Score, computed based on precision and recall: Precision
[M Marl
[, MaT
2. Precision Recall
Recall
Fl
[Mu]
[Marl
Precision Recall
(19)
DDI rate measures the ratio harmful drug-drup drug interactions among predicted medications:
K(d, d,) |d, d2 E M,, i i Did, d,1 = 1
DDI(M,.
(20)
(..))
C.3 Prompt Templates
In Section 4.3 we propose two-stage recommendation framework. which consists of: (I) drug-level
filtering and (2) list-level refinement. Below we provide the prompt templates used in each stage.
For the drug-level classifier (mas), formulate binary classification task over each candidate drug m E .M the set of drug candidates. Each drug is evaluated independently based on its appropriateness for the given patient. The model is prompted with the patient's clinical representation and a textual description of the candidate drug. Specifically, {[Patient Representation) includes both patient's current status and previous hospitalization records. {(Drug Candidate)} refers to the textual representation of single drug under evaluation.
Prompt Template for Drug level Filtering
You are about to evaluate a candidate drug for patient's clinical condition. You will be provided with the patient's current condition, as well as information about their previous hospitalization, and the candidate drug. Your task is determine whether the candidate drug is effective and safe for the patient. Please respond with < <Yes.> or <No.> without providing an explanation.
{[Patient Representationn)
Candidate drug: {(Drug Candidates)}
#HH Answer:
15

---

For the list-level refinemet policy (Tia.). we further optimize the drug set Mp produced by the drug-level classifier by modeling dependencies among drugs and refining the list through two complementary tasks: (1) adding missing yet clinically necessary drugs, and (2) removing drugs that are unnecessary or potentially unsafe for the patient. Each task is formulated as separate prompt, where the model is asked to revise pre-predicted drug list based the patient's condition and medical history.
The task instruction is injected into the prompt through the {(instruction)) placeholder, with the following two variants:
Add task: add any relevant drugs that are missing from the pre-predicted list.
Remove task: remove any drugs from the pre-predicted list that are unnecessary or unsuit- able for the patient's condition.
The unified prompt template used in both tasks is shown below:
Prompt Template for List-level Refinement
You are tasked with refining drug list for patient's clinical condition. You will be provided with the patient's condition, information about their previous hospitalization and pre- predicted drug list.
Your task is to: [(instruction)} Please output list of drugs that need to be {fadded or removed)), with each drug separated by newline.
{[Patient Representation))
###Pre-predicted Drug List: {(M,,))
###Druugs to {{Add or Remove)):
To enable comprehensive modeling of the patient's condition within each task prompt, we incorporate both current and historical clinical information into the ((Patient Representation') field when- ever available. Specifically, we differentiate between patients with and without prior hospitalization records. The formatting is as follows:
Prompt Template for Patient Representation
For patients with previous hospitalization records:
### Previous Hospitalization: {[Previous Representation))
### Current Clinical Condition: {[Current Representation))
For patients without previous hospitalization records:
### Current Clinical Condition: {[Current Representation))
For the patient's previous hospitalization representation {[Previous Representation)), we construct hybrid representation that integrates structured information from diagnoses, procedures, and medications, along with demographic data.
Prompt Template for Previous Representation
Age: {{Age of the patient, eg. 68}}.
Gender: ([Patient's biological sex, e.g., Male}},
Diagnoses: [[Hybrid diagnosis ropresentation)),
Procedures: {[Hybrid procedures representation)), ((Hybrid drugs representation)).
Drug names:
Here we further describe the format the {(Current Representation)), which captures the patient's clinical status at admission. This representation integrates both structured data- as diagnoses and procedures-aae unstructured information extracted from clinical notes, including the history of present illness, past medical history, allergies, and medications admission.
16

---

Prompt Template for Current Representation
Patient representation: {Lateent embedding)),
History of present illness: [(Unstructured text extracted from clinical notes)) {(Unstructured text extracted from clinical notes)), Allergies: KUUntrrucured text extracted from clinical notes)),
Past medical history:
Medications on admission: {(Unstructured text extracted from clinical notes)),
Diagnoses: [[Hybrid diagnosis representation)),
Procedures:
{[Hybrid procedures representation));
As introduced in Section our model leverages hybrid representations encode structured EHR bodds-namelyy diagnoses, procedures, and medications- combining the textual representations generated by LLM with collaborative embeddings produced by domain-speciff encoders. These hy- brid representations correspond to the placeholder fields Hiiiss- diagnosis representat on, Hybrid procedures representation, and Hybrid drugs representation- previouuly in- troduced our prompt templates. Here, we provide concrete examples of how each field instanti- ated.
Specifically, for each structured medical code, we first obtain a textual description suitable for LLM input. In parallel, apply domain-specinn encoder to the original structured code to extract collaborative embedding, which captures latent clinical semantics. This collaborative embedding is then projected into the LLM embedding space using learnable adapter, rather than being passed through the LLM's standard tokenizer and embedding layer. The final hybrid representation is thus composed of both the textual token sequence and its aligned embedding enhancement, More details of this embedding alignment can be found in Section 4.4.
The following illustrates the prompt-level instantiation of hybrid representations:
Prompt-level Instantiation of Hybrid Representations
Diagnoses: Hematochezia [DiagEmb-11] Acute Kidney Injury [DiagEmb-56), Heart Failure (DiagEmb-104]]
Procedures: Coronary Artery Stenting [ProEmb-8]. Three Vessel Procedure [ProEmb-79]. Stent Insertion [ProEmb-226]...
Drug names: Calcium gluconate [DrugEmb-2], Captopril [DrugEmb-31], Magnesium
sulfate [DrugEmb-3]..
D Limitations
While FLAME shows promising results, it also has several limitations that warrant further investiga- tion:
Computational cost and deployment challenges: The use of LLM introduces significant training and inference overhead. This can limit the practical deployment of the system in resource-constrainee clinical settings, especially in real-time or on-device applications.
Lack of automated multi-modal data integration: Although our framework incorporates both structured and unstructured patient data, it currently lacks an end-to-end automated mechanism for multi-modal data processing. In particular, the handling of unstructured patient narratives relies on manually invoking external APIs (e.g., GPT-4o), which limits scalability and consistency.
Static evaluation setting: Our model trained and evaluated on static EHR datasets, which do not fully capture the dynamic nature of real-world clinical workflows. Bridging this gap requires further investigation into how such models can adapted for continual learning or integration with live clinical decision support systems.
17

---

E
Broader Impacts
This work presents the potential for significant positive societal impact by advancing the integration of LLMs into clinical decision- decision-making. By providing accurate, context-aware, and safe medication rec- ommendations, our framework supports physicians rather than replacing them, offering interpretable suggestions that can complement and enhance clinical judgment. This coman-in--th-Baee reinforces trust and accountability in medical Al systems, ensuring that final decisions remain under professional oversight. Furthermore, the model's ability to handle both structured clinical data and unstructured frre-text inputs aligns naturally with real- wolldwrrld clinical workflows, where critical patient information often resides in narrative form. This flexibilityy reduces the burden of data curation and enables more seamless adoption in practice. addition, the model's demonstrated generalizability across time and institutional boundaries opens avenues for deployment in under-resourced settings, where access to experienced clinicians and high-quality clinical decision support limited. By bridging this gap. work contributes toward improving the equity and reach of healthcare services. Ultimately, we envision this research as step toward scalable, adaptive, and collaborative Al systems that respect clinical complexity while expanding access to safe and effective medical support across diverse populations.
design
However, despite its potential benefits, the deployment of FLAME also poses certain societal risks. First, the model's performance may be affected by biases the training data, which could lead to disparities in recommendation quality across different patient populations, particularly those underrepresented in historical records. Second, there is a risk of misuse if such systems are deployed without proper clinical oversight-for example, being used in unauthorized settings for non- medical purposes, potentially leading to harmful or misleading recommendations. These concerns underscore the need for responsible deployment practices, transparency, and safeguards to ensure that FLAME is used ethically and equitably in real- world clinical environments.
18

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: The abstract and introduction clearly state the key contributions of the paper, including the introduction of list-wise decision framework (FLAME), the design of step-wise GRPO for fine aene-grained reward optimization, and the integration of multi-level medical knowledge. These are all reflected in the method and validated through extensive experiments, matching both the theoretical and empirical scope of the work.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings. It is fine to include aspirational goals as motivation long it is clear that these goals are not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations the work performed by the authors?
Answer:
[Yes]
Justification: Provided in the appendix.
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g.. independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or speech-to-text system might not be used reliably provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairesss.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't acknowledged
the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
19

---

Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
Justification: The paper provides Theorem I with all necessary assumptions explicitly stated, and its proof is included in Appendix A.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: We provide the code to reproduce the experimental results, and include detailed descriptions of hyperparameter settings and prompts in the appendix ensure reproducibility.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken make their results reproducible verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to model. In general. releasing code and data often one good way to accomplish this, but reproducibilityy can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of large language model), releasing of a model checkpoint, other means that are appropriate to the research performed.
specific model and empirical evaluation, may
While NeurlPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm. (b) If the contribution is primarily the architecture clearly and fully.
model architecture, the paper should describe
(c) the contribution is a new model (c.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open- open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibiliity In the case of closed-source models, it may be that access to the model is limited in
20

---

some way (e.g., to registered users), but it should be possible for other researchers have some path reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification: We provide the code for reproducing experimental results, including scripts for processing the EHR datasets. Detailed instructions are included in the supplemental material to ensure faithful reproduction.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips.co cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https
wwtpps.ccpuullc//uiiieee guwdes/ CodeeSumiia ssi onyolicy) for more details.
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data,
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Question:
Answer: [Yes]
Justification: Full details provided in the appendix ensure clarity and reproducibility.
Guidelines:
The answer NA means that the paper does include experiments.
The experimental setting should be presented in the core the paper to a level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [No]
Justification: We did not perform statistical significance tests due to compute and time constraints, but ablation studies support the validity of our key findings.
Guidelines:
The answer NA means that the paper does not include experiments.
21

---

The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall
with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
should be clear whether the error bar is the standard deviation the standard error of the mean.
It is OK to report 1-sigma error bars, but should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful figures symmetric error bars that would yield results that are out of range (e.g. negative eтror rates).
show in tables or
error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
Justification: Provided in the appendix.
Guidelines:
The answer NA means that the paper does include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental rums as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (c.g., preliminary or failed experiments that didn't make it into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics lattps://neuripp:: pttp:://nourrss.cc/pubb///////t
Answer: [Yes]
Justification:
Fully compliant with the NeurlPS Code of Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special cireumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., if there is special consid- eration due to laws or regulations their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: We briefly mention the societal impact the conclusion and provide a more detailed discussion in the appendix.
22

---

Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied partieular applications, let alone deployments. However, if there is direct path any negative applications, the authors should point it out. For example, it is legitimate to point that improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of MI..
11. Safeguards
Does the paper deseribe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: Our work does not release high-risk data or model artifacts.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual- should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification:
We cite the dataset and respect its usage license and terms.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset used and, possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
23

---

For scraped data from a particular source (e.g., website). the copyright and terms of service of that souree should be provided.
If assets are released, the license, copyright information, and terms use in the package should be provided. For popular datasets, papprswipphaeeoooccaaappp thcode. com/ com/dataseta has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that re-packaged, both the original license and the license of the derived asset (if has changed) should be provided. If this information is not available online, the authors are encouraged to reach out to the asset's ' creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification:
Code is released with documentation and anonymized link.
Guidelines:
The answer NA means that the paper does release new assets.
Researchers should communicate the details the dataset/code/moodll as part their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: This paper does not involve crowdsourcing or any research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: This study does not involve crowdsoureing or research with human subjects, and therefore does not require IRB or equivalent approval.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
24

---

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if' applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Does the paper describe the usage of LLMs if is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, originality of the research, declaration is not required.
Question:
Answer: [Yes]
Justification: The LLM is an essential part of our core methodology and is thoroughly detailed in the paper.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, non-standard components.
Please refer to our LLM policy (https://neurl)) http:://eeerrrss.cc/oofeeeeeeeee /ct////oefeeeeneee/22////.. for what should or should not be described.
25


Reviews:
[{"review_id": "D6B6JW9DvO", "reviewer": "Reviewer_bQc3", "summary": "Existing medication recommendation methods follow point-wise paradigms that overlook synergistic effects and adverse drug-drug interactions (DDIs). This paper proposes FLAME, a list-wise, step-wise approach that applies Group Relative Policy Optimization (GRPO) with potential-based reward shaping. At each step, the model adds or removes a drug, and the advantage is computed using a potential function based on correctness, safety (DDI risk), and constraint adherence. Final recommendations are generated in two stages: an initial drug-level relevance classification, followed by list-wise refinement using the proposed step-wise GRPO. The model also incorporates multi-source knowledge by combining structured clinical features and collaborative embeddings with LLM representations. Experiments on MIMIC-III, MIMIC-IV, and eICU show that FLAME outperforms prior methods in recommendation accuracy while achieving better safety-control trade-offs.", "questions": "1. How is the candidate set Mc for \"adherence\" (refusal rate) measurement constructed?\n2. How is the drug-level classifier trained?\n3. Could improved tuning of the drug-level classifier alone achieve correctness comparable to the full FLAME model? What is the safety score of the classifier?\n\nGeneral questions:\n1. Can other risk factors, like patient-specific allergies or preferences to certain medications, be encoded in this framework?\n2. Could the ground truth labels based on historical prescriptions be affected by incorrect or suboptimal recommendations? If so, could incorporating external medical knowledge or expert validation help refine the training signal and improve model performance?", "limitations": "Yes", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 4, "originality": 4, "strengths_and_weaknesses": "Strengths:\n* The proposed method is both novel and well-suited to the clinical medication recommendation domain.\n* The authors compare against several strong existing baselines, and their proposed method outperforms them in both correctness and safety.\n\nWeaknesses:\n* The experiments indicate a trade-off between safety and correctness, but the paper does not explore the underlying cause of this behavior. For example, does prioritizing safety lead the model to make more conservative recommendations, excluding clinically relevant drugs?\n* Some experimental details are missing in the paper, such as the construction of the candidate set Mc and the training of the drug-level classifier. \n* From the ablation study (Table 2, rows a-b), it appears that much of the performance gain comes from the inclusion of the point-wise drug-level classification step. This raises the question of whether the GRPO-based list-wise refinement is truly necessary, or if similar improvements could be achieved through better tuning of the classifier alone. Reporting the safety score of the classifier can help clarify the added value of the GRPO step.\n* Additional clarifications requested in the \"Questions\" section.", "was_revised": true, "final_justification": "The authors have satisfactorily addressed my concerns, so I am raising my score to 5.\nThe clarification provided for point 3 is particularly important for understanding the paper’s main contribution. I strongly encourage the authors to include qualitative examples where point-wise methods fail to capture drug–drug interactions that list-wise methods successfully identify, even if they are from a dataset.", "rebuttal": "We sincerely thank the reviewer for their thorough assessment, constructive feedback, and recognition of our contributions. Below, we address each point of concern in detail and clarify the questions raised.  \n\n---\n\n### 1. On the Safety–Correctness Trade-off\n\n>  **W1**: The experiments indicate a trade-off between safety and correctness, but the paper does not explore the underlying cause of this behavior. For example, does prioritizing safety lead the model to make more conservative recommendations, excluding clinically relevant drugs?\n\nThank you for this insightful question. \nThe observed trade-off between safety and correctness primarily stems from the nature of real-world clinical data. Because datasets such as MIMIC‑III are derived from historical hospital records, the ground‑truth prescriptions inevitably include a non‑negligible proportion of drug combinations with known DDIs (e.g., a DDI rate of 13.69% in MIMIC‑III). Models trained purely to fit such data tend to reproduce these unsafe combinations, which explains why optimizing for safety often reduces label‑matching accuracy.\n\nFrom a clinical perspective, this phenomenon is also understandable. Certain drug pairs are intentionally co‑prescribed despite known interactions, as their combined effects can be therapeutically beneficial. A typical example is the Glycopyrronium–Neostigmine combination, which is widely used for neuromuscular blockade reversal to mitigate cardiovascular and secretory adverse effects; however, the combination itself can lead to anticholinergic side effects. As safety constraints are increased, FLAME may revise such co‑prescriptions by removing one of the interacting drugs, prioritizing safety over fidelity to historical labels. In addition, real‑world records inevitably contain variability and occasional inconsistencies in prescribing, further contributing to the correctness–safety discrepancy.\n\nTo illustrate this behavior, consider a patient with a history of aortic and tricuspid valve replacements (ID: 138212). Both Glycopyrronium and Neostigmine were co‑prescribed in the hospital record. When optimizing for accuracy, FLAME successfully recommends both drugs; when safety is prioritized, FLAME removes Glycopyrronium to reduce the DDI risk, leading to slightly lower label‑matching accuracy. This example demonstrates that the safety–correctness trade‑off is not merely a modeling artifact but reflects genuine clinical decision‑making dilemmas, while also highlighting FLAME’s ability to flexibly balance these competing objectives.\n\n---\n\n### 2. Construction of the Candidate Set *Mc* and Training of the Drug-level Classifier\n\n> **W2**: Some experimental details are missing in the paper, such as the construction of the candidate set Mc and the training of the drug-level classifier.\n>\n> **Q1**: How is the candidate set Mc for \"adherence\" (refusal rate) measurement constructed?\n>\n> **Q2**: How is the drug-level classifier trained?\n\n**Candidate Set *Mc*:**  \n*Mc* is all unique medications in the processed training data, ensuring clinically relevant drugs seen in training are available for inference while avoiding unseen drugs. Mc is also used for adherence (refusal‑rate) measurement in Eq. (8).\n\n**Drug-level Classifier Training:**  \nπ_cls is initialized from Llama3.1‑Aloe‑Beta‑8B and fine‑tuned with supervised learning. For each visit, every drug in Mc is paired with the patient context and labeled “Yes.” if it appears in the ground truth, otherwise “No.” Details are in Appendix C.1.\n\n---\n\n### 3. Necessity and Value of List-wise Refinement\n\n> **W3**: From the ablation study (Table 2, rows a-b), it appears that much of the performance gain comes from the inclusion of the point-wise drug-level classification step. This raises the question of whether the GRPO-based list-wise refinement is truly necessary, or if similar improvements could be achieved through better tuning of the classifier alone. Reporting the safety score of the classifier can help clarify the added value of the GRPO step.\n>\n> **Q3**: Could improved tuning of the drug-level classifier alone achieve correctness comparable to the full FLAME model? What is the safety score of the classifier?\n\nThank you for raising this important question.\n\n**To directly answer Q3:**\n(1) Merely improving the point‑wise classifier cannot achieve correctness or safety comparable to the full FLAME model, because it lacks the ability to model drug‑drug dependencies or optimize for safety trade‑offs.\n(2) On the test set, the classifier alone achieves Jaccard = 0.4785, F1 = 0.6354, and DDI = 0.1336 (close to the ground‑truth DDI = 0.1369), confirming that it essentially reproduces the dataset distribution without controllable safety adjustment.\n\nWe elaborate below on the distinct role and added value of the GRPO‑based list‑wise refinement step:\n\n- **Modeling Drug Dependencies (Necessity):**\nThe classifier makes independent decisions for each drug, missing co‑prescription patterns or redundancies. For example, Piperacillin is a broad‑spectrum β‑lactam penicillin used for severe infections, sharing overlapping indications with Cefepime, Meropenem, and Ampicillin. In a point‑wise setting, where each drug is predicted independently, the classifier (and baselines) cannot leverage signals from similar drugs, often leading to redundant recommendations or omission of appropriate alternatives. FLAME’s list‑wise policy explicitly edits the entire drug list, correcting such issues. On the test set, it performed 28 add/remove operations on Piperacillin‑containing lists with an 89% correctness rate, showing that it meaningfully rectifies classifier limitations.\n\n- **Enabling Safety–Accuracy Trade‑off (Necessity):**\nBecause π_cls is trained only to mimic ground truth, its DDI rate remains near the dataset’s (0.1336 vs. 0.1369). It cannot adapt to stricter safety constraints. The list‑wise policy, by contrast, optimizes for both correctness and safety via step‑wise GRPO, allowing explicit control of the trade‑off during inference.\n\n- **Why the Performance Gain Appears Modest:**\nThe refinement step is intentionally designed as a **post‑processing global editor**, making an average of 2.03 add/remove operations on lists of average length 20.83. These small but targeted changes resolve drug‑drug dependencies and redundancies that a point‑wise classifier cannot capture. The resulting improvements in both correctness and safety are therefore **qualitatively crucial** even if the overall accuracy gain appears numerically smaller.\n\nIn summary, the GRPO‑based list‑wise refinement is indispensable for capturing drug interactions and enabling controllable safety optimization—capabilities that cannot be achieved by tuning the classifier alone.\n\n---\n\n### 4. Encoding Other Risk Factors (e.g., Allergies, Preferences)\n\n> **General Q1**: Can other risk factors, like patient-specific allergies or preferences to certain medications, be encoded in this framework?\n\nYes, FLAME naturally supports encoding patient‑specific risk factors such as allergies and medication preferences.\n\nUnstructured clinical notes—including the history of present illness, past medical history, allergies, and medications on admission—are incorporated into the patient representation (see Fig. 4). Allergies are explicitly included as natural‑language fields (e.g., “Allergies: [Aspirin, Ibuprofen]”), enabling the model to avoid contraindicated drugs. Likewise, patient‑specific preferences or conditions (e.g., “The patient was anticoagulated with heparin and later switched to enoxaparin”) are encoded in free‑text form, allowing the model to infer suitable or preferred alternatives.\n\nThis design is a key strength of FLAME: unlike models restricted to structured codes, it can exploit rich, context‑dependent information and pass it to the list‑wise policy, enabling more personalized and clinically appropriate recommendations.\n\n---\n\n### 5. Ground Truth Limitations and Incorporation of External Knowledge\n\n> **General Q2**: Could the ground truth labels based on historical prescriptions be affected by incorrect or suboptimal recommendations? If so, could incorporating external medical knowledge or expert validation help refine the training signal and improve model performance?\n\nYes, ground‑truth prescriptions in real‑world data can include suboptimal or even erroneous decisions, which may bias both training and evaluation.\n\nTo mitigate this, FLAME already incorporates multi‑source external knowledge at several levels (Section 4.4, Fig. 4):\n\n- **Patient‑level:** contextual health representations from pretrained clinical models.\n\n- **Diagnosis & procedure‑level:** co‑occurrence information capturing real‑world treatment patterns.\n\n- **Medication‑level:** molecular substructure features that reflect pharmacological properties.\n\nThese signals are fused into the LLM embedding space via learnable adapters, allowing FLAME to reason beyond historical prescriptions alone and reduce dependence on potentially biased labels. Our ablation study confirms that each knowledge source contributes to improved accuracy:\n\n| Variant         | Jaccard | F1     |\n|-----------------|---------|--------|\n| w/o patient‑level enternal embedding    | 0.4693  | 0.6264 |\n| w/o diagnosis‑level enternal embedding  | 0.4779  | 0.6350 |\n| w/o procedure‑level enternal embedding  | 0.4814  | 0.6385 |\n| w/o medication‑level enternal embedding | 0.4784  | 0.6350 |\n| **FLAME**       | 0.4836  | 0.6408 |\n\nLooking forward, expert‑in‑the‑loop validation and integration of clinical guidelines or curated knowledge bases are promising directions to further refine the training signal and enhance reliability. These extensions are fully compatible with FLAME’s design and will be explored in future work.\n\n---\n\nWe hope these clarifications address the reviewer’s concerns. We are grateful for the constructive feedback, which helps us further refine and communicate our contributions."}, {"review_id": "sdj4elFo8N", "reviewer": "Reviewer_zoHJ", "summary": "The paper introduces FLAME, a novel list-wise medication recommendation framework that reformulates drug recommendation as a sequential decision-making process. FLAME also integrates multi-source clinical knowledge—structured codes, unstructured notes, and semantic labels. Extensive experiments demonstrate state-of-the-art accuracy and safety control on real-world EHR datasets.", "questions": "1. In real life there may exists drugs whose interation effects are unknown, and this may result in noisy DDI graph ***D***. How would FLAME performs on noisy DDI graph? \n2. The numbers under \"Evaluate each step\" and \"Step-wise advantages\" in Figure 1 are unclear to me. Could author provide more information on how these numbers are associated?", "limitations": "Yes", "rating": 4, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 2, "originality": 2, "strengths_and_weaknesses": "**Strength:**\n1. The overall theoretical analysis is well-organized\n2. Detailed architecture, datasets, and hyperparameters; appendix lists detailed prompt templates.\n3. Comprehensive ablations isolate the effect of different model variants\n\n**Weakness:**\n1. No error bars or standard deviation included. In particular, Table 2 reports several Jaccard/F1 scores that are very close (e.g., FLAME vs. variant (e) with standard GRPO).\n2. Although limitations about computational expense are mentioned in the appendix, it would still be beneficial to include the walltime for both training and inference, especially for inference which can be used to measure feasibility for real-time settings.\n3. The ablation study comparing the use of \"standard GRPO\" versus \"Step-wise GRPO\" shows no significant performance gap (i.e. <0.5% increase). This diminishes the practical significance of the proposed approach.", "was_revised": true, "final_justification": "The authors' responses have addressed most of my concerns and problems, and I am therefore willing to raise my score to 4.", "rebuttal": "We thank the reviewer for the detailed feedback and constructive questions. Below, we address each weakness and question in turn.\n\n---\n\n### 1. On Error Bars and Standard Deviation in Table 2  \n\n> **W1**: No error bars or standard deviation included. In particular, Table 2 reports several Jaccard/F1 scores that are very close (e.g., FLAME vs. variant (e) with standard GRPO).\n\nWe acknowledge that Table 2 does not report error bars or standard deviations. We have performed additional runs for FLAME and variant (e) (standard GRPO) and now include the corresponding standard deviations, confirming that the improvements are consistent.\n\nThe small numerical gaps for variants (b)–(e) stem from the design of the **list‑wise policy as a lightweight refinement module**. On the test set, the classifier outputs lists with an average length of 20.83, while the list‑wise policy makes only ~2.03 add/remove edits. Because this stage is intended to make targeted adjustments to an already strong classifier output, the absolute improvements in Jaccard and F1 remain modest. However, this step plays a critical role:\n\n- **Modeling drug dependencies:** The classifier makes independent decisions, missing co‑prescription patterns or redundancies. For example, Piperacillin is a broad‑spectrum β‑lactam penicillin used for severe infections, sharing overlapping indications with Cefepime, Meropenem, and Ampicillin. In a point‑wise setting, where each drug is predicted independently, the classifier (and baselines) cannot leverage signals from similar drugs, often leading to redundant recommendations or omission of appropriate alternatives. The list‑wise policy explicitly edits entire lists, correcting such issues. It performed 28 add/remove operations on Piperacillin‑containing lists with an 89% correctness rate.\n\n- **Enabling safety–accuracy trade‑off:** Because π_cls is trained only to mimic ground truth, its DDI rate (0.1336) remains close to the dataset’s (0.1369). The list‑wise policy, optimized with step‑wise GRPO, allows explicit control over safety versus correctness.\n\nThus, the refinement step yields qualitative benefits that classifier tuning alone cannot achieve, even if its numerical improvements seem modest.\n\nStandard GRPO further limits the effectiveness of list‑wise refinement because it assigns identical rewards to all tokens in a generated list, lacking fine‑grained supervision. Our proposed **step‑wise GRPO** addresses this by computing per‑step rewards based on each drug’s contribution to the final outcome. As shown in Fig. 6, this design provides **denser feedback, accelerates learning, and improves stability**.  \n\nNotably, the improvement achieved by list‑wise refinement over point‑wise classification increases from 0.0029 (using standard GRPO) to 0.0052 (using step‑wise GRPO). Our additional experiments with standard deviations confirm that step‑wise GRPO yields consistently higher and more stable performance than standard GRPO，as shown in the following table:\n\n| Variant           | Jaccard | F1     |\n|-------------------|---------|--------|\n| with standard GRPO| 0.4814 ± 0.0003 | 0.6387 ± 0.0004 |\n| **FLAME**         | 0.4837 ± 0.0004 | 0.6409 ± 0.0004 |\n\n---\n\n### 2. On Computational Cost and Walltime  \n\n> **W2**: Although limitations about computational expense are mentioned in the appendix, it would still be beneficial to include the walltime for both training and inference, especially for inference which can be used to measure feasibility for real-time settings.\n\nThank you for the suggestion. We agree that reporting walltime is important to evaluate computational feasibility, especially for real‑time inference.  \n\nAll experiments were conducted on NVIDIA A100‑SXM4‑80GB GPUs. The measured training and inference walltimes are as follows:\n\n**Training walltime**\n\n| #Admissions | Classifier SFT       | List-wise Policy SFT       | List-wise Policy step-wise GRPO       |\n|-------------|----------------------|------------------|-------------------|\n| **8,741**   | ~200 GPU‑h (8×25 h)  | ~25 GPU‑h (8×3 h)| ~100 GPU‑h (8×16 h)|\n\n**Inference walltime**\n\n| #Admissions | Total GPU Hours | Per Patient (1 GPU) | Per Patient (8 GPUs) |\n|-------------|-----------------|----------------------|-----------------------|\n| **2,369**   | ~9 GPU‑h        | ~13.6 s             | ~1.7 s               |\n\nWith 8 GPUs in parallel, inference time per patient is **~1.7 s**, making FLAME feasible for near real‑time clinical use. We will include these tables in the revised version to better illustrate computational cost.\n\n---\n\n### 3. On the Small Performance Gap Between Standard and Step-wise GRPO  \n\n> **W3**: The ablation study comparing the use of \"standard GRPO\" versus \"Step-wise GRPO\" shows no significant performance gap (i.e. <0.5% increase). This diminishes the practical significance of the proposed approach.\n\nAs explained in our response to **W1**, the numerical gain from list‑wise refinement is modest because it is intentionally designed as a lightweight post‑classification editor, performing only a few targeted add/remove operations on an already strong classifier output. Consequently, the overall Jaccard/F1 change is small even when the refinement strategy is improved.\n\nNevertheless, step‑wise GRPO remains practically meaningful. Unlike standard GRPO, which assigns identical rewards to all tokens in a generated list, step‑wise GRPO provides denser and more discriminative per‑step rewards, directly reflecting each drug’s contribution to the final outcome. This leads to faster convergence and more stable training, as shown in Fig. 6.\n\nMoreover, the ablation study (variant (e)) confirms that step‑wise GRPO increases the improvement achieved by list‑wise refinement—from 0.0028 (using standard GRPO) to 0.0051—demonstrating its utility even within a post‑processing setup. The benefit of step‑wise GRPO is therefore best understood in terms of training efficiency and stability, rather than solely the final Jaccard/F1 difference.\n\n---\n\n### 4. On Performance with Noisy DDI Graphs  \n\n> **Q1**: In real life there may exists drugs whose interation effects are unknown, and this may result in noisy DDI graph D. How would FLAME performs on noisy DDI graph?\n\nIn our approach, the DDI graph is integrated into both the training phase (as a safety penalty within the reward function) and the evaluation metrics. As such, noise or missing interactions in the DDI graph would naturally influence both the policy learned by the model and the measured safety of the recommendations. If the DDI information is incomplete or noisy, the model’s optimization for safety may become less effective—specifically, at a fixed level of accuracy, the achievable reduction in DDI risk will be limited by the quality of the underlying DDI graph. For this reason, we did not conduct additional experiments with artificially noised DDI graphs on MIMIC-III, since this effect is direct and expected, and the static benchmarks themselves already reflect the limitations and incompleteness of real-world DDI knowledge.\n\n**FLAME currently improves recommendation performance through a hybrid representation strategy with multi‑source knowledge fusion.** As shown in Table 2 (variants (i) and (j)), incorporating external drug knowledge—such as molecular substructure–function embeddings and pretrained clinical representations—consistently enhances accuracy and safety, indicating that the model can leverage richer drug features beyond historical prescriptions.\n\n**Building on this design, such external knowledge could also help mitigate the impact of unknown drug interactions in noisy DDI graphs.** With a backbone LLM pretrained on large‑scale medical corpora, FLAME can combine pharmacological understanding with molecular structure information to improve robustness. Furthermore, a pretrain‑then‑finetune paradigm could be adopted: the model could first be pretrained with known DDI relations and drug knowledge to better understand interaction patterns, and then finetuned on the recommendation task. These strategies would make FLAME more resilient to incomplete or noisy DDI graphs in real‑world deployments.\n\n---\n\n### 5. Clarification of Figure 1 (“Evaluate Each Step” and “Step-wise Advantages”)  \n\n> **Q2**: The numbers under \"Evaluate each step\" and \"Step-wise advantages\" in Figure 1 are unclear to me. Could author provide more information on how these numbers are associated?\n\nFigure 1 visualizes how step‑wise GRPO differs from standard GRPO in assigning rewards to each step.  \n\n- **“Evaluate each step”**: After decomposing the model output (a drug list) into steps based on individual drugs (Eq. 3), we compute a per‑step reward using a potential‑based shaping function. The numbers under “Evaluate each step” represent the **reward value for each drug**, calculated as the difference in potential values before and after adding/removing that drug. These values reflect the quality of each step in terms of correctness, safety, and constraint adherence.  \n\n- **“Step‑wise advantages”**: These are obtained by **combining the per‑step reward above with the outcome‑based advantage from standard GRPO (Eq. 6)**. In other words, the numbers under “Step‑wise advantages” can be interpreted as  \n\n  $\\text{Step‑wise Advantage} = \\text{Per‑step Reward (from “Evaluate each step”)} + \\text{Outcome‑based Advantage}.$\n  \nThus, the figure shows that step‑wise GRPO augments the standard GRPO signal with additional per‑step information, enabling finer‑grained credit assignment for each drug in the list.\n\n---\n\n### 6. Summary  \nWe will (i) report standard deviations and add error bars, (ii) include walltime for training and inference, (iii) expand discussion on the added value of step-wise GRPO, (iv) discuss potential robustness strategies for noisy DDI graphs, and (v) clarify the annotations in Figure 1.\n\nWe thank the reviewer again for these suggestions, which will help us make the paper more thorough and practically informative."}, {"review_id": "0eZ4ewlyyh", "reviewer": "Reviewer_TAqr", "summary": "In this paper, the authors focus on the challenge of medication recommendation and aim to address it by leveraging the substantial capabilities of large language models (LLMs), particularly in scenarios where multiple drugs interact synergistically. The paper introduces a novel approach called FLAME to tackle the medication recommendation problem while considering drug-drug interactions (DDIs) through an LLM framework. The authors model this issue as a list-wise decision-making problem. With the assistance of step-wise GRPO, it becomes feasible to conceptualize the medication recommendation process as a sequential state transition mechanism.", "questions": "1. Have you considered the order in which drugs are added or removed? In the training and testing sets of MIMIC and eICU, it appears that if no data augmentation is applied, the order is not taken into account.\n\n2. Regarding generalization, how similar are the training set and testing set? Is there any potential for data leakage?\n\n3. In a scenario where only one drug should be recommended—indicating that individual consideration from previous methods is appropriate—what would be the performance outcome?", "limitations": "yes, for the generalization ability.", "rating": 4, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:  \n1. The paper is clear and easy to understand.  \n2. The novelty of considering drug-drug interactions (DDI) in medication recommendations is evident and meaningful.\n3. The ablation study clearly shows the effectiveness of each component in their design.\n\nWeaknesses:  \n1. The approach of framing the problem as a sequence of decision-making may lead to issues, as different orders of adding or removing drugs could yield varying outcomes. For instance, the sequences m1, m2, m3 may produce different results compared to m2, m3, m1.  \n2. Figure 1 includes notations for m1, m2, and m3; however, these notations are explained in Section 3.  \n3. The first mention of GRPO in line 39 should be properly cited.", "was_revised": true, "final_justification": "The response has largely addressed my concerns, so I would like to maintain my positive rating.", "rebuttal": "We thank the reviewer for the constructive feedback and thoughtful questions. Below, we address each weakness and question in detail.\n\n---\n\n### 1. On the Sequential Decision-Making Formulation \n\n> **W1**: The approach of framing the problem as a sequence of decision-making may lead to issues, as different orders of adding or removing drugs could yield varying outcomes. For instance, the sequences m1, m2, m3 may produce different results compared to m2, m3, m1.\n \nThe concern regarding potential issues from different orders of adding or removing drugs is understandable. In FLAME, the list-wise policy ultimately outputs a **set** of medications, and evaluation is based on comparing this predicted set with the ground-truth set. Therefore, the final recommendation is **order-invariant**—different sequences that produce the same set lead to identical evaluation results.  \n\nDuring training with step-wise GRPO, the order of adding or removing drugs can lead to slightly different intermediate states (Eq. 7), resulting in minor variations in the step-wise advantage signals. However, this does not alter the fundamental reward assignment: correct drugs always receive positive rewards, and incorrect drugs receive negative rewards. Combining this fine-grained step-wise signal with the outcome-based reward provides richer supervision than outcome-based reward alone, helping mitigate the sparsity of process-level feedback.\n\n---\n\n### 2. Figure 1 Notations and GRPO Citation  \n\n> **W2**: Figure 1 includes notations for m1, m2, and m3; however, these notations are explained in Section 3.\n>\n> **W3**: The first mention of GRPO in line 39 should be properly cited.\n\nWe acknowledge that the notations *m1*, *m2*, and *m3* in Figure 1 are only explained in Section 3. We will revise the figure caption to clarify them earlier.  \nAdditionally, we agree that the first mention of GRPO (line 39) should be properly cited and will correct this in the final version.\n\n---\n\n### 3. On the Order of Add/Remove Operations  \n\n> **Q1**: Have you considered the order in which drugs are added or removed? In the training and testing sets of MIMIC and eICU, it appears that if no data augmentation is applied, the order is not taken into account.\n\nAs noted above, FLAME does not explicitly consider the order of add/remove operations because the policy is trained **on-policy**: rewards are computed by comparing the model’s output set with the ground-truth set, independent of the sequence in which drugs were added or removed. No trajectory augmentation is required, as the policy learns directly from its own outputs.\n\n---\n\n### 4. Generalization and Potential Data Leakage  \n\n> **Q2**: Regarding generalization, how similar are the training set and testing set? Is there any potential for data leakage?\n\n**MIMIC-III vs. MIMIC-IV:**  \n- MIMIC-III (2001–2012) and MIMIC-IV (2008–2019) were both collected at Beth Israel Deaconess Medical Center.  \n- These datasets differ in temporal coverage and structured coding systems (transition from ICD‑9 to ICD‑10).  \n- Because the collection periods overlap (2008–2012), there is a **small possibility of patient overlap** in the first two temporal bins of Fig. 5(b). However, such limited overlap has a negligible effect on the conclusions, as the temporal validation results still demonstrate that FLAME maintains strong generalization even when medical practice and coding standards evolve over time. Moreover, there is no overlap in the later two bins.\n\n\n**MIMIC-III vs. eICU:**  \n- eICU (2014–2015) is a multi-center database from 208 hospitals across the U.S., none of which are the source of MIMIC data.  \n- There is no temporal or institutional overlap, and thus no risk of data leakage.  \n- External validation on eICU confirms FLAME’s robustness to distributional and coding differences.\n\n---\n\n### 5. Performance When Only One Drug Should Be Recommended  \n\n> **Q3**: In a scenario where only one drug should be recommended—indicating that individual consideration from previous methods is appropriate—what would be the performance outcome?\n\nThe MIMIC datasets capture **critical care admissions**, where patients typically have complex conditions requiring multi-drug therapy. In our MIMIC-III dataset:  \n- The **average number of drugs per admission** is 23.36.  \n- The **maximum number** is 77.  \n- Only **0.14%** of admissions (20/14 207) involve exactly one drug.  \n\nTherefore, list-wise modeling is highly appropriate for the vast majority of cases. In the rare instances requiring only a single drug, the task naturally reduces to a simpler point-wise decision, where FLAME is still applicable.\n\n---\n\n### 6. Summary  \n- The sequential decision-making formulation does not introduce order-dependence in final recommendations because evaluation is based on sets.  \n- We will fix the missing GRPO citation and clarify notations in Figure 1.  \n- Generalization was evaluated using datasets with distinct temporal spans and coding systems. While MIMIC-III and MIMIC-IV have a slight temporal overlap, temporal validation on MIMIC-IV still shows that FLAME achieves robust performance compared to baselines as medical practice evolves. Additionally, external validation on eICU confirms FLAME’s strong robustness to distributional and coding differences.\n- Single-drug prescriptions are extremely rare in these datasets; when they occur, the task reduces to a trivial case of our list-wise formulation.\n\nWe appreciate the reviewer’s feedback, which will help us improve clarity and completeness in the final version."}, {"review_id": "lKeXmmcdnI", "reviewer": "Reviewer_2sXp", "summary": "The paper introduces **FLAME**, a medication recommendation framework that treats prescription generation as a **step-by-step list-wise decision process** using large language models. It proposes **step-wise Group Relative Policy Optimization (GRPO)** to assign fine-grained rewards for each drug added or removed, improving both accuracy and safety. FLAME also fuses structured and unstructured clinical data to enhance patient modeling. Experiments on multiple EHR datasets show that FLAME outperforms prior methods in multiple metrics.", "questions": "1. **Clinical Usability and Human Evaluation**\n   Have you considered conducting evaluations with clinicians to assess the interpretability and trustworthiness of FLAME’s recommendations? Including even small-scale feedback from practitioners would strengthen the paper's practical impact and address real-world usability concerns.\n   *Clarification on this could increase confidence in the significance and applicability of the work.*\n\n2. **Generalization to Other Domains**\nDo you plan to apply FLAME to other structured decision-making domains beyond medication recommendation (e.g., mathematical problems, programming problems)? Any preliminary results or insights would be helpful.", "limitations": "yes", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "**Strengths**\n\n* Proposes a novel **step-wise list-wise** medication recommendation framework (FLAME) with fine-grained reward shaping.\n* Achieves **state-of-the-art performance** on accuracy, safety control, and generalization across datasets.\n* Effectively integrates **structured and unstructured clinical data** via hybrid representations.\n* Methodology is **technically sound**, clearly explained, and supported by thorough experiments and ablations.\n\n**Weaknesses**\n\n* **No human or clinical evaluation**, limiting real-world validation.\n* **Generalizability of step-wise GRPO is not further evaluated**, the method could have a border impact, if the authors could further analyze it on more general tasks.", "was_revised": true, "final_justification": "All my questions have been resolved, yet, they are not he major concerns. So I will keep my rating.", "rebuttal": "We thank the reviewer for the positive assessment of our work and for highlighting important directions to strengthen FLAME’s practical impact and broader applicability. Below, we address the weaknesses and questions in detail.\n\n---\n\n### 1. On Clinical Usability and Human Evaluation  \n\n> **W1**: No human or clinical evaluation, limiting real-world validation.\n>\n> **Q1**: Clinical Usability and Human Evaluation Have you considered conducting evaluations with clinicians to assess the interpretability and trustworthiness of FLAME’s recommendations? Including even small-scale feedback from practitioners would strengthen the paper's practical impact and address real-world usability concerns. Clarification on this could increase confidence in the significance and applicability of the work.\n\n\n\nWe fully agree that involving clinicians to evaluate the interpretability and trustworthiness of FLAME’s recommendations would meaningfully enhance its real-world applicability. Such evaluations could provide valuable feedback on whether the generated prescriptions align with clinical reasoning, increase user trust, and identify cases where the model’s suggestions might need refinement.  \n\nHowever, due to practical constraints (e.g., lack of direct access to clinical collaborators during this study), we were unable to conduct human-in-the-loop evaluations in this work. We plan to pursue this in future studies by partnering with medical professionals to (i) assess prescription interpretability, (ii) collect qualitative feedback on usability, and (iii) design user-facing interfaces to facilitate clinical adoption. We appreciate the reviewer’s suggestion and believe it would substantially strengthen FLAME’s real-world value.\n\n---\n\n### 2. On Generalization of Step-wise GRPO to Other Domains \n\n> **W2**: Generalizability of step-wise GRPO is not further evaluated, the method could have a border impact, if the authors could further analyze it on more general tasks.\n>\n> **Q2**: Generalization to Other Domains Do you plan to apply FLAME to other structured decision-making domains beyond medication recommendation (e.g., mathematical problems, programming problems)? Any preliminary results or insights would be helpful.\n \nFLAME leverages **step-wise GRPO** to provide fine-grained reward signals for medication recommendation. This design is natural in our setting because:  \n1. Generated outputs can be clearly decomposed into steps based on individual drugs.  \n2. The quality of each intermediate state can be objectively evaluated using a potential function defined by changes in accuracy, safety, and validity.  \n\nThis principle can be extended to other structured decision-making tasks where outputs can be sequentially decomposed, such as **sequential recommendation** (e.g., movies, books). However, designing the potential function becomes more challenging: unlike medication recommendation, which has objective correctness and safety metrics, sequential recommendation often involves subjective factors like user preferences or diversity. Extending step-wise GRPO to these domains would require additional work to encode such criteria into the potential function.  \n\nSimilarly, we envision applying step-wise GRPO to **mathematical reasoning** or **code generation**, where a model’s output can be divided into intermediate sub-steps according to logical or syntactic structure. In these cases, the reward function would need to be redesigned to assess the quality of each sub-step, such as correctness of intermediate reasoning or partial program execution results.  \n\nWe view these directions as promising future work and appreciate the reviewer’s suggestion to broaden the impact of our approach.\n\n---\n\n### 3. Summary  \nWe acknowledge the absence of clinician-involved evaluation and plan to address this in follow-up studies. We also recognize the potential of step-wise GRPO beyond medication recommendation and have outlined how it could be extended to other structured decision-making tasks. We thank the reviewer for highlighting these valuable avenues for future research."}]

Author rebuttal:

Meta-review: This paper introduces FLAME, a fine-grained list-wise alignment framework for generative medication recommendation. The central idea is to reformulate prescription generation as a sequential decision-making process, where each drug is added or removed step by step, and to optimize the process via step-wise Group Relative Policy Optimization (GRPO) with potential-based reward shaping. The framework explicitly models drug–drug interactions (DDIs), integrates structured and unstructured patient information into LLM representations, and provides controllable trade-offs between accuracy and safety. Empirical evaluations on multiple benchmark EHR datasets (MIMIC-III, MIMIC-IV, eICU) demonstrate state-of-the-art performance across correctness, safety, and generalization. The paper makes both methodological and practical contributions by offering a clinically motivated formulation of medication recommendation and validating its effectiveness with extensive ablation studies.

The reviewers consistently praised the novelty of the step-wise, list-wise formulation and the introduction of fine-grained reward shaping, which go beyond conventional point-wise prediction paradigms. The work is technically sound, clearly written, and well-supported with thorough experiments and ablations. It addresses a clinically important problem by explicitly balancing safety and accuracy in drug recommendation, a feature often overlooked in prior work. The integration of multiple knowledge sources (structured EHR data, unstructured notes, and molecular drug features) further strengthens patient modeling and improves robustness. Rebuttal clarifications highlighted the qualitative importance of list-wise refinement, illustrating cases where drug dependencies and redundancies cannot be captured by point-wise models alone.

Some concerns were raised about the magnitude of numerical gains from the step-wise GRPO compared to standard GRPO, with improvements appearing modest in absolute terms. Reviewers also noted missing details in the initial submission (e.g., construction of candidate sets, classifier training, standard deviations, walltime analysis), and the lack of human/clinical evaluation limits immediate real-world impact. In addition, generalization of the proposed GRPO variant beyond the medication recommendation task was not fully explored. However, these issues were effectively addressed in rebuttal: the authors added detailed clarifications on experimental procedures, reported standard deviations and runtime measurements, and explained that the gains from list-wise refinement are qualitatively critical even if numerically small.

During the discussion phase, reviewers acknowledged that their concerns were satisfactorily addressed. Reviewer zoHJ explicitly raised their score after seeing clarifications on GRPO benefits and error bars. Reviewer bQc3 also increased their rating after detailed explanations on the necessity of list-wise refinement. Reviewer TAqr’s concerns about order-dependence and data leakage were resolved with clarifications on the set-based evaluation and dataset splits. Reviewer 2sXp remained supportive after the authors explained plans for future clinician-involved evaluation and broader applicability of step-wise GRPO. Overall, the rebuttal successfully strengthened the paper’s standing, leading to a positive consensus.

I recommend Accept. The paper makes a timely and significant contribution by bridging reinforcement learning–based alignment with clinical medication recommendation, demonstrating both methodological novelty and practical relevance. Despite some limitations, the work is technically solid, convincingly validated, and well-defended during rebuttal. Its explicit modeling of drug dependencies and safety–accuracy trade-offs positions it as an impactful step forward for trustworthy clinical AI, meriting recognition beyond a standard acceptance.
Decision: Accept (spotlight)