Title: Fine-grained List-wise Alignment for Generative Medication Recommendation
Venue: NeurIPS 2025
Accepted: yes
Keywords: Medication Recommendation

Large Language Models

Reinforcement Learning

Electronic Health Record

Reward Shaping
Abstract: Accurate and safe medication recommendations are critical for effective clinical decision-making, especially in multimorbidity cases. However, existing systems rely on point-wise prediction paradigms that overlook synergistic drug effects and potential adverse drug-drug interactions (DDIs).  We propose FLAME, a fine-grained list-wise alignment framework for large language models (LLMs), enabling drug-by-drug generation of drug lists.  FLAME formulates recommendation as a sequential decision process, where each step adds or removes a single drug.  To provide fine-grained learning signals, we devise step-wise Group Relative Policy Optimization (GRPO) with potential-based reward shaping, which explicitly models DDIs and optimizes the contribution of each drug to the overall prescription. Furthermore, FLAME enhances patient modeling by integrating structured clinical knowledge and collaborative information into the representation space of LLMs. Experiments on benchmark datasets demonstrate that FLAME achieves state-of-the-art performance, delivering superior accuracy, controllable safety–accuracy trade-offs, and strong generalization across diverse clinical scenarios. Our code is available at https://github.com/cxfann/Flame.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper introduces **FLAME**, a medication recommendation framework that treats prescription generation as a **step-by-step list-wise decision process** using large language models. It proposes **step-wise Group Relative Policy Optimization (GRPO)** to assign fine-grained rewards for each drug added or removed, improving both accuracy and safety. FLAME also fuses structured and unstructured clinical data to enhance patient modeling. Experiments on multiple EHR datasets show that FLAME outperforms prior methods in multiple metrics.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths**\n\n* Proposes a novel **step-wise list-wise** medication recommendation framework (FLAME) with fine-grained reward shaping.\n* Achieves **state-of-the-art performance** on accuracy, safety control, and generalization across datasets.\n* Effectively integrates **structured and unstructured clinical data** via hybrid representations.\n* Methodology is **technically sound**, clearly explained, and supported by thorough experiments and ablations.\n\n**Weaknesses**\n\n* **No human or clinical evaluation**, limiting real-world validation.\n* **Generalizability of step-wise GRPO is not further evaluated**, the method could have a border impact, if the authors could further analyze it on more general tasks.\n\nQUESTIONS:\n1. **Clinical Usability and Human Evaluation**\n   Have you considered conducting evaluations with clinicians to assess the interpretability and trustworthiness of FLAME’s recommendations? Including even small-scale feedback from practitioners would strengthen the paper's practical impact and address real-world usability concerns.\n   *Clarification on this could increase confidence in the significance and applicability of the work.*\n\n2. **Generalization to Other Domains**\nDo you plan to apply FLAME to other structured decision-making domains beyond medication recommendation (e.g., mathematical problems, programming problems)? Any preliminary results or insights would be helpful.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-18"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nIn this paper, the authors focus on the challenge of medication recommendation and aim to address it by leveraging the substantial capabilities of large language models (LLMs), particularly in scenarios where multiple drugs interact synergistically. The paper introduces a novel approach called FLAME to tackle the medication recommendation problem while considering drug-drug interactions (DDIs) through an LLM framework. The authors model this issue as a list-wise decision-making problem. With the assistance of step-wise GRPO, it becomes feasible to conceptualize the medication recommendation process as a sequential state transition mechanism.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:  \n1. The paper is clear and easy to understand.  \n2. The novelty of considering drug-drug interactions (DDI) in medication recommendations is evident and meaningful.\n3. The ablation study clearly shows the effectiveness of each component in their design.\n\nWeaknesses:  \n1. The approach of framing the problem as a sequence of decision-making may lead to issues, as different orders of adding or removing drugs could yield varying outcomes. For instance, the sequences m1, m2, m3 may produce different results compared to m2, m3, m1.  \n2. Figure 1 includes notations for m1, m2, and m3; however, these notations are explained in Section 3.  \n3. The first mention of GRPO in line 39 should be properly cited.\n\nQUESTIONS:\n1. Have you considered the order in which drugs are added or removed? In the training and testing sets of MIMIC and eICU, it appears that if no data augmentation is applied, the order is not taken into account.\n\n2. Regarding generalization, how similar are the training set and testing set? Is there any potential for data leakage?\n\n3. In a scenario where only one drug should be recommended—indicating that individual consideration from previous methods is appropriate—what would be the performance outcome?\n\nLIMITATIONS:\nyes, for the generalization ability.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 2, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-01"}

comments: SUMMARY:
The paper introduces FLAME, a novel list-wise medication recommendation framework that reformulates drug recommendation as a sequential decision-making process. FLAME also integrates multi-source clinical knowledge—structured codes, unstructured notes, and semantic labels. Extensive experiments demonstrate state-of-the-art accuracy and safety control on real-world EHR datasets.

STRENGTHS AND WEAKNESSES:
**Strength:**
1. The overall theoretical analysis is well-organized
2. Detailed architecture, datasets, and hyperparameters; appendix lists detailed prompt templates.
3. Comprehensive ablations isolate the effect of different model variants

**Weakness:**
1. No error bars or standard deviation included. In particular, Table 2 reports several Jaccard/F1 scores that are very close (e.g., FLAME vs. variant (e) with standard GRPO).
2. Although limitations about computational expense are mentioned in the appendix, it would still be beneficial to include the walltime for both training and inference, especially for inference which can be used to measure feasibility for real-time settings.
3. The ablation study comparing the use of "standard GRPO" versus "Step-wise GRPO" shows no significant performance gap (i.e. <0.5% increase). This diminishes the practical significance of the proposed approach.

QUESTIONS:
1. In real life there may exists drugs whose interation effects are unknown, and this may result in noisy DDI graph ***D***. How would FLAME performs on noisy DDI graph? 
2. The numbers under "Evaluate each step" and "Step-wise advantages" in Figure 1 are unclear to me. Could author provide more information on how these numbers are associated?

LIMITATIONS:
Yes
TITLE: Official Review
CLARITY: 2
IMPACT: 2
ORIGINALITY: 2
DATE: 2025-07-02

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nExisting medication recommendation methods follow point-wise paradigms that overlook synergistic effects and adverse drug-drug interactions (DDIs). This paper proposes FLAME, a list-wise, step-wise approach that applies Group Relative Policy Optimization (GRPO) with potential-based reward shaping. At each step, the model adds or removes a drug, and the advantage is computed using a potential function based on correctness, safety (DDI risk), and constraint adherence. Final recommendations are generated in two stages: an initial drug-level relevance classification, followed by list-wise refinement using the proposed step-wise GRPO. The model also incorporates multi-source knowledge by combining structured clinical features and collaborative embeddings with LLM representations. Experiments on MIMIC-III, MIMIC-IV, and eICU show that FLAME outperforms prior methods in recommendation accuracy while achieving better safety-control trade-offs.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n* The proposed method is both novel and well-suited to the clinical medication recommendation domain.\n* The authors compare against several strong existing baselines, and their proposed method outperforms them in both correctness and safety.\n\nWeaknesses:\n* The experiments indicate a trade-off between safety and correctness, but the paper does not explore the underlying cause of this behavior. For example, does prioritizing safety lead the model to make more conservative recommendations, excluding clinically relevant drugs?\n* Some experimental details are missing in the paper, such as the construction of the candidate set Mc and the training of the drug-level classifier. \n* From the ablation study (Table 2, rows a-b), it appears that much of the performance gain comes from the inclusion of the point-wise drug-level classification step. This raises the question of whether the GRPO-based list-wise refinement is truly necessary, or if similar improvements could be achieved through better tuning of the classifier alone. Reporting the safety score of the classifier can help clarify the added value of the GRPO step.\n* Additional clarifications requested in the \"Questions\" section.\n\nQUESTIONS:\n1. How is the candidate set Mc for \"adherence\" (refusal rate) measurement constructed?\n2. How is the drug-level classifier trained?\n3. Could improved tuning of the drug-level classifier alone achieve correctness comparable to the full FLAME model? What is the safety score of the classifier?\n\nGeneral questions:\n1. Can other risk factors, like patient-specific allergies or preferences to certain medications, be encoded in this framework?\n2. Could the ground truth labels based on historical prescriptions be affected by incorrect or suboptimal recommendations? If so, could incorporating external medical knowledge or expert validation help refine the training signal and improve model performance?\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 4, "ORIGINALITY": 4, "DATE": "2025-07-03"}