Title: Magical: Medical Lay Language Generation via Semantic Invariance and Layperson-tailored Adaptation
Venue: NeurIPS 2025
Accepted: yes
Keywords: Medical Lay Language Generation

Large Language Model

LoRA

PEFT
Abstract: Medical Lay Language Generation (MLLG) plays a vital role in improving the accessibility of complex scientific content for broader audiences. Recent literature to MLLG commonly employ parameter-efficient fine-tuning methods such as Low-Rank Adaptation (LoRA) to fine-tuning large language models (LLMs) using paired expert-lay language datasets. However, LoRA struggles with the challenges posed by multi-source heterogeneous MLLG datasets. Specifically, through a series of exploratory experiments, we reveal that standard LoRA fail to meet the requirement for semantic fidelity and diverse lay-style generation in MLLG task. To address these limitations, we propose Magical, an asymmetric LoRA architecture tailored for MLLG under heterogeneous data scenarios. Magical employs a shared matrix A for abstractive summarization, along with multiple isolated matrices B for diverse lay-style generation. To preserve semantic fidelity during the lay language generation process, Magical introduces a Semantic Invariance Constraint to mitigate semantic subspace shifts on matrix A. Furthermore, to better adapt to diverse lay-style generation, Magical incorporates the Recommendation-guided Switch, an externally interface to prompt the LLM to switch between different matrices B. Experimental results on three real-world lay language generation datasets demonstrate that Magical consistently outperforms prompt-based methods, vanilla LoRA, and its recent variants, while also reducing trainable parameters by 31.66%. Our code is publicly available at https://github.com/tianlwang/Magical.git.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper addresses the challenge of MLLG, focusing on transforming biomedical content into layman-accessible language. The authors critique the use of standard LoRA in handling semantic fidelity and stylistic diversity across heterogeneous multi-source datasets. They propose Magical, a novel asymmetric LoRA architecture that separates semantic abstraction (via shared matrix A) and lay-style adaptation (via multiple isolated matrices B). The contributions mainly focus on 1) Semantic invariance constraint on matrix A using semantic-relevant layer identification and contrastive learning to preserve semantic fidelity. 2) Layperson-tailored adaptation via dataset-specific B matrices and a recommendation-guided switch for selective matrix activation. 3) Empirical results on three MLLG datasets showing Magical consistently outperforms LoRA and its recent variants in both semantic fidelity and style diversity.\n\nSTRENGTHS AND WEAKNESSES:\n## Quality\n### Strengths:\n- Rigorous evaluation across three real-world datasets, including Cochrane, eLife, Plos_genetics.\n- Very detailed ablation studies, which effectively isolate component contributions.\n### Weaknesses:\n- Due to the subjective nature of layman-oriented content comprehension, human evaluation metrics are necessary in this task setting to assess whether the model-generated content is acceptable from the perspective of real-world target users.\n\n## Clarity\n### Strengths：\n- The framework is well-illustrated through Figure 3 and the proposed methodology and key components are clearly presented in Section 3.2 and 3.3.\n### Weaknesses:\n- Figure 2 lack detailed captions or axis labels, reducing their standalone interpretability.\n- There is a lack of impact analysis of the number of B matrices and the choice of top-K semantic layers.\n\n## Significance\n### Strengths：\n- The paper addresses a meaningful social need, which is improving public access to biomedical knowledge.\n### Weaknesses:\n- Despite its significance, the model has limited generalizability beyond MLLG without adaptation, especially since it depends on labeled expert-lay pairs.\n- As the provided method is very domain-specific, there is limited discussion on how it can be integrated into existing clinical or public health communication pipelines.\n- A main concern is that, while the proposed method is specifically designed within the MLLG domain, many of its core innovations, such as asymmetric low-rank adaptation, semantic invariance constraints, and lay-style routing, are generally applicable to other text generation tasks that require semantic preservation and stylistic transformation. However, the paper currently does not explore this broader applicability. This could make the method appear overly domain-specific, which may limit its perceived impact and usefulness to the broader NLP community.\n\n## Originality\n### Strengths：\n- It seems to be the first to explicitly decouple semantic abstraction and stylistic transformation in MLLG via asymmetric LoRA.\n### Weaknesses:\n- While inspired by HydraLoRA, the originality of some technical aspects, such as the use of external switching agents is somewhat incremental.\n\nQUESTIONS:\n1. Could Magical be extended to more domains such as multi-lingual lay language generation and other domains including legal simplification or educational content generation, and what changes would be needed to handle such complexity?\n2. Have you conducted or considered conducting any human subject evaluations to assess whether generated lay texts are truly easier to understand?\n3. How sensitive is Magical’s performance to the choice of the number of B matrices and the top-K semantic layers?\n\nLIMITATIONS:\nThe paper does acknowledge that the Recommendation Agent is not implemented, and it also discusses challenges around modeling layperson profiles and proposes future directions such as meta-learning and RL. However, it could be further strengthened by discussing scalability, domain generalizability, data imbalance, and ethical risks caused by hallucination or oversimplification of the generated medical content.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-13"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces Magical, a novel asymmetric LoRA-based architecture designed for Medical Lay Language Generation, a task that involves simplifying complex biomedical texts for non-expert audiences. The authors propose: 1) A shared low-rank matrix A for semantic content modeling (abstractive summarization). 2) Multiple isolated low-rank matrices B for stylistic variation corresponding to heterogeneous lay styles. 3) A Semantic Invariance Constraint applied on matrix A using contrastive learning and semantic-relevant layer identification. 4) A Recommendation-guided Switch that leverages an external agent to dynamically select the appropriate B matrix based on the target lay style. Experiments are conducted across three real-world datasets (Cochrane, eLife, Plos_genetics) and three LLMs (LLaMA3.2-3B, LLaMA3.1-8B, Qwen2.5-7B). Magical achieves state-of-the-art performance compared to various baselines, including rsLoRA, DoRA, PiSSA, and HydraLoRA.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths**:\n\n1 - MLLG is a high-impact application with societal relevance. The use of PEFT methods such as LoRA (and Magical) enables feasible deployment on limited-resource settings.\n\n2 - The authors perform a solid preliminary investigation showing that multi-source MLLG datasets are highly heterogeneous in structure and style. They identify specific shortcomings of single-LoRA, supporting their architectural decisions with empirical evidence (e.g., Table 1).\n\n3 - Magical achieves significant improvements in both ROUGE and BLEU, while reducing trainable parameters by ~31.66% (Table 2). Moreover, ablation studies (Table 3) are thorough and support the importance of each component.\n\n\n**Weaknesses**:\n\n1 - The core idea (separate low-rank branches + a selector) is incremental over HydraLoRA [24] and Mixture-of-LoRAs [32]. The only new twist is the semantic contrastive loss on A, whose technical depth is modest.\n\n2 - The Recommendation-guided Switch is central to the claim of “layperson-tailored adaptation,” yet the recommender is manually simulated. As stated in Section 4.1 and 5, “we adopt a manually specified matrix B,” which significantly reduces real-world applicability and novelty\n\n3 - Minor issues: two figures (Figs. 6-7) appear in the appendix but are referenced in Sec. 4 without page cues; typos (“BLUE” for BLEU in multiple tables).\n\n4 - While KDE plots (Fig. 4) are  good qualitative results; no quantitative semantic similarity metric (e.g., BERTScore, CheXplain) is provided to verify “semantic invariance”.\n\nQUESTIONS:\n1 - The proposed asymmetric LoRA framework closely resembles HydraLoRA [24] and Mixture-of-LoRAs [32], both of which use separate low-rank branches and routing mechanisms. Can the authors clarify in more detail how Magical provides a non-trivial architectural or theoretical advancement over these prior approaches? In particular, is there any scenario where HydraLoRA fails but Magical succeeds (beyond empirical results)?\n\n\n2 - The Recommendation-guided Switch is presented as a key component enabling layperson-tailored adaptation. However, the current implementation relies on manual selection of matrix B. Could the authors elaborate on why a basic learning-based selector (e.g., using dataset metadata, prefix tuning, or retrieval) was not attempted? Additionally, how would performance degrade under imperfect or noisy recommendation in real-world settings?\n\n3 - In Equation (2), the aggregation of BiAx using α_i is central, yet the mechanism for computing α_i during training and inference is under-specified. Are α_i values learned, externally set, or inferred dynamically? If the switch is hard-coded during training, how does this affect generalization to unseen or mixed-style data?\n\n4 - The proposed design is evaluated solely on medical lay language generation. Can the authors provide any evidence or discussion regarding how the method would generalize to other domains requiring stylistic or semantic transformation (e.g., legal summarization, educational simplification)?\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 2, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces Magical, a novel approach to improve how medical texts are simplified for non-experts readers. The authors found that standard LoRA has two critical limitations when applied to medical lay language generation: semantic drift, where LoRA fails to preserve the semantic fidelity during simplification, and style heterogeneity, where different medical datasets require different simplification styles, but standard LoRA struggles to adapt to this diversity. The authors propose use to an asymmetric LoRA architecture to address these issues, using a shared matrix A to handle abstractive summarization while maintaining semantic accuracy, and multiple matrices B to enable adaptation to diverse simplification needs. The authors report Magical outperformed standard LoRA and its variants across three medical datasets (Cochrane, eLife, Plos_genetics).\n\nSTRENGTHS AND WEAKNESSES:\n### Strengths:\n* The paper is well-motivated and addressed an important problem - converting complex medical texts into language that non-experts can understand. This is important for improving public health literacy and patient adherence to medical advice.\n* The proposed asymmetric LoRA architecture is used in a novel setting.  \n* The paper is well-written, well-structured, and easy to follow.\n\n\n\n### Weakness\n\n* The authors use BLEU and ROUGE as the main metrics to evaluate the quality of medical lay language generation. However, since these metrics rely on surface-form n-gram overlap, they may not capture the true quality of the generated medical lay language. Human evaluation may be needed, and using a strong language model as a judge—such as o3—could also be a reasonable proxy.\n\n* The authors focus on comparing against standard LoRA as the baseline, but a comparison to full-parameter fine-tuning should have been included as well.\n\n* Following my previous two points, I’m not fully convinced by the reported effectiveness of the proposed asymmetric LoRA in medical lay language generation.\n\nQUESTIONS:\nThe LLMs evaluated in the paper are relatively small (under 8B). Have you benchmarked the performance of stronger models, such as o3-mini or Claude?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 3, "REVIEWER_CONFIDENCE": 2, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-07-06"}

comments: SUMMARY:
The authors aim to address the limitations of style transfer in LoRA models, particularly in the medical domain for expert to lay styles. The goal of th paper is to specifically address heterogenous datasets where the content can be quite different. The proposed method, "Magical", implements LoRA with a single A matrix and multiple B matrices for each different style or dataset.

The authors evaluate this method on 3 biomedical corpora across three backbone LLMs and shows performance improvements on top of vanilla LoRA.

STRENGTHS AND WEAKNESSES:
This is a useful and interesting paper that other researchers will appreciate reading, and represents a positive step in an area with limited attention, namely, tackling heterogeneous biomedical datasets. The method itself can be implemented with relatively few steps, and the experiments are thorough.

There are some limitations with how realistic the evaluations are, given the static nature and limited scope of the datasets, but these are general problems with medical LLM evaluation.

QUESTIONS:
Are there external validations for the automated metrics? I understand the need for deterministic ways to evaluate quality, but how do we know the proposed methods do not just overfit to the measured quantity vs. actual perceived quality? For example, a small panel of physicians and patients evaluating quality.

LIMITATIONS:
Yes
TITLE: Official Review
CLARITY: 3
IMPACT: 4
ORIGINALITY: 3
DATE: 2025-07-07