Gazal-R1: Achieving State-of-the-Art Medical Reasoning with
  Parameter-Efficient Two-Stage Training
Authors: name: Ahmed M. Adly; name: Mostafa Samy; name: Amr Fawzy
Published: 2025-06-18 09:44:21

Abstract: We present Gazal-R1, a 32-billion-parameter language model that achieves
state-of-the-art performance in medical reasoning while providing transparent,
step-by-step explanations for clinical decision-making. Built upon Qwen3 32B,
our model demonstrates that strategic training can enable mid-sized models to
outperform significantly larger counterparts in specialized domains. We
developed a novel two-stage training pipeline: first, supervised fine-tuning on
a carefully curated dataset of 107,033 synthetic medical reasoning examples
that teaches structured clinical thinking, enhanced by advanced
parameter-efficient techniques including Weight-Decomposed Low-Rank Adaptation
(DoRA) and Rank-Stabilized LoRA (rsLoRA); second, reinforcement learning using
Group Relative Policy Optimization (GRPO) with a sophisticated multi-component
reward system that refines accuracy, format adherence, and reasoning quality.
Gazal-R1 achieves exceptional performance across medical benchmarks, scoring
87.1% on MedQA, 81.6% on MMLU Pro (Medical), and 79.6% on PubMedQA, surpassing
models up to 12x larger. Beyond its strong empirical results, this work
provides detailed insights into the challenges of training reasoning-capable
models in specialized domains, including issues with reward hacking, training
instability, and the fundamental tension between factual recall and detailed
reasoning. Our methodology offers a reproducible framework for developing
high-capability, domain-specific language models that balance performance,
efficiency, and explainability.