OphVLM-R1

Efficient Ophthalmic Reasoning via Curriculum Reinforcement Learning

English · 简体中文

Overview

OphVLM-R1 is a lightweight 2B-parameter vision-language model for ophthalmic multimodal reasoning. Starting from InternVL3.5-2B, it first acquires ophthalmic knowledge through LoRA supervised fine-tuning (SFT), then develops progressively harder clinical reasoning skills through four-stage curriculum reinforcement learning. Group Sequence-level Policy Optimization (GSPO) and hard-sample dynamic backtracking are used to improve optimization over long reasoning trajectories and difficult long-tail cases.

This README focuses on the model training pipeline, architecture and algorithms, and experimental results. For dataset construction and distribution details, see the OphReason-Vision dataset repositories below.

Project Resources

• Project page: https://qizishi.github.io/OphVLM-R1/
• OphVLM-R1 model: Hugging Face | ModelScope
• OphReason-Vision dataset: Hugging Face | ModelScope
• OphAgent system: GitHub

Model and Training Framework

OphVLM-R1 uses InternVL3.5-2B as its backbone. Its 2B parameter scale targets deployment in resource-constrained settings while retaining multimodal clinical reasoning capacity. Training consists of two stages.

Stage 1: LoRA Supervised Fine-Tuning

The cold-start stage injects ophthalmic domain knowledge with Low-Rank Adaptation:

$$W_0+\mathrm{\Delta }W=W_0+BA,h=W_{0}x+\frac{\alpha }{r}BAxW\_0\; +\; \backslash Delta\; W\; =\; W\_0\; +\; BA,\; \backslash qquad\; h\; =\; W\_0x\; +\; \backslash frac\{\backslash alpha\}\{r\}BAx$$

where $B\in {\mathbb{R}}^{d\times r}B\; \backslash in\; \backslash mathbb\{R\}^\{d\; \backslash times\; r\}$, $A\in {\mathbb{R}}^{r\times k}A\; \backslash in\; \backslash mathbb\{R\}^\{r\; \backslash times\; k\}$, and $r\ll \min (d,k)r\; \backslash ll\; \backslash min(d,k)$. The SFT objective is

$${\mathcal{L}}_{\mathrm{S}\mathrm{F}\mathrm{T}}=-\mathbb{E}[\sum _{t=1}^{\mid y\mid }\log {\pi }_{\varphi }(y_t\mid x,y_{<t})]\mathrm{.}\backslash mathcal\{L\}\_\{\backslash mathrm\{SFT\}\}=-\backslash mathbb\{E\}\backslash left[\backslash sum\_\{t=1\}^\{\backslash lvert\; y\backslash rvert\}\backslash log\backslash pi\_\backslash phi(y\_t\backslash mid\; x,y\_\{\backslash lt\; t\})\backslash right].$$

Setting	Value
Training subset	3,418 cold-start samples
LoRA rank / scaling	$r=64r=64$, $\alpha =128\backslash alpha=128$
Target projections	$W_{q}W\_q$, $W_{k}W\_k$, $W_{v}W\_v$, $W_{o}W\_o$
Learning rate	$1\times {10}^{-4}1\backslash times10^\{-4\}$ with cosine annealing
Batch size / epochs	32 / 3
Trainable parameters	Approximately 0.5%

Stage 2: Curriculum Reinforcement Learning

GSPO computes the policy ratio at sequence level rather than independently clipping token-level ratios:

$$s_i(\varphi )={\left(\frac{{\pi }_{\varphi }(y_i\mid x)}{{\pi }_{{\varphi }_{\mathrm{o}\mathrm{l}\mathrm{d}}}(y_i\mid x)}\right)}^{\frac{1}{\mid y_i\mid }}=\exp (\frac{1}{\mid y_i\mid }\sum _{t=1}^{\mid y_i\mid }\log \frac{{\pi }_{\varphi }(y_{i,t}\mid x,y_{i,<t})}{{\pi }_{{\varphi }_{\mathrm{o}\mathrm{l}\mathrm{d}}}(y_{i,t}\mid x,y_{i,<t})})\mathrm{.}s\_i(\backslash phi)=\backslash left(\backslash frac\{\backslash pi\_\backslash phi(y\_i\backslash mid\; x)\}\{\backslash pi\_\{\backslash phi\_\{\backslash mathrm\{old\}\}\}(y\_i\backslash mid\; x)\}\backslash right)^\{\backslash frac\{1\}\{\backslash lvert\; y\_i\backslash rvert\}\}=\backslash exp\backslash left(\backslash frac\{1\}\{\backslash lvert\; y\_i\backslash rvert\}\backslash sum\_\{t=1\}^\{\backslash lvert\; y\_i\backslash rvert\}\backslash log\backslash frac\{\backslash pi\_\backslash phi(y\_\{i,t\}\backslash mid\; x,y\_\{i,\backslash lt\; t\})\}\{\backslash pi\_\{\backslash phi\_\{\backslash mathrm\{old\}\}\}(y\_\{i,t\}\backslash mid\; x,y\_\{i,\backslash lt\; t\})\}\backslash right).$$

The PPO-style objective uses group-normalized advantages:

$${\mathcal{J}}_{\mathrm{G}\mathrm{S}\mathrm{P}\mathrm{O}}(\varphi )=\mathbb{E}[\frac{1}{G}\sum _{i=1}^G\min (s_i{\widehat{A}}_i,\mathrm{clip}(s_i,1-\epsilon ,1+\epsilon ){\widehat{A}}_i)]\mathrm{.}\backslash mathcal\{J\}\_\{\backslash mathrm\{GSPO\}\}(\backslash phi)=\backslash mathbb\{E\}\backslash left[\backslash frac\{1\}\{G\}\backslash sum\_\{i=1\}^\{G\}\backslash min\backslash left(s\_i\backslash hat\; A\_i,\backslash operatorname\{clip\}(s\_i,1-\backslash varepsilon,1+\backslash varepsilon)\backslash hat\; A\_i\backslash right)\backslash right].$$

Each stage uses a mixed reward

$$r(x,y)={\lambda }_1{r}_{\mathrm{r}\mathrm{u}\mathrm{l}\mathrm{e}}(x,y)+{\lambda }_2{r}_{\mathrm{j}\mathrm{u}\mathrm{d}\mathrm{g}\mathrm{e}}(x,y),r(x,y)=\backslash lambda\_1r\_\{\backslash mathrm\{rule\}\}(x,y)+\backslash lambda\_2r\_\{\backslash mathrm\{judge\}\}(x,y),$$

where the judge reward is produced by Intern-S1-mini. The curriculum follows increasing clinical complexity:

1. Lesion Localization — single-image visual perception.
2. Multi-image Selection — cross-image comparison.
3. Report Generation — structured, long-form synthesis.
4. Knowledge Q&A — integration of visual findings and clinical knowledge.

Setting	Value
Group size / clipping	$G=8G=8$, $\epsilon =0.2\backslash varepsilon=0.2$
Learning rate	$5\times {10}^{-6}5\backslash times10^\{-6\}$
KL coefficient	${\beta }_{\mathrm{K}\mathrm{L}}=0.04\backslash beta\_\{\backslash mathrm\{KL\}\}=0.04$
Reward weights	${\lambda }_1=0.6\backslash lambda\_1=0.6$, ${\lambda }_2=0.4\backslash lambda\_2=0.4$
Training length	2 epochs per curriculum stage

Hard-Sample Dynamic Backtracking

Prompts that repeatedly receive low rewards during the most recent $k=5k=5$ rounds are resampled on-policy:

$$P_{\mathrm{s}\mathrm{a}\mathrm{m}\mathrm{p}\mathrm{l}\mathrm{e}}(x,y)\propto f(x,y{)}^{\beta },P\_\{\backslash mathrm\{sample\}\}(x,y)\backslash propto\; f(x,y)^\backslash beta,$$

where $f(x,y)f(x,y)$ is the consecutive failure count and $\beta =2.0\backslash beta=2.0$. Resampled prompts always receive fresh rollouts, and the resampling share is capped at 30% of each batch.

Training Stack

• Hardware: 8 NVIDIA GeForce RTX 4090 GPUs (24 GB each).
• SFT: 4 GPUs.
• Reinforcement learning: 6 GPUs for training and 2 GPUs for vLLM rollout generation.
• Software: ms-swift, DeepSpeed ZeRO-3, AdamW, vLLM, and EvalScope.

Experiments

Evaluation Benchmarks

Benchmark	Samples	Tasks / sources	Role
In-Domain	2,000	4 tasks	Held-out clinical reasoning
Fundus-MMBench	620	31 tasks	Fine-grained fundus analysis
OmniMedVQA-Eye	10,044	11 sources	Out-of-domain VQA

Main Results

Accuracy is reported in percent. The cross-benchmark average is reference-only because the benchmarks differ in task format, difficulty, and random baseline; per-benchmark comparisons are primary.

Model	In-Domain	Fundus	Omni-Eye	Avg.*
InternVL3.5-2B	34.50	36.61	55.47	42.19
InternVL3.5-4B	36.23	42.10	77.51	51.95
MedVLM-R1-2B	27.80	20.81	68.06	38.89
Lingshu-7B	44.20	41.29	87.42	57.64
HuatuoGPT-Vision-7B	38.30	28.06	71.78	46.05
FundusExpert-8B	31.20	54.84	64.71	50.25
OphthaReason-Intern-2B	31.00	35.48	79.61	48.70
OphthaReason-Qwen-3B	36.60	38.87	86.86	54.11
OphVLM-R1-2B (ours)	38.40	42.58	88.24	56.41

OphVLM-R1 reaches 88.24% on OmniMedVQA-Eye and 42.58% on Fundus-MMBench. Its 56.41% reference average is 4.46 percentage points above InternVL3.5-4B and 2.30 points above OphthaReason-Qwen-3B. Comparisons with off-the-shelf 7B/8B models should be interpreted cautiously because data exposure and parameter scale are not controlled.

Ablation Results

Configuration	In-Domain	Fundus	Omni-Eye	$\mathrm{\Delta }\backslash Delta$ Omni
OphVLM-R1 (full)	38.40	42.58	88.24	—
SFT only	37.52	34.47	62.03	-26.21
SFT + RL one-shot	37.96	38.62	78.14	-10.10
SFT + RL shuffled	37.73	37.85	76.48	-11.76
Without Stage 1	38.14	40.43	85.62	-2.62
Without Stage 2	38.02	40.17	85.13	-3.11
Without Stage 3	37.88	39.72	84.38	-3.86
Without Stage 4	38.07	40.31	85.47	-2.77
Token-level GRPO	37.84	39.76	84.52	-3.72
Without hard-sample backtracking	38.11	40.83	86.12	-2.12

The ablations show that ordered curriculum RL, sequence-level optimization, and hard-sample backtracking make independent contributions. All reported results are single runs without confidence intervals or significance tests. In addition, the theoretical variance-reduction argument for GSPO is exact only under i.i.d. per-token log-ratio assumptions and is approximate for correlated natural-language tokens.

Citation

@inproceedings{qi2026ophvlm,
  title={OphVLM-R1: Efficient Ophthalmic Reasoning via Curriculum Reinforcement Learning},
  author={Qi, Zishi and Hu, Xiaoya and Pan, Huilin and Gao, Ang and Hou, Jiaxin and Li, Jiankun and Qian, Yongao},
  booktitle={Proceedings of the World Artificial Intelligence Conference Academic (WAICA)},
  year={2026}
}

Acknowledgements

We thank OpenGVLab for the InternVL foundation model and the ophthalmologists who contributed to data review and quality assurance.