Hugging Face's logo

QiZishi
/
OphReason-Vision

Reinforcement Learning
English
medical
ophthalmology
multimodal
chain-of-thought
Model card Files
xet
 Community

OphReason-Vision Dataset

πŸ”— Open Source Links

πŸ“° News

  • 2026.07.07 πŸŽ‰ Paper "OphVLM-R1: Efficient Ophthalmic Reasoning via Curriculum RL" accepted by WAICA 2026!
  • 2025.11.28 πŸ“Š OphReason-Vision dataset subset open-sourced

Introduction

OphReason-Vision is a high-quality dataset constructed specifically for ophthalmic multimodal reasoning, containing 15,418 expert-verified Chain-of-Thought reasoning trajectories. Built from 100K+ clinical cases and 30+ public datasets, with automated quality control and expert-collaborative review achieving inter-rater agreement Cohen's $\kappa = 0.82$.

The currently open-sourced subset contains 3,418 samples for model cold-start supervised fine-tuning (Cold-Start SFT).

Dataset Construction

OphReason-Vision uses a three-stage closed-loop pipeline, addressing three deficiencies in existing ophthalmic multimodal training data: high heterogeneity across sources with incompatible formats, absence of structured reasoning chains where public datasets provide only image-level labels, and skewed difficulty distribution that overrepresents common conditions while underserving rare diseases.

Data Pipeline

Stage 1: Data Standardization

Integrates 100K+ clinical cases with 30+ public datasets using a dual-stream strategy:

  • Text stream: Parses unstructured electronic medical records into standardized JSON, resolving inconsistent terminology through a curated ophthalmic synonym table
  • Visual stream: Generates detailed textual descriptions for datasets that contain only image-level labels, enriching visual information available for reasoning synthesis

Stage 2: Structured Reasoning Synthesis

Uses Intern-S1 to generate multi-dimensional instructions covering lesion localization, multimodal diagnosis, and knowledge question answering. For each instruction, produces a Chain-of-Thought reasoning chain following the clinical diagnostic workflow: visual sign identification, knowledge retrieval, pathological analysis, and clinical decision.

Given input $x$, reasoning chain $z = (z_1, \ldots, z_n)$, and answer $y$, the chain probability is:

P(z∣x)=∏t=1∣z∣P(zt∣x,z<t)P(z|x) = \prod_{t=1}^{|z|} P(z_t | x, z_{<t})

Quality control employs an LVLM-as-a-Judge mechanism using Intern-S1 with threshold $\tau = 0.7$, determined via a pilot study on 500 expert-reviewed samples to maximize F1. The judge evaluates four dimensions: medical correctness, reasoning consistency, step completeness, and clarity.

Stage 3: Expert-Collaborative Optimization

Three board-certified ophthalmologists review the 18% of samples flagged as difficult, achieving inter-rater agreement Cohen's $\kappa = 0.82$. Disagreements are resolved through discussion until consensus.

Difficulty grading partitions the dataset by the base model's perplexity:

d(x,y)=1βˆ’PΞΈ(y∣x),Level(x,y)={Easy,d<Ο„1Medium,Ο„1≀d<Ο„2Hard,dβ‰₯Ο„2d(x, y) = 1 - P_\theta(y|x), \quad \text{Level}(x, y) = \begin{cases} \text{Easy}, & d < \tau_1 \\ \text{Medium}, & \tau_1 \leq d < \tau_2 \\ \text{Hard}, & d \geq \tau_2 \end{cases}

Dataset Statistics

The final dataset contains 15,418 records:

  • Training set: 13,418 samples
  • Evaluation set: 2,000 samples (held-out clinical cases with no shared patient IDs)

To mitigate data contamination with external benchmarks, three checks are performed: perceptual hashing to detect near-duplicate images, cross-referencing source dataset identifiers to exclude overlapping samples, and manual audit of shared source institutions.

The currently open-sourced subset contains 3,418 samples for cold-start supervised fine-tuning.

Data Quality Assurance

  • Automated quality control: LVLM-as-a-Judge mechanism with threshold $\tau = 0.7$
  • Expert-collaborative review: Three board-certified ophthalmologists review difficult samples
  • Inter-rater agreement: Cohen's $\kappa = 0.82$
  • Data contamination detection: Perceptual hashing, source identifier cross-referencing, manual audit

Usage Example 

from datasets import load_dataset

# Load dataset
dataset = load_dataset("QiZishi/OphReason-Vision")

# View sample
print(dataset["train"][0])

Ethics Statement

OphReason-Vision draws from two data sources with appropriate ethical authorization:

  • 30+ public datasets: Published research with original IRB approval, used in accordance with their respective licenses
  • 100K+ clinical cases: Collected under IRB/Ethics Committee approval with research data use authorization

Comprehensive de-identification was performed, including removal of protected health information (PHI), metadata scrubbing, and facial cropping to ensure patient privacy. A waiver of informed consent was approved by the IRB for this retrospective research. All expert reviewers involved in data quality assessment are board-certified ophthalmologists who participated under institutional review protocols.

Citation

If you use OphReason-Vision, please cite our work:

@inproceedings{qi2026ophvlmr1,
  title={OphVLM-R1: Efficient Ophthalmic Reasoning via Curriculum RL},
  author={Qi, Zishi and Hu, Xiaoya and Pan, Huilin and Gao, Ang and Hou, Jiaxin and Li, Jiankun and Qian, Yongao},
  booktitle={Proceedings of the World Artificial Intelligence Conference (WAICA)},
  year={2026}
}

License

This dataset is licensed under CC-BY-NC-SA-4.0.

Related Resources

Downloads last month

-

Downloads are not tracked for this model. How to track
Video Preview
Reinforcement Learning
loading