NetraV4

NetraV4 is a 4B-parameter medical vision-language model that reasons step by step before answering. It reads medical images (radiology, pathology, clinical photos, figures) together with text and is tuned for medical visual question answering and medical exam-style reasoning.

It is built on Qwen3-VL-4B and trained with reinforcement learning on medical reasoning tasks.

Not a medical device. NetraV4 is a research model. Its answers can be wrong, and it must not be used for diagnosis, treatment or any clinical decision without review by a qualified professional.

Highlights

• Ahead of published open models under 15B parameters on most of the benchmarks below, including the 3.5× larger HealthGPT-14B, Lingshu-7B and MedGemma-4B.
• Above the published GPT-4.1 and Claude Sonnet 4 results on PMC-VQA, SLAKE (closed) and VQA-RAD (closed), and level with GPT-4o on GMAI-MMBench.
• The newest frontier models (GPT-5, Gemini 3 Pro) remain far ahead on expert-level reasoning (MedXpertQA, MedQA).

Training

• Base: Qwen3-VL-4B (Apache-2.0), with medical domain tuning.
• Method: reinforcement learning with a multi-reward policy optimisation objective on top of a PiSSA adapter (rank 64), followed by merging the adapter into the base weights (BF16).
• Rewards: answer correctness against verified references, plus reasoning-step quality judged by independent teacher models.
• Data: public medical VQA and medical reasoning datasets. The official test splits of every benchmark below were held out from training; benchmark questions were checked against the training data for text and image overlap.

Results

Accuracy (%). NetraV4 was evaluated with greedy decoding, up to 4,096 new tokens, and its own prompt format (below). The other columns are published results from each model's paper or report, obtained with their own prompts and settings. They are an indicative comparison, not a same-setup head-to-head.

Medical vision

Benchmark	NetraV4 (4B)	Frontier models (published)	Open models (published)
MedXpertQA-MM	26.5	GPT-5 71.2 · GPT-4.1 45.2 · Claude Sonnet 4 43.3	Lingshu-7B 26.7 · HealthGPT-14B 24.7 · MedGemma-4B 22.3 · Qwen2.5-VL-7B 22.3
GMAI-MMBench ¹	54.0	GPT-4o 54.0	—
VQA-RAD (closed) ²	74.3	GPT-5 70.9 · Claude Sonnet 4 67.6 · GPT-4.1 65.0	MedGemma-4B 72.5 · Lingshu-7B 67.9 · HuatuoGPT-V-7B 67.0 · HealthGPT-14B 65.0
SLAKE (closed) ²	76.2	GPT-4.1 72.2 · Claude Sonnet 4 70.6	Lingshu-7B 83.1 · MedGemma-4B 76.4 · HealthGPT-14B 66.1
PMC-VQA	57.9	GPT-4.1 55.2 · Claude Sonnet 4 54.4	HealthGPT-14B 56.4 · Lingshu-7B 56.3 · MedGemma-4B 49.9
MMMU-Med	64.1	GPT-4.1 75.2 · Claude Sonnet 4 74.6	InternVL3-8B 59.2 · Lingshu-7B 54.0 · HealthGPT-14B 49.6 · MedGemma-4B 43.7
VQA-RAD (open) ³	39.2	—	—
SLAKE (open) ³	51.4	—	—

Medical text

Benchmark	NetraV4 (4B)	Frontier models (published)	Open models (published)
MedQA (4-option)	76.8	Gemini 3 Pro 96.0 · GPT-5 95.8 · Claude Sonnet 4 92.1 · GPT-4.1 89.1	MedGemma-27B 87.7 · HealthGPT-14B 66.2 · MedGemma-4B 64.4 · Lingshu-7B 63.3
MedMCQA	63.5	Claude Sonnet 4 79.3 · GPT-4.1 77.7	HealthGPT-14B 63.4 · Lingshu-7B 55.9 · MedGemma-4B 55.7
PubMedQA	77.6	Claude Sonnet 4 78.6 · GPT-4.1 75.6	Lingshu-7B 76.6 · MedGemma-4B 73.4 · HealthGPT-14B 68.0
MMLU-Med	83.8	GPT-5 96.1 · Claude Sonnet 4 91.3 · GPT-4.1 89.6	HealthGPT-14B 80.2 · Lingshu-7B 74.5 · MedGemma-4B 70.0
MedXpertQA-Text	17.3	GPT-5 55.9 · Claude Sonnet 4 33.6 · GPT-4.1 30.9	Lingshu-7B 16.5 · MedGemma-4B 14.2 · HealthGPT-14B 11.3

¹ GMAI-MMBench: NetraV4 on the validation split (the test answers are not public); GPT-4o's published figure is on the test split. ² Closed (yes/no) questions only. Published VQA-RAD/SLAKE figures from the Lingshu report combine closed and open questions. ³ Open questions: mean token recall of the reference answer inside the model's <answer> tag; not comparable with published accuracy figures.

Sources of published results: GPT-5: Wang et al. 2025, Capabilities of GPT-5 on Multimodal Medical Reasoning (arXiv:2508.08224; MedXpertQA and MMLU-Med as the mean of the reported sub-scores). Gemini 3 Pro: Vals AI MedQA leaderboard. GPT-4.1, Claude Sonnet 4, Lingshu, HealthGPT-14B, HuatuoGPT-V, InternVL3, Qwen2.5-VL: Lingshu technical report (arXiv:2506.07044). MedGemma: MedGemma technical report (arXiv:2507.05201). GPT-4o on GMAI-MMBench: GMAI-MMBench (NeurIPS 2024).

Usage

import torch
from PIL import Image
from transformers import AutoModelForImageTextToText, AutoProcessor

model_id = "dipankar500/NetraV4"
model = AutoModelForImageTextToText.from_pretrained(model_id, torch_dtype=torch.bfloat16, device_map="auto")
processor = AutoProcessor.from_pretrained(model_id)

question = "Is there evidence of pleural effusion?"
instruction = ("Think step-by-step and enclose your reasoning in <think>...</think> tags. "
               "Then provide your answer in <answer>...</answer> tags.")
messages = [{"role": "user", "content": [
    {"type": "image", "image": Image.open("chest_xray.png")},
    {"type": "text", "text": question + "\n" + instruction},
]}]
prompt = processor.apply_chat_template(messages, tokenize=False, add_generation_prompt=True) + "<think>\n"
inputs = processor(text=[prompt], images=[messages[0]["content"][0]["image"]], return_tensors="pt").to(model.device)
output = model.generate(**inputs, max_new_tokens=4096, do_sample=False)
print(processor.decode(output[0, inputs["input_ids"].shape[1]:], skip_special_tokens=True))

For multiple-choice questions, list the options after the question, one per line (A. ..., B. ...); the model answers with the option inside <answer>...</answer>. The model also works with vLLM.

Limitations

• Expert-level clinical reasoning (MedXpertQA) remains far below frontier models.
• Long reasoning: some answers need more than 4,096 tokens and are cut off.
• Trained and evaluated mainly in English.
• May produce confident but incorrect statements; always verify against clinical sources.

License

Apache-2.0, following the base model Qwen3-VL-4B (Apache-2.0).