Instructions to use dipankar500/Netra-Med-V3-7B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use dipankar500/Netra-Med-V3-7B with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="dipankar500/Netra-Med-V3-7B", trust_remote_code=True)
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# Load model directly
from transformers import AutoModelForCausalLM
model = AutoModelForCausalLM.from_pretrained("dipankar500/Netra-Med-V3-7B", trust_remote_code=True, device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use dipankar500/Netra-Med-V3-7B with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "dipankar500/Netra-Med-V3-7B"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "dipankar500/Netra-Med-V3-7B",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/dipankar500/Netra-Med-V3-7B

SGLang

How to use dipankar500/Netra-Med-V3-7B with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "dipankar500/Netra-Med-V3-7B" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "dipankar500/Netra-Med-V3-7B",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "dipankar500/Netra-Med-V3-7B" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "dipankar500/Netra-Med-V3-7B",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Docker Model Runner
How to use dipankar500/Netra-Med-V3-7B with Docker Model Runner:
```
docker model run hf.co/dipankar500/Netra-Med-V3-7B
```

Access Netra-Med-V3-7B

This model is released for NON-COMMERCIAL RESEARCH USE ONLY. It was fine-tuned on data that includes MIMIC-CXR (PhysioNet credentialed data) and PathCap (CC-BY-NC-2.0). You must confirm the statements below before downloading.

By requesting access you confirm that: (1) you will use these weights for non-commercial research only; (2) you will NOT use them to inform clinical decisions or patient care; (3) if your use touches MIMIC-CXR-derived behaviour you hold, or will obtain, the relevant PhysioNet credentialing and accept the corresponding Data Use Agreement; (4) you will not attempt to re-identify any individual from model outputs.

Netra-Med-V3-7B

A medical vision–language model fine-tuned from ZJU-AI4H/Hulu-Med-7B on a curated 50,000-row multimodal medical corpus spanning 8 imaging modalities and 32 source datasets.

These are merged full weights. You do not need the base model, PEFT, or an adapter — a single from_pretrained is enough.

Not a medical device. This model must not be used for diagnosis, triage, or any decision affecting patient care. It makes confident errors.

Results

Evaluated with a single fixed harness (greedy decoding, identical prompts) against the stock base model. Every number below was produced by the same harness, so the two columns are directly comparable to each other — but not to numbers published elsewhere, which use different prompts, decoding and scoring.

Benchmark	Hulu-Med-7B (base)	Netra-Med-V3-7B	Δ
OmniMedVQA	59.80	81.60	+21.80
SLAKE	68.17	86.20	+18.03
MMMU-Med	37.33	50.67	+13.34
PMC-VQA	49.80	57.40	+7.60
PathVQA	79.60	83.22	+3.62
VQA-RAD	76.49	76.10	−0.39
MedXpertQA	28.80	27.45	−1.35
Mean	57.14	66.09	+8.95

The larger gain is on open-ended answering, which the headline metric understates:

Open-ended recall	base	Netra-Med-V3-7B	Δ
SLAKE	22.07	77.92	+55.85
VQA-RAD	32.11	50.35	+18.24
PathVQA	16.78	34.19	+17.41

Two benchmarks regressed slightly (VQA-RAD −0.39, MedXpertQA −1.35). Both are within or near the measurement error and are reported rather than hidden.

Comparison with frontier and specialist models

Read the caveat before the table. The Netra-Med-V3-7B row was measured by our harness. The other rows are numbers those authors published using their harness. Prompt wording, answer matching and decoding all differ between harnesses, and that difference is not small.

Concrete evidence of exactly that: on MedXpertQA-MM, MedVLThinker reports GPT-4o at 35.95, while the benchmark's own official leaderboard reports GPT-4o at 42.80. Same model, same benchmark, ~7 points apart purely from protocol. So treat the table as indicative positioning, not a controlled head-to-head.

Reference numbers below are from MedVLThinker (arXiv:2508.02669), Table 2, which used greedy decoding averaged over 3 runs — the same decoding strategy this model was scored with.

Model	Params	PMC-VQA	MMMU-Med	MedXpert-MM	PathVQA	SLAKE	VQA-RAD	Mean
GPT-4o	—	58.55	68.82	35.95	72.43	76.44	70.22	63.74
Netra-Med-V3-7B	8B	57.40	50.67	27.45	83.22	86.20	76.10	63.51
GPT-4o-mini	—	51.90	63.53	28.55	63.33	75.24	66.91	58.24
MedVLThinker-7B (RL)	7B	50.67	56.86	24.43	66.83	65.79	64.71	54.88
HuatuoGPT-Vision-7B	7B	53.39	50.59	22.00	63.53	75.00	63.60	54.69
Qwen2.5-VL-7B-Instruct	7B	49.30	52.94	18.89	65.39	65.71	68.75	53.50
Hulu-Med-7B (our base)	8B	49.80	37.33	28.80	79.60	68.17	76.49	56.70

Where this model stands, stated plainly:

Ahead of GPT-4o on PathVQA (+10.8), SLAKE (+9.8) and VQA-RAD (+5.9) — the image-grounded perception benchmarks.
Behind GPT-4o on MMMU-Med (−18.2) and MedXpertQA (−8.5) — the benchmarks that require multi-step expert reasoning rather than reading the image. This is the honest weakness: an 8B model fine-tuned for perception does not acquire expert clinical reasoning.
Mean 63.51 vs GPT-4o's 63.74 — level on average, but the averages are made of very different strengths, and averaging across benchmarks of unequal difficulty flatters whichever model happens to win the easy ones.
Ahead of every listed open 7B model, including the specialist medical ones, by ~9 points mean.

On MedXpertQA specifically, the official leaderboard places o1 at 56.28, GPT-4o at 42.80, Gemini-2.0-Flash at 37.20, Claude-3.5-Sonnet at 33.20 and GPT-4o-mini at 28.05. Our 27.45 sits at the bottom of that range. Expert-level medical reasoning is not what this model is good at, and no amount of averaging should obscure that.

Usage

import torch
from transformers import AutoModelForCausalLM, AutoProcessor
from PIL import Image

model_id = "dipankar500/Netra-Med-V3-7B"
processor = AutoProcessor.from_pretrained(model_id, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
    model_id, trust_remote_code=True, dtype=torch.bfloat16, device_map="cuda")
model.eval()

image = Image.open("chest_xray.png")
conversation = [{"role": "user", "content": [
    {"type": "image", "image": image},
    {"type": "text",  "text": "Describe this image in detail and report all findings, "
                              "including any that are normal."},
]}]
inputs = processor.apply_chat_template(
    conversation, add_generation_prompt=True, tokenize=True,
    return_dict=True, return_tensors="pt").to("cuda")
out = model.generate(**inputs, max_new_tokens=512, do_sample=False)
print(processor.batch_decode(out[:, inputs["input_ids"].shape[1]:],
                             skip_special_tokens=True)[0])

trust_remote_code=True is required: Hulu-Med defines a custom architecture. The necessary modeling files are included in this repository, so nothing else needs to be fetched.

Prompting notes

Open-ended: ask for a report, not a label — "Describe this image in detail and report all findings, including any that are normal." The model was trained to produce structured Findings/Impression prose, and a terse prompt yields a terse answer.
Multiple choice: include the options in the prompt and end with "Answer with the letter of the correct option." Withholding the options costs a large amount of accuracy.

Training


Base	ZJU-AI4H/Hulu-Med-7B @ `258594714a0d3835eb2c9e4cc165a4242e606d71`
Method	AdaLoRA (`init_r=32`, `target_r=16`), merged into the base for release
Precision	bf16
Optimiser	AdamW, cosine schedule
Corpus	50,000 curated rows · 32 sources · 8 modalities
Hardware	1× NVIDIA A10G (22 GB)
Selection	lowest held-out validation loss (0.89925)

Classification datasets were not trained as bare labels. A label like malignant is not language, and training image → "malignant" teaches a model to emit one token rather than to describe evidence. Every classification source was rendered into a question/answer pair with varied phrasing, and multi-label chest-X-ray observations were rendered as Findings/Impression reports that state pertinent negatives. Uncertain labels became explicit "cannot be determined from this image" rather than being dropped or forced positive.

The corpus was contamination-filtered against all evaluation sets (measured overlap: 0).

Training data and licences

This model inherits obligations from its training data. Principal sources in the 50k corpus:

Source	Share	Terms
eltorio/ROCOv2-radiology	16.0%	research use
UCSC-VLAA/MedTrinity-25M	16.0%	research use
YYama0/RadFig-VQA	16.0%	research use
danjacobellis/chexpert	16.0%	Stanford CheXpert research-use terms
jamessyx/PathCap	16.0%	CC-BY-NC-2.0 — non-commercial
MLforHealthcare/mimic-cxr	8.8%	PhysioNet credentialed DUA
redlessone/Derm1M	4.3%	research use
25 further sources	6.9%	research use

Because of PathCap and MIMIC-CXR in particular, these weights are released for non-commercial research only and access is gated. Commercial use is not permitted.

Limitations

Not validated clinically. No prospective study, no regulatory clearance, no evidence of safety or efficacy in any care setting.
MedXpertQA 27.45 — hard expert-level reasoning remains weak, and slightly below base.
Benchmark gains do not transfer automatically to real clinical distributions; the evaluation sets are public benchmarks, not prospective patient data.
Trained predominantly on English text.
Will hallucinate findings when the image is ambiguous, low quality, or out of distribution.
Demographic performance was not audited; disparate error rates across patient groups are likely and unmeasured.

Citation

@misc{netra_med_v3_7b,
  title  = {Netra-Med-V3-7B: a medical vision-language model fine-tuned from Hulu-Med-7B},
  author = {Dipankar},
  year   = {2026},
  url    = {https://huggingface.co/dipankar500/Netra-Med-V3-7B}
}

Please also cite the base model, Hulu-Med, and the underlying datasets.

Downloads last month: -

Safetensors

Model size

8B params

Tensor type

BF16

Model tree for dipankar500/Netra-Med-V3-7B

Base model

ZJU-AI4H/Hulu-Med-7B

Finetuned

(1)

this model

Paper for dipankar500/Netra-Med-V3-7B

MedVLThinker: Simple Baselines for Multimodal Medical Reasoning

Paper • 2508.02669 • Published Aug 4, 2025