Instructions to use elliot-mllm/elliot-ov2-sft-9k with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use elliot-mllm/elliot-ov2-sft-9k with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="elliot-mllm/elliot-ov2-sft-9k", trust_remote_code=True)
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# Load model directly
from transformers import AutoModelForCausalLM
model = AutoModelForCausalLM.from_pretrained("elliot-mllm/elliot-ov2-sft-9k", trust_remote_code=True, device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use elliot-mllm/elliot-ov2-sft-9k with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "elliot-mllm/elliot-ov2-sft-9k"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "elliot-mllm/elliot-ov2-sft-9k",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/elliot-mllm/elliot-ov2-sft-9k

SGLang

How to use elliot-mllm/elliot-ov2-sft-9k with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "elliot-mllm/elliot-ov2-sft-9k" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "elliot-mllm/elliot-ov2-sft-9k",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "elliot-mllm/elliot-ov2-sft-9k" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "elliot-mllm/elliot-ov2-sft-9k",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Docker Model Runner
How to use elliot-mllm/elliot-ov2-sft-9k with Docker Model Runner:
```
docker model run hf.co/elliot-mllm/elliot-ov2-sft-9k
```

You need to agree to share your contact information to access this model

This repository is publicly accessible, but you have to accept the conditions to access its files and content.

Elliot OV2 SFT 9k

ElliotVL vision-language checkpoint, exported to standard Hugging Face format from the 9k-step Megatron checkpoint.


Parameters	2B
Language tower	Qwen3.5-2B (hybrid full + linear-attention, `model_type: qwen3_5_text` in this checkpoint's own `config.json`)
Vision tower	SigLIP2 NaViT (native-resolution)
Precision	BF16 / F32
Task	Image-Text-to-Text
Loading	requires `trust_remote_code=True`

Benchmarks

All ElliotVL numbers below were measured on this checkpoint with VLMEvalKit in no-think mode.

Benchmark	Elliot OV2 SFT 9k	Qwen2.5-VL-3B	Qwen3-VL-2B	Qwen3.5-2B (no-think)	InternVL3.5-2B
OCRBench	78.00	(79.2)	(85.8)	83.30*	(83.6)
OCRBench v2 (avg / En / Zh)	24.42 / 30.07 / 18.76	36.82* / 34.48 / 39.16	41.18* / 43.54 / 38.82	–	–
RealWorldQA	66.01	(60.0)	(63.9)	68.76*	(62.0)
MMBench DEV_EN_V1.1	75.15	76.63* (78.0)	(78.4)	73.14*	(76.6)
MME perception	1510.06	1579.33*	1508.94*	1587.27*	(sum 2123.3)
MME cognition	451.43	603.57*	488.57*	548.57*	(sum 2123.3)
MME-RealWorld EN	57.22	(51.6)	44.19*	30.89*	(49.7)
MMStar	54.47	(55.9)	(58.3)	50.53*	(62.7)
AI2D	78.40	(78.6 / 90.7)	(76.9)	78.11*	(78.8 / 89.1)
ChartQA TEST	68.84 raw · 83.72 format-audited	(83.4)	78.36* (79.1)	78.56*	(80.7)
HallusionBench (aAcc / qAcc)	60.46 / 35.82	60.67* / 34.95*	61.83* / 36.04* (51.4)	67.61* / 44.62*	(avg 48.6)
MMMU DEV_VAL	43.22	48.78* val / 50.00* dev (53.1 Qwen; 46.4 LLaVA)	41.67* val / 46.00* dev (53.4)	33.78* val / 37.33* dev	(59.0)
TextVQA VAL	79.65	79.25*	79.72*	79.15*	(76.5)
DocVQA VAL	90.31	–	–	92.53*	–
SEEDBench IMG	75.48	(74.8)	74.89*	75.77*	–
SEED-2 Plus	65.39	68.95*	67.59*	–	(68.0)
MathVista MINI †	60.40	–	–	–	–
MathVerse MINI †	25.69	–	–	–	–
MathVision †	19.97	–	–	–	–

* reproduced by us · (parentheses) value reported by the model's authors or a third party · – not measured.

Read the numbers with these caveats:

† Math rows are not official scores. They come from a local exact-match scorer with the judge API disabled, so they are not comparable to published MathVista/MathVerse/MathVision figures, which use an LLM judge.
ChartQA is reported twice. 68.84 is the raw VLMEvalKit relaxed-accuracy result. Its non-numeric answers require an exact string match, and a format audit found many otherwise-correct predictions carrying trailing punctuation or units; normalising those conservatively gives 83.72. The raw number is the one to compare against other raw numbers.
MME is split into perception and cognition (VLMEvalKit labels the latter reasoning in its raw CSV). InternVL3.5 reports only a sum, so its cells are not split.
OCRBench for Qwen3-VL is normalised from the reported 858 to 85.8.
MMMU reruns use an answer-only prompt; earlier exact-match values were invalid because the prompt produced long truncated explanations.
DocVQA official reports are on the test split; VAL and TEST are different rows and are not interchangeable.
CharXiv RQ/DQ were not run — they require a judge API that was disabled.

Usage

Requires Transformers 5.8.0.dev0 (development build from GitHub), plus torch, pillow, accelerate, safetensors.

from transformers import AutoModelForCausalLM, AutoProcessor

model = AutoModelForCausalLM.from_pretrained(
    "elliot-mllm/elliot-ov2-sft-9k", trust_remote_code=True, torch_dtype="bfloat16")
processor = AutoProcessor.from_pretrained(
    "elliot-mllm/elliot-ov2-sft-9k", trust_remote_code=True)

Text-only prompt:

prompt = (
    "<|im_start|>system\nYou are a helpful assistant.<|im_end|>\n"
    "<|im_start|>user\nAnswer briefly: what is a checkpoint?<|im_end|>\n"
    "<|im_start|>assistant\n<think>\n\n</think>\n\n"
)

For multimodal prompts, include one <image> placeholder in the user text per image passed to the processor.

Repository contents

config.json with auto_map entries for AutoConfig, AutoModelForCausalLM and AutoProcessor
Sharded safetensors weights with model.safetensors.index.json
Tokenizer, processor, image processor, chat template and remote-code modules

Export validation

Check	Result
Loaded tensors	761
Missing HF tensors	0
Missing Megatron tensors	0
Shape mismatches	0

This is a research checkpoint.

Downloads last month: 2

Safetensors

Model size

2B params

Tensor type

BF16

F32