Instructions to use datalab-to/surya-parsebench-rl with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use datalab-to/surya-parsebench-rl with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="datalab-to/surya-parsebench-rl")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("datalab-to/surya-parsebench-rl")
model = AutoModelForMultimodalLM.from_pretrained("datalab-to/surya-parsebench-rl", device_map="auto")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use datalab-to/surya-parsebench-rl with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "datalab-to/surya-parsebench-rl"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "datalab-to/surya-parsebench-rl",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/datalab-to/surya-parsebench-rl

SGLang

How to use datalab-to/surya-parsebench-rl with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "datalab-to/surya-parsebench-rl" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "datalab-to/surya-parsebench-rl",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "datalab-to/surya-parsebench-rl" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "datalab-to/surya-parsebench-rl",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Docker Model Runner
How to use datalab-to/surya-parsebench-rl with Docker Model Runner:
```
docker model run hf.co/datalab-to/surya-parsebench-rl
```

Surya ParseBench RL

This is a research checkpoint, not a product model. We built it to test one question: how much of a ParseBench score comes from reading the page, and how much comes from matching the benchmark's output conventions?

We took Surya OCR 2, a 650M-parameter OCR model, and trained it with 70 steps of RL. We did not train on any ParseBench documents. It went from 23rd to 3rd on the ParseBench leaderboard (mean of the four non-chart dimensions). Almost all of the gain came from formatting. Content faithfulness, the score closest to "did the model read the page correctly", moved 0.4 points.

For OCR, use Surya OCR 2 or Chandra OCR 2, not this checkpoint.

Results

ParseBench, our run of the public harness (parse-bench==1.0.4), with the provider in code/. Charts are left out of every number.

	Tables	Content	Semantic formatting	Visual grounding	Mean of 4
Surya OCR 2, before RL	82.10	89.00	74.34	68.11	78.39
This checkpoint (70 RL steps)	83.04	89.38	85.24	70.33	82.00
This checkpoint + loop retries and table fixes	84.14	90.00	86.14	72.31	83.15
LlamaParse Agentic Plus (leaderboard 1st)	93.37	92.25	87.12	84.09	89.21

On the leaderboard at commit b72c714, 83.15 ranks 3rd and 82.00 ranks 5th.

What moved:

Semantic formatting: +10.9 from RL.
Content faithfulness: +0.4.
Tables: +0.9 from RL, and +1.1 more from the inference fixes.
Visual grounding: +2.2 from RL, and +2.0 more from the inference fixes.

Notes:

These are our runs, not the leaderboard's pipeline.
The "before RL" row uses the same converter as the other rows, so the rows compare the model only.
The parse-bench layout loader cannot score 64 of the 500 layout documents for any provider. Visual grounding averages over the other 436.

Training

70 steps of RL from Surya OCR 2. We did not train on any ParseBench documents.

How to run it

The score needs the model, the exact prompt, the retry rule, and the converter in code/.

Serve with vLLM:

vllm serve datalab-to/surya-parsebench-rl --served-model-name surya --max-model-len 32768 \
  --limit-mm-per-prompt '{"image":1}'

Render each PDF page at 192 DPI. Send the image with this exact prompt: OCR this image to HTML. Each block is a div with data-label and data-bbox (x0 y0 x1 y1, normalized 0-1000).
Decode greedy with max_tokens 28000. If the output does not end with </div>, the page looped. Retry with repetition_penalty 1.1, then 1.25, and keep the first output that closes.
Convert the layout HTML with code/surya_convert.py. The ParseBench provider in code/pbext/__init__.py does steps 2 to 4. code/bench.sh shows how we ran the four groups and built the scorecard.

config.json sets text_config.eos_token_id to 2. This checkpoint has no MTP head, so run it without speculative decoding.

License

The model weights use a modified AI Pubs Open Rail-M license, the same as Surya OCR 2: free for research, personal use, and startups under $5M in funding or revenue. The code in code/ is Apache 2.0.

Downloads last month: 11

Safetensors

Model size

0.7B params

Tensor type

BF16

Model tree for datalab-to/surya-parsebench-rl

Base model

datalab-to/surya-ocr-2

Finetuned

(7)

this model