Instructions to use Geong/DARL with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use Geong/DARL with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="Geong/DARL", trust_remote_code=True)
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# Load model directly
from transformers import AutoModelForCausalLM
model = AutoModelForCausalLM.from_pretrained("Geong/DARL", trust_remote_code=True, device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use Geong/DARL with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Geong/DARL"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Geong/DARL",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/Geong/DARL

SGLang

How to use Geong/DARL with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "Geong/DARL" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Geong/DARL",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "Geong/DARL" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Geong/DARL",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Docker Model Runner
How to use Geong/DARL with Docker Model Runner:
```
docker model run hf.co/Geong/DARL
```

DARL: Efficient Document-to-Markup Generation via Look-Ahead Diffusion Trajectory Sampling

Pre-trained weights for DARL (ECCV 2026), a hybrid diffusion-autoregressive framework for document-to-markup generation. DARL reaches 2.3× inference speedup on document parsing tasks while keeping accuracy comparable to autoregressive baselines.

📄 Paper: ECCV 2026
💻 Code: GitHub

Method

DARL introduces two components on top of a vision-language backbone:

Online Monte Carlo Trajectory Generation (OMTG): real-time trajectory sampling with a sliding window mechanism
Diffusion Trajectory Preference Optimization (DTPO): reinforcement learning with longest common prefix (LCP) rewards

At inference, multiple tokens are generated in parallel via special initialization tokens, then verified incrementally from left to right by the sliding window.

Model Details


Base model	dots.ocr
Architecture	`DotsOCRForCausalLM` (`dots_ocr`)
Parameters	~3B (bfloat16)
Language model	28 layers, hidden size 1536, 12 heads (2 KV heads)
Vision encoder	`dots_vit`, 42 layers, patch size 14
Context length	131072
Precision	bfloat16

Usage

Requires trust_remote_code=True since the model ships custom modeling code.

Note: This repo ships the weights, config and tokenizer only. The image preprocessor config is not included — load the processor from the base model dots-studio/dots.ocr as shown below, or copy preprocessor_config.json from there into this repo.

from transformers import AutoModelForCausalLM, AutoProcessor
from PIL import Image

model = AutoModelForCausalLM.from_pretrained(
    "your-org/DARL",
    trust_remote_code=True,
    torch_dtype="bfloat16",
    attn_implementation="flash_attention_2",
    device_map="cuda",
)
# processor comes from the base model, see note above
processor = AutoProcessor.from_pretrained("dots-studio/dots.ocr", trust_remote_code=True)

image = Image.open("document.jpg")
prompt = "Recognize image as Markdown format"

messages = [{"role": "user", "content": [
    {"type": "image", "image": image},
    {"type": "text", "text": prompt},
]}]
text = processor.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = processor(text=[text], images=[image], return_tensors="pt").to(model.device)

outputs = model.generate(**inputs, max_new_tokens=4096)
print(processor.batch_decode(outputs[:, inputs.input_ids.shape[1]:], skip_special_tokens=True)[0])

Requirements

Python 3.8+, CUDA 12.1+
Flash Attention 2.x
transformers >= 4.51.3

Evaluation

Evaluated on document parsing benchmarks:

OmniDocBench-1.5
olmOCR-Bench

Speedup scales with model size, up to 2.78× on 14B backbones. See the paper and repository for full numbers.

Citation

@inproceedings{darl2026,
  title={DARL: Efficient Document-to-Markup Generation via Look-Ahead Diffusion Trajectory Sampling},
  author={Yang, Wentao and Shi, Yongxin and Tang, Rui and Zhang, Peirong and Wu, Shihang and He, Huiguo and Huang, Zheng and Peng, Dezhi and Liao, Minghui and Jin, Lianwen},
  booktitle={European Conference on Computer Vision (ECCV)},
  year={2026}
}

Acknowledgements

Built on top of dots.ocr and Qwen-VL. Supported by the National Natural Science Foundation of China (62476093) and the Natural Science Foundation of Guangdong Province (2026A1515012038).

License

Apache 2.0

Contact

wente_young@foxmail.com

Downloads last month: -

Model tree for Geong/DARL

Base model

dots-studio/dots.ocr

Finetuned

(7)

this model