Instructions to use shbaydadaev/qwen-ocr-standalone-fp8 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use shbaydadaev/qwen-ocr-standalone-fp8 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="shbaydadaev/qwen-ocr-standalone-fp8")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("shbaydadaev/qwen-ocr-standalone-fp8")
model = AutoModelForMultimodalLM.from_pretrained("shbaydadaev/qwen-ocr-standalone-fp8", device_map="auto")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use shbaydadaev/qwen-ocr-standalone-fp8 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "shbaydadaev/qwen-ocr-standalone-fp8"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "shbaydadaev/qwen-ocr-standalone-fp8",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/shbaydadaev/qwen-ocr-standalone-fp8

SGLang

How to use shbaydadaev/qwen-ocr-standalone-fp8 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "shbaydadaev/qwen-ocr-standalone-fp8" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "shbaydadaev/qwen-ocr-standalone-fp8",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "shbaydadaev/qwen-ocr-standalone-fp8" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "shbaydadaev/qwen-ocr-standalone-fp8",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Docker Model Runner
How to use shbaydadaev/qwen-ocr-standalone-fp8 with Docker Model Runner:
```
docker model run hf.co/shbaydadaev/qwen-ocr-standalone-fp8
```

You need to agree to share your Hugging Face information to access this model

This repository is publicly accessible, but you have to accept the conditions to access its files and content.

Qwen OCR Standalone FP8

Standalone Persian/Arabic text-line OCR model. The trained OCR LoRA adapter is already merged into the base weights. No separate adapter or base download is required. Do not apply the adapter again.

Base: Qwen/Qwen3.8-27B-FP8, revision 017b9c7af6b5689d5dd426a76e0bc077eb5ca20a.
Approximately 30.9 GB across 66 Safetensors shards.
FP8 E4M3 block-quantized weights with BF16 components and BF16 inference compute.
Merge: dequantize affected weights to FP32, add rank-16 LoRA updates (alpha 32), then requantize the 400 affected FP8 layers. The other 96 affected BF16 layers remain BF16. Untargeted tensor values are preserved.
merge_provenance.json records the export method, source revision, adapter hash and package versions. Its export-time quality status predates the evaluation below.

Evaluation

Evaluation on 2026-11-01 used the same 100 held-out Persian/Arabic text lines as the earlier BF16+LoRA evaluation: 3,150 reference characters and 609 words.

Metric	Previous BF16 + LoRA	Standalone FP8
CER (lower is better)	0.8889%	1.1111%
WER (lower is better)	2.7915%	3.1199%
Exact transcriptions	88/100	86/100
Valid JSON/schema responses	100/100	100/100

No incomplete responses or HTTP failures. CER uses Unicode codepoints; WER splits on whitespace. No normalization or transcription repair was applied, so spacing and combining-mark order differences count as errors. See evaluation_summary.json.

This is a pilot evaluation, not the full 164,995-line publisher Test split. Existing reference approvals have unspecified reviewer provenance. Source language labels are unspecified, so separate Persian and Arabic scores are unavailable. Runtime and precision also changed relative to the baseline; this does not isolate the effect of merging. Full-page layout analysis, language identification and broader multilingual quality were not validated.

Download and verify

For this private repository, authenticate with an account that has access:

hf auth login
hf download shbaydadaev/qwen-ocr-standalone-fp8 \
  --local-dir models/qwen-ocr-standalone-fp8
cd models/qwen-ocr-standalone-fp8
sha256sum -c SHA256SUMS
cd ../..

Serve

Tested using vLLM 0.32.0 on an NVIDIA RTX PRO 6000 Blackwell GPU with approximately 96 GB VRAM. GPU memory needs include activations, image tokens and KV cache in addition to weights; checkpoint disk size is not a VRAM requirement.

vllm serve models/qwen-ocr-standalone-fp8 \
  --served-model-name ocr \
  --dtype bfloat16 --quantization fp8 \
  --max-model-len 4096 --max-num-seqs 1 \
  --limit-mm-per-prompt '{"image":1,"video":0}' \
  --default-chat-template-kwargs '{"enable_thinking":false}' \
  --gpu-memory-utilization 0.60 --enforce-eager \
  --host 127.0.0.1 --port 8000

Send a cropped text-line image to POST /v1/chat/completions using "model": "ocr", temperature 0, max_tokens: 1024, and chat_template_kwargs: {"enable_thinking": false}. Include an image URL or base64 data URL in the user message alongside the prompt from ocr_prompt.txt; replace {width} and {height} with the actual image dimensions.

The training output is JSON with text-r001.original_text. Language/script fields may be und and should not be treated as reliable language identification. Validate JSON and check the finish reason before consuming a response. For full pages, detect/crop text lines and assemble their reading order separately.

Files and license

Model shards, configuration, tokenizer, preprocessing settings, chat template, merge provenance, OCR prompt and aggregate evaluation are included. Raw training/test images and annotations are not included. SHA256SUMS covers the release files other than itself.

The bundled source license is Apache 2.0; retain the license and attribution when redistributing. This checkpoint is a modified OCR derivative, not an official Qwen release.

Downloads last month: 2

Safetensors

Model size

28B params

Tensor type

BF16

F8_E4M3

Model tree for shbaydadaev/qwen-ocr-standalone-fp8

Base model

Qwen/Qwen3.8-27B

Quantized

Qwen/Qwen3.8-27B-FP8

Finetuned

(9)

this model