Instructions to use moneebaa/qwen2vl-arabic-ocr-lora with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use moneebaa/qwen2vl-arabic-ocr-lora with PEFT:

from peft import PeftModel
from transformers import AutoModelForCausalLM

base_model = AutoModelForCausalLM.from_pretrained("Qwen/Qwen2-VL-2B-Instruct")
model = PeftModel.from_pretrained(base_model, "moneebaa/qwen2vl-arabic-ocr-lora")

Transformers

How to use moneebaa/qwen2vl-arabic-ocr-lora with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="moneebaa/qwen2vl-arabic-ocr-lora")

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoModel
model = AutoModel.from_pretrained("moneebaa/qwen2vl-arabic-ocr-lora", device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use moneebaa/qwen2vl-arabic-ocr-lora with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "moneebaa/qwen2vl-arabic-ocr-lora"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "moneebaa/qwen2vl-arabic-ocr-lora",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/moneebaa/qwen2vl-arabic-ocr-lora

SGLang

How to use moneebaa/qwen2vl-arabic-ocr-lora with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "moneebaa/qwen2vl-arabic-ocr-lora" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "moneebaa/qwen2vl-arabic-ocr-lora",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "moneebaa/qwen2vl-arabic-ocr-lora" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "moneebaa/qwen2vl-arabic-ocr-lora",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use moneebaa/qwen2vl-arabic-ocr-lora with Docker Model Runner:
```
docker model run hf.co/moneebaa/qwen2vl-arabic-ocr-lora
```

Qwen2-VL-2B Arabic OCR (LoRA adapter)

LoRA adapter for Qwen2-VL-2B-Instruct, fine-tuned on 300 samples from the KITAB-Bench synthesizear subset. This is a small-scale learning project and not an official KITAB-Bench result.

Developed by: Moneeba Abrar
Base model: Qwen/Qwen2-VL-2B-Instruct
Code, results and failure analysis: https://github.com/aka-Trophy-Hunter/Arabic-OCR-with-Qwen2-VL-LoRA-fine-tuning-

Results

	CER	WER
Base, in-domain	0.349	0.589
LoRA, in-domain	0.043	0.167
Base, out-of-domain	0.873	0.950
LoRA, out-of-domain	0.691	0.840

In-domain: 30 held-out samples from arocrbench_synthesizear. Out-of-domain: 30 samples from arocrbench_hindawi. Scores use KITAB-Bench's Arabic text normalization and jiwer.

Training details

LoRA r=16, alpha=32, dropout 0.05 on attention and MLP projections (18.5M trainable parameters)
300 training samples, 1 epoch (38 steps), learning rate 1e-4, effective batch size 8
fp16 on a Tesla T4; loss computed on answer tokens only
Prompt: "Extract the text in the image. Give me the final text, nothing else."

How to use

import torch
from transformers import Qwen2VLForConditionalGeneration, AutoProcessor
from peft import PeftModel

base = Qwen2VLForConditionalGeneration.from_pretrained(
    "Qwen/Qwen2-VL-2B-Instruct", torch_dtype=torch.float16, device_map="auto"
)
model = PeftModel.from_pretrained(base, "YOUR-USERNAME/qwen2vl-arabic-ocr-lora")
processor = AutoProcessor.from_pretrained("Qwen/Qwen2-VL-2B-Instruct")

For a full inference example, see the notebook in the GitHub repo.

Limitations

Tiny test sets (30 samples each) and a single run, so treat results as indicative only
The in-domain test is synthetic text matching the training distribution
Out-of-domain performance on book pages is still weak
KITAB-Bench evaluates 7B models; this is a 2B model, so numbers are not comparable with the paper

Credits

KITAB-Bench (Heakl et al., MBZUAI-Oryx), Qwen2-VL (Alibaba), Hugging Face Transformers and PEFT.

Framework versions

PEFT 0.21.2

Downloads last month: -

Model tree for moneebaa/qwen2vl-arabic-ocr-lora

Base model

Qwen/Qwen2-VL-2B

Finetuned

Qwen/Qwen2-VL-2B-Instruct

Adapter

(206)

this model