MohamedQiqa/legal-documents-ocr-parser-1.0

📜 Arabic Legal Documents OCR Parser

Gemma-3-4B-IT — Fine-Tuned for Structured OCR Extraction from Arabic Legal Documents

---

A multimodal vision-language model specialized in extracting structured metadata from scanned Arabic legal documents, including government regulations, official correspondence, and institutional records.

---

Table of Contents

• Model Details
  • Model Description
  • Model Sources
• Uses
  • Direct Use
  • Downstream Use
  • Out-of-Scope Use
• How to Get Started
• Training Details
  • Training Data
  • Training Procedure
  • Training Hyperparameters
• Evaluation
• Technical Specifications
• Bias, Risks, and Limitations
• Environmental Impact
• Citation
• Model Card Authors
• Model Card Contact

---

Model Details

Model Description

Arabic Legal Documents OCR Parser is a multimodal vision-language model fine-tuned from Google's Gemma-3-4B-IT for the task of structured metadata extraction from scanned Arabic legal documents. Given an image of a legal document page, the model outputs a comprehensive JSON object containing classified document metadata — including document type, issuing authority, physical properties, official seals/stamps, signatures, routing information, and quality assessments.

The model was fine-tuned using LoRA (Low-Rank Adaptation) via LLaMA-Factory. After training, the LoRA adapter was merged back into the base model — this repository contains the full-precision merged weights, ready for direct inference without any adapter loading or quantization.

Attribute	Details
Developed by	MohamedSamyAI
Model type	Multimodal Vision-Language Model (Image-to-Text)
Language(s)	Arabic (primary), English (secondary)
License	Apache 2.0
Fine-tuned from	google/gemma-3-4b-it
Fine-tuning method	LoRA (adapter merged into base model)
Model format	Full-precision merged weights (no quantization)
Fine-tuning framework	LLaMA-Factory

Model Sources

• Repository: MohamedSamyAI/arabic-legal-documents-ocr-parser-1.0
• Base model: google/gemma-3-4b-it
• Fine-tuning framework: LLaMA-Factory

---

Uses

Direct Use

This model is designed for automated extraction of structured metadata from scanned images of Arabic legal documents. It is suitable for:

• Document digitization pipelines — Automating the cataloging and indexing of scanned legal archives.
• Legal document triage — Quickly classifying document types (regulations, correspondence, memos, etc.) and identifying issuing authorities.
• Metadata extraction — Extracting seals, stamps, signatures, dates, reference numbers, and routing information from scanned pages.
• Quality assessment — Evaluating scan quality, completeness, and legibility for downstream archival processes.

Downstream Use

The model can be integrated into larger systems:

• Document management systems (DMS) — As a preprocessing component that auto-populates metadata fields.
• Legal research platforms — To index and search across large corpora of Arabic legal documents.
• Government digitization initiatives — For bulk processing of ministerial and regulatory archives.
• RAG pipelines — As a structured extraction layer feeding into retrieval-augmented generation systems.

Out-of-Scope Use

This model is NOT intended for:

• ❌ Full-text OCR — The model extracts structured metadata, not the full body text of documents.
• ❌ Legal advice or interpretation — The model classifies and extracts metadata; it does not provide legal analysis.
• ❌ Non-Arabic documents — While the model understands English prompts, it is specialized for Arabic legal documents.
• ❌ Handwritten text recognition — The model is trained on printed/typed scanned documents; handwriting extraction is not supported.
• ❌ Real-time or safety-critical applications — The model may produce incorrect extractions and should not be used where errors have significant legal consequences without human review.

---

How to Get Started with the Model

Using Transformers

from transformers import AutoProcessor, AutoModelForImageTextToText
from PIL import Image
import torch
import json

# Load the merged model directly — no adapter loading needed
model_id = "MohamedSamyAI/arabic-legal-documents-ocr-parser-1.0"
processor = AutoProcessor.from_pretrained(model_id)
model = AutoModelForImageTextToText.from_pretrained(
    model_id,
    torch_dtype=torch.float16,
    device_map="auto",
)

# Load your document image
image = Image.open("path/to/legal_document.jpg")

# Prepare the prompt
prompt = """<image>You are a professional OCR Details Extractor.
Your rule to extract the: document_classification, source, physical_properties, official_marks, signatures_authorization, routing_distribution, attachments_references, condition_notes and confidence_quality of the document.
Extract the final output into a json format.
Do not generate any introduction or conclusion."""

messages = [
    {"role": "user", "content": [
        {"type": "image"},
        {"type": "text", "text": prompt}
    ]}
]

# Generate
inputs = processor.apply_chat_template(messages, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=2048)
result = processor.decode(outputs[0], skip_special_tokens=True)

# Parse the JSON output
parsed = json.loads(result)
print(json.dumps(parsed, indent=2, ensure_ascii=False))

💡 Note: This is a fully merged model — you load it directly like any other model. No adapter paths or LoRA configuration required.

Using LLaMA-Factory CLI (Inference)

llamafactory-cli chat \
  --model_name_or_path MohamedSamyAI/arabic-legal-documents-ocr-parser-1.0 \
  --template gemma3

---

Training Details

Training Data

The model was fine-tuned on a custom dataset of Arabic legal documents, including:

• Document types: Government regulations, ministerial correspondence, official circulars, legal decrees, and institutional records.
• Source: Scanned images of documents from various Saudi Arabian government ministries (e.g., Ministry of Justice).
• Format: ShareGPT-style conversation pairs (human → image + instruction, gpt → structured JSON output).
• Annotations: Each document image is paired with a comprehensive JSON annotation covering 9 extraction categories.

Split	Description
Train	Primary training set with diverse legal document types
Validation	Held-out set for monitoring overfitting during training

Training Procedure

The model was fine-tuned using Supervised Fine-Tuning (SFT) with QLoRA (4-bit quantized LoRA). After training, the LoRA adapter was merged back into the base model to produce the full merged weights published in this repository.

• Framework: LLaMA-Factory (commit 762b480)
• Method: LoRA applied to all linear layers (lora_target: all)
• Quantization (training only): 4-bit NF4 via bitsandbytes (used during training for memory efficiency)
• Published model: Full merged weights (FP16) — no quantization applied to the released model
• Vision tower: Frozen (pretrained vision encoder preserved)
• Gradient checkpointing: Enabled (memory optimization)
• Optimizer: Paged AdamW 8-bit (paged_adamw_8bit)

Training Hyperparameters

Parameter	Value
LoRA rank	16
LoRA target	All linear layers
Quantization (training)	4-bit NF4 (bitsandbytes) — training only, merged model is unquantized
Batch size	1 (per device)
Gradient accumulation	8 steps
Effective batch size	8
Learning rate	1e-4
LR scheduler	Cosine
Warmup ratio	0.1
Epochs	3
Max sequence length	2048 tokens
Precision	FP16
Gradient checkpointing	✅ Enabled
Vision tower	🧊 Frozen

Preprocessing

• Document images are processed through Gemma-3's native vision encoder.
• Text inputs use the gemma3 chat template via LLaMA-Factory.
• Target outputs are structured JSON strings with Arabic content (ensure_ascii=False).

---

Evaluation

Evaluation was performed on the held-out validation set during training, with metrics logged via Weights & Biases.

Metric	Details
Eval strategy	Every 50 training steps
Eval batch size	1
Tracking	WandB (yt-ocr-finetune-llamafactory)

Output Schema

The model produces structured JSON with the following extraction categories:

├── document_classification   → type, subtype, category, languages
├── source                    → issuing authority, department, dates, references
├── physical_properties       → page info, scan quality, orientation, watermarks
├── official_marks            → seals, stamps, barcodes/QR codes
├── signatures_authorization  → signatories, approval chains
├── routing_distribution      → addressees, CC, forwarding info
├── attachments_references    → mentioned attachments, referenced documents
├── condition_notes           → completeness, legibility, damage, annotations
└── confidence_quality        → overall confidence, uncertain elements, review flags

---

Technical Specifications

Model Architecture and Objective

• Architecture: Gemma-3-4B-IT (multimodal vision-language model) — full merged model
• Objective: Supervised fine-tuning for structured JSON generation from document images
• Parameters: ~4 billion (full model, adapter merged into base weights)
• Trainable parameters (during training): Significantly fewer due to rank-16 LoRA on all linear layers

Compute Infrastructure

Component	Specification
Platform	Kaggle Notebooks
GPU	NVIDIA P100 / T4 (16 GB VRAM)
Precision	FP16 mixed-precision
Memory optimization (training)	4-bit QLoRA + gradient checkpointing + paged optimizer
Runtime	Python 3.12, PyTorch 2.8.0, Transformers 4.57.6

---

Bias, Risks, and Limitations

Known Limitations

• Domain specificity: The model is trained exclusively on Arabic legal documents. Performance on other document types (medical, financial, academic) is not guaranteed.
• Language bias: While the model understands English prompts, extracted content is predominantly Arabic. Multi-language documents may have reduced accuracy.
• Scan quality sensitivity: Low-resolution, heavily damaged, or poorly scanned documents may produce incomplete or inaccurate extractions.
• Hallucination risk: Like all generative models, this model may occasionally produce plausible-sounding but incorrect metadata fields.
• Date/number extraction: Calendar conversions (Hijri ↔ Gregorian) and number extraction from degraded scans remain challenging.

Risks

• Over-reliance: Users should not treat model outputs as ground truth for legal proceedings without human verification.
• Privacy: The training data contains scanned government documents. Users should ensure compliance with applicable data protection regulations when deploying this model.

Recommendations

• ✅ Always include human-in-the-loop verification for critical metadata fields.
• ✅ Monitor the confidence_quality field in model outputs to flag uncertain extractions.
• ✅ Use the requires_manual_review flag to route low-confidence documents to human reviewers.
• ✅ Fine-tune further on your specific document corpus for best results.

---

Environmental Impact

Carbon emissions were estimated using the ML CO2 Impact Calculator.

Factor	Estimate
Hardware type	NVIDIA P100 (16 GB)
Hours used	~3–4 hours
Cloud provider	Kaggle (Google Cloud)
Compute region	US
Carbon emitted	~0.5 kg CO₂ eq. (estimated)

---

Citation

If you use this model in your research or applications, please cite:

BibTeX

@misc{mohamedsamyai2026arabiclegaldocparser,
  title        = {Arabic Legal Documents OCR Parser},
  author       = {Mohamed Samy},
  year         = {2026},
  publisher    = {Hugging Face},
  howpublished = {\url{https://huggingface.co/MohamedSamyAI/arabic-legal-documents-ocr-parser-1.0}},
  note         = {Fine-tuned from google/gemma-3-4b-it using QLoRA via LLaMA-Factory, adapter merged into base}
}

APA

Mohamed Samy. (2026). Arabic Legal Documents OCR Parser [Multimodal Language Model]. Hugging Face. https://huggingface.co/MohamedSamyAI/arabic-legal-documents-ocr-parser-1.0

---

Model Card Authors

• MohamedSamyAI

Model Card Contact

For questions, suggestions, or issues with this model card, please open a discussion on the model repository or contact the author via Hugging Face.

---

Built with ❤️ using Gemma 3 and LLaMA-Factory