SenOCR-Vi-HTR

Decoder-only LoRA on VietAlphaLabs/SenOCR-Vi (PaddleOCR-VL-1.6) for Vietnamese handwriting.

Non-commercial / research. Training data includes handwriting sources that are CC-BY-NC or research-grey. Do not use this adapter in a commercial product.

Technical report (PDF): vi_htr_lora_report.pdf

Results (micro CER, lower is better)

Handwriting bench, 562 images. Ours vs frozen SenOCR-Vi, PaddleOCR-VL-1.6, and VietOCR VGG-Transformer.

Split	n	Ours	SenOCR-Vi	VL-1.6	VietOCR
Overall HW	562	0.040	0.083	0.210	0.572
UIT-HWDB official test (unseen)	158	0.074	0.112	0.267	0.270
Cinnamon leftover addresses	200	0.021	0.082	0.181	0.203
Mix-val HW (no printed replay)	204	0.035	0.075	0.202	0.782

Exact-line accuracy overall: 40.6% (SenOCR-Vi 8.7%, VL-1.6 0.0%, VietOCR 0.2%).

Adapter

Base	VietAlphaLabs/SenOCR-Vi
Type	LoRA, decoder only (q/k/v/o/gate/up/down_proj)
Rank / alpha / dropout	32 / 64 / 0.05
Trainable	12,091,392 params, 126 modules
Frozen	vision encoder, merger, embeddings, LM head
Train	3 epochs, 366 steps, batch 64, LR 1e-4 cosine min 1e-5, BF16, 1× RTX 3090 (2 h 28 m)
Eval loss	0.169 @ step 200 → 0.146 @ step 366

Training mix (this run)

Intended SenOCR-Vi recipe: 55% real HW / 20% style / 10% Cinnamon / 15% printed replay.

Gated 5CD-AI/VietHTR-Line and 5CD-AI/Viet-Handwriting-OCR-v2 were not available, so the 55% lane is iAmHieu handwriting.

Lane	Share	Source	Train n
Heavy HW	55%	iAmHieu	4,268
Style	20%	UIT-HWDB-line	1,552
Domain	10%	Cinnamon addresses	776
Replay	15%	400 local printed synth lines, upsampled	1,164
Train / val			7,760 / 240

Unicode is NFC. Prompt is OCR:.

Usage

Load the native Transformers PaddleOCR-VL implementation (transformers>=5). Hub trust_remote_code configs from 4.55 crash on vision RoPE in 5.20. Turn KV cache on (the base config ships use_cache: false).

import torch
from PIL import Image
from transformers import AutoConfig, AutoModelForImageTextToText, AutoProcessor
from peft import PeftModel

base_id = "VietAlphaLabs/SenOCR-Vi"
adapter_id = "trunghlt/senocr-vi-htr"

processor = AutoProcessor.from_pretrained(adapter_id)
processor.tokenizer.padding_side = "left"

config = AutoConfig.from_pretrained(base_id)
config.tie_word_embeddings = False
config.use_cache = True
if getattr(config, "text_config", None) is not None:
    config.text_config.use_cache = True
    if getattr(config.vision_config, "rope_parameters", None) is None:
        config.vision_config.rope_parameters = {"rope_type": "axial", "rope_theta": 10000.0}

model = AutoModelForImageTextToText.from_pretrained(
    base_id, config=config, torch_dtype=torch.bfloat16, attn_implementation="sdpa"
)
model = PeftModel.from_pretrained(model, adapter_id).eval().cuda()
model.config.use_cache = True
model.generation_config.use_cache = True

image = Image.open("line.jpg").convert("RGB")
messages = [{"role": "user", "content": [
    {"type": "image", "image": image},
    {"type": "text", "text": "OCR:"},
]}]
inputs = processor.apply_chat_template(
    messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt"
).to("cuda")
out = model.generate(**inputs, max_new_tokens=96, do_sample=False, use_cache=True)
print(processor.decode(out[0][inputs["input_ids"].shape[-1]:], skip_special_tokens=True))

Files

• adapter_model.safetensors / adapter_config.json — LoRA
• tokenizer.json, chat_template.jinja, processor configs
• vi_htr_lora_report.pdf — full technical report
• vi_htr_lora_report.tex — report source

License

CC-BY-NC-4.0. The SenOCR-Vi / PaddleOCR-VL-1.6 weights are Apache 2.0; this adapter is restricted by the fine-tuning data.

Citation

See vi_htr_lora_report.pdf. Base model: VietAlpha Lab, SenOCR-Vi, 2026.