shrew-ocr-preview-lora

LoRA adapter for shrew-ocr-preview — per-page document image → structured JSON (metadata, summary, RAG-ready semantic chunks, figures/tables with bounding boxes and HTML), fine-tuned from ibm-granite/granite-vision-4.1-4b.

This repo contains only the adapter: r=256, α=512, dropout 0.05, language-model decoder only (q/k/v/o and gate/up/down projections across all 40 layers). The vision tower and projectors are untouched — all task adaptation lives in the language blocks.

Use this repo for adapter composition, inspecting the fine-tune delta, or continued training. For inference, use shrew-ocr-preview (merged bf16) or shrew-ocr-preview-GPTQ-8bit: adapter-path serving at rank 256 is slower (measured ~590 tok/s decode ceiling for base + adapter vs 850–1,400 tok/s for the merged INT8 variant on the same hardware). The reference serving pipeline for the merged variants is shrew-server (PDF in, structured JSON out; see the merged model card).

Applying the adapter

The base model's stock image tiling does not match the adapter's training. Set the bucket tile grids in both the model config and the image processor before inference; without this, output quality degrades severely:

from transformers import AutoModelForImageTextToText, AutoProcessor
from peft import PeftModel

PINPOINTS = [[1536, 1152], [2304, 1536], [3072, 2304], [1152, 1152]]

base = AutoModelForImageTextToText.from_pretrained(
    "ibm-granite/granite-vision-4.1-4b", dtype="bfloat16", trust_remote_code=True)
model = PeftModel.from_pretrained(base, "btbtyler09/shrew-ocr-preview-lora")

processor = AutoProcessor.from_pretrained(
    "ibm-granite/granite-vision-4.1-4b", trust_remote_code=True)
model.config.image_grid_pinpoints = PINPOINTS
processor.image_processor.image_grid_pinpoints = PINPOINTS

(The merged repos ship config.json/preprocessor_config.json with these values already set.)

All other usage requirements apply unchanged: system prompt set to the literal string structured_extraction, temperature 0, and glyph-routed page preparation into the bucket resolutions above. The prepare_page reference implementation, output schema, serving guidance, and streaming repetition guard are documented in the merged model card. Read it before deploying.

For vLLM adapter serving, pass --enable-lora --max-lora-rank 256; the config patch above still applies to the base checkpoint. The merged variants serve faster.

Results and limitations

Measured results on the OHR-Bench document-RAG corpus (8,561 pages; retrieval hit@5/MRR@10 vs human ground truth, MinerU, and PaddleOCR), together with known failure modes (dense broadsheet scans, non-Latin scripts, broadsheet reading order), are documented in the merged model card — the merged bf16 model is mathematically identical to base + this adapter.

Versions

variant	precision	size	notes
shrew-ocr-preview	bf16	7.5 GB	reference quality
shrew-ocr-preview-GPTQ-8bit	INT8 LM / bf16 vision	4.9 GB	~1.8× serving throughput, +0.25% domain perplexity; serve with --dtype half
shrew-ocr-preview-GGUF	Q8_0 or f16 LM / f16 vision	3.6–6.8 GB	llama.cpp; full context per slot required (-c = N × 32768)
shrew-ocr-preview-lora	LoRA adapter (r=256, bf16) (this repo)	2.0 GB	for composition / continued training

This is a preview: weights update in place under these names as the model improves, and the three repos are always pushed in lockstep from the same training generation. Each push's commit message records the generation — pin a commit (revision=) for reproducibility.

Base model: ibm-granite/granite-vision-4.1-4b (Apache 2.0), trained with peft 0.18.1.