KoDistill-VL-2B

A Korean-focused Qwen3.5-2B fine-tune with bilingual distillation, SFT, and GRPO, followed by Korean OCR continuation. Fully merged BF16 safetensors; no adapter loading is required.

On our matched evaluation, K-STR official word accuracy improves 67.62% → 71.26% (+3.64 pp), while CER falls 32.14% → 15.00%. English MMStar and TextVQA scores also exceed the remeasured base.

Benchmarks

Benchmark	Metric	Qwen3.5-2B	Final	Δ vs Qwen (pp)
K-MMStar	Accuracy ↑	44.07	54.20	+10.13
MTVQA-KO	Strict EM ↑	26.16	29.21	+3.05
K-DTCBench	Accuracy ↑	72.08	80.83	+8.75
MMStar (EN)	Accuracy ↑	40.73	58.40	+17.67
TextVQA (EN)	VQA accuracy ↑	75.27	78.22	+2.95
K-STR	Official word accuracy ↑	67.62	71.26	+3.64
K-STR	CER ↓	32.14	15.00	-17.14
K-STR	WER ↓	46.79	33.47	-13.32

Scores are percentages; changes are percentage points. These are our matched, non-thinking, greedy evaluations, not Qwen's published leaderboard scores. The five VQA benchmarks contain 8,798 questions; K-STR uses all 20,086 released KAIST/GIST crops. CER/WER are auxiliary corpus errors on 19,855 crops without □ reference characters; CER retains internal spaces. Short output budgets and answer-format compliance affect the VQA comparison.

Training and scope

The final model continues an earlier bilingual SFT/GRPO checkpoint (V2). OCR continuation uses 33,371 SFT rows and 2,683 GRPO prompts, one completed pass each, on one RTX 5080 (16 GB). Development selection chooses SFT step 1,800 and GRPO step 600; actual selected exposure is 28,800 rows and 2,400 prompts. Language-side rank-16 LoRA leaves vision, embeddings, and the output head frozen.

Versus V2, K-STR gains +3.45 pp (paired scene-bootstrap 95% CI [+3.04, +3.84] pp). MMStar is unchanged and TextVQA gains +0.10 pp, while MTVQA-KO strict EM falls −0.18 pp. This is a single-seed follow-up on previously seen public tests; it neither establishes broad English preservation nor isolates a GRPO benefit.

Research article · SFT logs · GRPO logs · Plot data · Evaluation results

Use

Tested with Transformers 5.19.0 and PyTorch 2.14.1. Install transformers==5.19.0, torch, accelerate, and pillow; provide your own image as your-image.png.

import torch
from PIL import Image
from transformers import AutoProcessor, Qwen3_5ForConditionalGeneration

model_id = "drlee1/kodistill-vl-2b"
processor = AutoProcessor.from_pretrained(model_id)
model = Qwen3_5ForConditionalGeneration.from_pretrained(
    model_id, dtype=torch.bfloat16, device_map="cuda", attn_implementation="sdpa"
).eval()
processor.image_processor.size = {"shortest_edge": 64 * 32**2, "longest_edge": 512 * 32**2}
image = Image.open("your-image.png").convert("RGB")
messages = [{"role": "user", "content": [
    {"type": "image"},
    {"type": "text", "text": "Transcribe the text in the image. Return only the text."},
]}]
prompt = processor.apply_chat_template(
    messages, tokenize=False, add_generation_prompt=True, enable_thinking=False
)
inputs = processor(text=[prompt], images=[image], return_tensors="pt").to(model.device)
with torch.inference_mode():
    output = model.generate(**inputs, do_sample=False, max_new_tokens=256)
print(processor.decode(output[0, inputs["input_ids"].shape[1]:], skip_special_tokens=True))

License

Noncommercial research terms apply to our modifications and original materials: LICENSE. The original Qwen model remains Apache-2.0. Source datasets retain their own terms; training images and teacher-answer corpora are not redistributed. See NOTICE.