Apprentice --- Gemma 4 E4B LoRA (receipt extraction)
Second model family for the same Apprentice benchmark task: LoRA adapter fine-tuned on 140
golden examples (Voxel51/scanned_receipts, real SROIE scanned receipts, human-annotated,
CC-BY-4.0) to extract company, address, date, total from receipt OCR text as JSON.
Results (60 held-out rows, field-level F1)
| Model | Score |
|---|---|
| gpt-4o-mini, plain prompt | 72.92 |
| gpt-4o-mini, GEPA-optimized prompt | 84.17 |
| gpt-5.4-mini, plain prompt | 72.92 |
| gpt-5.4-mini, GEPA-optimized prompt | 79.58 |
| Qwen3.5-4B raw | 42.50 |
| Qwen3.5-4B + LoRA | 89.17 |
| Gemma 4 E4B raw | 79.17 |
| Gemma 4 E4B + this adapter | 87.08 |
The fine-tuned Gemma 4 E4B beats both GEPA-optimized teachers: +2.91 over gpt-4o-mini and
+7.50 over the newer gpt-5.4-mini, on the same 60 held-out rows -- though it lands 2.09 below
the fine-tuned Qwen3.5-4B on this task. Reproduce everything (data prep, GEPA runs, both
fine-tunes): apprentice-benchmark,
tasks/receipt-extraction/.
Training
LoRA r=16, alpha 16, 3 epochs, lr 2e-4, batch 2 x grad-accum 4, Unsloth 4-bit, Colab GPU. Train/eval split: seed 42, 140/60 from 200 sampled rows -- identical split to the Qwen3.5-4B adapter above, so the two are directly comparable.
Usage
Load with PEFT on top of google/gemma-4-E4B-it, or serve via vLLM with --enable-lora.
Caveat: evaluated on 60 rows of a public benchmark with exact-string field matching ---
re-validate before production use.
Built with Gemma (Apache-2.0).
- Downloads last month
- -