YAML Metadata Warning:empty or missing yaml metadata in repo card
Check out the documentation for more information.
formcheck-vlm v4 (detection-first)
~17M-parameter detection-first model: ViT-8L/384 encoder + per-cell detection head predicting presence, bbox, type, state, and label (closed 736-label vocab) for every control on the page. JSON is assembled rule-based from detections, so any prompt template maps to output without relying on autoregressive grounding.
Results
Synthetic val (48 pages, 12k steps on 40k pages)
| metric | v3 (text VLM + aux) | v4 (detection-first) |
|---|---|---|
| detection recall / precision | — | 0.940 / 0.986 |
| per-control state accuracy | 0.036 | 0.994 |
| label accuracy | — | 0.653 |
| full-page exact match | 0.104 | 0.042* |
| mean box-count error | — | 0.060 |
*v4's page_exact requires all labels AND states AND counts correct; its state accuracy is near-perfect and its JSON output is valid by construction (rule-based emitter covers all 6 prompt templates, including single-control lookup).
Real scans (Apache-2.0 CommonForms, 200 pages)
| metric | v3 (text VLM) | v4 (detection) |
|---|---|---|
| JSON parse / detection P-R | 0.125 / — | — / 0.022–0.059 |
| box-count within ±1 | 0.010 | 0.095 |
The real-scan domain gap is now the dominant error for every version. Clean synthetic pages transfer poorly to real scans (unseen fonts, densities, box styles, 384px squashed full pages). Next lever: domain adaptation — mix CommonForms pages (crop + synthetic fill for state pairs, the Apache-2.0-blessed trick from wendys-llc) at 5-10% into training, and/or pseudo-label real pages with wendys-llc/checkbox-detector.
Spec
- Heads per 16px cell: presence (BCE, x2 weight), bbox (L1), type (CE x3), state (CE x9), label (CE x736); losses averaged on occupied cells
- Inference: cells with sigmoid > 0.5 → detections sorted in reading order → rule-based JSON
- Config:
{"grid": 24, "cell": 16, "img": 384, "labels": 736, "params_m": 17.4}
Files
pytorch_model.bin · config.json · job_train.py (renderer with detection records) ·
det_train.py (model, losses, emitter, eval)
Licensing
Everything used is commercially usable: synthetic pipeline (own code), eval data CommonForms (Apache-2.0). FUNSD/XFUND/CheckboxQA/RVL-CDIP excluded as non-commercial.
- Downloads last month
- 37