perlious-Savage/layoutlmv3-cord-extraction

LayoutLMv3 fine-tuned on CORD-v2 for receipt field extraction

Token classification over receipt images, extracting 59 BIO-tagged field types (menu items, quantities, unit prices, subtotals, tax, service charges, payment amounts).

Part of a document-intelligence pipeline: Perlious-Savage/financial-doc-intelligence.

Results

Evaluated on the CORD-v2 test split (100 documents, 1,309 entities).

method	precision	recall	F1
keyword + position rules, no model	0.557	0.468	0.509
this model	0.942	0.955	0.948

Scoring is entity-level: a prediction counts only when its type and its full span match the reference exactly. Partial extractions score zero, because a partially extracted field is not a usable one. Implementation: src/metrics.py.

Where the gain comes from

The rule baseline handles fields printed next to their own label and scores exactly zero on fields identified by position alone. Those are the fields this model exists for:

field	support	rules F1	this model
menu.cnt	220	0.000	0.984
menu.unitprice	67	0.000	0.933
menu.sub.nm	36	0.000	0.892
menu.sub.price	20	0.000	0.947
menu.sub.cnt	17	0.000	0.944

Known weaknesses

Every remaining weak field is a rare class: menu.etc (3 test examples, F1 0.000), total.total_etc (3, 0.000), menu.itemsubtotal (6, 0.000), total.emoneyprice (2, 0.400), menu.discountprice (10, 0.571). Two are catch-all etc categories, where scarcity compounds with genuine ambiguity about what belongs in them. This is a data-coverage limitation, not an architectural one.

Training

base model	microsoft/layoutlmv3-base
epochs	8
learning rate	5e-5
batch size	4
training documents	800
label classes	59
hardware	A100 (Colab), 711s

Validation F1 by epoch: 0.820, 0.923, 0.921, 0.950, 0.946, 0.952, 0.955, 0.955 — plateaus from epoch 6, so the final two epochs added nothing.

Limitations

• Single training run. No repeated seeds, no confidence intervals. With 100 test documents the interval around 0.948 is not tight; a rerun would move the third decimal.
• Receipts, not KYC. CORD is Indonesian retail receipts. This measures document extraction, not regulatory or compliance performance.
• Indonesian number formatting. Amounts use . as a thousands separator (12.000 is twelve thousand). Parsing logic assumes this convention.
• Comparable published work (nielsr/layoutlmv3-finetuned-cord) reports 0.964 F1 under a different evaluation setup — context, not a like-for-like comparison.

Usage

from transformers import AutoProcessor, LayoutLMv3ForTokenClassification

processor = AutoProcessor.from_pretrained("perlious-Savage/layoutlmv3-cord-extraction", apply_ocr=True)
model = LayoutLMv3ForTokenClassification.from_pretrained("perlious-Savage/layoutlmv3-cord-extraction")

License

Inherits CC-BY-NC-4.0 from microsoft/layoutlmv3-base (non-commercial). Training data: CORD-v2, CC-BY-4.0.