AyoubChLin/DistilmBERT-CompanyDocs-v2

CompanyDocs DistilmBERT v2

Classify business documents from their extracted text. CompanyDocs DistilmBERT v2 is a fine-tuned multilingual DistilBERT model for identifying document types such as invoices, purchase orders, quotes, receipts, and shipping orders.

It was trained on CompanyDocuments v2, using the dataset's file_content field as input and document_type as the target. The uploaded training notebook reports 100.00% test accuracy, 100.00% macro-F1, and approximately 135.3 million parameters.

The training data includes English and French documents. The recorded run reports perfect test scores in both languages. These results describe this benchmark and do not establish perfect performance on real-world documents.

Model details

Property	Value
Author	Ayoub Cherguelaine (AyoubChLin)
Base checkpoint	distilbert/distilbert-base-multilingual-cased
Architecture	multilingual DistilBERT with a sequence classification head
Task	Single-label document type classification
Input	Extracted document text
Output	A document type label and model score
Maximum input length	512 tokens, including special tokens
Parameters	Approximately 135.3M
Training framework	PyTorch and Hugging Face Transformers Trainer

Dataset and training relationship

Training dataset: AyoubChLin/Company-document-dataset-v2, configuration all.

The training pipeline reads three fields:

• file_content: text extracted from the document.
• document_type: the classification target.
• locale: used for language-specific evaluation, rather than as a model input.

Empty texts are removed. Documents are tokenized with truncation at 512 tokens, preserving the beginning of each document. Batches use dynamic padding. The classifier does not consume embedded PDFs, document images, bounding boxes, or structured extraction annotations.

The recorded run contains these split sizes:

Split	Documents	English	French
Train	259,497	163,994	95,503
Validation	47,114	30,167	16,947
Test	46,969	9,496	37,473

The notebook describes the dataset splits as grouped by issuing company, with unseen issuers in validation and test. This grouping is a property of the dataset's provided splits; the notebook does not independently audit it. Its saved output reports no classes missing from training, so no class redistribution occurred in that recorded run.

Document types

The notebook's saved training output lists 13 labels:

ID in the recorded run	Label
0	account_statement
1	credit_note
2	employment_certificate
3	goods_received_note
4	inventory_report
5	invoice
6	packing_slip
7	payslip
8	purchase_order
9	quote
10	receipt
11	shipping_order
12	work_order

Checkpoint verification: the notebook's current source also contains an unexecuted rare-class filter that would remove four of these labels. Its saved outputs still show 13 labels and the full split sizes. This card documents those recorded outputs. For the downloaded checkpoint, treat model.config.id2label as authoritative; verify it against this table before publishing the card as checkpoint-verified documentation.

Evaluation

Results below come from the multilingual DistilBERT entry in the uploaded notebook's saved outputs. They are not a new evaluation of the hosted checkpoint.

Test metric	Result
Accuracy	100.00%
Macro-F1	100.00%
Weighted-F1	100.00%
Macro precision	100.00%
Macro recall	100.00%
English accuracy	100.00%
French accuracy	100.00%

The test set contains substantially more French documents than English documents. Macro-F1 gives each class equal weight and is useful here because the class distribution is highly uneven: the recorded test set includes 6,807 purchase orders but only 10 inventory reports. Perfect scores on this dataset warrant further testing on independent documents, noisy OCR, and examples without explicit document titles. No independent leakage or duplicate audit was supplied.

The notebook also reports a training time of approximately 7.4 minutes, peak allocated GPU memory of 15.7 GB, and approximately 2,288.9 test documents/second. These measurements belong to that notebook run, configured for a single A100 80 GB GPU. Throughput was measured using batched Trainer.predict; it is not a guarantee of service latency or CPU performance.

Training configuration

The notebook source specifies the following configuration:

Setting	Value
Epochs	1
Learning rate	3e-5
Training batch size	128
Evaluation batch size	256
Gradient accumulation steps	1
Optimizer on CUDA	Fused AdamW (adamw_torch_fused)
Weight decay	0.01
Learning rate schedule	Linear
Warmup	10% of configured training steps
Random seed	42
Checkpoint selection	Best validation macro-F1
Precision	BF16 when supported on CUDA; otherwise FP16 on CUDA

Exact package versions, dataset commit, and hosted checkpoint commit were not recorded in the information used to prepare this card.

Quick start

pip install torch transformers

import torch
from transformers import pipeline

# Current repository ID. After renaming the repository, use:
# MODEL_ID = "AyoubChLin/companydocs-distilmbert-v2"
MODEL_ID = "AyoubChLin/companydocs-v2-doc-cls-distilmbert"

classifier = pipeline(
    "text-classification",
    model=MODEL_ID,
    device=0 if torch.cuda.is_available() else -1,
)

texts = [
    "INVOICE\nInvoice number: INV-1042\nTotal amount due: EUR 1,250.00",
    "PURCHASE ORDER\nPO number: PO-218\nPlease supply 20 office chairs.",
]

predictions = classifier(texts, truncation=True, max_length=512)
for prediction in predictions:
    print(prediction["label"], prediction["score"])

# Inspect the labels actually supported by this checkpoint.
print(classifier.model.config.id2label)

Extract text from a PDF or run OCR before passing the resulting string to the classifier. The examples above illustrate the interface; their predictions are not precomputed benchmark results.

Intended uses and limitations

Use this model for document routing, indexing, and selecting a downstream extraction workflow based on a predicted document type.

• Language coverage: training and evaluation cover English and French. Other languages were not evaluated in the recorded run.
• Benchmark scope: both English and French accuracy are 100% in the recorded run. Evaluate each language on representative production documents before deployment.
• Text only: the model does not process visual layout, handwriting, or raw PDF files. OCR errors can change its predictions.
• Truncation: content after the first 512 tokens is omitted. Documents whose identifying information appears later may be misclassified.
• Closed label set: an unsupported document type will still receive a supported label. There is no trained unknown-document class.
• Benchmark shortcuts: document titles such as “INVOICE” or “FACTURE” may make classification easier. Issuer-separated splits alone do not establish robustness to missing titles, new templates, or noisy real-world scans.
• Scores: prediction scores have not been shown to be calibrated probabilities. Validate any review threshold on representative deployment data.

License and provenance

The repository's current license could not be verified when preparing this card, so no license is asserted in the metadata. Check the model repository's license and the dataset card before reuse or redistribution.

Training settings, label lists, and results are sourced from companydocs-v2-doc-classification.ipynb. Its saved outputs and current source are not fully synchronized, including a later error-analysis cell reporting a different test size. The results here use the multilingual DistilBERT training/evaluation output and its associated recorded split counts. Verify the hosted model configuration and rerun evaluation against a pinned dataset revision to establish fully reproducible checkpoint-specific results.