pierreguillou's picture
Update README.md
6dd6afe
metadata
language:
  - multilingual
  - en
  - de
  - fr
  - ja
license: mit
tags:
  - object-detection
  - vision
  - generated_from_trainer
  - DocLayNet
  - COCO
  - PDF
  - IBM
  - Financial-Reports
  - Finance
  - Manuals
  - Scientific-Articles
  - Science
  - Laws
  - Law
  - Regulations
  - Patents
  - Government-Tenders
  - object-detection
  - image-segmentation
  - token-classification
inference: false
datasets:
  - pierreguillou/DocLayNet-base
metrics:
  - precision
  - recall
  - f1
  - accuracy
model-index:
  - name: layout-xlm-base-finetuned-with-DocLayNet-base-at-linelevel-ml384
    results:
      - task:
          name: Token Classification
          type: token-classification
        metrics:
          - name: f1
            type: f1
            value: 0.7336
          - name: accuracy
            type: accuracy
            value: 0.9373

Document Understanding model (finetuned LayoutXLM base at line level on DocLayNet base)

This model is a fine-tuned version of microsoft/layoutxlm-base with the DocLayNet base dataset. It achieves the following results on the evaluation set:

  • Loss: 0.2364
  • Precision: 0.7260
  • Recall: 0.7415
  • F1: 0.7336
  • Accuracy: 0.9373

References

Blog posts

Notebooks (paragraph level)

Notebooks (line level)

APP

You can test this model with this APP in Hugging Face Spaces: Inference APP for Document Understanding at line level (v2).

Inference APP for Document Understanding at line level (v2)

DocLayNet dataset

DocLayNet dataset (IBM) provides page-by-page layout segmentation ground-truth using bounding-boxes for 11 distinct class labels on 80863 unique pages from 6 document categories.

Until today, the dataset can be downloaded through direct links or as a dataset from Hugging Face datasets:

Paper: DocLayNet: A Large Human-Annotated Dataset for Document-Layout Analysis (06/02/2022)

Model description

The model was finetuned at line level on chunk of 384 tokens with overlap of 128 tokens. Thus, the model was trained with all layout and text data of all pages of the dataset.

At inference time, a calculation of best probabilities give the label to each line bounding boxes.

Inference

See notebook: Document AI | Inference at line level with a Document Understanding model (LayoutXLM base fine-tuned on DocLayNet dataset)

Training and evaluation data

See notebook: Document AI | Fine-tune LayoutXLM base on DocLayNet base in any language at line level (chunk of 384 tokens with overlap)

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 2e-05
  • train_batch_size: 8
  • eval_batch_size: 16
  • seed: 42
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: linear
  • lr_scheduler_warmup_ratio: 0.1
  • num_epochs: 3
  • mixed_precision_training: Native AMP

Training results

Training Loss Epoch Step Accuracy F1 Validation Loss Precision Recall
No log 0.12 300 0.8413 0.1311 0.5185 0.1437 0.1205
0.9231 0.25 600 0.8751 0.5031 0.4108 0.4637 0.5498
0.9231 0.37 900 0.8887 0.5206 0.3911 0.5076 0.5343
0.369 0.5 1200 0.8724 0.5365 0.4118 0.5094 0.5667
0.2737 0.62 1500 0.8960 0.6033 0.3328 0.6046 0.6020
0.2737 0.75 1800 0.9186 0.6404 0.2984 0.6062 0.6787
0.2542 0.87 2100 0.9163 0.6593 0.3115 0.6324 0.6887
0.2542 1.0 2400 0.9198 0.6537 0.2878 0.6160 0.6962
0.1938 1.12 2700 0.9165 0.6752 0.3414 0.6673 0.6833
0.1581 1.25 3000 0.9193 0.6871 0.3611 0.6868 0.6875
0.1581 1.37 3300 0.9256 0.6822 0.2763 0.6988 0.6663
0.1428 1.5 3600 0.9287 0.7084 0.3065 0.7246 0.6929
0.1428 1.62 3900 0.9194 0.6812 0.2942 0.6866 0.6760
0.1025 1.74 4200 0.9347 0.7223 0.2990 0.7315 0.7133
0.1225 1.87 4500 0.9360 0.7048 0.2729 0.7249 0.6858
0.1225 1.99 4800 0.9396 0.7222 0.2826 0.7497 0.6966
0.108 2.12 5100 0.9301 0.7193 0.3071 0.7022 0.7372
0.108 2.24 5400 0.9334 0.7243 0.2999 0.7250 0.7237
0.0799 2.37 5700 0.9382 0.7254 0.2710 0.7310 0.7198
0.0793 2.49 6000 0.9329 0.7228 0.3201 0.7352 0.7108
0.0793 2.62 6300 0.9373 0.7336 0.3035 0.7260 0.7415
0.0696 2.74 6600 0.9374 0.7275 0.3137 0.7313 0.7237
0.0696 2.87 6900 0.9381 0.7253 0.3242 0.7369 0.7142
0.0866 2.99 7200 0.2473 0.7439 0.7207 0.7321 0.9407

Framework versions

  • Transformers 4.26.1
  • Pytorch 1.10.0+cu111
  • Datasets 2.10.1
  • Tokenizers 0.13.2

Other models