|
--- |
|
license: mit |
|
base_model: microsoft/layoutlm-base-uncased |
|
tags: |
|
- generated_from_trainer |
|
datasets: |
|
- funsd |
|
model-index: |
|
- name: layoutlm-funsd1 |
|
results: [] |
|
--- |
|
|
|
<!-- This model card has been generated automatically according to the information the Trainer had access to. You |
|
should probably proofread and complete it, then remove this comment. --> |
|
|
|
# layoutlm-funsd1 |
|
|
|
This model is a fine-tuned version of [microsoft/layoutlm-base-uncased](https://huggingface.co/microsoft/layoutlm-base-uncased) on the funsd dataset. |
|
It achieves the following results on the evaluation set: |
|
- Loss: 0.6511 |
|
- Answer: {'precision': 0.6761487964989059, 'recall': 0.7639060568603214, 'f1': 0.7173534532791643, 'number': 809} |
|
- Header: {'precision': 0.24545454545454545, 'recall': 0.226890756302521, 'f1': 0.23580786026200873, 'number': 119} |
|
- Question: {'precision': 0.7472245943637916, 'recall': 0.8215962441314554, 'f1': 0.7826475849731663, 'number': 1065} |
|
- Overall Precision: 0.6925 |
|
- Overall Recall: 0.7627 |
|
- Overall F1: 0.7259 |
|
- Overall Accuracy: 0.7992 |
|
|
|
## Model description |
|
|
|
More information needed |
|
|
|
## Intended uses & limitations |
|
|
|
More information needed |
|
|
|
## Training and evaluation data |
|
|
|
More information needed |
|
|
|
## Training procedure |
|
|
|
### Training hyperparameters |
|
|
|
The following hyperparameters were used during training: |
|
- learning_rate: 3e-05 |
|
- train_batch_size: 16 |
|
- eval_batch_size: 8 |
|
- seed: 42 |
|
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08 |
|
- lr_scheduler_type: linear |
|
- num_epochs: 10 |
|
- mixed_precision_training: Native AMP |
|
|
|
### Training results |
|
|
|
| Training Loss | Epoch | Step | Validation Loss | Answer | Header | Question | Overall Precision | Overall Recall | Overall F1 | Overall Accuracy | |
|
|:-------------:|:-----:|:----:|:---------------:|:----------------------------------------------------------------------------------------------------------:|:--------------------------------------------------------------------------------------------------------------:|:---------------------------------------------------------------------------------------------------------:|:-----------------:|:--------------:|:----------:|:----------------:| |
|
| 1.7571 | 1.0 | 10 | 1.5405 | {'precision': 0.0392156862745098, 'recall': 0.0519159456118665, 'f1': 0.04468085106382978, 'number': 809} | {'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119} | {'precision': 0.23129251700680273, 'recall': 0.3511737089201878, 'f1': 0.27889634601044, 'number': 1065} | 0.1548 | 0.2087 | 0.1777 | 0.4539 | |
|
| 1.4002 | 2.0 | 20 | 1.2087 | {'precision': 0.21976592977893367, 'recall': 0.2088998763906057, 'f1': 0.21419518377693283, 'number': 809} | {'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119} | {'precision': 0.4806934594168637, 'recall': 0.5727699530516432, 'f1': 0.5227077977720652, 'number': 1065} | 0.3822 | 0.3909 | 0.3865 | 0.5991 | |
|
| 1.0781 | 3.0 | 30 | 0.9612 | {'precision': 0.437219730941704, 'recall': 0.4820766378244747, 'f1': 0.4585537918871252, 'number': 809} | {'precision': 0.030303030303030304, 'recall': 0.008403361344537815, 'f1': 0.013157894736842105, 'number': 119} | {'precision': 0.6361233480176212, 'recall': 0.6779342723004694, 'f1': 0.6563636363636363, 'number': 1065} | 0.5403 | 0.5585 | 0.5492 | 0.6934 | |
|
| 0.8462 | 4.0 | 40 | 0.7985 | {'precision': 0.5972515856236786, 'recall': 0.6983930778739185, 'f1': 0.6438746438746439, 'number': 809} | {'precision': 0.11363636363636363, 'recall': 0.04201680672268908, 'f1': 0.06134969325153375, 'number': 119} | {'precision': 0.6884955752212389, 'recall': 0.7305164319248826, 'f1': 0.7088838268792711, 'number': 1065} | 0.6358 | 0.6764 | 0.6555 | 0.7564 | |
|
| 0.6873 | 5.0 | 50 | 0.7161 | {'precision': 0.6699779249448123, 'recall': 0.7503090234857849, 'f1': 0.707871720116618, 'number': 809} | {'precision': 0.23529411764705882, 'recall': 0.16806722689075632, 'f1': 0.19607843137254902, 'number': 119} | {'precision': 0.6994022203245089, 'recall': 0.7690140845070422, 'f1': 0.7325581395348838, 'number': 1065} | 0.6688 | 0.7255 | 0.6960 | 0.7858 | |
|
| 0.5786 | 6.0 | 60 | 0.6912 | {'precision': 0.6480505795574288, 'recall': 0.7601977750309024, 'f1': 0.6996587030716724, 'number': 809} | {'precision': 0.2638888888888889, 'recall': 0.15966386554621848, 'f1': 0.19895287958115182, 'number': 119} | {'precision': 0.7293700088731144, 'recall': 0.7718309859154929, 'f1': 0.7499999999999999, 'number': 1065} | 0.6778 | 0.7306 | 0.7032 | 0.7848 | |
|
| 0.5389 | 7.0 | 70 | 0.6760 | {'precision': 0.6835722160970231, 'recall': 0.7663782447466008, 'f1': 0.7226107226107226, 'number': 809} | {'precision': 0.21978021978021978, 'recall': 0.16806722689075632, 'f1': 0.1904761904761905, 'number': 119} | {'precision': 0.7195723684210527, 'recall': 0.8215962441314554, 'f1': 0.7672073651907059, 'number': 1065} | 0.6843 | 0.7602 | 0.7202 | 0.7929 | |
|
| 0.491 | 8.0 | 80 | 0.6643 | {'precision': 0.6782608695652174, 'recall': 0.7713226205191595, 'f1': 0.7218045112781956, 'number': 809} | {'precision': 0.2708333333333333, 'recall': 0.2184873949579832, 'f1': 0.24186046511627907, 'number': 119} | {'precision': 0.757847533632287, 'recall': 0.7934272300469484, 'f1': 0.7752293577981653, 'number': 1065} | 0.7015 | 0.7501 | 0.7250 | 0.7969 | |
|
| 0.4543 | 9.0 | 90 | 0.6519 | {'precision': 0.6808743169398908, 'recall': 0.7700865265760197, 'f1': 0.722737819025522, 'number': 809} | {'precision': 0.24509803921568626, 'recall': 0.21008403361344538, 'f1': 0.22624434389140272, 'number': 119} | {'precision': 0.7564102564102564, 'recall': 0.8309859154929577, 'f1': 0.7919463087248323, 'number': 1065} | 0.7010 | 0.7692 | 0.7335 | 0.8003 | |
|
| 0.4461 | 10.0 | 100 | 0.6511 | {'precision': 0.6761487964989059, 'recall': 0.7639060568603214, 'f1': 0.7173534532791643, 'number': 809} | {'precision': 0.24545454545454545, 'recall': 0.226890756302521, 'f1': 0.23580786026200873, 'number': 119} | {'precision': 0.7472245943637916, 'recall': 0.8215962441314554, 'f1': 0.7826475849731663, 'number': 1065} | 0.6925 | 0.7627 | 0.7259 | 0.7992 | |
|
|
|
|
|
### Framework versions |
|
|
|
- Transformers 4.41.2 |
|
- Pytorch 2.3.1+cu121 |
|
- Datasets 2.19.2 |
|
- Tokenizers 0.19.1 |
|
|
