|
---
|
|
license: mit
|
|
base_model: microsoft/layoutlm-base-uncased
|
|
tags:
|
|
- generated_from_trainer
|
|
datasets:
|
|
- funsd
|
|
model-index:
|
|
- name: layoutlm-funsd
|
|
results: []
|
|
---
|
|
|
|
<!-- This model card has been generated automatically according to the information the Trainer had access to. You
|
|
should probably proofread and complete it, then remove this comment. -->
|
|
|
|
# layoutlm-funsd
|
|
|
|
This model is a fine-tuned version of [microsoft/layoutlm-base-uncased](https://huggingface.co/microsoft/layoutlm-base-uncased) on the funsd dataset.
|
|
It achieves the following results on the evaluation set:
|
|
- Loss: 0.6930
|
|
- Answer: {'precision': 0.705114254624592, 'recall': 0.8009888751545118, 'f1': 0.7499999999999999, 'number': 809}
|
|
- Header: {'precision': 0.2642857142857143, 'recall': 0.31092436974789917, 'f1': 0.28571428571428575, 'number': 119}
|
|
- Question: {'precision': 0.7760141093474426, 'recall': 0.8262910798122066, 'f1': 0.8003638017280582, 'number': 1065}
|
|
- Overall Precision: 0.7136
|
|
- Overall Recall: 0.7852
|
|
- Overall F1: 0.7477
|
|
- Overall Accuracy: 0.8082
|
|
|
|
## Model description
|
|
|
|
More information needed
|
|
|
|
## Intended uses & limitations
|
|
|
|
More information needed
|
|
|
|
## Training and evaluation data
|
|
|
|
More information needed
|
|
|
|
## Training procedure
|
|
|
|
### Training hyperparameters
|
|
|
|
The following hyperparameters were used during training:
|
|
- learning_rate: 3e-05
|
|
- train_batch_size: 16
|
|
- eval_batch_size: 8
|
|
- seed: 42
|
|
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
|
|
- lr_scheduler_type: linear
|
|
- num_epochs: 15
|
|
- mixed_precision_training: Native AMP
|
|
|
|
### Training results
|
|
|
|
| Training Loss | Epoch | Step | Validation Loss | Answer | Header | Question | Overall Precision | Overall Recall | Overall F1 | Overall Accuracy |
|
|
|:-------------:|:-----:|:----:|:---------------:|:-------------------------------------------------------------------------------------------------------------:|:-------------------------------------------------------------------------------------------------------------:|:----------------------------------------------------------------------------------------------------------:|:-----------------:|:--------------:|:----------:|:----------------:|
|
|
| 1.812 | 1.0 | 10 | 1.5657 | {'precision': 0.026246719160104987, 'recall': 0.024721878862793572, 'f1': 0.02546148949713558, 'number': 809} | {'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119} | {'precision': 0.1956521739130435, 'recall': 0.1267605633802817, 'f1': 0.15384615384615385, 'number': 1065} | 0.1067 | 0.0778 | 0.0900 | 0.3859 |
|
|
| 1.4244 | 2.0 | 20 | 1.2288 | {'precision': 0.14189189189189189, 'recall': 0.103831891223733, 'f1': 0.11991434689507495, 'number': 809} | {'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119} | {'precision': 0.42844202898550726, 'recall': 0.444131455399061, 'f1': 0.43614568925772246, 'number': 1065} | 0.3284 | 0.2795 | 0.3020 | 0.5784 |
|
|
| 1.1038 | 3.0 | 30 | 0.9813 | {'precision': 0.4468629961587708, 'recall': 0.43139678615574784, 'f1': 0.4389937106918239, 'number': 809} | {'precision': 0.03225806451612903, 'recall': 0.008403361344537815, 'f1': 0.013333333333333332, 'number': 119} | {'precision': 0.6163522012578616, 'recall': 0.644131455399061, 'f1': 0.6299357208448118, 'number': 1065} | 0.5382 | 0.5198 | 0.5288 | 0.7101 |
|
|
| 0.8512 | 4.0 | 40 | 0.8085 | {'precision': 0.5877192982456141, 'recall': 0.6625463535228677, 'f1': 0.6228936664729808, 'number': 809} | {'precision': 0.109375, 'recall': 0.058823529411764705, 'f1': 0.07650273224043715, 'number': 119} | {'precision': 0.6793760831889082, 'recall': 0.7361502347417841, 'f1': 0.7066246056782335, 'number': 1065} | 0.6230 | 0.6658 | 0.6437 | 0.7566 |
|
|
| 0.6646 | 5.0 | 50 | 0.7071 | {'precision': 0.6478723404255319, 'recall': 0.7527812113720643, 'f1': 0.6963979416809606, 'number': 809} | {'precision': 0.21052631578947367, 'recall': 0.16806722689075632, 'f1': 0.1869158878504673, 'number': 119} | {'precision': 0.6853658536585366, 'recall': 0.7915492957746478, 'f1': 0.7346405228758169, 'number': 1065} | 0.6499 | 0.7386 | 0.6914 | 0.7871 |
|
|
| 0.5615 | 6.0 | 60 | 0.6934 | {'precision': 0.6427840327533265, 'recall': 0.7762669962917181, 'f1': 0.7032474804031356, 'number': 809} | {'precision': 0.2191780821917808, 'recall': 0.13445378151260504, 'f1': 0.16666666666666669, 'number': 119} | {'precision': 0.7584973166368515, 'recall': 0.7962441314553991, 'f1': 0.7769125057260651, 'number': 1065} | 0.6882 | 0.7486 | 0.7171 | 0.8008 |
|
|
| 0.4852 | 7.0 | 70 | 0.6675 | {'precision': 0.6806451612903226, 'recall': 0.7824474660074165, 'f1': 0.7280046003450259, 'number': 809} | {'precision': 0.2421875, 'recall': 0.2605042016806723, 'f1': 0.2510121457489879, 'number': 119} | {'precision': 0.7596759675967597, 'recall': 0.7924882629107981, 'f1': 0.775735294117647, 'number': 1065} | 0.6953 | 0.7566 | 0.7247 | 0.8098 |
|
|
| 0.4261 | 8.0 | 80 | 0.6601 | {'precision': 0.6707818930041153, 'recall': 0.8059332509270705, 'f1': 0.7321729365524987, 'number': 809} | {'precision': 0.23770491803278687, 'recall': 0.24369747899159663, 'f1': 0.24066390041493776, 'number': 119} | {'precision': 0.7515257192676548, 'recall': 0.8093896713615023, 'f1': 0.779385171790235, 'number': 1065} | 0.6885 | 0.7742 | 0.7289 | 0.8027 |
|
|
| 0.3798 | 9.0 | 90 | 0.6595 | {'precision': 0.6950431034482759, 'recall': 0.7972805933250927, 'f1': 0.7426597582037997, 'number': 809} | {'precision': 0.2727272727272727, 'recall': 0.2773109243697479, 'f1': 0.27499999999999997, 'number': 119} | {'precision': 0.7698343504795118, 'recall': 0.8291079812206573, 'f1': 0.7983725135623869, 'number': 1065} | 0.7108 | 0.7832 | 0.7453 | 0.8120 |
|
|
| 0.366 | 10.0 | 100 | 0.6659 | {'precision': 0.6912393162393162, 'recall': 0.799752781211372, 'f1': 0.7415472779369628, 'number': 809} | {'precision': 0.29310344827586204, 'recall': 0.2857142857142857, 'f1': 0.2893617021276596, 'number': 119} | {'precision': 0.7822222222222223, 'recall': 0.8262910798122066, 'f1': 0.8036529680365297, 'number': 1065} | 0.7170 | 0.7832 | 0.7487 | 0.8196 |
|
|
| 0.3112 | 11.0 | 110 | 0.6790 | {'precision': 0.674562306900103, 'recall': 0.8096415327564895, 'f1': 0.7359550561797752, 'number': 809} | {'precision': 0.2890625, 'recall': 0.31092436974789917, 'f1': 0.29959514170040485, 'number': 119} | {'precision': 0.7867383512544803, 'recall': 0.8244131455399061, 'f1': 0.8051352590554791, 'number': 1065} | 0.7088 | 0.7878 | 0.7462 | 0.8022 |
|
|
| 0.3003 | 12.0 | 120 | 0.6876 | {'precision': 0.7192393736017897, 'recall': 0.7948084054388134, 'f1': 0.7551379917792131, 'number': 809} | {'precision': 0.2824427480916031, 'recall': 0.31092436974789917, 'f1': 0.29600000000000004, 'number': 119} | {'precision': 0.7788546255506608, 'recall': 0.8300469483568075, 'f1': 0.8036363636363637, 'number': 1065} | 0.7241 | 0.7847 | 0.7532 | 0.8069 |
|
|
| 0.28 | 13.0 | 130 | 0.6905 | {'precision': 0.7013963480128894, 'recall': 0.8071693448702101, 'f1': 0.7505747126436783, 'number': 809} | {'precision': 0.2923076923076923, 'recall': 0.31932773109243695, 'f1': 0.3052208835341365, 'number': 119} | {'precision': 0.7860340196956133, 'recall': 0.8244131455399061, 'f1': 0.8047662694775436, 'number': 1065} | 0.7204 | 0.7873 | 0.7523 | 0.8104 |
|
|
| 0.2654 | 14.0 | 140 | 0.6952 | {'precision': 0.7069154774972558, 'recall': 0.796044499381953, 'f1': 0.7488372093023256, 'number': 809} | {'precision': 0.2569444444444444, 'recall': 0.31092436974789917, 'f1': 0.28136882129277563, 'number': 119} | {'precision': 0.7758164165931156, 'recall': 0.8253521126760563, 'f1': 0.7998180163785259, 'number': 1065} | 0.7130 | 0.7827 | 0.7462 | 0.8068 |
|
|
| 0.2629 | 15.0 | 150 | 0.6930 | {'precision': 0.705114254624592, 'recall': 0.8009888751545118, 'f1': 0.7499999999999999, 'number': 809} | {'precision': 0.2642857142857143, 'recall': 0.31092436974789917, 'f1': 0.28571428571428575, 'number': 119} | {'precision': 0.7760141093474426, 'recall': 0.8262910798122066, 'f1': 0.8003638017280582, 'number': 1065} | 0.7136 | 0.7852 | 0.7477 | 0.8082 |
|
|
|
|
|
|
### Framework versions
|
|
|
|
- Transformers 4.41.2
|
|
- Pytorch 2.3.1+cu118
|
|
- Datasets 2.19.2
|
|
- Tokenizers 0.19.1
|
|
|
