File size: 9,334 Bytes
7c5e1a3 5d1ba10 7c5e1a3 5d1ba10 7c5e1a3 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 |
---
license: mit
base_model: microsoft/layoutlm-base-uncased
tags:
- generated_from_trainer
model-index:
- name: layoutlm-funsd
results: []
---
<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->
# layoutlm-funsd
This model is a fine-tuned version of [microsoft/layoutlm-base-uncased](https://huggingface.co/microsoft/layoutlm-base-uncased) on an unknown dataset.
It achieves the following results on the evaluation set:
- Loss: 0.7207
- Answer: {'precision': 0.7114754098360656, 'recall': 0.8046971569839307, 'f1': 0.7552204176334106, 'number': 809}
- Header: {'precision': 0.3025210084033613, 'recall': 0.3025210084033613, 'f1': 0.3025210084033613, 'number': 119}
- Question: {'precision': 0.7707061900610288, 'recall': 0.8300469483568075, 'f1': 0.7992766726943942, 'number': 1065}
- Overall Precision: 0.7203
- Overall Recall: 0.7883
- Overall F1: 0.7528
- Overall Accuracy: 0.7971
## Model description
More information needed
## Intended uses & limitations
More information needed
## Training and evaluation data
More information needed
## Training procedure
### Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 3e-05
- train_batch_size: 16
- eval_batch_size: 8
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 15
- mixed_precision_training: Native AMP
### Training results
| Training Loss | Epoch | Step | Validation Loss | Answer | Header | Question | Overall Precision | Overall Recall | Overall F1 | Overall Accuracy |
|:-------------:|:-----:|:----:|:---------------:|:-----------------------------------------------------------------------------------------------------------:|:------------------------------------------------------------------------------------------------------------:|:------------------------------------------------------------------------------------------------------------:|:-----------------:|:--------------:|:----------:|:----------------:|
| 1.8417 | 1.0 | 10 | 1.6166 | {'precision': 0.028741328047571853, 'recall': 0.03584672435105068, 'f1': 0.0319031903190319, 'number': 809} | {'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119} | {'precision': 0.16827852998065765, 'recall': 0.16338028169014085, 'f1': 0.16579323487374942, 'number': 1065} | 0.0993 | 0.1019 | 0.1006 | 0.3810 |
| 1.4476 | 2.0 | 20 | 1.2599 | {'precision': 0.15711947626841244, 'recall': 0.11866501854140915, 'f1': 0.13521126760563382, 'number': 809} | {'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119} | {'precision': 0.46172441579371476, 'recall': 0.5380281690140845, 'f1': 0.4969644405897658, 'number': 1065} | 0.3612 | 0.3357 | 0.3480 | 0.5747 |
| 1.1227 | 3.0 | 30 | 0.9780 | {'precision': 0.4661214953271028, 'recall': 0.4932014833127318, 'f1': 0.4792792792792793, 'number': 809} | {'precision': 0.16279069767441862, 'recall': 0.058823529411764705, 'f1': 0.08641975308641975, 'number': 119} | {'precision': 0.5979020979020979, 'recall': 0.6422535211267606, 'f1': 0.6192847442281576, 'number': 1065} | 0.5335 | 0.5469 | 0.5401 | 0.7036 |
| 0.8596 | 4.0 | 40 | 0.7946 | {'precision': 0.5789473684210527, 'recall': 0.6526576019777504, 'f1': 0.6135967460778617, 'number': 809} | {'precision': 0.24193548387096775, 'recall': 0.12605042016806722, 'f1': 0.16574585635359115, 'number': 119} | {'precision': 0.6658141517476556, 'recall': 0.7333333333333333, 'f1': 0.6979445933869526, 'number': 1065} | 0.6167 | 0.6643 | 0.6396 | 0.7589 |
| 0.6705 | 5.0 | 50 | 0.7132 | {'precision': 0.6424759871931697, 'recall': 0.7441285537700866, 'f1': 0.6895761741122567, 'number': 809} | {'precision': 0.3333333333333333, 'recall': 0.21008403361344538, 'f1': 0.2577319587628866, 'number': 119} | {'precision': 0.6747376916868443, 'recall': 0.7849765258215963, 'f1': 0.7256944444444444, 'number': 1065} | 0.6499 | 0.7341 | 0.6894 | 0.7767 |
| 0.5653 | 6.0 | 60 | 0.6840 | {'precision': 0.653125, 'recall': 0.7750309023485785, 'f1': 0.7088750706613907, 'number': 809} | {'precision': 0.30952380952380953, 'recall': 0.2184873949579832, 'f1': 0.2561576354679803, 'number': 119} | {'precision': 0.7077814569536424, 'recall': 0.8028169014084507, 'f1': 0.7523097228332599, 'number': 1065} | 0.6696 | 0.7566 | 0.7105 | 0.7846 |
| 0.4959 | 7.0 | 70 | 0.6684 | {'precision': 0.6872964169381107, 'recall': 0.7824474660074165, 'f1': 0.7317919075144509, 'number': 809} | {'precision': 0.2815533980582524, 'recall': 0.24369747899159663, 'f1': 0.26126126126126126, 'number': 119} | {'precision': 0.734006734006734, 'recall': 0.8187793427230047, 'f1': 0.7740790057700843, 'number': 1065} | 0.6935 | 0.7697 | 0.7296 | 0.7950 |
| 0.4343 | 8.0 | 80 | 0.6696 | {'precision': 0.6898395721925134, 'recall': 0.7972805933250927, 'f1': 0.7396788990825688, 'number': 809} | {'precision': 0.26956521739130435, 'recall': 0.2605042016806723, 'f1': 0.264957264957265, 'number': 119} | {'precision': 0.7495769881556683, 'recall': 0.831924882629108, 'f1': 0.7886070315976857, 'number': 1065} | 0.6998 | 0.7837 | 0.7394 | 0.7987 |
| 0.375 | 9.0 | 90 | 0.6760 | {'precision': 0.7105549510337323, 'recall': 0.8071693448702101, 'f1': 0.755787037037037, 'number': 809} | {'precision': 0.25, 'recall': 0.2605042016806723, 'f1': 0.25514403292181076, 'number': 119} | {'precision': 0.7758007117437722, 'recall': 0.8187793427230047, 'f1': 0.7967108268615805, 'number': 1065} | 0.7180 | 0.7807 | 0.7481 | 0.7992 |
| 0.3532 | 10.0 | 100 | 0.6802 | {'precision': 0.7147470398277718, 'recall': 0.8207663782447466, 'f1': 0.7640966628308401, 'number': 809} | {'precision': 0.3018867924528302, 'recall': 0.2689075630252101, 'f1': 0.28444444444444444, 'number': 119} | {'precision': 0.7761061946902655, 'recall': 0.8234741784037559, 'f1': 0.7990888382687927, 'number': 1065} | 0.7266 | 0.7893 | 0.7566 | 0.8046 |
| 0.3265 | 11.0 | 110 | 0.6995 | {'precision': 0.6968716289104638, 'recall': 0.7985166872682324, 'f1': 0.7442396313364056, 'number': 809} | {'precision': 0.308411214953271, 'recall': 0.2773109243697479, 'f1': 0.29203539823008845, 'number': 119} | {'precision': 0.7589134125636672, 'recall': 0.8394366197183099, 'f1': 0.7971466785555059, 'number': 1065} | 0.7111 | 0.7893 | 0.7482 | 0.7973 |
| 0.3023 | 12.0 | 120 | 0.7053 | {'precision': 0.7027896995708155, 'recall': 0.8096415327564895, 'f1': 0.7524411257897761, 'number': 809} | {'precision': 0.30303030303030304, 'recall': 0.33613445378151263, 'f1': 0.3187250996015936, 'number': 119} | {'precision': 0.769434628975265, 'recall': 0.8178403755868544, 'f1': 0.7928994082840236, 'number': 1065} | 0.7131 | 0.7858 | 0.7477 | 0.7991 |
| 0.2927 | 13.0 | 130 | 0.7080 | {'precision': 0.7024704618689581, 'recall': 0.8084054388133498, 'f1': 0.7517241379310345, 'number': 809} | {'precision': 0.3125, 'recall': 0.29411764705882354, 'f1': 0.30303030303030304, 'number': 119} | {'precision': 0.7679033649698016, 'recall': 0.8356807511737089, 'f1': 0.8003597122302158, 'number': 1065} | 0.7171 | 0.7923 | 0.7528 | 0.7999 |
| 0.2756 | 14.0 | 140 | 0.7128 | {'precision': 0.7081967213114754, 'recall': 0.8009888751545118, 'f1': 0.7517401392111368, 'number': 809} | {'precision': 0.32142857142857145, 'recall': 0.3025210084033613, 'f1': 0.3116883116883117, 'number': 119} | {'precision': 0.7720524017467248, 'recall': 0.8300469483568075, 'f1': 0.7999999999999999, 'number': 1065} | 0.7219 | 0.7868 | 0.7529 | 0.7984 |
| 0.2741 | 15.0 | 150 | 0.7207 | {'precision': 0.7114754098360656, 'recall': 0.8046971569839307, 'f1': 0.7552204176334106, 'number': 809} | {'precision': 0.3025210084033613, 'recall': 0.3025210084033613, 'f1': 0.3025210084033613, 'number': 119} | {'precision': 0.7707061900610288, 'recall': 0.8300469483568075, 'f1': 0.7992766726943942, 'number': 1065} | 0.7203 | 0.7883 | 0.7528 | 0.7971 |
### Framework versions
- Transformers 4.36.2
- Pytorch 2.1.0+cu121
- Datasets 2.16.0
- Tokenizers 0.15.0
|