|
--- |
|
tags: |
|
- generated_from_trainer |
|
datasets: |
|
- funsd |
|
model-index: |
|
- name: layoutlm-funsd |
|
results: [] |
|
--- |
|
|
|
<!-- This model card has been generated automatically according to the information the Trainer had access to. You |
|
should probably proofread and complete it, then remove this comment. --> |
|
|
|
# layoutlm-funsd |
|
|
|
This model is a fine-tuned version of [microsoft/layoutlm-base-uncased](https://huggingface.co/microsoft/layoutlm-base-uncased) on the funsd dataset. |
|
It achieves the following results on the evaluation set: |
|
- Loss: 0.7034 |
|
- Answer: {'precision': 0.6974697469746974, 'recall': 0.7836835599505563, 'f1': 0.7380675203725262, 'number': 809} |
|
- Header: {'precision': 0.3106060606060606, 'recall': 0.3445378151260504, 'f1': 0.32669322709163345, 'number': 119} |
|
- Question: {'precision': 0.7759226713532513, 'recall': 0.8291079812206573, 'f1': 0.8016341352700863, 'number': 1065} |
|
- Overall Precision: 0.7150 |
|
- Overall Recall: 0.7817 |
|
- Overall F1: 0.7469 |
|
- Overall Accuracy: 0.8170 |
|
|
|
## Model description |
|
|
|
More information needed |
|
|
|
## Intended uses & limitations |
|
|
|
More information needed |
|
|
|
## Training and evaluation data |
|
|
|
More information needed |
|
|
|
## Training procedure |
|
|
|
### Training hyperparameters |
|
|
|
The following hyperparameters were used during training: |
|
- learning_rate: 3e-05 |
|
- train_batch_size: 16 |
|
- eval_batch_size: 8 |
|
- seed: 42 |
|
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08 |
|
- lr_scheduler_type: linear |
|
- num_epochs: 15 |
|
|
|
### Training results |
|
|
|
| Training Loss | Epoch | Step | Validation Loss | Answer | Header | Question | Overall Precision | Overall Recall | Overall F1 | Overall Accuracy | |
|
|:-------------:|:-----:|:----:|:---------------:|:----------------------------------------------------------------------------------------------------------------:|:----------------------------------------------------------------------------------------------------------:|:---------------------------------------------------------------------------------------------------------:|:-----------------:|:--------------:|:----------:|:----------------:| |
|
| 1.817 | 1.0 | 10 | 1.6092 | {'precision': 0.002197802197802198, 'recall': 0.0012360939431396785, 'f1': 0.0015822784810126582, 'number': 809} | {'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119} | {'precision': 0.22, 'recall': 0.07230046948356808, 'f1': 0.10883392226148411, 'number': 1065} | 0.0968 | 0.0391 | 0.0557 | 0.3198 | |
|
| 1.4741 | 2.0 | 20 | 1.2390 | {'precision': 0.22916666666666666, 'recall': 0.24474660074165636, 'f1': 0.23670053795576806, 'number': 809} | {'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119} | {'precision': 0.4344262295081967, 'recall': 0.5474178403755868, 'f1': 0.4844204403822185, 'number': 1065} | 0.3540 | 0.3919 | 0.3720 | 0.6109 | |
|
| 1.1045 | 3.0 | 30 | 0.9397 | {'precision': 0.493006993006993, 'recall': 0.522867737948084, 'f1': 0.5074985002999399, 'number': 809} | {'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119} | {'precision': 0.5923515052888527, 'recall': 0.6835680751173709, 'f1': 0.6346992153443767, 'number': 1065} | 0.5473 | 0.5775 | 0.5620 | 0.7110 | |
|
| 0.8369 | 4.0 | 40 | 0.7900 | {'precision': 0.6087408949011447, 'recall': 0.723114956736712, 'f1': 0.6610169491525424, 'number': 809} | {'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119} | {'precision': 0.6610025488530161, 'recall': 0.7305164319248826, 'f1': 0.6940231935771632, 'number': 1065} | 0.6229 | 0.6839 | 0.6520 | 0.7567 | |
|
| 0.6855 | 5.0 | 50 | 0.7185 | {'precision': 0.6293706293706294, 'recall': 0.7787391841779975, 'f1': 0.696132596685083, 'number': 809} | {'precision': 0.1, 'recall': 0.06722689075630252, 'f1': 0.08040201005025126, 'number': 119} | {'precision': 0.6999125109361329, 'recall': 0.7511737089201878, 'f1': 0.7246376811594203, 'number': 1065} | 0.6466 | 0.7215 | 0.6820 | 0.7805 | |
|
| 0.5663 | 6.0 | 60 | 0.6795 | {'precision': 0.6485655737704918, 'recall': 0.7824474660074165, 'f1': 0.7092436974789915, 'number': 809} | {'precision': 0.1728395061728395, 'recall': 0.11764705882352941, 'f1': 0.13999999999999999, 'number': 119} | {'precision': 0.6875502008032128, 'recall': 0.8037558685446009, 'f1': 0.7411255411255411, 'number': 1065} | 0.6529 | 0.7541 | 0.6999 | 0.7940 | |
|
| 0.4995 | 7.0 | 70 | 0.6814 | {'precision': 0.6630552546045504, 'recall': 0.7564894932014833, 'f1': 0.7066974595842955, 'number': 809} | {'precision': 0.21929824561403508, 'recall': 0.21008403361344538, 'f1': 0.2145922746781116, 'number': 119} | {'precision': 0.721465076660988, 'recall': 0.7953051643192488, 'f1': 0.7565877623939258, 'number': 1065} | 0.6712 | 0.7446 | 0.7060 | 0.7994 | |
|
| 0.454 | 8.0 | 80 | 0.6688 | {'precision': 0.6716738197424893, 'recall': 0.7737948084054388, 'f1': 0.7191269385410684, 'number': 809} | {'precision': 0.24324324324324326, 'recall': 0.226890756302521, 'f1': 0.23478260869565218, 'number': 119} | {'precision': 0.7363481228668942, 'recall': 0.8103286384976526, 'f1': 0.7715690657130085, 'number': 1065} | 0.6844 | 0.7607 | 0.7205 | 0.8080 | |
|
| 0.4132 | 9.0 | 90 | 0.6665 | {'precision': 0.6782231852654388, 'recall': 0.7737948084054388, 'f1': 0.7228637413394918, 'number': 809} | {'precision': 0.29508196721311475, 'recall': 0.3025210084033613, 'f1': 0.2987551867219917, 'number': 119} | {'precision': 0.739460370994941, 'recall': 0.8234741784037559, 'f1': 0.7792092403376277, 'number': 1065} | 0.6898 | 0.7722 | 0.7287 | 0.8095 | |
|
| 0.3671 | 10.0 | 100 | 0.6719 | {'precision': 0.6879049676025918, 'recall': 0.7873918417799752, 'f1': 0.7342939481268012, 'number': 809} | {'precision': 0.319672131147541, 'recall': 0.3277310924369748, 'f1': 0.32365145228215775, 'number': 119} | {'precision': 0.7710526315789473, 'recall': 0.8253521126760563, 'f1': 0.7972789115646259, 'number': 1065} | 0.7107 | 0.7802 | 0.7438 | 0.8165 | |
|
| 0.334 | 11.0 | 110 | 0.6857 | {'precision': 0.6862955032119914, 'recall': 0.792336217552534, 'f1': 0.7355134825014343, 'number': 809} | {'precision': 0.3584905660377358, 'recall': 0.31932773109243695, 'f1': 0.3377777777777778, 'number': 119} | {'precision': 0.7785778577857786, 'recall': 0.812206572769953, 'f1': 0.7950367647058822, 'number': 1065} | 0.7178 | 0.7747 | 0.7452 | 0.8160 | |
|
| 0.3234 | 12.0 | 120 | 0.6966 | {'precision': 0.6926454445664105, 'recall': 0.7799752781211372, 'f1': 0.7337209302325581, 'number': 809} | {'precision': 0.3125, 'recall': 0.33613445378151263, 'f1': 0.3238866396761134, 'number': 119} | {'precision': 0.7707786526684165, 'recall': 0.8272300469483568, 'f1': 0.7980072463768116, 'number': 1065} | 0.7113 | 0.7787 | 0.7435 | 0.8154 | |
|
| 0.3019 | 13.0 | 130 | 0.6940 | {'precision': 0.7010869565217391, 'recall': 0.7972805933250927, 'f1': 0.746096009253904, 'number': 809} | {'precision': 0.3252032520325203, 'recall': 0.33613445378151263, 'f1': 0.3305785123966942, 'number': 119} | {'precision': 0.7724444444444445, 'recall': 0.815962441314554, 'f1': 0.7936073059360731, 'number': 1065} | 0.7168 | 0.7797 | 0.7469 | 0.8166 | |
|
| 0.2888 | 14.0 | 140 | 0.7011 | {'precision': 0.6946564885496184, 'recall': 0.7873918417799752, 'f1': 0.7381228273464657, 'number': 809} | {'precision': 0.29927007299270075, 'recall': 0.3445378151260504, 'f1': 0.3203125, 'number': 119} | {'precision': 0.7726872246696035, 'recall': 0.8234741784037559, 'f1': 0.7972727272727272, 'number': 1065} | 0.7104 | 0.7802 | 0.7437 | 0.8170 | |
|
| 0.2838 | 15.0 | 150 | 0.7034 | {'precision': 0.6974697469746974, 'recall': 0.7836835599505563, 'f1': 0.7380675203725262, 'number': 809} | {'precision': 0.3106060606060606, 'recall': 0.3445378151260504, 'f1': 0.32669322709163345, 'number': 119} | {'precision': 0.7759226713532513, 'recall': 0.8291079812206573, 'f1': 0.8016341352700863, 'number': 1065} | 0.7150 | 0.7817 | 0.7469 | 0.8170 | |
|
|
|
|
|
### Framework versions |
|
|
|
- Transformers 4.27.4 |
|
- Pytorch 1.13.1 |
|
- Datasets 2.11.0 |
|
- Tokenizers 0.13.3 |
|
|
