|
--- |
|
tags: |
|
- generated_from_trainer |
|
datasets: |
|
- funsd |
|
model-index: |
|
- name: layoutlm-funsd |
|
results: [] |
|
--- |
|
|
|
<!-- This model card has been generated automatically according to the information the Trainer had access to. You |
|
should probably proofread and complete it, then remove this comment. --> |
|
|
|
# layoutlm-funsd |
|
|
|
This model is a fine-tuned version of [microsoft/layoutlm-base-uncased](https://huggingface.co/microsoft/layoutlm-base-uncased) on the funsd dataset. |
|
It achieves the following results on the evaluation set: |
|
- Loss: 0.6650 |
|
- Answer: {'precision': 0.7158712541620422, 'recall': 0.7972805933250927, 'f1': 0.7543859649122808, 'number': 809} |
|
- Header: {'precision': 0.2982456140350877, 'recall': 0.2857142857142857, 'f1': 0.2918454935622318, 'number': 119} |
|
- Question: {'precision': 0.7667238421955404, 'recall': 0.8394366197183099, 'f1': 0.8014343343792021, 'number': 1065} |
|
- Overall Precision: 0.7212 |
|
- Overall Recall: 0.7893 |
|
- Overall F1: 0.7537 |
|
- Overall Accuracy: 0.8191 |
|
|
|
## Model description |
|
|
|
More information needed |
|
|
|
## Intended uses & limitations |
|
|
|
More information needed |
|
|
|
## Training and evaluation data |
|
|
|
More information needed |
|
|
|
## Training procedure |
|
|
|
### Training hyperparameters |
|
|
|
The following hyperparameters were used during training: |
|
- learning_rate: 3e-05 |
|
- train_batch_size: 16 |
|
- eval_batch_size: 8 |
|
- seed: 42 |
|
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08 |
|
- lr_scheduler_type: linear |
|
- num_epochs: 15 |
|
|
|
### Training results |
|
|
|
| Training Loss | Epoch | Step | Validation Loss | Answer | Header | Question | Overall Precision | Overall Recall | Overall F1 | Overall Accuracy | |
|
|:-------------:|:-----:|:----:|:---------------:|:-----------------------------------------------------------------------------------------------------------:|:------------------------------------------------------------------------------------------------------------:|:------------------------------------------------------------------------------------------------------------:|:-----------------:|:--------------:|:----------:|:----------------:| |
|
| 1.7902 | 1.0 | 10 | 1.6058 | {'precision': 0.0174496644295302, 'recall': 0.016069221260815822, 'f1': 0.01673101673101673, 'number': 809} | {'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119} | {'precision': 0.24484848484848484, 'recall': 0.18967136150234742, 'f1': 0.21375661375661376, 'number': 1065} | 0.1369 | 0.1079 | 0.1207 | 0.3425 | |
|
| 1.4512 | 2.0 | 20 | 1.2477 | {'precision': 0.22826086956521738, 'recall': 0.23362175525339926, 'f1': 0.23091020158827122, 'number': 809} | {'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119} | {'precision': 0.4611066559743384, 'recall': 0.539906103286385, 'f1': 0.49740484429065746, 'number': 1065} | 0.3680 | 0.3833 | 0.3755 | 0.5802 | |
|
| 1.0772 | 3.0 | 30 | 0.9579 | {'precision': 0.47790055248618785, 'recall': 0.4276885043263288, 'f1': 0.45140247879973905, 'number': 809} | {'precision': 0.05555555555555555, 'recall': 0.01680672268907563, 'f1': 0.025806451612903226, 'number': 119} | {'precision': 0.6270125223613596, 'recall': 0.6582159624413145, 'f1': 0.6422354557947779, 'number': 1065} | 0.5586 | 0.5263 | 0.5420 | 0.6919 | |
|
| 0.8282 | 4.0 | 40 | 0.7735 | {'precision': 0.6132368148914168, 'recall': 0.7330037082818294, 'f1': 0.6677927927927928, 'number': 809} | {'precision': 0.17647058823529413, 'recall': 0.10084033613445378, 'f1': 0.1283422459893048, 'number': 119} | {'precision': 0.6726649528706083, 'recall': 0.7370892018779343, 'f1': 0.703405017921147, 'number': 1065} | 0.6312 | 0.6974 | 0.6627 | 0.7621 | |
|
| 0.6763 | 5.0 | 50 | 0.7086 | {'precision': 0.6333333333333333, 'recall': 0.7515451174289246, 'f1': 0.6873940079140758, 'number': 809} | {'precision': 0.325, 'recall': 0.2184873949579832, 'f1': 0.26130653266331655, 'number': 119} | {'precision': 0.6769731489015459, 'recall': 0.7812206572769953, 'f1': 0.7253705318221447, 'number': 1065} | 0.6461 | 0.7356 | 0.6879 | 0.7869 | |
|
| 0.5577 | 6.0 | 60 | 0.6736 | {'precision': 0.6542155816435432, 'recall': 0.757725587144623, 'f1': 0.7021764032073311, 'number': 809} | {'precision': 0.32926829268292684, 'recall': 0.226890756302521, 'f1': 0.26865671641791045, 'number': 119} | {'precision': 0.6952822892498066, 'recall': 0.844131455399061, 'f1': 0.7625106022052586, 'number': 1065} | 0.6657 | 0.7722 | 0.7150 | 0.7955 | |
|
| 0.4901 | 7.0 | 70 | 0.6510 | {'precision': 0.6706263498920086, 'recall': 0.7676143386897404, 'f1': 0.7158501440922191, 'number': 809} | {'precision': 0.27927927927927926, 'recall': 0.2605042016806723, 'f1': 0.26956521739130435, 'number': 119} | {'precision': 0.7412765957446809, 'recall': 0.8178403755868544, 'f1': 0.7776785714285714, 'number': 1065} | 0.6885 | 0.7642 | 0.7244 | 0.7998 | |
|
| 0.4474 | 8.0 | 80 | 0.6389 | {'precision': 0.6828478964401294, 'recall': 0.7824474660074165, 'f1': 0.7292626728110598, 'number': 809} | {'precision': 0.3137254901960784, 'recall': 0.2689075630252101, 'f1': 0.2895927601809955, 'number': 119} | {'precision': 0.7523564695801199, 'recall': 0.8244131455399061, 'f1': 0.7867383512544801, 'number': 1065} | 0.7026 | 0.7742 | 0.7367 | 0.8049 | |
|
| 0.4055 | 9.0 | 90 | 0.6371 | {'precision': 0.6855277475516867, 'recall': 0.7787391841779975, 'f1': 0.7291666666666666, 'number': 809} | {'precision': 0.288135593220339, 'recall': 0.2857142857142857, 'f1': 0.2869198312236287, 'number': 119} | {'precision': 0.7368852459016394, 'recall': 0.844131455399061, 'f1': 0.7868708971553611, 'number': 1065} | 0.6925 | 0.7842 | 0.7355 | 0.8111 | |
|
| 0.3597 | 10.0 | 100 | 0.6547 | {'precision': 0.7027932960893855, 'recall': 0.7775030902348579, 'f1': 0.7382629107981221, 'number': 809} | {'precision': 0.25925925925925924, 'recall': 0.29411764705882354, 'f1': 0.2755905511811024, 'number': 119} | {'precision': 0.7463330457290768, 'recall': 0.812206572769953, 'f1': 0.7778776978417264, 'number': 1065} | 0.6985 | 0.7672 | 0.7312 | 0.8070 | |
|
| 0.3295 | 11.0 | 110 | 0.6618 | {'precision': 0.709070796460177, 'recall': 0.792336217552534, 'f1': 0.7483946293053124, 'number': 809} | {'precision': 0.3333333333333333, 'recall': 0.25210084033613445, 'f1': 0.28708133971291866, 'number': 119} | {'precision': 0.7857142857142857, 'recall': 0.8366197183098592, 'f1': 0.8103683492496588, 'number': 1065} | 0.7340 | 0.7837 | 0.7581 | 0.8106 | |
|
| 0.3169 | 12.0 | 120 | 0.6639 | {'precision': 0.7094972067039106, 'recall': 0.7849196538936959, 'f1': 0.7453051643192488, 'number': 809} | {'precision': 0.3017241379310345, 'recall': 0.29411764705882354, 'f1': 0.29787234042553185, 'number': 119} | {'precision': 0.7582417582417582, 'recall': 0.8422535211267606, 'f1': 0.7980427046263344, 'number': 1065} | 0.7142 | 0.7863 | 0.7485 | 0.8152 | |
|
| 0.2951 | 13.0 | 130 | 0.6653 | {'precision': 0.7094972067039106, 'recall': 0.7849196538936959, 'f1': 0.7453051643192488, 'number': 809} | {'precision': 0.3063063063063063, 'recall': 0.2857142857142857, 'f1': 0.2956521739130435, 'number': 119} | {'precision': 0.7784588441330998, 'recall': 0.8347417840375587, 'f1': 0.805618486633439, 'number': 1065} | 0.7253 | 0.7817 | 0.7525 | 0.8167 | |
|
| 0.2872 | 14.0 | 140 | 0.6667 | {'precision': 0.7116022099447514, 'recall': 0.796044499381953, 'f1': 0.751458576429405, 'number': 809} | {'precision': 0.2982456140350877, 'recall': 0.2857142857142857, 'f1': 0.2918454935622318, 'number': 119} | {'precision': 0.7737162750217581, 'recall': 0.8347417840375587, 'f1': 0.803071364046974, 'number': 1065} | 0.7228 | 0.7863 | 0.7532 | 0.8179 | |
|
| 0.2779 | 15.0 | 150 | 0.6650 | {'precision': 0.7158712541620422, 'recall': 0.7972805933250927, 'f1': 0.7543859649122808, 'number': 809} | {'precision': 0.2982456140350877, 'recall': 0.2857142857142857, 'f1': 0.2918454935622318, 'number': 119} | {'precision': 0.7667238421955404, 'recall': 0.8394366197183099, 'f1': 0.8014343343792021, 'number': 1065} | 0.7212 | 0.7893 | 0.7537 | 0.8191 | |
|
|
|
|
|
### Framework versions |
|
|
|
- Transformers 4.25.1 |
|
- Pytorch 1.12.1 |
|
- Datasets 2.8.0 |
|
- Tokenizers 0.13.2 |
|
|
