layoutlm-funsd
This model is a fine-tuned version of microsoft/layoutlm-base-uncased on the funsd dataset. It achieves the following results on the evaluation set:
- Loss: 0.6940
- Answer: {'precision': 0.721978021978022, 'recall': 0.8121137206427689, 'f1': 0.7643979057591623, 'number': 809}
- Header: {'precision': 0.2662337662337662, 'recall': 0.3445378151260504, 'f1': 0.30036630036630035, 'number': 119}
- Question: {'precision': 0.7816091954022989, 'recall': 0.8300469483568075, 'f1': 0.8051001821493625, 'number': 1065}
- Overall Precision: 0.7207
- Overall Recall: 0.7938
- Overall F1: 0.7555
- Overall Accuracy: 0.8073
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 3e-05
- train_batch_size: 16
- eval_batch_size: 8
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 15
Training results
| Training Loss | Epoch | Step | Validation Loss | Answer | Header | Question | Overall Precision | Overall Recall | Overall F1 | Overall Accuracy |
|---|---|---|---|---|---|---|---|---|---|---|
| 1.755 | 1.0 | 10 | 1.5815 | {'precision': 0.026919242273180457, 'recall': 0.03337453646477132, 'f1': 0.02980132450331126, 'number': 809} | {'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119} | {'precision': 0.20780487804878048, 'recall': 0.2, 'f1': 0.20382775119617225, 'number': 1065} | 0.1183 | 0.1204 | 0.1194 | 0.3885 |
| 1.4375 | 2.0 | 20 | 1.2088 | {'precision': 0.28227848101265823, 'recall': 0.27564894932014833, 'f1': 0.2789243277048155, 'number': 809} | {'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119} | {'precision': 0.4782964782964783, 'recall': 0.5483568075117371, 'f1': 0.5109361329833771, 'number': 1065} | 0.4013 | 0.4049 | 0.4031 | 0.6223 |
| 1.0595 | 3.0 | 30 | 0.9379 | {'precision': 0.503954802259887, 'recall': 0.5512978986402967, 'f1': 0.526564344746163, 'number': 809} | {'precision': 0.0425531914893617, 'recall': 0.01680672268907563, 'f1': 0.024096385542168672, 'number': 119} | {'precision': 0.6126205083260298, 'recall': 0.6563380281690141, 'f1': 0.6337262012692656, 'number': 1065} | 0.5533 | 0.5755 | 0.5642 | 0.7194 |
| 0.8139 | 4.0 | 40 | 0.7735 | {'precision': 0.6280041797283177, 'recall': 0.7428924598269468, 'f1': 0.680634201585504, 'number': 809} | {'precision': 0.13432835820895522, 'recall': 0.07563025210084033, 'f1': 0.09677419354838708, 'number': 119} | {'precision': 0.6600688468158348, 'recall': 0.72018779342723, 'f1': 0.688819039066008, 'number': 1065} | 0.6299 | 0.6909 | 0.6590 | 0.7636 |
| 0.664 | 5.0 | 50 | 0.7245 | {'precision': 0.6519453207150369, 'recall': 0.7663782447466008, 'f1': 0.7045454545454546, 'number': 809} | {'precision': 0.24719101123595505, 'recall': 0.18487394957983194, 'f1': 0.21153846153846156, 'number': 119} | {'precision': 0.7090909090909091, 'recall': 0.7690140845070422, 'f1': 0.7378378378378379, 'number': 1065} | 0.6656 | 0.7331 | 0.6977 | 0.7757 |
| 0.5505 | 6.0 | 60 | 0.6956 | {'precision': 0.6834061135371179, 'recall': 0.7737948084054388, 'f1': 0.7257971014492753, 'number': 809} | {'precision': 0.28205128205128205, 'recall': 0.18487394957983194, 'f1': 0.2233502538071066, 'number': 119} | {'precision': 0.723421926910299, 'recall': 0.8178403755868544, 'f1': 0.7677390921110622, 'number': 1065} | 0.6911 | 0.7622 | 0.7249 | 0.7888 |
| 0.4759 | 7.0 | 70 | 0.6712 | {'precision': 0.6844396082698585, 'recall': 0.7775030902348579, 'f1': 0.7280092592592592, 'number': 809} | {'precision': 0.2727272727272727, 'recall': 0.2773109243697479, 'f1': 0.27499999999999997, 'number': 119} | {'precision': 0.7472527472527473, 'recall': 0.8300469483568075, 'f1': 0.786476868327402, 'number': 1065} | 0.6955 | 0.7757 | 0.7334 | 0.7975 |
| 0.4276 | 8.0 | 80 | 0.6765 | {'precision': 0.6889375684556407, 'recall': 0.7775030902348579, 'f1': 0.7305458768873403, 'number': 809} | {'precision': 0.28205128205128205, 'recall': 0.2773109243697479, 'f1': 0.2796610169491525, 'number': 119} | {'precision': 0.7527333894028595, 'recall': 0.8403755868544601, 'f1': 0.7941437444543035, 'number': 1065} | 0.7017 | 0.7812 | 0.7393 | 0.8021 |
| 0.3788 | 9.0 | 90 | 0.6653 | {'precision': 0.7081930415263749, 'recall': 0.7799752781211372, 'f1': 0.7423529411764707, 'number': 809} | {'precision': 0.2647058823529412, 'recall': 0.3025210084033613, 'f1': 0.2823529411764706, 'number': 119} | {'precision': 0.7667238421955404, 'recall': 0.8394366197183099, 'f1': 0.8014343343792021, 'number': 1065} | 0.7118 | 0.7832 | 0.7458 | 0.8049 |
| 0.3466 | 10.0 | 100 | 0.6838 | {'precision': 0.7005464480874317, 'recall': 0.792336217552534, 'f1': 0.7436194895591649, 'number': 809} | {'precision': 0.2706766917293233, 'recall': 0.3025210084033613, 'f1': 0.28571428571428564, 'number': 119} | {'precision': 0.7728055077452668, 'recall': 0.8431924882629108, 'f1': 0.8064660978895375, 'number': 1065} | 0.7127 | 0.7903 | 0.7495 | 0.8047 |
| 0.3142 | 11.0 | 110 | 0.6795 | {'precision': 0.6997816593886463, 'recall': 0.792336217552534, 'f1': 0.7431884057971013, 'number': 809} | {'precision': 0.2857142857142857, 'recall': 0.3025210084033613, 'f1': 0.2938775510204082, 'number': 119} | {'precision': 0.7994628469113697, 'recall': 0.8384976525821596, 'f1': 0.8185151237396883, 'number': 1065} | 0.7272 | 0.7878 | 0.7563 | 0.8067 |
| 0.2978 | 12.0 | 120 | 0.6922 | {'precision': 0.6927194860813705, 'recall': 0.799752781211372, 'f1': 0.7423981640849111, 'number': 809} | {'precision': 0.2585034013605442, 'recall': 0.31932773109243695, 'f1': 0.2857142857142857, 'number': 119} | {'precision': 0.7768090671316478, 'recall': 0.8366197183098592, 'f1': 0.8056057866184448, 'number': 1065} | 0.7074 | 0.7908 | 0.7467 | 0.8026 |
| 0.2824 | 13.0 | 130 | 0.6960 | {'precision': 0.7184357541899441, 'recall': 0.7948084054388134, 'f1': 0.754694835680751, 'number': 809} | {'precision': 0.2611464968152866, 'recall': 0.3445378151260504, 'f1': 0.2971014492753623, 'number': 119} | {'precision': 0.7757255936675461, 'recall': 0.828169014084507, 'f1': 0.8010899182561309, 'number': 1065} | 0.7154 | 0.7858 | 0.7489 | 0.8045 |
| 0.2696 | 14.0 | 140 | 0.6917 | {'precision': 0.7164667393675027, 'recall': 0.8121137206427689, 'f1': 0.7612977983777521, 'number': 809} | {'precision': 0.2708333333333333, 'recall': 0.3277310924369748, 'f1': 0.2965779467680608, 'number': 119} | {'precision': 0.7833775419982316, 'recall': 0.831924882629108, 'f1': 0.8069216757741348, 'number': 1065} | 0.7217 | 0.7938 | 0.7560 | 0.8067 |
| 0.2674 | 15.0 | 150 | 0.6940 | {'precision': 0.721978021978022, 'recall': 0.8121137206427689, 'f1': 0.7643979057591623, 'number': 809} | {'precision': 0.2662337662337662, 'recall': 0.3445378151260504, 'f1': 0.30036630036630035, 'number': 119} | {'precision': 0.7816091954022989, 'recall': 0.8300469483568075, 'f1': 0.8051001821493625, 'number': 1065} | 0.7207 | 0.7938 | 0.7555 | 0.8073 |
Framework versions
- Transformers 4.30.2
- Pytorch 2.0.1+cu118
- Datasets 2.13.1
- Tokenizers 0.13.3
- Downloads last month
- 8
This model does not have enough activity to be deployed to Inference API (serverless) yet. Increase its social
visibility and check back later, or deploy to Inference Endpoints (dedicated)
instead.