metadata
license: mit
base_model: microsoft/layoutlm-base-uncased
tags:
- generated_from_trainer
datasets:
- funsd
model-index:
- name: layoutlm-funsd
results: []
layoutlm-funsd
This model is a fine-tuned version of microsoft/layoutlm-base-uncased on the funsd dataset. It achieves the following results on the evaluation set:
- Loss: 1.0890
- Answer: {'precision': 0.38420107719928187, 'recall': 0.5290482076637825, 'f1': 0.4451378055122205, 'number': 809}
- Header: {'precision': 0.28888888888888886, 'recall': 0.2184873949579832, 'f1': 0.24880382775119617, 'number': 119}
- Question: {'precision': 0.48959136468774095, 'recall': 0.596244131455399, 'f1': 0.5376799322607958, 'number': 1065}
- Overall Precision: 0.4354
- Overall Recall: 0.5464
- Overall F1: 0.4846
- Overall Accuracy: 0.6258
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 3e-05
- train_batch_size: 16
- eval_batch_size: 8
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 15
- mixed_precision_training: Native AMP
Training results
| Training Loss | Epoch | Step | Validation Loss | Answer | Header | Question | Overall Precision | Overall Recall | Overall F1 | Overall Accuracy |
|---|---|---|---|---|---|---|---|---|---|---|
| 1.7643 | 1.0 | 10 | 1.5177 | {'precision': 0.052202283849918436, 'recall': 0.07911001236093942, 'f1': 0.06289926289926291, 'number': 809} | {'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119} | {'precision': 0.2581360946745562, 'recall': 0.3276995305164319, 'f1': 0.2887877534133223, 'number': 1065} | 0.1602 | 0.2072 | 0.1807 | 0.3823 |
| 1.4448 | 2.0 | 20 | 1.3359 | {'precision': 0.18779342723004694, 'recall': 0.39555006180469715, 'f1': 0.2546756864305611, 'number': 809} | {'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119} | {'precision': 0.2733245729303548, 'recall': 0.39061032863849765, 'f1': 0.321608040201005, 'number': 1065} | 0.2273 | 0.3693 | 0.2814 | 0.4206 |
| 1.2967 | 3.0 | 30 | 1.2160 | {'precision': 0.2261437908496732, 'recall': 0.4276885043263288, 'f1': 0.29585292860196666, 'number': 809} | {'precision': 0.02040816326530612, 'recall': 0.008403361344537815, 'f1': 0.011904761904761904, 'number': 119} | {'precision': 0.34987113402061853, 'recall': 0.5098591549295775, 'f1': 0.41497898356897206, 'number': 1065} | 0.2843 | 0.4466 | 0.3474 | 0.4803 |
| 1.172 | 4.0 | 40 | 1.1080 | {'precision': 0.2609299097848716, 'recall': 0.4647713226205192, 'f1': 0.3342222222222222, 'number': 809} | {'precision': 0.2, 'recall': 0.12605042016806722, 'f1': 0.15463917525773196, 'number': 119} | {'precision': 0.39096126255380204, 'recall': 0.5117370892018779, 'f1': 0.4432696217974787, 'number': 1065} | 0.3216 | 0.4696 | 0.3818 | 0.5682 |
| 1.0668 | 5.0 | 50 | 1.1224 | {'precision': 0.2859304084720121, 'recall': 0.4672435105067985, 'f1': 0.3547630220553731, 'number': 809} | {'precision': 0.2571428571428571, 'recall': 0.15126050420168066, 'f1': 0.19047619047619044, 'number': 119} | {'precision': 0.39935691318327976, 'recall': 0.5830985915492958, 'f1': 0.47404580152671755, 'number': 1065} | 0.3451 | 0.5103 | 0.4117 | 0.5719 |
| 1.0053 | 6.0 | 60 | 1.0842 | {'precision': 0.31098430813124106, 'recall': 0.5389369592088998, 'f1': 0.3943916779737675, 'number': 809} | {'precision': 0.3333333333333333, 'recall': 0.17647058823529413, 'f1': 0.23076923076923078, 'number': 119} | {'precision': 0.4626998223801066, 'recall': 0.4892018779342723, 'f1': 0.4755819260611593, 'number': 1065} | 0.3775 | 0.4907 | 0.4267 | 0.5869 |
| 0.9367 | 7.0 | 70 | 1.0354 | {'precision': 0.33884297520661155, 'recall': 0.4561186650185414, 'f1': 0.38883034773445735, 'number': 809} | {'precision': 0.27848101265822783, 'recall': 0.18487394957983194, 'f1': 0.2222222222222222, 'number': 119} | {'precision': 0.4579100145137881, 'recall': 0.5924882629107981, 'f1': 0.5165779778960293, 'number': 1065} | 0.4014 | 0.5128 | 0.4503 | 0.6069 |
| 0.8736 | 8.0 | 80 | 1.0367 | {'precision': 0.3433583959899749, 'recall': 0.5080346106304079, 'f1': 0.4097706879361914, 'number': 809} | {'precision': 0.24675324675324675, 'recall': 0.15966386554621848, 'f1': 0.19387755102040818, 'number': 119} | {'precision': 0.4403292181069959, 'recall': 0.6028169014084507, 'f1': 0.5089179548156956, 'number': 1065} | 0.3924 | 0.5379 | 0.4538 | 0.6083 |
| 0.8322 | 9.0 | 90 | 1.0585 | {'precision': 0.38257575757575757, 'recall': 0.49938195302843014, 'f1': 0.43324396782841823, 'number': 809} | {'precision': 0.1919191919191919, 'recall': 0.15966386554621848, 'f1': 0.17431192660550457, 'number': 119} | {'precision': 0.48465266558966075, 'recall': 0.5633802816901409, 'f1': 0.5210594876248372, 'number': 1065} | 0.4275 | 0.5133 | 0.4665 | 0.6171 |
| 0.8201 | 10.0 | 100 | 1.0589 | {'precision': 0.3753527751646284, 'recall': 0.4932014833127318, 'f1': 0.42628205128205127, 'number': 809} | {'precision': 0.275, 'recall': 0.18487394957983194, 'f1': 0.22110552763819097, 'number': 119} | {'precision': 0.4782945736434108, 'recall': 0.5793427230046948, 'f1': 0.5239915074309979, 'number': 1065} | 0.4266 | 0.5208 | 0.4690 | 0.6086 |
| 0.7451 | 11.0 | 110 | 1.0393 | {'precision': 0.3754716981132076, 'recall': 0.4919653893695921, 'f1': 0.42589620117710003, 'number': 809} | {'precision': 0.2804878048780488, 'recall': 0.19327731092436976, 'f1': 0.22885572139303487, 'number': 119} | {'precision': 0.4541832669322709, 'recall': 0.6422535211267606, 'f1': 0.5320886814469078, 'number': 1065} | 0.4173 | 0.5544 | 0.4762 | 0.6132 |
| 0.7445 | 12.0 | 120 | 1.0649 | {'precision': 0.3752166377816291, 'recall': 0.5352286773794809, 'f1': 0.4411614875191034, 'number': 809} | {'precision': 0.2653061224489796, 'recall': 0.2184873949579832, 'f1': 0.23963133640552997, 'number': 119} | {'precision': 0.49351701782820095, 'recall': 0.571830985915493, 'f1': 0.5297955632883862, 'number': 1065} | 0.4296 | 0.5359 | 0.4769 | 0.6145 |
| 0.7064 | 13.0 | 130 | 1.1267 | {'precision': 0.3775933609958506, 'recall': 0.5624227441285538, 'f1': 0.45183714001986097, 'number': 809} | {'precision': 0.3116883116883117, 'recall': 0.20168067226890757, 'f1': 0.24489795918367344, 'number': 119} | {'precision': 0.5072094995759118, 'recall': 0.5615023474178403, 'f1': 0.5329768270944741, 'number': 1065} | 0.4376 | 0.5404 | 0.4836 | 0.6174 |
| 0.6846 | 14.0 | 140 | 1.0692 | {'precision': 0.3945841392649903, 'recall': 0.5043263288009888, 'f1': 0.44275637547476937, 'number': 809} | {'precision': 0.29411764705882354, 'recall': 0.21008403361344538, 'f1': 0.2450980392156863, 'number': 119} | {'precision': 0.48787878787878786, 'recall': 0.6046948356807512, 'f1': 0.5400419287211741, 'number': 1065} | 0.4416 | 0.5404 | 0.4860 | 0.6198 |
| 0.6688 | 15.0 | 150 | 1.0890 | {'precision': 0.38420107719928187, 'recall': 0.5290482076637825, 'f1': 0.4451378055122205, 'number': 809} | {'precision': 0.28888888888888886, 'recall': 0.2184873949579832, 'f1': 0.24880382775119617, 'number': 119} | {'precision': 0.48959136468774095, 'recall': 0.596244131455399, 'f1': 0.5376799322607958, 'number': 1065} | 0.4354 | 0.5464 | 0.4846 | 0.6258 |
Framework versions
- Transformers 4.38.2
- Pytorch 2.2.1+cu121
- Datasets 2.18.0
- Tokenizers 0.15.2