layoutlm-funsd3
This model is a fine-tuned version of microsoft/layoutlm-base-uncased on the funsd dataset. It achieves the following results on the evaluation set:
- Loss: 0.6649
- Answer: {'precision': 0.6862955032119914, 'recall': 0.792336217552534, 'f1': 0.7355134825014343, 'number': 809}
- Header: {'precision': 0.2782608695652174, 'recall': 0.2689075630252101, 'f1': 0.2735042735042735, 'number': 119}
- Question: {'precision': 0.731418918918919, 'recall': 0.8131455399061033, 'f1': 0.7701200533570476, 'number': 1065}
- Overall Precision: 0.6892
- Overall Recall: 0.7722
- Overall F1: 0.7283
- Overall Accuracy: 0.8077
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 2e-05
- train_batch_size: 16
- eval_batch_size: 8
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 500
- num_epochs: 25
- mixed_precision_training: Native AMP
Training results
| Training Loss | Epoch | Step | Validation Loss | Answer | Header | Question | Overall Precision | Overall Recall | Overall F1 | Overall Accuracy |
|---|---|---|---|---|---|---|---|---|---|---|
| 1.9006 | 1.0 | 10 | 1.9148 | {'precision': 0.034013605442176874, 'recall': 0.11742892459826947, 'f1': 0.05274847307051638, 'number': 809} | {'precision': 0.007547169811320755, 'recall': 0.01680672268907563, 'f1': 0.010416666666666666, 'number': 119} | {'precision': 0.035897435897435895, 'recall': 0.06572769953051644, 'f1': 0.04643449419568822, 'number': 1065} | 0.0333 | 0.0838 | 0.0477 | 0.1975 |
| 1.8905 | 2.0 | 20 | 1.9045 | {'precision': 0.03486238532110092, 'recall': 0.11742892459826947, 'f1': 0.05376344086021505, 'number': 809} | {'precision': 0.004761904761904762, 'recall': 0.008403361344537815, 'f1': 0.006079027355623101, 'number': 119} | {'precision': 0.03635432667690732, 'recall': 0.06666666666666667, 'f1': 0.04705102717031146, 'number': 1065} | 0.0342 | 0.0838 | 0.0485 | 0.2074 |
| 1.8811 | 3.0 | 30 | 1.8873 | {'precision': 0.032742681047765794, 'recall': 0.10506798516687268, 'f1': 0.0499265785609398, 'number': 809} | {'precision': 0.00684931506849315, 'recall': 0.008403361344537815, 'f1': 0.007547169811320755, 'number': 119} | {'precision': 0.03967027305512622, 'recall': 0.07230046948356808, 'f1': 0.051230871590153035, 'number': 1065} | 0.0348 | 0.0818 | 0.0488 | 0.2231 |
| 1.8598 | 4.0 | 40 | 1.8641 | {'precision': 0.029242174629324547, 'recall': 0.08776266996291718, 'f1': 0.043867778807537845, 'number': 809} | {'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119} | {'precision': 0.045021186440677964, 'recall': 0.07981220657276995, 'f1': 0.057568574331188616, 'number': 1065} | 0.0355 | 0.0783 | 0.0489 | 0.2434 |
| 1.8352 | 5.0 | 50 | 1.8359 | {'precision': 0.027777777777777776, 'recall': 0.07416563658838071, 'f1': 0.040417649040080834, 'number': 809} | {'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119} | {'precision': 0.05329512893982808, 'recall': 0.08732394366197183, 'f1': 0.06619217081850534, 'number': 1065} | 0.0389 | 0.0768 | 0.0516 | 0.2684 |
| 1.805 | 6.0 | 60 | 1.8038 | {'precision': 0.021965952773201538, 'recall': 0.049443757725587144, 'f1': 0.030418250950570342, 'number': 809} | {'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119} | {'precision': 0.056172436316133244, 'recall': 0.08075117370892018, 'f1': 0.06625577812018489, 'number': 1065} | 0.0374 | 0.0632 | 0.0470 | 0.2894 |
| 1.7726 | 7.0 | 70 | 1.7692 | {'precision': 0.02216252518468771, 'recall': 0.0407911001236094, 'f1': 0.028720626631853784, 'number': 809} | {'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119} | {'precision': 0.06599378881987578, 'recall': 0.07981220657276995, 'f1': 0.07224819379515512, 'number': 1065} | 0.0424 | 0.0592 | 0.0494 | 0.3088 |
| 1.7332 | 8.0 | 80 | 1.7277 | {'precision': 0.018210609659540775, 'recall': 0.02843016069221261, 'f1': 0.022200772200772198, 'number': 809} | {'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119} | {'precision': 0.08497316636851521, 'recall': 0.0892018779342723, 'f1': 0.08703618873110398, 'number': 1065} | 0.0496 | 0.0592 | 0.0540 | 0.3301 |
| 1.6941 | 9.0 | 90 | 1.6821 | {'precision': 0.024411508282476024, 'recall': 0.034610630407911, 'f1': 0.028629856850715743, 'number': 809} | {'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119} | {'precision': 0.11352885525070956, 'recall': 0.11267605633802817, 'f1': 0.11310084825636192, 'number': 1065} | 0.0672 | 0.0743 | 0.0705 | 0.3529 |
| 1.6579 | 10.0 | 100 | 1.6290 | {'precision': 0.03211805555555555, 'recall': 0.04573547589616811, 'f1': 0.03773584905660377, 'number': 809} | {'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119} | {'precision': 0.1650902837489252, 'recall': 0.18028169014084508, 'f1': 0.17235188509874327, 'number': 1065} | 0.0989 | 0.1149 | 0.1063 | 0.3897 |
| 1.5882 | 11.0 | 110 | 1.5600 | {'precision': 0.06073943661971831, 'recall': 0.08529048207663782, 'f1': 0.07095115681233934, 'number': 809} | {'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119} | {'precision': 0.22230538922155688, 'recall': 0.27887323943661974, 'f1': 0.2473969179508538, 'number': 1065} | 0.1481 | 0.1836 | 0.1639 | 0.4485 |
| 1.5164 | 12.0 | 120 | 1.4778 | {'precision': 0.111, 'recall': 0.13720642768850433, 'f1': 0.12271973466003316, 'number': 809} | {'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119} | {'precision': 0.28520877565463554, 'recall': 0.3784037558685446, 'f1': 0.3252623083131558, 'number': 1065} | 0.2130 | 0.2579 | 0.2333 | 0.5018 |
| 1.4203 | 13.0 | 130 | 1.3796 | {'precision': 0.1891304347826087, 'recall': 0.21508034610630408, 'f1': 0.20127241179872757, 'number': 809} | {'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119} | {'precision': 0.37259100642398285, 'recall': 0.49014084507042255, 'f1': 0.4233576642335766, 'number': 1065} | 0.2999 | 0.3492 | 0.3227 | 0.5474 |
| 1.2916 | 14.0 | 140 | 1.2617 | {'precision': 0.27813852813852813, 'recall': 0.3176761433868974, 'f1': 0.29659549913444894, 'number': 809} | {'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119} | {'precision': 0.4349112426035503, 'recall': 0.5521126760563381, 'f1': 0.48655357881671496, 'number': 1065} | 0.3713 | 0.4240 | 0.3959 | 0.5943 |
| 1.1747 | 15.0 | 150 | 1.1279 | {'precision': 0.3726775956284153, 'recall': 0.4215080346106304, 'f1': 0.39559164733178653, 'number': 809} | {'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119} | {'precision': 0.5030816640986132, 'recall': 0.6131455399061033, 'f1': 0.5526872619551417, 'number': 1065} | 0.4482 | 0.4987 | 0.4721 | 0.6467 |
| 1.0441 | 16.0 | 160 | 0.9940 | {'precision': 0.46846846846846846, 'recall': 0.5784919653893696, 'f1': 0.5176991150442478, 'number': 809} | {'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119} | {'precision': 0.5588709677419355, 'recall': 0.6507042253521127, 'f1': 0.6013015184381778, 'number': 1065} | 0.5126 | 0.5825 | 0.5453 | 0.7049 |
| 0.9042 | 17.0 | 170 | 0.8718 | {'precision': 0.567703109327984, 'recall': 0.6996291718170581, 'f1': 0.6267995570321152, 'number': 809} | {'precision': 0.020833333333333332, 'recall': 0.008403361344537815, 'f1': 0.011976047904191616, 'number': 119} | {'precision': 0.6192468619246861, 'recall': 0.6948356807511737, 'f1': 0.6548672566371683, 'number': 1065} | 0.5835 | 0.6558 | 0.6175 | 0.7473 |
| 0.7845 | 18.0 | 180 | 0.7760 | {'precision': 0.597478176527643, 'recall': 0.761433868974042, 'f1': 0.6695652173913043, 'number': 809} | {'precision': 0.16981132075471697, 'recall': 0.07563025210084033, 'f1': 0.10465116279069768, 'number': 119} | {'precision': 0.6678082191780822, 'recall': 0.7323943661971831, 'f1': 0.6986117330944918, 'number': 1065} | 0.6239 | 0.7050 | 0.6620 | 0.7693 |
| 0.7023 | 19.0 | 190 | 0.7265 | {'precision': 0.619188921859545, 'recall': 0.7737948084054388, 'f1': 0.6879120879120879, 'number': 809} | {'precision': 0.22580645161290322, 'recall': 0.11764705882352941, 'f1': 0.15469613259668508, 'number': 119} | {'precision': 0.6943722943722944, 'recall': 0.7530516431924883, 'f1': 0.7225225225225225, 'number': 1065} | 0.6472 | 0.7235 | 0.6833 | 0.7783 |
| 0.6331 | 20.0 | 200 | 0.7139 | {'precision': 0.6457446808510638, 'recall': 0.7503090234857849, 'f1': 0.6941109205260149, 'number': 809} | {'precision': 0.25609756097560976, 'recall': 0.17647058823529413, 'f1': 0.208955223880597, 'number': 119} | {'precision': 0.6934548467274234, 'recall': 0.7859154929577464, 'f1': 0.7367957746478873, 'number': 1065} | 0.6572 | 0.7351 | 0.6940 | 0.7900 |
| 0.5789 | 21.0 | 210 | 0.6960 | {'precision': 0.6496815286624203, 'recall': 0.7564894932014833, 'f1': 0.6990291262135921, 'number': 809} | {'precision': 0.25274725274725274, 'recall': 0.19327731092436976, 'f1': 0.21904761904761905, 'number': 119} | {'precision': 0.706081081081081, 'recall': 0.7849765258215963, 'f1': 0.7434415295686971, 'number': 1065} | 0.6635 | 0.7381 | 0.6988 | 0.7929 |
| 0.5417 | 22.0 | 220 | 0.6774 | {'precision': 0.6699346405228758, 'recall': 0.7601977750309024, 'f1': 0.7122177185871453, 'number': 809} | {'precision': 0.211864406779661, 'recall': 0.21008403361344538, 'f1': 0.2109704641350211, 'number': 119} | {'precision': 0.6981907894736842, 'recall': 0.7971830985915493, 'f1': 0.7444103463393249, 'number': 1065} | 0.6612 | 0.7471 | 0.7015 | 0.7959 |
| 0.481 | 23.0 | 230 | 0.6671 | {'precision': 0.6748400852878464, 'recall': 0.7824474660074165, 'f1': 0.7246708643388666, 'number': 809} | {'precision': 0.2540983606557377, 'recall': 0.2605042016806723, 'f1': 0.2572614107883818, 'number': 119} | {'precision': 0.718013468013468, 'recall': 0.8009389671361502, 'f1': 0.7572126054150022, 'number': 1065} | 0.6748 | 0.7612 | 0.7154 | 0.8022 |
| 0.4419 | 24.0 | 240 | 0.6534 | {'precision': 0.6799140708915145, 'recall': 0.7824474660074165, 'f1': 0.7275862068965516, 'number': 809} | {'precision': 0.2818181818181818, 'recall': 0.2605042016806723, 'f1': 0.27074235807860264, 'number': 119} | {'precision': 0.7332185886402753, 'recall': 0.8, 'f1': 0.7651549169286035, 'number': 1065} | 0.6882 | 0.7607 | 0.7226 | 0.8054 |
| 0.406 | 25.0 | 250 | 0.6649 | {'precision': 0.6862955032119914, 'recall': 0.792336217552534, 'f1': 0.7355134825014343, 'number': 809} | {'precision': 0.2782608695652174, 'recall': 0.2689075630252101, 'f1': 0.2735042735042735, 'number': 119} | {'precision': 0.731418918918919, 'recall': 0.8131455399061033, 'f1': 0.7701200533570476, 'number': 1065} | 0.6892 | 0.7722 | 0.7283 | 0.8077 |
Framework versions
- Transformers 4.41.2
- Pytorch 2.3.1+cu121
- Datasets 2.19.2
- Tokenizers 0.19.1
- Downloads last month
- 4
Model tree for Benedict-L/layoutlm-funsd3
Base model
microsoft/layoutlm-base-uncased