Edit model card

layoutlm-funsd3

This model is a fine-tuned version of microsoft/layoutlm-base-uncased on the funsd dataset. It achieves the following results on the evaluation set:

  • Loss: 0.6649
  • Answer: {'precision': 0.6862955032119914, 'recall': 0.792336217552534, 'f1': 0.7355134825014343, 'number': 809}
  • Header: {'precision': 0.2782608695652174, 'recall': 0.2689075630252101, 'f1': 0.2735042735042735, 'number': 119}
  • Question: {'precision': 0.731418918918919, 'recall': 0.8131455399061033, 'f1': 0.7701200533570476, 'number': 1065}
  • Overall Precision: 0.6892
  • Overall Recall: 0.7722
  • Overall F1: 0.7283
  • Overall Accuracy: 0.8077

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 2e-05
  • train_batch_size: 16
  • eval_batch_size: 8
  • seed: 42
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: linear
  • lr_scheduler_warmup_steps: 500
  • num_epochs: 25
  • mixed_precision_training: Native AMP

Training results

Training Loss Epoch Step Validation Loss Answer Header Question Overall Precision Overall Recall Overall F1 Overall Accuracy
1.9006 1.0 10 1.9148 {'precision': 0.034013605442176874, 'recall': 0.11742892459826947, 'f1': 0.05274847307051638, 'number': 809} {'precision': 0.007547169811320755, 'recall': 0.01680672268907563, 'f1': 0.010416666666666666, 'number': 119} {'precision': 0.035897435897435895, 'recall': 0.06572769953051644, 'f1': 0.04643449419568822, 'number': 1065} 0.0333 0.0838 0.0477 0.1975
1.8905 2.0 20 1.9045 {'precision': 0.03486238532110092, 'recall': 0.11742892459826947, 'f1': 0.05376344086021505, 'number': 809} {'precision': 0.004761904761904762, 'recall': 0.008403361344537815, 'f1': 0.006079027355623101, 'number': 119} {'precision': 0.03635432667690732, 'recall': 0.06666666666666667, 'f1': 0.04705102717031146, 'number': 1065} 0.0342 0.0838 0.0485 0.2074
1.8811 3.0 30 1.8873 {'precision': 0.032742681047765794, 'recall': 0.10506798516687268, 'f1': 0.0499265785609398, 'number': 809} {'precision': 0.00684931506849315, 'recall': 0.008403361344537815, 'f1': 0.007547169811320755, 'number': 119} {'precision': 0.03967027305512622, 'recall': 0.07230046948356808, 'f1': 0.051230871590153035, 'number': 1065} 0.0348 0.0818 0.0488 0.2231
1.8598 4.0 40 1.8641 {'precision': 0.029242174629324547, 'recall': 0.08776266996291718, 'f1': 0.043867778807537845, 'number': 809} {'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119} {'precision': 0.045021186440677964, 'recall': 0.07981220657276995, 'f1': 0.057568574331188616, 'number': 1065} 0.0355 0.0783 0.0489 0.2434
1.8352 5.0 50 1.8359 {'precision': 0.027777777777777776, 'recall': 0.07416563658838071, 'f1': 0.040417649040080834, 'number': 809} {'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119} {'precision': 0.05329512893982808, 'recall': 0.08732394366197183, 'f1': 0.06619217081850534, 'number': 1065} 0.0389 0.0768 0.0516 0.2684
1.805 6.0 60 1.8038 {'precision': 0.021965952773201538, 'recall': 0.049443757725587144, 'f1': 0.030418250950570342, 'number': 809} {'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119} {'precision': 0.056172436316133244, 'recall': 0.08075117370892018, 'f1': 0.06625577812018489, 'number': 1065} 0.0374 0.0632 0.0470 0.2894
1.7726 7.0 70 1.7692 {'precision': 0.02216252518468771, 'recall': 0.0407911001236094, 'f1': 0.028720626631853784, 'number': 809} {'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119} {'precision': 0.06599378881987578, 'recall': 0.07981220657276995, 'f1': 0.07224819379515512, 'number': 1065} 0.0424 0.0592 0.0494 0.3088
1.7332 8.0 80 1.7277 {'precision': 0.018210609659540775, 'recall': 0.02843016069221261, 'f1': 0.022200772200772198, 'number': 809} {'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119} {'precision': 0.08497316636851521, 'recall': 0.0892018779342723, 'f1': 0.08703618873110398, 'number': 1065} 0.0496 0.0592 0.0540 0.3301
1.6941 9.0 90 1.6821 {'precision': 0.024411508282476024, 'recall': 0.034610630407911, 'f1': 0.028629856850715743, 'number': 809} {'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119} {'precision': 0.11352885525070956, 'recall': 0.11267605633802817, 'f1': 0.11310084825636192, 'number': 1065} 0.0672 0.0743 0.0705 0.3529
1.6579 10.0 100 1.6290 {'precision': 0.03211805555555555, 'recall': 0.04573547589616811, 'f1': 0.03773584905660377, 'number': 809} {'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119} {'precision': 0.1650902837489252, 'recall': 0.18028169014084508, 'f1': 0.17235188509874327, 'number': 1065} 0.0989 0.1149 0.1063 0.3897
1.5882 11.0 110 1.5600 {'precision': 0.06073943661971831, 'recall': 0.08529048207663782, 'f1': 0.07095115681233934, 'number': 809} {'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119} {'precision': 0.22230538922155688, 'recall': 0.27887323943661974, 'f1': 0.2473969179508538, 'number': 1065} 0.1481 0.1836 0.1639 0.4485
1.5164 12.0 120 1.4778 {'precision': 0.111, 'recall': 0.13720642768850433, 'f1': 0.12271973466003316, 'number': 809} {'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119} {'precision': 0.28520877565463554, 'recall': 0.3784037558685446, 'f1': 0.3252623083131558, 'number': 1065} 0.2130 0.2579 0.2333 0.5018
1.4203 13.0 130 1.3796 {'precision': 0.1891304347826087, 'recall': 0.21508034610630408, 'f1': 0.20127241179872757, 'number': 809} {'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119} {'precision': 0.37259100642398285, 'recall': 0.49014084507042255, 'f1': 0.4233576642335766, 'number': 1065} 0.2999 0.3492 0.3227 0.5474
1.2916 14.0 140 1.2617 {'precision': 0.27813852813852813, 'recall': 0.3176761433868974, 'f1': 0.29659549913444894, 'number': 809} {'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119} {'precision': 0.4349112426035503, 'recall': 0.5521126760563381, 'f1': 0.48655357881671496, 'number': 1065} 0.3713 0.4240 0.3959 0.5943
1.1747 15.0 150 1.1279 {'precision': 0.3726775956284153, 'recall': 0.4215080346106304, 'f1': 0.39559164733178653, 'number': 809} {'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119} {'precision': 0.5030816640986132, 'recall': 0.6131455399061033, 'f1': 0.5526872619551417, 'number': 1065} 0.4482 0.4987 0.4721 0.6467
1.0441 16.0 160 0.9940 {'precision': 0.46846846846846846, 'recall': 0.5784919653893696, 'f1': 0.5176991150442478, 'number': 809} {'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119} {'precision': 0.5588709677419355, 'recall': 0.6507042253521127, 'f1': 0.6013015184381778, 'number': 1065} 0.5126 0.5825 0.5453 0.7049
0.9042 17.0 170 0.8718 {'precision': 0.567703109327984, 'recall': 0.6996291718170581, 'f1': 0.6267995570321152, 'number': 809} {'precision': 0.020833333333333332, 'recall': 0.008403361344537815, 'f1': 0.011976047904191616, 'number': 119} {'precision': 0.6192468619246861, 'recall': 0.6948356807511737, 'f1': 0.6548672566371683, 'number': 1065} 0.5835 0.6558 0.6175 0.7473
0.7845 18.0 180 0.7760 {'precision': 0.597478176527643, 'recall': 0.761433868974042, 'f1': 0.6695652173913043, 'number': 809} {'precision': 0.16981132075471697, 'recall': 0.07563025210084033, 'f1': 0.10465116279069768, 'number': 119} {'precision': 0.6678082191780822, 'recall': 0.7323943661971831, 'f1': 0.6986117330944918, 'number': 1065} 0.6239 0.7050 0.6620 0.7693
0.7023 19.0 190 0.7265 {'precision': 0.619188921859545, 'recall': 0.7737948084054388, 'f1': 0.6879120879120879, 'number': 809} {'precision': 0.22580645161290322, 'recall': 0.11764705882352941, 'f1': 0.15469613259668508, 'number': 119} {'precision': 0.6943722943722944, 'recall': 0.7530516431924883, 'f1': 0.7225225225225225, 'number': 1065} 0.6472 0.7235 0.6833 0.7783
0.6331 20.0 200 0.7139 {'precision': 0.6457446808510638, 'recall': 0.7503090234857849, 'f1': 0.6941109205260149, 'number': 809} {'precision': 0.25609756097560976, 'recall': 0.17647058823529413, 'f1': 0.208955223880597, 'number': 119} {'precision': 0.6934548467274234, 'recall': 0.7859154929577464, 'f1': 0.7367957746478873, 'number': 1065} 0.6572 0.7351 0.6940 0.7900
0.5789 21.0 210 0.6960 {'precision': 0.6496815286624203, 'recall': 0.7564894932014833, 'f1': 0.6990291262135921, 'number': 809} {'precision': 0.25274725274725274, 'recall': 0.19327731092436976, 'f1': 0.21904761904761905, 'number': 119} {'precision': 0.706081081081081, 'recall': 0.7849765258215963, 'f1': 0.7434415295686971, 'number': 1065} 0.6635 0.7381 0.6988 0.7929
0.5417 22.0 220 0.6774 {'precision': 0.6699346405228758, 'recall': 0.7601977750309024, 'f1': 0.7122177185871453, 'number': 809} {'precision': 0.211864406779661, 'recall': 0.21008403361344538, 'f1': 0.2109704641350211, 'number': 119} {'precision': 0.6981907894736842, 'recall': 0.7971830985915493, 'f1': 0.7444103463393249, 'number': 1065} 0.6612 0.7471 0.7015 0.7959
0.481 23.0 230 0.6671 {'precision': 0.6748400852878464, 'recall': 0.7824474660074165, 'f1': 0.7246708643388666, 'number': 809} {'precision': 0.2540983606557377, 'recall': 0.2605042016806723, 'f1': 0.2572614107883818, 'number': 119} {'precision': 0.718013468013468, 'recall': 0.8009389671361502, 'f1': 0.7572126054150022, 'number': 1065} 0.6748 0.7612 0.7154 0.8022
0.4419 24.0 240 0.6534 {'precision': 0.6799140708915145, 'recall': 0.7824474660074165, 'f1': 0.7275862068965516, 'number': 809} {'precision': 0.2818181818181818, 'recall': 0.2605042016806723, 'f1': 0.27074235807860264, 'number': 119} {'precision': 0.7332185886402753, 'recall': 0.8, 'f1': 0.7651549169286035, 'number': 1065} 0.6882 0.7607 0.7226 0.8054
0.406 25.0 250 0.6649 {'precision': 0.6862955032119914, 'recall': 0.792336217552534, 'f1': 0.7355134825014343, 'number': 809} {'precision': 0.2782608695652174, 'recall': 0.2689075630252101, 'f1': 0.2735042735042735, 'number': 119} {'precision': 0.731418918918919, 'recall': 0.8131455399061033, 'f1': 0.7701200533570476, 'number': 1065} 0.6892 0.7722 0.7283 0.8077

Framework versions

  • Transformers 4.41.2
  • Pytorch 2.3.1+cu121
  • Datasets 2.19.2
  • Tokenizers 0.19.1
Downloads last month
18
Safetensors
Model size
113M params
Tensor type
F32
·
Unable to determine this model’s pipeline type. Check the docs .

Finetuned from