layoutlm-funsd / README.md
richardsdexa's picture
End of training
d710f0c verified
metadata
license: mit
base_model: microsoft/layoutlm-base-uncased
tags:
  - generated_from_trainer
datasets:
  - funsd
model-index:
  - name: layoutlm-funsd
    results: []

layoutlm-funsd

This model is a fine-tuned version of microsoft/layoutlm-base-uncased on the funsd dataset. It achieves the following results on the evaluation set:

  • Loss: 1.1050
  • Answer: {'precision': 0.37133808392715756, 'recall': 0.5797280593325093, 'f1': 0.45270270270270274, 'number': 809}
  • Header: {'precision': 0.32926829268292684, 'recall': 0.226890756302521, 'f1': 0.26865671641791045, 'number': 119}
  • Question: {'precision': 0.49682539682539684, 'recall': 0.5877934272300469, 'f1': 0.538494623655914, 'number': 1065}
  • Overall Precision: 0.4307
  • Overall Recall: 0.5630
  • Overall F1: 0.4880
  • Overall Accuracy: 0.6093

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 3e-05
  • train_batch_size: 16
  • eval_batch_size: 8
  • seed: 42
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: linear
  • num_epochs: 15
  • mixed_precision_training: Native AMP

Training results

Training Loss Epoch Step Validation Loss Answer Header Question Overall Precision Overall Recall Overall F1 Overall Accuracy
1.8038 1.0 10 1.5073 {'precision': 0.06441476826394343, 'recall': 0.10135970333745364, 'f1': 0.07877041306436118, 'number': 809} {'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119} {'precision': 0.24326241134751772, 'recall': 0.3220657276995305, 'f1': 0.2771717171717171, 'number': 1065} 0.1584 0.2132 0.1818 0.3843
1.4521 2.0 20 1.3396 {'precision': 0.20421753607103219, 'recall': 0.45488257107540175, 'f1': 0.28188433550363845, 'number': 809} {'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119} {'precision': 0.2649350649350649, 'recall': 0.38309859154929576, 'f1': 0.31324376199616116, 'number': 1065} 0.2321 0.3894 0.2909 0.4184
1.278 3.0 30 1.2050 {'precision': 0.2645794966236955, 'recall': 0.5327564894932015, 'f1': 0.3535684987694832, 'number': 809} {'precision': 0.12903225806451613, 'recall': 0.06722689075630252, 'f1': 0.08839779005524862, 'number': 119} {'precision': 0.34989503149055284, 'recall': 0.4694835680751174, 'f1': 0.400962309542903, 'number': 1065} 0.3010 0.4711 0.3673 0.4760
1.1503 4.0 40 1.1044 {'precision': 0.28089080459770116, 'recall': 0.48331273176761436, 'f1': 0.3552930486142663, 'number': 809} {'precision': 0.2391304347826087, 'recall': 0.18487394957983194, 'f1': 0.2085308056872038, 'number': 119} {'precision': 0.4, 'recall': 0.5295774647887324, 'f1': 0.45575757575757575, 'number': 1065} 0.3376 0.4902 0.3998 0.5630
1.07 5.0 50 1.1546 {'precision': 0.30014025245441794, 'recall': 0.5290482076637825, 'f1': 0.38299776286353465, 'number': 809} {'precision': 0.3188405797101449, 'recall': 0.18487394957983194, 'f1': 0.23404255319148937, 'number': 119} {'precision': 0.4058373870743572, 'recall': 0.5483568075117371, 'f1': 0.4664536741214057, 'number': 1065} 0.3524 0.5188 0.4197 0.5383
0.9914 6.0 60 1.0507 {'precision': 0.3119065010956903, 'recall': 0.5278121137206427, 'f1': 0.3921028466483012, 'number': 809} {'precision': 0.2345679012345679, 'recall': 0.15966386554621848, 'f1': 0.18999999999999997, 'number': 119} {'precision': 0.4122938530734633, 'recall': 0.5164319248826291, 'f1': 0.45852438516048355, 'number': 1065} 0.3578 0.4997 0.4170 0.6002
0.9373 7.0 70 1.0652 {'precision': 0.3710691823899371, 'recall': 0.43757725587144625, 'f1': 0.4015882019285309, 'number': 809} {'precision': 0.25510204081632654, 'recall': 0.21008403361344538, 'f1': 0.23041474654377883, 'number': 119} {'precision': 0.4739583333333333, 'recall': 0.5981220657276995, 'f1': 0.5288501452885015, 'number': 1065} 0.4240 0.5098 0.4630 0.6006
0.8833 8.0 80 1.0389 {'precision': 0.3351605324980423, 'recall': 0.5290482076637825, 'f1': 0.4103547459252157, 'number': 809} {'precision': 0.375, 'recall': 0.20168067226890757, 'f1': 0.2622950819672132, 'number': 119} {'precision': 0.44528301886792454, 'recall': 0.5539906103286385, 'f1': 0.49372384937238495, 'number': 1065} 0.3908 0.5228 0.4473 0.6143
0.8029 9.0 90 1.0520 {'precision': 0.3685612788632327, 'recall': 0.5129789864029666, 'f1': 0.4289405684754522, 'number': 809} {'precision': 0.28695652173913044, 'recall': 0.2773109243697479, 'f1': 0.2820512820512821, 'number': 119} {'precision': 0.4902874902874903, 'recall': 0.5924882629107981, 'f1': 0.5365646258503401, 'number': 1065} 0.4268 0.5414 0.4773 0.6023
0.7658 10.0 100 1.0764 {'precision': 0.3386511965192168, 'recall': 0.5772558714462299, 'f1': 0.42687385740402195, 'number': 809} {'precision': 0.3709677419354839, 'recall': 0.19327731092436976, 'f1': 0.2541436464088398, 'number': 119} {'precision': 0.4847986852917009, 'recall': 0.5539906103286385, 'f1': 0.5170902716914987, 'number': 1065} 0.4063 0.5419 0.4644 0.6066
0.7112 11.0 110 1.0675 {'precision': 0.3728963684676705, 'recall': 0.5203955500618047, 'f1': 0.43446852425180593, 'number': 809} {'precision': 0.3333333333333333, 'recall': 0.21008403361344538, 'f1': 0.2577319587628866, 'number': 119} {'precision': 0.4918032786885246, 'recall': 0.5915492957746479, 'f1': 0.5370843989769821, 'number': 1065} 0.4330 0.5399 0.4806 0.6124
0.6875 12.0 120 1.1100 {'precision': 0.37746256895193064, 'recall': 0.5920889987639061, 'f1': 0.46102021174205965, 'number': 809} {'precision': 0.33783783783783783, 'recall': 0.21008403361344538, 'f1': 0.25906735751295334, 'number': 119} {'precision': 0.514554794520548, 'recall': 0.564319248826291, 'f1': 0.5382892969099866, 'number': 1065} 0.4401 0.5544 0.4907 0.6102
0.6571 13.0 130 1.0804 {'precision': 0.36231884057971014, 'recall': 0.5253399258343634, 'f1': 0.4288597376387487, 'number': 809} {'precision': 0.313953488372093, 'recall': 0.226890756302521, 'f1': 0.2634146341463415, 'number': 119} {'precision': 0.46940244780417567, 'recall': 0.612206572769953, 'f1': 0.5313773431132844, 'number': 1065} 0.4169 0.5539 0.4758 0.6141
0.6564 14.0 140 1.0934 {'precision': 0.37943262411347517, 'recall': 0.5290482076637825, 'f1': 0.44192049561177077, 'number': 809} {'precision': 0.37662337662337664, 'recall': 0.24369747899159663, 'f1': 0.29591836734693877, 'number': 119} {'precision': 0.49803613511390415, 'recall': 0.5953051643192488, 'f1': 0.542343883661249, 'number': 1065} 0.4403 0.5474 0.4880 0.6215
0.6558 15.0 150 1.1050 {'precision': 0.37133808392715756, 'recall': 0.5797280593325093, 'f1': 0.45270270270270274, 'number': 809} {'precision': 0.32926829268292684, 'recall': 0.226890756302521, 'f1': 0.26865671641791045, 'number': 119} {'precision': 0.49682539682539684, 'recall': 0.5877934272300469, 'f1': 0.538494623655914, 'number': 1065} 0.4307 0.5630 0.4880 0.6093

Framework versions

  • Transformers 4.36.2
  • Pytorch 2.1.2+cu118
  • Datasets 2.18.0
  • Tokenizers 0.15.0