layoutlm-funsd

This model is a fine-tuned version of microsoft/layoutlm-base-uncased on the funsd dataset. It achieves the following results on the evaluation set:

• Loss: 0.7158
• Answer: {'precision': 0.7149220489977728, 'recall': 0.7935723114956736, 'f1': 0.7521968365553603, 'number': 809}
• Header: {'precision': 0.3063063063063063, 'recall': 0.2857142857142857, 'f1': 0.2956521739130435, 'number': 119}
• Question: {'precision': 0.7892857142857143, 'recall': 0.8300469483568075, 'f1': 0.8091533180778031, 'number': 1065}
• Overall Precision: 0.7327
• Overall Recall: 0.7827
• Overall F1: 0.7569
• Overall Accuracy: 0.8108

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 3e-05
• train_batch_size: 16
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 15
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Answer	Header	Question	Overall Precision	Overall Recall	Overall F1	Overall Accuracy
1.8132	1.0	10	1.6191	{'precision': 0.015122873345935728, 'recall': 0.019777503090234856, 'f1': 0.01713979646491698, 'number': 809}	{'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119}	{'precision': 0.1685508735868448, 'recall': 0.1539906103286385, 'f1': 0.16094210009813542, 'number': 1065}	0.0886	0.0903	0.0895	0.3534
1.4783	2.0	20	1.2483	{'precision': 0.12857142857142856, 'recall': 0.12237330037082818, 'f1': 0.1253958201393287, 'number': 809}	{'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119}	{'precision': 0.4541955350269438, 'recall': 0.5539906103286385, 'f1': 0.4991539763113368, 'number': 1065}	0.3330	0.3457	0.3392	0.5682
1.1072	3.0	30	0.9718	{'precision': 0.42777155655095184, 'recall': 0.4721878862793572, 'f1': 0.4488836662749706, 'number': 809}	{'precision': 0.04, 'recall': 0.008403361344537815, 'f1': 0.01388888888888889, 'number': 119}	{'precision': 0.6266205704407951, 'recall': 0.6807511737089202, 'f1': 0.6525652565256526, 'number': 1065}	0.5340	0.5559	0.5447	0.7070
0.8444	4.0	40	0.7957	{'precision': 0.6296296296296297, 'recall': 0.7354758961681088, 'f1': 0.6784492588369442, 'number': 809}	{'precision': 0.19230769230769232, 'recall': 0.08403361344537816, 'f1': 0.11695906432748539, 'number': 119}	{'precision': 0.6831168831168831, 'recall': 0.7408450704225352, 'f1': 0.7108108108108109, 'number': 1065}	0.6478	0.6994	0.6726	0.7651
0.6845	5.0	50	0.7443	{'precision': 0.6530612244897959, 'recall': 0.7515451174289246, 'f1': 0.6988505747126437, 'number': 809}	{'precision': 0.23684210526315788, 'recall': 0.15126050420168066, 'f1': 0.1846153846153846, 'number': 119}	{'precision': 0.7318181818181818, 'recall': 0.755868544600939, 'f1': 0.74364896073903, 'number': 1065}	0.6792	0.7180	0.6980	0.7736
0.5597	6.0	60	0.6918	{'precision': 0.6673706441393875, 'recall': 0.7812113720642769, 'f1': 0.7198177676537586, 'number': 809}	{'precision': 0.2857142857142857, 'recall': 0.15126050420168066, 'f1': 0.1978021978021978, 'number': 119}	{'precision': 0.7344150298889838, 'recall': 0.8075117370892019, 'f1': 0.7692307692307693, 'number': 1065}	0.6923	0.7577	0.7235	0.7933
0.4929	7.0	70	0.6803	{'precision': 0.6694825765575502, 'recall': 0.7836835599505563, 'f1': 0.7220956719817767, 'number': 809}	{'precision': 0.21818181818181817, 'recall': 0.20168067226890757, 'f1': 0.2096069868995633, 'number': 119}	{'precision': 0.7467134092900964, 'recall': 0.8, 'f1': 0.772438803263826, 'number': 1065}	0.6870	0.7577	0.7206	0.7945
0.4447	8.0	80	0.6814	{'precision': 0.6866158868335147, 'recall': 0.7799752781211372, 'f1': 0.7303240740740741, 'number': 809}	{'precision': 0.26506024096385544, 'recall': 0.18487394957983194, 'f1': 0.21782178217821785, 'number': 119}	{'precision': 0.7810283687943262, 'recall': 0.8272300469483568, 'f1': 0.8034655722754217, 'number': 1065}	0.7202	0.7697	0.7441	0.8024
0.3953	9.0	90	0.6739	{'precision': 0.7015765765765766, 'recall': 0.7700865265760197, 'f1': 0.7342368886269888, 'number': 809}	{'precision': 0.2920353982300885, 'recall': 0.2773109243697479, 'f1': 0.28448275862068967, 'number': 119}	{'precision': 0.7753496503496503, 'recall': 0.8328638497652582, 'f1': 0.8030783159800814, 'number': 1065}	0.7193	0.7742	0.7458	0.8115
0.3538	10.0	100	0.6853	{'precision': 0.7081497797356828, 'recall': 0.7948084054388134, 'f1': 0.7489807804309844, 'number': 809}	{'precision': 0.32673267326732675, 'recall': 0.2773109243697479, 'f1': 0.30000000000000004, 'number': 119}	{'precision': 0.7804878048780488, 'recall': 0.8413145539906103, 'f1': 0.8097605061003164, 'number': 1065}	0.7288	0.7888	0.7576	0.8152
0.3262	11.0	110	0.6948	{'precision': 0.7058177826564215, 'recall': 0.7948084054388134, 'f1': 0.7476744186046511, 'number': 809}	{'precision': 0.35051546391752575, 'recall': 0.2857142857142857, 'f1': 0.3148148148148148, 'number': 119}	{'precision': 0.8032638259292838, 'recall': 0.831924882629108, 'f1': 0.8173431734317343, 'number': 1065}	0.7404	0.7842	0.7617	0.8129
0.3094	12.0	120	0.6989	{'precision': 0.7128603104212861, 'recall': 0.7948084054388134, 'f1': 0.7516072472238456, 'number': 809}	{'precision': 0.3333333333333333, 'recall': 0.29411764705882354, 'f1': 0.3125, 'number': 119}	{'precision': 0.7987364620938628, 'recall': 0.8309859154929577, 'f1': 0.8145421076852278, 'number': 1065}	0.7390	0.7842	0.7610	0.8138
0.2941	13.0	130	0.7134	{'precision': 0.7239819004524887, 'recall': 0.7911001236093943, 'f1': 0.7560543414057885, 'number': 809}	{'precision': 0.32710280373831774, 'recall': 0.29411764705882354, 'f1': 0.3097345132743363, 'number': 119}	{'precision': 0.7998204667863554, 'recall': 0.8366197183098592, 'f1': 0.8178063331803579, 'number': 1065}	0.7439	0.7858	0.7643	0.8115
0.2813	14.0	140	0.7138	{'precision': 0.7106710671067107, 'recall': 0.7985166872682324, 'f1': 0.7520372526193247, 'number': 809}	{'precision': 0.3119266055045872, 'recall': 0.2857142857142857, 'f1': 0.2982456140350877, 'number': 119}	{'precision': 0.7935656836461126, 'recall': 0.8338028169014085, 'f1': 0.8131868131868133, 'number': 1065}	0.7337	0.7868	0.7593	0.8109
0.2812	15.0	150	0.7158	{'precision': 0.7149220489977728, 'recall': 0.7935723114956736, 'f1': 0.7521968365553603, 'number': 809}	{'precision': 0.3063063063063063, 'recall': 0.2857142857142857, 'f1': 0.2956521739130435, 'number': 119}	{'precision': 0.7892857142857143, 'recall': 0.8300469483568075, 'f1': 0.8091533180778031, 'number': 1065}	0.7327	0.7827	0.7569	0.8108

Framework versions

• Transformers 4.24.0
• Pytorch 1.12.1+cu113
• Datasets 2.6.1
• Tokenizers 0.13.1