layoutlm-funsd

This model is a fine-tuned version of microsoft/layoutlm-base-uncased on the funsd dataset. It achieves the following results on the evaluation set:

• Loss: 0.7037
• Answer: {'precision': 0.7206703910614525, 'recall': 0.7972805933250927, 'f1': 0.7570422535211268, 'number': 809}
• Header: {'precision': 0.3006993006993007, 'recall': 0.36134453781512604, 'f1': 0.3282442748091603, 'number': 119}
• Question: {'precision': 0.7585004359197908, 'recall': 0.8169014084507042, 'f1': 0.7866184448462928, 'number': 1065}
• Overall Precision: 0.7130
• Overall Recall: 0.7817
• Overall F1: 0.7458
• Overall Accuracy: 0.7989

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 3e-05
• train_batch_size: 16
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 15

Training results

Training Loss	Epoch	Step	Validation Loss	Answer	Header	Question	Overall Precision	Overall Recall	Overall F1	Overall Accuracy
1.7815	1.0	10	1.5703	{'precision': 0.022222222222222223, 'recall': 0.022249690976514216, 'f1': 0.022235948116121063, 'number': 809}	{'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119}	{'precision': 0.21046643913538113, 'recall': 0.17370892018779344, 'f1': 0.19032921810699588, 'number': 1065}	0.1202	0.1019	0.1103	0.3789
1.4352	2.0	20	1.2331	{'precision': 0.12166172106824925, 'recall': 0.10135970333745364, 'f1': 0.11058664868509778, 'number': 809}	{'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119}	{'precision': 0.4863247863247863, 'recall': 0.5342723004694836, 'f1': 0.50917225950783, 'number': 1065}	0.3530	0.3266	0.3393	0.5662
1.0804	3.0	30	0.9725	{'precision': 0.4528985507246377, 'recall': 0.4635352286773795, 'f1': 0.4581551618814906, 'number': 809}	{'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119}	{'precision': 0.6272806255430061, 'recall': 0.6779342723004694, 'f1': 0.6516245487364621, 'number': 1065}	0.5447	0.5504	0.5475	0.6845
0.8495	4.0	40	0.7990	{'precision': 0.5910973084886129, 'recall': 0.7058096415327565, 'f1': 0.6433802816901407, 'number': 809}	{'precision': 0.05970149253731343, 'recall': 0.03361344537815126, 'f1': 0.04301075268817204, 'number': 119}	{'precision': 0.6702222222222223, 'recall': 0.707981220657277, 'f1': 0.6885844748858447, 'number': 1065}	0.6158	0.6668	0.6403	0.7510
0.6866	5.0	50	0.7357	{'precision': 0.6541436464088398, 'recall': 0.7317676143386898, 'f1': 0.6907817969661612, 'number': 809}	{'precision': 0.2235294117647059, 'recall': 0.15966386554621848, 'f1': 0.18627450980392157, 'number': 119}	{'precision': 0.7028619528619529, 'recall': 0.784037558685446, 'f1': 0.7412339103417664, 'number': 1065}	0.6639	0.7255	0.6934	0.7698
0.5626	6.0	60	0.6982	{'precision': 0.6594871794871795, 'recall': 0.7948084054388134, 'f1': 0.7208520179372198, 'number': 809}	{'precision': 0.28378378378378377, 'recall': 0.17647058823529413, 'f1': 0.21761658031088082, 'number': 119}	{'precision': 0.6939417781274587, 'recall': 0.828169014084507, 'f1': 0.7551369863013697, 'number': 1065}	0.6664	0.7757	0.7169	0.7872
0.4875	7.0	70	0.6710	{'precision': 0.6905286343612335, 'recall': 0.7750309023485785, 'f1': 0.7303436225975539, 'number': 809}	{'precision': 0.2336448598130841, 'recall': 0.21008403361344538, 'f1': 0.22123893805309733, 'number': 119}	{'precision': 0.7287145242070117, 'recall': 0.819718309859155, 'f1': 0.7715422006186478, 'number': 1065}	0.6891	0.7652	0.7252	0.7924
0.4499	8.0	80	0.6635	{'precision': 0.6888412017167382, 'recall': 0.7935723114956736, 'f1': 0.7375071797817346, 'number': 809}	{'precision': 0.25210084033613445, 'recall': 0.25210084033613445, 'f1': 0.25210084033613445, 'number': 119}	{'precision': 0.7314814814814815, 'recall': 0.815962441314554, 'f1': 0.771415889924545, 'number': 1065}	0.6883	0.7732	0.7283	0.7977
0.3939	9.0	90	0.6686	{'precision': 0.709070796460177, 'recall': 0.792336217552534, 'f1': 0.7483946293053124, 'number': 809}	{'precision': 0.24817518248175183, 'recall': 0.2857142857142857, 'f1': 0.265625, 'number': 119}	{'precision': 0.7311557788944724, 'recall': 0.819718309859155, 'f1': 0.7729083665338645, 'number': 1065}	0.6926	0.7767	0.7323	0.7970
0.3522	10.0	100	0.6728	{'precision': 0.7094668117519043, 'recall': 0.8059332509270705, 'f1': 0.7546296296296295, 'number': 809}	{'precision': 0.3135593220338983, 'recall': 0.31092436974789917, 'f1': 0.31223628691983124, 'number': 119}	{'precision': 0.7573149741824441, 'recall': 0.8262910798122066, 'f1': 0.7903008531656939, 'number': 1065}	0.7135	0.7873	0.7486	0.8034
0.3124	11.0	110	0.6859	{'precision': 0.7041800643086816, 'recall': 0.8121137206427689, 'f1': 0.7543053960964409, 'number': 809}	{'precision': 0.3076923076923077, 'recall': 0.3025210084033613, 'f1': 0.30508474576271183, 'number': 119}	{'precision': 0.7731316725978647, 'recall': 0.815962441314554, 'f1': 0.793969849246231, 'number': 1065}	0.7185	0.7837	0.7497	0.8006
0.306	12.0	120	0.6947	{'precision': 0.720489977728285, 'recall': 0.799752781211372, 'f1': 0.7580550673696543, 'number': 809}	{'precision': 0.2773722627737226, 'recall': 0.31932773109243695, 'f1': 0.296875, 'number': 119}	{'precision': 0.7567332754126846, 'recall': 0.8178403755868544, 'f1': 0.7861010830324908, 'number': 1065}	0.7118	0.7807	0.7447	0.7987
0.283	13.0	130	0.6948	{'precision': 0.7201783723522854, 'recall': 0.7985166872682324, 'f1': 0.7573270808909731, 'number': 809}	{'precision': 0.30597014925373134, 'recall': 0.3445378151260504, 'f1': 0.3241106719367589, 'number': 119}	{'precision': 0.7585004359197908, 'recall': 0.8169014084507042, 'f1': 0.7866184448462928, 'number': 1065}	0.7149	0.7812	0.7466	0.8000
0.2726	14.0	140	0.7002	{'precision': 0.7119205298013245, 'recall': 0.7972805933250927, 'f1': 0.7521865889212828, 'number': 809}	{'precision': 0.3049645390070922, 'recall': 0.36134453781512604, 'f1': 0.3307692307692308, 'number': 119}	{'precision': 0.762532981530343, 'recall': 0.8140845070422535, 'f1': 0.787465940054496, 'number': 1065}	0.7120	0.7802	0.7446	0.8001
0.264	15.0	150	0.7037	{'precision': 0.7206703910614525, 'recall': 0.7972805933250927, 'f1': 0.7570422535211268, 'number': 809}	{'precision': 0.3006993006993007, 'recall': 0.36134453781512604, 'f1': 0.3282442748091603, 'number': 119}	{'precision': 0.7585004359197908, 'recall': 0.8169014084507042, 'f1': 0.7866184448462928, 'number': 1065}	0.7130	0.7817	0.7458	0.7989

Framework versions

• Transformers 4.26.0
• Pytorch 1.13.1+cu116
• Datasets 2.9.0
• Tokenizers 0.13.2