layoutlm-funsd

This model is a fine-tuned version of microsoft/layoutlm-base-uncased on the funsd dataset. It achieves the following results on the evaluation set:

• Loss: 0.6897
• Answer: {'precision': 0.7033707865168539, 'recall': 0.7737948084054388, 'f1': 0.736904061212478, 'number': 809}
• Header: {'precision': 0.3697478991596639, 'recall': 0.3697478991596639, 'f1': 0.3697478991596639, 'number': 119}
• Question: {'precision': 0.7738825591586328, 'recall': 0.8291079812206573, 'f1': 0.800543970988214, 'number': 1065}
• Overall Precision: 0.7223
• Overall Recall: 0.7792
• Overall F1: 0.7497
• Overall Accuracy: 0.8012

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 3e-05
• train_batch_size: 16
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 15
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Answer	Header	Question	Overall Precision	Overall Recall	Overall F1	Overall Accuracy
1.8092	1.0	10	1.6181	{'precision': 0.013850415512465374, 'recall': 0.006180469715698393, 'f1': 0.008547008547008546, 'number': 809}	{'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119}	{'precision': 0.26586102719033233, 'recall': 0.08262910798122065, 'f1': 0.12607449856733524, 'number': 1065}	0.1344	0.0467	0.0693	0.3144
1.4937	2.0	20	1.2877	{'precision': 0.1342925659472422, 'recall': 0.138442521631644, 'f1': 0.1363359707851491, 'number': 809}	{'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119}	{'precision': 0.34902488231338263, 'recall': 0.48732394366197185, 'f1': 0.4067398119122257, 'number': 1065}	0.2719	0.3166	0.2925	0.5718
1.1349	3.0	30	0.9891	{'precision': 0.45396825396825397, 'recall': 0.5302843016069221, 'f1': 0.48916761687571264, 'number': 809}	{'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119}	{'precision': 0.5338582677165354, 'recall': 0.6366197183098592, 'f1': 0.5807280513918629, 'number': 1065}	0.4998	0.5554	0.5261	0.6801
0.8493	4.0	40	0.8234	{'precision': 0.580259222333001, 'recall': 0.7194066749072929, 'f1': 0.6423841059602649, 'number': 809}	{'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119}	{'precision': 0.6639928698752228, 'recall': 0.6995305164319249, 'f1': 0.6812985825331503, 'number': 1065}	0.6189	0.6658	0.6415	0.7323
0.6989	5.0	50	0.7466	{'precision': 0.6263048016701461, 'recall': 0.7416563658838071, 'f1': 0.6791171477079797, 'number': 809}	{'precision': 0.11688311688311688, 'recall': 0.07563025210084033, 'f1': 0.09183673469387756, 'number': 119}	{'precision': 0.7050043898156277, 'recall': 0.7539906103286385, 'f1': 0.7286751361161524, 'number': 1065}	0.6495	0.7085	0.6777	0.7609
0.5795	6.0	60	0.6809	{'precision': 0.6511387163561076, 'recall': 0.7775030902348579, 'f1': 0.7087323943661973, 'number': 809}	{'precision': 0.13095238095238096, 'recall': 0.09243697478991597, 'f1': 0.10837438423645321, 'number': 119}	{'precision': 0.705229793977813, 'recall': 0.8356807511737089, 'f1': 0.7649333906317147, 'number': 1065}	0.6618	0.7677	0.7108	0.7872
0.508	7.0	70	0.6786	{'precision': 0.667375132837407, 'recall': 0.7762669962917181, 'f1': 0.7177142857142856, 'number': 809}	{'precision': 0.1875, 'recall': 0.17647058823529413, 'f1': 0.1818181818181818, 'number': 119}	{'precision': 0.7431972789115646, 'recall': 0.8206572769953052, 'f1': 0.7800089245872378, 'number': 1065}	0.6833	0.7642	0.7215	0.7871
0.4558	8.0	80	0.6540	{'precision': 0.6932314410480349, 'recall': 0.7849196538936959, 'f1': 0.736231884057971, 'number': 809}	{'precision': 0.24, 'recall': 0.20168067226890757, 'f1': 0.2191780821917808, 'number': 119}	{'precision': 0.7464195450716091, 'recall': 0.831924882629108, 'f1': 0.7868561278863234, 'number': 1065}	0.7013	0.7752	0.7364	0.7997
0.3978	9.0	90	0.6650	{'precision': 0.6969026548672567, 'recall': 0.7787391841779975, 'f1': 0.7355516637478109, 'number': 809}	{'precision': 0.2644628099173554, 'recall': 0.2689075630252101, 'f1': 0.2666666666666667, 'number': 119}	{'precision': 0.7474662162162162, 'recall': 0.8309859154929577, 'f1': 0.7870164517563362, 'number': 1065}	0.7003	0.7762	0.7363	0.7951
0.3612	10.0	100	0.6679	{'precision': 0.7168742921857305, 'recall': 0.7824474660074165, 'f1': 0.7482269503546098, 'number': 809}	{'precision': 0.30701754385964913, 'recall': 0.29411764705882354, 'f1': 0.30042918454935624, 'number': 119}	{'precision': 0.7629757785467128, 'recall': 0.828169014084507, 'f1': 0.7942368302566412, 'number': 1065}	0.7199	0.7777	0.7477	0.8026
0.3168	11.0	110	0.6831	{'precision': 0.6914778856526429, 'recall': 0.792336217552534, 'f1': 0.7384792626728109, 'number': 809}	{'precision': 0.3238095238095238, 'recall': 0.2857142857142857, 'f1': 0.30357142857142855, 'number': 119}	{'precision': 0.7770979020979021, 'recall': 0.8347417840375587, 'f1': 0.8048890900860118, 'number': 1065}	0.7188	0.7847	0.7503	0.7978
0.3054	12.0	120	0.6884	{'precision': 0.6985539488320356, 'recall': 0.7762669962917181, 'f1': 0.7353629976580796, 'number': 809}	{'precision': 0.3559322033898305, 'recall': 0.35294117647058826, 'f1': 0.35443037974683544, 'number': 119}	{'precision': 0.7755102040816326, 'recall': 0.8206572769953052, 'f1': 0.7974452554744526, 'number': 1065}	0.7201	0.7747	0.7464	0.7985
0.2913	13.0	130	0.6794	{'precision': 0.7107344632768362, 'recall': 0.7775030902348579, 'f1': 0.7426210153482882, 'number': 809}	{'precision': 0.36134453781512604, 'recall': 0.36134453781512604, 'f1': 0.36134453781512604, 'number': 119}	{'precision': 0.7737478411053541, 'recall': 0.8413145539906103, 'f1': 0.8061178587494376, 'number': 1065}	0.7253	0.7868	0.7548	0.8039
0.2775	14.0	140	0.6870	{'precision': 0.7040358744394619, 'recall': 0.7762669962917181, 'f1': 0.7383891828336272, 'number': 809}	{'precision': 0.3728813559322034, 'recall': 0.3697478991596639, 'f1': 0.37130801687763715, 'number': 119}	{'precision': 0.7757255936675461, 'recall': 0.828169014084507, 'f1': 0.8010899182561309, 'number': 1065}	0.7238	0.7797	0.7507	0.8014
0.2706	15.0	150	0.6897	{'precision': 0.7033707865168539, 'recall': 0.7737948084054388, 'f1': 0.736904061212478, 'number': 809}	{'precision': 0.3697478991596639, 'recall': 0.3697478991596639, 'f1': 0.3697478991596639, 'number': 119}	{'precision': 0.7738825591586328, 'recall': 0.8291079812206573, 'f1': 0.800543970988214, 'number': 1065}	0.7223	0.7792	0.7497	0.8012

Framework versions

• Transformers 4.29.2
• Pytorch 2.0.1+cu118
• Datasets 2.12.0
• Tokenizers 0.13.3