layoutlm-funsd

This model is a fine-tuned version of microsoft/layoutlm-base-uncased on the funsd dataset. It achieves the following results on the evaluation set:

• Loss: 0.7043
• Answer: {'precision': 0.7119021134593994, 'recall': 0.7911001236093943, 'f1': 0.7494145199063232, 'number': 809}
• Header: {'precision': 0.37209302325581395, 'recall': 0.40336134453781514, 'f1': 0.3870967741935484, 'number': 119}
• Question: {'precision': 0.7965796579657966, 'recall': 0.8309859154929577, 'f1': 0.8134191176470588, 'number': 1065}
• Overall Precision: 0.7354
• Overall Recall: 0.7893
• Overall F1: 0.7614
• Overall Accuracy: 0.8036

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 3e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 10
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Answer	Header	Question	Overall Precision	Overall Recall	Overall F1	Overall Accuracy
1.5459	1.0	38	1.0013	{'precision': 0.4444444444444444, 'recall': 0.5735475896168108, 'f1': 0.500809498111171, 'number': 809}	{'precision': 0.05263157894736842, 'recall': 0.008403361344537815, 'f1': 0.014492753623188406, 'number': 119}	{'precision': 0.5643024162120032, 'recall': 0.67981220657277, 'f1': 0.616695059625213, 'number': 1065}	0.5068	0.5966	0.5481	0.6648
0.8508	2.0	76	0.7119	{'precision': 0.6081081081081081, 'recall': 0.7787391841779975, 'f1': 0.6829268292682927, 'number': 809}	{'precision': 0.16455696202531644, 'recall': 0.1092436974789916, 'f1': 0.1313131313131313, 'number': 119}	{'precision': 0.6774703557312253, 'recall': 0.8046948356807512, 'f1': 0.7356223175965665, 'number': 1065}	0.6303	0.7526	0.6860	0.7691
0.6131	3.0	114	0.6432	{'precision': 0.6631689401888772, 'recall': 0.7812113720642769, 'f1': 0.717366628830874, 'number': 809}	{'precision': 0.248, 'recall': 0.2605042016806723, 'f1': 0.2540983606557377, 'number': 119}	{'precision': 0.7474048442906575, 'recall': 0.8112676056338028, 'f1': 0.7780279153534444, 'number': 1065}	0.6835	0.7662	0.7225	0.7837
0.4734	4.0	152	0.6196	{'precision': 0.6882845188284519, 'recall': 0.8133498145859085, 'f1': 0.7456090651558074, 'number': 809}	{'precision': 0.2569444444444444, 'recall': 0.31092436974789917, 'f1': 0.28136882129277563, 'number': 119}	{'precision': 0.763716814159292, 'recall': 0.8103286384976526, 'f1': 0.7863325740318907, 'number': 1065}	0.6987	0.7817	0.7379	0.8005
0.3721	5.0	190	0.6197	{'precision': 0.6894343649946638, 'recall': 0.7985166872682324, 'f1': 0.7399770904925544, 'number': 809}	{'precision': 0.31007751937984496, 'recall': 0.33613445378151263, 'f1': 0.3225806451612903, 'number': 119}	{'precision': 0.7683566433566433, 'recall': 0.8253521126760563, 'f1': 0.7958352195563604, 'number': 1065}	0.7081	0.7852	0.7447	0.8005
0.2989	6.0	228	0.6483	{'precision': 0.6992316136114161, 'recall': 0.7873918417799752, 'f1': 0.7406976744186047, 'number': 809}	{'precision': 0.35766423357664234, 'recall': 0.4117647058823529, 'f1': 0.3828125, 'number': 119}	{'precision': 0.7854578096947935, 'recall': 0.8215962441314554, 'f1': 0.8031206975676914, 'number': 1065}	0.7220	0.7832	0.7514	0.7987
0.2437	7.0	266	0.6707	{'precision': 0.7067415730337079, 'recall': 0.7775030902348579, 'f1': 0.7404355503237198, 'number': 809}	{'precision': 0.34057971014492755, 'recall': 0.3949579831932773, 'f1': 0.36575875486381326, 'number': 119}	{'precision': 0.7804232804232805, 'recall': 0.8309859154929577, 'f1': 0.8049113233287858, 'number': 1065}	0.7220	0.7832	0.7514	0.7993
0.2008	8.0	304	0.6904	{'precision': 0.7038251366120218, 'recall': 0.796044499381953, 'f1': 0.7470997679814385, 'number': 809}	{'precision': 0.3356643356643357, 'recall': 0.40336134453781514, 'f1': 0.366412213740458, 'number': 119}	{'precision': 0.7885304659498208, 'recall': 0.8262910798122066, 'f1': 0.8069692801467218, 'number': 1065}	0.7231	0.7888	0.7545	0.7990
0.1802	9.0	342	0.7072	{'precision': 0.7161862527716186, 'recall': 0.7985166872682324, 'f1': 0.7551139684395091, 'number': 809}	{'precision': 0.34459459459459457, 'recall': 0.42857142857142855, 'f1': 0.38202247191011235, 'number': 119}	{'precision': 0.7896174863387978, 'recall': 0.8140845070422535, 'f1': 0.8016643550624134, 'number': 1065}	0.7281	0.7847	0.7554	0.7989
0.1681	10.0	380	0.7043	{'precision': 0.7119021134593994, 'recall': 0.7911001236093943, 'f1': 0.7494145199063232, 'number': 809}	{'precision': 0.37209302325581395, 'recall': 0.40336134453781514, 'f1': 0.3870967741935484, 'number': 119}	{'precision': 0.7965796579657966, 'recall': 0.8309859154929577, 'f1': 0.8134191176470588, 'number': 1065}	0.7354	0.7893	0.7614	0.8036

Framework versions

• Transformers 4.40.2
• Pytorch 2.3.0+cu121
• Datasets 2.19.1
• Tokenizers 0.19.1