layoutlm-funsd

This model is a fine-tuned version of microsoft/layoutlm-base-uncased on the funsd dataset. It achieves the following results on the evaluation set:

• Loss: 0.7189
• Answer: {'precision': 0.6983783783783784, 'recall': 0.7985166872682324, 'f1': 0.7450980392156863, 'number': 809}
• Header: {'precision': 0.28368794326241137, 'recall': 0.33613445378151263, 'f1': 0.3076923076923077, 'number': 119}
• Question: {'precision': 0.7754199823165341, 'recall': 0.8234741784037559, 'f1': 0.7987249544626595, 'number': 1065}
• Overall Precision: 0.7114
• Overall Recall: 0.7842
• Overall F1: 0.7461
• Overall Accuracy: 0.8074

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 3e-05
• train_batch_size: 16
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 15
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Answer	Header	Question	Overall Precision	Overall Recall	Overall F1	Overall Accuracy
1.561	1.0	10	1.3641	{'precision': 0.05998125585754452, 'recall': 0.07911001236093942, 'f1': 0.06823027718550106, 'number': 809}	{'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119}	{'precision': 0.39123867069486407, 'recall': 0.4863849765258216, 'f1': 0.43365424863959817, 'number': 1065}	0.2434	0.2920	0.2655	0.4879
1.1891	2.0	20	0.9802	{'precision': 0.43872778297474274, 'recall': 0.5797280593325093, 'f1': 0.4994675186368477, 'number': 809}	{'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119}	{'precision': 0.5793269230769231, 'recall': 0.6788732394366197, 'f1': 0.6251621271076524, 'number': 1065}	0.5062	0.5981	0.5483	0.6922
0.8513	3.0	30	0.8015	{'precision': 0.5782520325203252, 'recall': 0.7033374536464772, 'f1': 0.6346904629113218, 'number': 809}	{'precision': 0.10204081632653061, 'recall': 0.04201680672268908, 'f1': 0.05952380952380952, 'number': 119}	{'precision': 0.6831421006178288, 'recall': 0.7267605633802817, 'f1': 0.7042766151046406, 'number': 1065}	0.6223	0.6764	0.6482	0.7473
0.702	4.0	40	0.7279	{'precision': 0.6275303643724697, 'recall': 0.7663782447466008, 'f1': 0.6900389538119087, 'number': 809}	{'precision': 0.1411764705882353, 'recall': 0.10084033613445378, 'f1': 0.11764705882352941, 'number': 119}	{'precision': 0.6978354978354978, 'recall': 0.7568075117370892, 'f1': 0.726126126126126, 'number': 1065}	0.6454	0.7215	0.6814	0.7727
0.6059	5.0	50	0.7065	{'precision': 0.6370530877573131, 'recall': 0.7268232385661311, 'f1': 0.6789838337182448, 'number': 809}	{'precision': 0.19607843137254902, 'recall': 0.16806722689075632, 'f1': 0.18099547511312217, 'number': 119}	{'precision': 0.7024106400665004, 'recall': 0.7934272300469484, 'f1': 0.7451499118165786, 'number': 1065}	0.6522	0.7291	0.6885	0.7809
0.5133	6.0	60	0.6761	{'precision': 0.6592592592592592, 'recall': 0.7700865265760197, 'f1': 0.7103762827822121, 'number': 809}	{'precision': 0.19791666666666666, 'recall': 0.15966386554621848, 'f1': 0.17674418604651165, 'number': 119}	{'precision': 0.7100638977635783, 'recall': 0.8347417840375587, 'f1': 0.7673716012084592, 'number': 1065}	0.6677	0.7682	0.7144	0.7927
0.4539	7.0	70	0.6811	{'precision': 0.6793893129770993, 'recall': 0.7700865265760197, 'f1': 0.7219003476245655, 'number': 809}	{'precision': 0.23387096774193547, 'recall': 0.24369747899159663, 'f1': 0.23868312757201646, 'number': 119}	{'precision': 0.7506361323155216, 'recall': 0.8309859154929577, 'f1': 0.7887700534759358, 'number': 1065}	0.6923	0.7712	0.7296	0.7970
0.4175	8.0	80	0.6604	{'precision': 0.6727664155005382, 'recall': 0.7725587144622992, 'f1': 0.7192174913693901, 'number': 809}	{'precision': 0.26785714285714285, 'recall': 0.25210084033613445, 'f1': 0.2597402597402597, 'number': 119}	{'precision': 0.7596899224806202, 'recall': 0.828169014084507, 'f1': 0.7924528301886793, 'number': 1065}	0.6980	0.7712	0.7328	0.8022
0.3711	9.0	90	0.6827	{'precision': 0.7034559643255296, 'recall': 0.7799752781211372, 'f1': 0.7397420867526378, 'number': 809}	{'precision': 0.2482758620689655, 'recall': 0.3025210084033613, 'f1': 0.2727272727272727, 'number': 119}	{'precision': 0.7497872340425532, 'recall': 0.8272300469483568, 'f1': 0.7866071428571428, 'number': 1065}	0.6982	0.7767	0.7354	0.8049
0.3346	10.0	100	0.6881	{'precision': 0.688367129135539, 'recall': 0.7972805933250927, 'f1': 0.738831615120275, 'number': 809}	{'precision': 0.2845528455284553, 'recall': 0.29411764705882354, 'f1': 0.2892561983471075, 'number': 119}	{'precision': 0.7693661971830986, 'recall': 0.8206572769953052, 'f1': 0.79418446160836, 'number': 1065}	0.7077	0.7797	0.7419	0.8076
0.3003	11.0	110	0.7039	{'precision': 0.6928104575163399, 'recall': 0.7861557478368356, 'f1': 0.7365373480023161, 'number': 809}	{'precision': 0.3008130081300813, 'recall': 0.31092436974789917, 'f1': 0.3057851239669422, 'number': 119}	{'precision': 0.7776801405975395, 'recall': 0.8309859154929577, 'f1': 0.8034498411257376, 'number': 1065}	0.7150	0.7817	0.7469	0.8095
0.2878	12.0	120	0.7100	{'precision': 0.6923913043478261, 'recall': 0.7873918417799752, 'f1': 0.736842105263158, 'number': 809}	{'precision': 0.2826086956521739, 'recall': 0.3277310924369748, 'f1': 0.3035019455252918, 'number': 119}	{'precision': 0.780053428317008, 'recall': 0.8225352112676056, 'f1': 0.8007312614259597, 'number': 1065}	0.7116	0.7787	0.7437	0.8066
0.2724	13.0	130	0.7137	{'precision': 0.6792249730893434, 'recall': 0.7799752781211372, 'f1': 0.7261219792865363, 'number': 809}	{'precision': 0.2846715328467153, 'recall': 0.3277310924369748, 'f1': 0.3046875, 'number': 119}	{'precision': 0.7838565022421524, 'recall': 0.8206572769953052, 'f1': 0.8018348623853212, 'number': 1065}	0.7079	0.7747	0.7398	0.8028
0.2582	14.0	140	0.7174	{'precision': 0.6964477933261571, 'recall': 0.799752781211372, 'f1': 0.7445339470655927, 'number': 809}	{'precision': 0.28368794326241137, 'recall': 0.33613445378151263, 'f1': 0.3076923076923077, 'number': 119}	{'precision': 0.7772848269742679, 'recall': 0.8225352112676056, 'f1': 0.7992700729927008, 'number': 1065}	0.7114	0.7842	0.7461	0.8073
0.2569	15.0	150	0.7189	{'precision': 0.6983783783783784, 'recall': 0.7985166872682324, 'f1': 0.7450980392156863, 'number': 809}	{'precision': 0.28368794326241137, 'recall': 0.33613445378151263, 'f1': 0.3076923076923077, 'number': 119}	{'precision': 0.7754199823165341, 'recall': 0.8234741784037559, 'f1': 0.7987249544626595, 'number': 1065}	0.7114	0.7842	0.7461	0.8074

Framework versions

• Transformers 4.29.0
• Pytorch 2.0.1+cu118
• Datasets 2.12.0
• Tokenizers 0.13.3