layoutlm-funsd

This model is a fine-tuned version of microsoft/layoutlm-base-uncased on the funsd dataset. It achieves the following results on the evaluation set:

• Loss: 0.7053
• Answer: {'precision': 0.7111597374179431, 'recall': 0.8034610630407911, 'f1': 0.7544979686593152, 'number': 809}
• Header: {'precision': 0.3697478991596639, 'recall': 0.3697478991596639, 'f1': 0.3697478991596639, 'number': 119}
• Question: {'precision': 0.7862254025044723, 'recall': 0.8253521126760563, 'f1': 0.8053137883646359, 'number': 1065}
• Overall Precision: 0.7313
• Overall Recall: 0.7893
• Overall F1: 0.7592
• Overall Accuracy: 0.8115

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 3e-05
• train_batch_size: 16
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 15
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Answer	Header	Question	Overall Precision	Overall Recall	Overall F1	Overall Accuracy
1.8129	1.0	10	1.6175	{'precision': 0.024783147459727387, 'recall': 0.024721878862793572, 'f1': 0.02475247524752475, 'number': 809}	{'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119}	{'precision': 0.24563318777292575, 'recall': 0.2112676056338028, 'f1': 0.2271580010095911, 'number': 1065}	0.1422	0.1229	0.1319	0.3619
1.4587	2.0	20	1.2242	{'precision': 0.14423076923076922, 'recall': 0.12978986402966625, 'f1': 0.13662979830839295, 'number': 809}	{'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119}	{'precision': 0.45927075252133437, 'recall': 0.5558685446009389, 'f1': 0.5029736618521666, 'number': 1065}	0.3456	0.3497	0.3476	0.5892
1.0781	3.0	30	0.9399	{'precision': 0.4616252821670429, 'recall': 0.5055624227441285, 'f1': 0.4825958702064897, 'number': 809}	{'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119}	{'precision': 0.5979381443298969, 'recall': 0.6535211267605634, 'f1': 0.6244952893674293, 'number': 1065}	0.5300	0.5544	0.5419	0.6934
0.8159	4.0	40	0.7947	{'precision': 0.5964912280701754, 'recall': 0.7144622991347342, 'f1': 0.6501687289088864, 'number': 809}	{'precision': 0.125, 'recall': 0.07563025210084033, 'f1': 0.09424083769633507, 'number': 119}	{'precision': 0.6864175022789426, 'recall': 0.7070422535211267, 'f1': 0.696577243293247, 'number': 1065}	0.6268	0.6724	0.6488	0.7517
0.6615	5.0	50	0.7330	{'precision': 0.6485042735042735, 'recall': 0.7503090234857849, 'f1': 0.695702005730659, 'number': 809}	{'precision': 0.28205128205128205, 'recall': 0.18487394957983194, 'f1': 0.2233502538071066, 'number': 119}	{'precision': 0.7291857273559011, 'recall': 0.748356807511737, 'f1': 0.7386468952734011, 'number': 1065}	0.6768	0.7155	0.6956	0.7756
0.5414	6.0	60	0.6814	{'precision': 0.6427850655903128, 'recall': 0.7873918417799752, 'f1': 0.7077777777777777, 'number': 809}	{'precision': 0.2857142857142857, 'recall': 0.16806722689075632, 'f1': 0.21164021164021166, 'number': 119}	{'precision': 0.726039016115352, 'recall': 0.8037558685446009, 'f1': 0.7629233511586453, 'number': 1065}	0.6754	0.7592	0.7149	0.7878
0.4787	7.0	70	0.6756	{'precision': 0.6776947705442903, 'recall': 0.7849196538936959, 'f1': 0.7273768613974799, 'number': 809}	{'precision': 0.3402061855670103, 'recall': 0.2773109243697479, 'f1': 0.3055555555555556, 'number': 119}	{'precision': 0.7390557939914163, 'recall': 0.8084507042253521, 'f1': 0.7721973094170403, 'number': 1065}	0.6953	0.7672	0.7295	0.8014
0.4379	8.0	80	0.6724	{'precision': 0.6952695269526953, 'recall': 0.7812113720642769, 'f1': 0.7357392316647265, 'number': 809}	{'precision': 0.3448275862068966, 'recall': 0.25210084033613445, 'f1': 0.2912621359223301, 'number': 119}	{'precision': 0.7552264808362369, 'recall': 0.8140845070422535, 'f1': 0.7835517397198374, 'number': 1065}	0.7132	0.7672	0.7392	0.8063
0.3864	9.0	90	0.6771	{'precision': 0.6915584415584416, 'recall': 0.7898640296662547, 'f1': 0.7374495095210617, 'number': 809}	{'precision': 0.32142857142857145, 'recall': 0.3025210084033613, 'f1': 0.3116883116883117, 'number': 119}	{'precision': 0.7570573139435415, 'recall': 0.8309859154929577, 'f1': 0.792300805729633, 'number': 1065}	0.7075	0.7827	0.7432	0.7962
0.3486	10.0	100	0.6774	{'precision': 0.6862955032119914, 'recall': 0.792336217552534, 'f1': 0.7355134825014343, 'number': 809}	{'precision': 0.3557692307692308, 'recall': 0.31092436974789917, 'f1': 0.33183856502242154, 'number': 119}	{'precision': 0.7725284339457568, 'recall': 0.8291079812206573, 'f1': 0.7998188405797102, 'number': 1065}	0.7157	0.7832	0.7480	0.8027
0.3138	11.0	110	0.6960	{'precision': 0.6893203883495146, 'recall': 0.7898640296662547, 'f1': 0.7361751152073734, 'number': 809}	{'precision': 0.3619047619047619, 'recall': 0.31932773109243695, 'f1': 0.33928571428571425, 'number': 119}	{'precision': 0.7870619946091644, 'recall': 0.8225352112676056, 'f1': 0.8044077134986226, 'number': 1065}	0.7240	0.7792	0.7506	0.8066
0.303	12.0	120	0.6989	{'precision': 0.6927194860813705, 'recall': 0.799752781211372, 'f1': 0.7423981640849111, 'number': 809}	{'precision': 0.3783783783783784, 'recall': 0.35294117647058826, 'f1': 0.3652173913043478, 'number': 119}	{'precision': 0.7902350813743219, 'recall': 0.8206572769953052, 'f1': 0.8051589129433441, 'number': 1065}	0.7266	0.7842	0.7543	0.8054
0.2823	13.0	130	0.7023	{'precision': 0.7027322404371584, 'recall': 0.7948084054388134, 'f1': 0.7459396751740139, 'number': 809}	{'precision': 0.36134453781512604, 'recall': 0.36134453781512604, 'f1': 0.36134453781512604, 'number': 119}	{'precision': 0.7818343722172751, 'recall': 0.8244131455399061, 'f1': 0.8025594149908593, 'number': 1065}	0.7251	0.7847	0.7537	0.8080
0.2707	14.0	140	0.7048	{'precision': 0.7040261153427638, 'recall': 0.799752781211372, 'f1': 0.7488425925925926, 'number': 809}	{'precision': 0.35833333333333334, 'recall': 0.36134453781512604, 'f1': 0.35983263598326365, 'number': 119}	{'precision': 0.7774822695035462, 'recall': 0.8234741784037559, 'f1': 0.7998176014591885, 'number': 1065}	0.7231	0.7863	0.7534	0.8094
0.2678	15.0	150	0.7053	{'precision': 0.7111597374179431, 'recall': 0.8034610630407911, 'f1': 0.7544979686593152, 'number': 809}	{'precision': 0.3697478991596639, 'recall': 0.3697478991596639, 'f1': 0.3697478991596639, 'number': 119}	{'precision': 0.7862254025044723, 'recall': 0.8253521126760563, 'f1': 0.8053137883646359, 'number': 1065}	0.7313	0.7893	0.7592	0.8115

Framework versions

• Transformers 4.28.0
• Pytorch 2.0.1+cu118
• Datasets 2.12.0
• Tokenizers 0.13.3