RJZauner/layoutml_funsd_rjz

layoutmlv2_funsd_rjz

This model is a fine-tuned version of microsoft/layoutlm-base-uncased on the funsd dataset. It achieves the following results on the evaluation set:

• Loss: 0.9422
• Answer: {'precision': 0.7382857142857143, 'recall': 0.7985166872682324, 'f1': 0.7672209026128266, 'number': 809}
• Header: {'precision': 0.42758620689655175, 'recall': 0.5210084033613446, 'f1': 0.4696969696969697, 'number': 119}
• Question: {'precision': 0.8075160403299725, 'recall': 0.8272300469483568, 'f1': 0.8172541743970314, 'number': 1065}
• Overall Precision: 0.7527
• Overall Recall: 0.7973
• Overall F1: 0.7744
• Overall Accuracy: 0.8096

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 3e-05
• train_batch_size: 16
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 15
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Answer	Header	Question	Overall Precision	Overall Recall	Overall F1	Overall Accuracy
0.3143	1.0	10	0.7685	{'precision': 0.7, 'recall': 0.7700865265760197, 'f1': 0.7333725721012359, 'number': 809}	{'precision': 0.2986111111111111, 'recall': 0.36134453781512604, 'f1': 0.32699619771863114, 'number': 119}	{'precision': 0.7693032015065914, 'recall': 0.7671361502347418, 'f1': 0.768218147625764, 'number': 1065}	0.7075	0.7441	0.7254	0.7924
0.2816	2.0	20	0.7829	{'precision': 0.7162315550510783, 'recall': 0.7799752781211372, 'f1': 0.7467455621301775, 'number': 809}	{'precision': 0.33152173913043476, 'recall': 0.5126050420168067, 'f1': 0.40264026402640263, 'number': 119}	{'precision': 0.7855839416058394, 'recall': 0.8084507042253521, 'f1': 0.7968533086534013, 'number': 1065}	0.7186	0.7792	0.7477	0.7976
0.2216	3.0	30	0.7825	{'precision': 0.7016806722689075, 'recall': 0.8257107540173053, 'f1': 0.7586598523566157, 'number': 809}	{'precision': 0.35570469798657717, 'recall': 0.44537815126050423, 'f1': 0.39552238805970147, 'number': 119}	{'precision': 0.7851985559566786, 'recall': 0.8169014084507042, 'f1': 0.8007363092498849, 'number': 1065}	0.7202	0.7983	0.7573	0.7942
0.1973	4.0	40	0.7683	{'precision': 0.7095032397408207, 'recall': 0.8121137206427689, 'f1': 0.7573487031700288, 'number': 809}	{'precision': 0.3968253968253968, 'recall': 0.42016806722689076, 'f1': 0.40816326530612246, 'number': 119}	{'precision': 0.802367941712204, 'recall': 0.8272300469483568, 'f1': 0.8146093388811835, 'number': 1065}	0.7386	0.7968	0.7666	0.8143
0.1671	5.0	50	0.7918	{'precision': 0.7269585253456221, 'recall': 0.7799752781211372, 'f1': 0.7525342874180083, 'number': 809}	{'precision': 0.4076923076923077, 'recall': 0.44537815126050423, 'f1': 0.42570281124497994, 'number': 119}	{'precision': 0.7848888888888889, 'recall': 0.8291079812206573, 'f1': 0.8063926940639269, 'number': 1065}	0.7381	0.7863	0.7614	0.8139
0.1342	6.0	60	0.8295	{'precision': 0.7234972677595628, 'recall': 0.8182941903584673, 'f1': 0.7679814385150812, 'number': 809}	{'precision': 0.37857142857142856, 'recall': 0.44537815126050423, 'f1': 0.4092664092664093, 'number': 119}	{'precision': 0.7939339875111507, 'recall': 0.8356807511737089, 'f1': 0.8142726440988106, 'number': 1065}	0.7376	0.8053	0.7700	0.8120
0.1212	7.0	70	0.8632	{'precision': 0.7337883959044369, 'recall': 0.7972805933250927, 'f1': 0.764218009478673, 'number': 809}	{'precision': 0.4084507042253521, 'recall': 0.48739495798319327, 'f1': 0.4444444444444445, 'number': 119}	{'precision': 0.8137347130761995, 'recall': 0.812206572769953, 'f1': 0.8129699248120301, 'number': 1065}	0.7524	0.7868	0.7692	0.8082
0.1131	8.0	80	0.9081	{'precision': 0.7244785949506037, 'recall': 0.8158220024721878, 'f1': 0.7674418604651163, 'number': 809}	{'precision': 0.40131578947368424, 'recall': 0.5126050420168067, 'f1': 0.4501845018450184, 'number': 119}	{'precision': 0.8097876269621422, 'recall': 0.8234741784037559, 'f1': 0.8165735567970206, 'number': 1065}	0.7446	0.8018	0.7722	0.8011
0.1043	9.0	90	0.9021	{'precision': 0.7308132875143184, 'recall': 0.788627935723115, 'f1': 0.7586206896551724, 'number': 809}	{'precision': 0.425531914893617, 'recall': 0.5042016806722689, 'f1': 0.4615384615384615, 'number': 119}	{'precision': 0.7914818101153505, 'recall': 0.8375586854460094, 'f1': 0.8138686131386863, 'number': 1065}	0.7426	0.7978	0.7692	0.8075
0.0884	10.0	100	0.9126	{'precision': 0.7231450719822813, 'recall': 0.8071693448702101, 'f1': 0.7628504672897196, 'number': 809}	{'precision': 0.40939597315436244, 'recall': 0.5126050420168067, 'f1': 0.4552238805970149, 'number': 119}	{'precision': 0.819718309859155, 'recall': 0.819718309859155, 'f1': 0.819718309859155, 'number': 1065}	0.7496	0.7963	0.7723	0.8094
0.084	11.0	110	0.9354	{'precision': 0.7502944640753828, 'recall': 0.7873918417799752, 'f1': 0.7683956574185766, 'number': 809}	{'precision': 0.4140127388535032, 'recall': 0.5462184873949579, 'f1': 0.47101449275362317, 'number': 119}	{'precision': 0.7946428571428571, 'recall': 0.8356807511737089, 'f1': 0.8146453089244852, 'number': 1065}	0.7488	0.7988	0.7730	0.8064
0.0794	12.0	120	0.9323	{'precision': 0.7244785949506037, 'recall': 0.8158220024721878, 'f1': 0.7674418604651163, 'number': 809}	{'precision': 0.4172661870503597, 'recall': 0.48739495798319327, 'f1': 0.4496124031007752, 'number': 119}	{'precision': 0.8152985074626866, 'recall': 0.8206572769953052, 'f1': 0.8179691155825924, 'number': 1065}	0.7502	0.7988	0.7738	0.8094
0.0803	13.0	130	0.9429	{'precision': 0.7401129943502824, 'recall': 0.8096415327564895, 'f1': 0.7733175914994096, 'number': 809}	{'precision': 0.42592592592592593, 'recall': 0.5798319327731093, 'f1': 0.49110320284697506, 'number': 119}	{'precision': 0.8110599078341014, 'recall': 0.8262910798122066, 'f1': 0.8186046511627907, 'number': 1065}	0.7523	0.8048	0.7777	0.8085
0.0754	14.0	140	0.9393	{'precision': 0.7425629290617849, 'recall': 0.8022249690976514, 'f1': 0.7712418300653594, 'number': 809}	{'precision': 0.4225352112676056, 'recall': 0.5042016806722689, 'f1': 0.45977011494252873, 'number': 119}	{'precision': 0.8018099547511313, 'recall': 0.831924882629108, 'f1': 0.816589861751152, 'number': 1065}	0.7520	0.8003	0.7754	0.8106
0.0732	15.0	150	0.9422	{'precision': 0.7382857142857143, 'recall': 0.7985166872682324, 'f1': 0.7672209026128266, 'number': 809}	{'precision': 0.42758620689655175, 'recall': 0.5210084033613446, 'f1': 0.4696969696969697, 'number': 119}	{'precision': 0.8075160403299725, 'recall': 0.8272300469483568, 'f1': 0.8172541743970314, 'number': 1065}	0.7527	0.7973	0.7744	0.8096

Framework versions

• Transformers 4.27.2
• Pytorch 1.13.1+cu116
• Datasets 2.10.1
• Tokenizers 0.13.2