layoutlmv3-real_triplet

This model is a fine-tuned version of microsoft/layoutlmv3-base on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.0279
• Item: {'precision': 0.9425511197663097, 'recall': 0.7378048780487805, 'f1': 0.827704147071398, 'number': 2624}
• Aption: {'precision': 0.8349913494809689, 'recall': 0.7924876847290641, 'f1': 0.81318449873631, 'number': 4872}
• Ootnote: {'precision': 0.7846153846153846, 'recall': 0.8360655737704918, 'f1': 0.8095238095238095, 'number': 122}
• Ormula: {'precision': 0.9865976241242765, 'recall': 0.9920367534456356, 'f1': 0.9893097128894318, 'number': 3265}
• Overall Precision: 0.9056
• Overall Recall: 0.8397
• Overall F1: 0.8714
• Overall Accuracy: 0.9961

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 3e-05
• train_batch_size: 8
• eval_batch_size: 4
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 10

Training results

Training Loss	Epoch	Step	Validation Loss	Item	Aption	Ootnote	Ormula	Overall Precision	Overall Recall	Overall F1	Overall Accuracy
0.0166	1.0	8507	0.0252	{'precision': 0.9114948731786292, 'recall': 0.6436737804878049, 'f1': 0.7545231181594819, 'number': 2624}	{'precision': 0.778075463273052, 'recall': 0.715311986863711, 'f1': 0.7453748262217944, 'number': 4872}	{'precision': 0.8598130841121495, 'recall': 0.7540983606557377, 'f1': 0.8034934497816593, 'number': 122}	{'precision': 0.976365868631062, 'recall': 0.9742725880551302, 'f1': 0.9753181051663345, 'number': 3265}	0.8711	0.7762	0.8209	0.9948
0.0075	2.0	17014	0.0318	{'precision': 0.9079837618403248, 'recall': 0.5114329268292683, 'f1': 0.6543149683081424, 'number': 2624}	{'precision': 0.7194696441032798, 'recall': 0.6348522167487685, 'f1': 0.6745175008177952, 'number': 4872}	{'precision': 0.9207920792079208, 'recall': 0.7622950819672131, 'f1': 0.8340807174887892, 'number': 122}	{'precision': 0.9831132944427388, 'recall': 0.9807044410413476, 'f1': 0.9819073903710519, 'number': 3265}	0.8462	0.7103	0.7723	0.9938
0.0057	3.0	25521	0.0338	{'precision': 0.9227359088030399, 'recall': 0.5552591463414634, 'f1': 0.6933142993100166, 'number': 2624}	{'precision': 0.7442236598890942, 'recall': 0.6611247947454844, 'f1': 0.7002173913043479, 'number': 4872}	{'precision': 0.8859649122807017, 'recall': 0.8278688524590164, 'f1': 0.8559322033898306, 'number': 122}	{'precision': 0.9791538933169834, 'recall': 0.9782542113323124, 'f1': 0.9787038455645779, 'number': 3265}	0.8589	0.7326	0.7907	0.9942
0.004	4.0	34028	0.0615	{'precision': 0.9321486268174475, 'recall': 0.43978658536585363, 'f1': 0.5976178146038322, 'number': 2624}	{'precision': 0.6900404088424055, 'recall': 0.5958538587848933, 'f1': 0.639497742042075, 'number': 4872}	{'precision': 0.8738738738738738, 'recall': 0.7950819672131147, 'f1': 0.832618025751073, 'number': 122}	{'precision': 0.9880660954712362, 'recall': 0.9889739663093415, 'f1': 0.9885198224399205, 'number': 3265}	0.8367	0.6784	0.7493	0.9932
0.0027	5.0	42535	0.0227	{'precision': 0.9356973995271868, 'recall': 0.7541920731707317, 'f1': 0.8351972990082295, 'number': 2624}	{'precision': 0.843103448275862, 'recall': 0.8029556650246306, 'f1': 0.8225399495374264, 'number': 4872}	{'precision': 0.8571428571428571, 'recall': 0.7868852459016393, 'f1': 0.8205128205128205, 'number': 122}	{'precision': 0.9850655288021944, 'recall': 0.9898928024502297, 'f1': 0.9874732661167125, 'number': 3265}	0.9085	0.8471	0.8767	0.9963
0.0021	6.0	51042	0.0165	{'precision': 0.9341987466427932, 'recall': 0.7953506097560976, 'f1': 0.859201317414574, 'number': 2624}	{'precision': 0.856687898089172, 'recall': 0.8282019704433498, 'f1': 0.8422041327489042, 'number': 4872}	{'precision': 0.9174311926605505, 'recall': 0.819672131147541, 'f1': 0.8658008658008659, 'number': 122}	{'precision': 0.9736523319200484, 'recall': 0.9846860643185299, 'f1': 0.9791381148165067, 'number': 3265}	0.9113	0.8671	0.8887	0.9966
0.0015	7.0	59549	0.0271	{'precision': 0.9294605809128631, 'recall': 0.6829268292682927, 'f1': 0.7873462214411249, 'number': 2624}	{'precision': 0.8111135515045025, 'recall': 0.7580049261083743, 'f1': 0.7836604774535808, 'number': 4872}	{'precision': 0.8389830508474576, 'recall': 0.8114754098360656, 'f1': 0.825, 'number': 122}	{'precision': 0.9880879657910813, 'recall': 0.9908116385911179, 'f1': 0.9894479278177092, 'number': 3265}	0.8932	0.8103	0.8498	0.9956
0.0012	8.0	68056	0.0231	{'precision': 0.9250706880301602, 'recall': 0.7480945121951219, 'f1': 0.8272229245680573, 'number': 2624}	{'precision': 0.8451156812339332, 'recall': 0.8097290640394089, 'f1': 0.8270440251572327, 'number': 4872}	{'precision': 0.8962264150943396, 'recall': 0.7786885245901639, 'f1': 0.8333333333333333, 'number': 122}	{'precision': 0.9889739663093415, 'recall': 0.9889739663093415, 'f1': 0.9889739663093415, 'number': 3265}	0.9086	0.8483	0.8774	0.9962
0.0009	9.0	76563	0.0224	{'precision': 0.9263715110683349, 'recall': 0.7336128048780488, 'f1': 0.8188005104210974, 'number': 2624}	{'precision': 0.835820895522388, 'recall': 0.7931034482758621, 'f1': 0.8139020537124803, 'number': 4872}	{'precision': 0.832, 'recall': 0.8524590163934426, 'f1': 0.8421052631578947, 'number': 122}	{'precision': 0.9829787234042553, 'recall': 0.9905053598774886, 'f1': 0.9867276887871854, 'number': 3265}	0.9022	0.8386	0.8693	0.9960
0.0007	10.0	85070	0.0279	{'precision': 0.9425511197663097, 'recall': 0.7378048780487805, 'f1': 0.827704147071398, 'number': 2624}	{'precision': 0.8349913494809689, 'recall': 0.7924876847290641, 'f1': 0.81318449873631, 'number': 4872}	{'precision': 0.7846153846153846, 'recall': 0.8360655737704918, 'f1': 0.8095238095238095, 'number': 122}	{'precision': 0.9865976241242765, 'recall': 0.9920367534456356, 'f1': 0.9893097128894318, 'number': 3265}	0.9056	0.8397	0.8714	0.9961

Framework versions

• Transformers 4.26.0
• Pytorch 1.12.1
• Datasets 2.9.0
• Tokenizers 0.13.2