lilt-en-funsd-2

This model is a fine-tuned version of SCUT-DLVCLab/lilt-roberta-en-base on the funsd-layoutlmv3 dataset. It achieves the following results on the evaluation set:

• Loss: 1.7012
• Answer: {'precision': 0.901985111662531, 'recall': 0.8898408812729498, 'f1': 0.8958718422674061, 'number': 817}
• Header: {'precision': 0.6371681415929203, 'recall': 0.6050420168067226, 'f1': 0.6206896551724138, 'number': 119}
• Question: {'precision': 0.8736027515047291, 'recall': 0.9433611884865367, 'f1': 0.9071428571428571, 'number': 1077}
• Overall Precision: 0.8718
• Overall Recall: 0.9016
• Overall F1: 0.8864
• Overall Accuracy: 0.8041

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• training_steps: 2500
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Answer	Header	Question	Overall Precision	Overall Recall	Overall F1	Overall Accuracy
0.4401	10.53	200	0.9136	{'precision': 0.8668252080856124, 'recall': 0.8922888616891065, 'f1': 0.879372738238842, 'number': 817}	{'precision': 0.512, 'recall': 0.5378151260504201, 'f1': 0.5245901639344263, 'number': 119}	{'precision': 0.8825622775800712, 'recall': 0.9210770659238626, 'f1': 0.9014084507042255, 'number': 1077}	0.8541	0.8867	0.8701	0.8093
0.0458	21.05	400	1.2043	{'precision': 0.879415347137637, 'recall': 0.8837209302325582, 'f1': 0.8815628815628815, 'number': 817}	{'precision': 0.6153846153846154, 'recall': 0.5378151260504201, 'f1': 0.5739910313901345, 'number': 119}	{'precision': 0.8856121537086684, 'recall': 0.9201485608170845, 'f1': 0.9025500910746811, 'number': 1077}	0.8694	0.8828	0.8760	0.8042
0.0127	31.58	600	1.3936	{'precision': 0.880722891566265, 'recall': 0.8947368421052632, 'f1': 0.8876745598057073, 'number': 817}	{'precision': 0.6019417475728155, 'recall': 0.5210084033613446, 'f1': 0.5585585585585585, 'number': 119}	{'precision': 0.8826714801444043, 'recall': 0.9080779944289693, 'f1': 0.8951945080091533, 'number': 1077}	0.8677	0.8798	0.8737	0.8098
0.0082	42.11	800	1.3872	{'precision': 0.8771498771498771, 'recall': 0.8739290085679314, 'f1': 0.8755364806866953, 'number': 817}	{'precision': 0.6428571428571429, 'recall': 0.5294117647058824, 'f1': 0.5806451612903226, 'number': 119}	{'precision': 0.8669565217391304, 'recall': 0.9257195914577531, 'f1': 0.895374943870678, 'number': 1077}	0.8603	0.8813	0.8707	0.8067
0.0035	52.63	1000	1.6235	{'precision': 0.8825665859564165, 'recall': 0.8922888616891065, 'f1': 0.887401095556908, 'number': 817}	{'precision': 0.49044585987261147, 'recall': 0.6470588235294118, 'f1': 0.5579710144927537, 'number': 119}	{'precision': 0.8833333333333333, 'recall': 0.8857938718662952, 'f1': 0.8845618915159944, 'number': 1077}	0.8531	0.8743	0.8636	0.7953
0.0031	63.16	1200	1.6677	{'precision': 0.9051833122629582, 'recall': 0.8763769889840881, 'f1': 0.890547263681592, 'number': 817}	{'precision': 0.48484848484848486, 'recall': 0.6722689075630253, 'f1': 0.5633802816901409, 'number': 119}	{'precision': 0.8893023255813953, 'recall': 0.8876508820798514, 'f1': 0.8884758364312269, 'number': 1077}	0.8626	0.8703	0.8665	0.7994
0.0014	73.68	1400	1.7012	{'precision': 0.901985111662531, 'recall': 0.8898408812729498, 'f1': 0.8958718422674061, 'number': 817}	{'precision': 0.6371681415929203, 'recall': 0.6050420168067226, 'f1': 0.6206896551724138, 'number': 119}	{'precision': 0.8736027515047291, 'recall': 0.9433611884865367, 'f1': 0.9071428571428571, 'number': 1077}	0.8718	0.9016	0.8864	0.8041
0.0011	84.21	1600	1.6779	{'precision': 0.8715814506539834, 'recall': 0.8971848225214198, 'f1': 0.8841978287092883, 'number': 817}	{'precision': 0.6413043478260869, 'recall': 0.4957983193277311, 'f1': 0.5592417061611374, 'number': 119}	{'precision': 0.8754448398576512, 'recall': 0.9136490250696379, 'f1': 0.894139027714675, 'number': 1077}	0.8634	0.8823	0.8727	0.8067
0.0009	94.74	1800	1.6159	{'precision': 0.8729216152019003, 'recall': 0.8996328029375765, 'f1': 0.8860759493670887, 'number': 817}	{'precision': 0.5740740740740741, 'recall': 0.5210084033613446, 'f1': 0.5462555066079295, 'number': 119}	{'precision': 0.8681898066783831, 'recall': 0.9173630454967502, 'f1': 0.8920993227990971, 'number': 1077}	0.8549	0.8867	0.8705	0.8060
0.0007	105.26	2000	1.5876	{'precision': 0.8733572281959379, 'recall': 0.8947368421052632, 'f1': 0.8839177750906893, 'number': 817}	{'precision': 0.5982142857142857, 'recall': 0.5630252100840336, 'f1': 0.5800865800865801, 'number': 119}	{'precision': 0.8783783783783784, 'recall': 0.9052924791086351, 'f1': 0.8916323731138546, 'number': 1077}	0.8611	0.8808	0.8708	0.8091
0.0003	115.79	2200	1.6529	{'precision': 0.8662721893491124, 'recall': 0.8959608323133414, 'f1': 0.8808664259927798, 'number': 817}	{'precision': 0.5714285714285714, 'recall': 0.5378151260504201, 'f1': 0.554112554112554, 'number': 119}	{'precision': 0.8662587412587412, 'recall': 0.9201485608170845, 'f1': 0.8923908149482216, 'number': 1077}	0.8505	0.8877	0.8687	0.8039
0.0002	126.32	2400	1.6602	{'precision': 0.8699763593380615, 'recall': 0.9008567931456548, 'f1': 0.8851473241130486, 'number': 817}	{'precision': 0.5775862068965517, 'recall': 0.5630252100840336, 'f1': 0.5702127659574467, 'number': 119}	{'precision': 0.8725663716814159, 'recall': 0.9155060352831941, 'f1': 0.8935206162211146, 'number': 1077}	0.8552	0.8887	0.8716	0.8024

Framework versions

• Transformers 4.29.2
• Pytorch 2.0.1+cu118
• Datasets 2.12.0
• Tokenizers 0.13.3