lilt-en-funsd

This model is a fine-tuned version of SCUT-DLVCLab/lilt-roberta-en-base on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 1.6939
• Answer: {'precision': 0.8763505402160864, 'recall': 0.8935128518971848, 'f1': 0.8848484848484848, 'number': 817}
• Header: {'precision': 0.6355140186915887, 'recall': 0.5714285714285714, 'f1': 0.6017699115044248, 'number': 119}
• Question: {'precision': 0.8895814781834372, 'recall': 0.9275766016713092, 'f1': 0.9081818181818182, 'number': 1077}
• Overall Precision: 0.8711
• Overall Recall: 0.8927
• Overall F1: 0.8817
• Overall Accuracy: 0.8074

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• training_steps: 2500
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Answer	Header	Question	Overall Precision	Overall Recall	Overall F1	Overall Accuracy
0.417	10.5263	200	1.1962	{'precision': 0.7863247863247863, 'recall': 0.9008567931456548, 'f1': 0.8397033656588704, 'number': 817}	{'precision': 0.5725806451612904, 'recall': 0.5966386554621849, 'f1': 0.5843621399176955, 'number': 119}	{'precision': 0.8825161887141536, 'recall': 0.8857938718662952, 'f1': 0.8841519925857275, 'number': 1077}	0.8225	0.8748	0.8479	0.7827
0.0465	21.0526	400	1.2754	{'precision': 0.8172972972972973, 'recall': 0.9253365973072215, 'f1': 0.8679678530424798, 'number': 817}	{'precision': 0.5531914893617021, 'recall': 0.4369747899159664, 'f1': 0.48826291079812206, 'number': 119}	{'precision': 0.8708220415537489, 'recall': 0.8950789229340761, 'f1': 0.8827838827838828, 'number': 1077}	0.8335	0.8803	0.8562	0.8044
0.0163	31.5789	600	1.4723	{'precision': 0.8574821852731591, 'recall': 0.8837209302325582, 'f1': 0.8704038577456299, 'number': 817}	{'precision': 0.5514705882352942, 'recall': 0.6302521008403361, 'f1': 0.5882352941176471, 'number': 119}	{'precision': 0.8746543778801843, 'recall': 0.8811513463324049, 'f1': 0.877890841813136, 'number': 1077}	0.8463	0.8674	0.8567	0.8020
0.0069	42.1053	800	1.5279	{'precision': 0.8382352941176471, 'recall': 0.9069767441860465, 'f1': 0.8712522045855379, 'number': 817}	{'precision': 0.5736434108527132, 'recall': 0.6218487394957983, 'f1': 0.596774193548387, 'number': 119}	{'precision': 0.9050814956855225, 'recall': 0.8765088207985144, 'f1': 0.8905660377358492, 'number': 1077}	0.8555	0.8738	0.8646	0.7997
0.0043	52.6316	1000	1.5809	{'precision': 0.8454332552693209, 'recall': 0.8837209302325582, 'f1': 0.8641532016756434, 'number': 817}	{'precision': 0.6016949152542372, 'recall': 0.5966386554621849, 'f1': 0.5991561181434599, 'number': 119}	{'precision': 0.9004651162790698, 'recall': 0.8987929433611885, 'f1': 0.8996282527881042, 'number': 1077}	0.8603	0.8748	0.8675	0.8078
0.0032	63.1579	1200	1.6442	{'precision': 0.8728179551122195, 'recall': 0.8567931456548348, 'f1': 0.8647313156269302, 'number': 817}	{'precision': 0.6458333333333334, 'recall': 0.5210084033613446, 'f1': 0.5767441860465117, 'number': 119}	{'precision': 0.871244635193133, 'recall': 0.9424326833797586, 'f1': 0.9054415700267618, 'number': 1077}	0.8614	0.8828	0.8719	0.8009
0.0012	73.6842	1400	1.6784	{'precision': 0.8463302752293578, 'recall': 0.9033047735618115, 'f1': 0.8738898756660746, 'number': 817}	{'precision': 0.5877192982456141, 'recall': 0.5630252100840336, 'f1': 0.5751072961373391, 'number': 119}	{'precision': 0.8979591836734694, 'recall': 0.8987929433611885, 'f1': 0.8983758700696056, 'number': 1077}	0.8590	0.8808	0.8698	0.8057
0.0007	84.2105	1600	1.6625	{'precision': 0.8316939890710382, 'recall': 0.9314565483476133, 'f1': 0.8787528868360277, 'number': 817}	{'precision': 0.6, 'recall': 0.4789915966386555, 'f1': 0.5327102803738317, 'number': 119}	{'precision': 0.9060150375939849, 'recall': 0.8950789229340761, 'f1': 0.9005137786081271, 'number': 1077}	0.8592	0.8852	0.8720	0.8082
0.0007	94.7368	1800	1.6939	{'precision': 0.8763505402160864, 'recall': 0.8935128518971848, 'f1': 0.8848484848484848, 'number': 817}	{'precision': 0.6355140186915887, 'recall': 0.5714285714285714, 'f1': 0.6017699115044248, 'number': 119}	{'precision': 0.8895814781834372, 'recall': 0.9275766016713092, 'f1': 0.9081818181818182, 'number': 1077}	0.8711	0.8927	0.8817	0.8074
0.0006	105.2632	2000	1.7306	{'precision': 0.8548199767711963, 'recall': 0.9008567931456548, 'f1': 0.8772348033373063, 'number': 817}	{'precision': 0.6526315789473685, 'recall': 0.5210084033613446, 'f1': 0.5794392523364486, 'number': 119}	{'precision': 0.8832442067736186, 'recall': 0.9201485608170845, 'f1': 0.901318781264211, 'number': 1077}	0.8609	0.8887	0.8746	0.8049
0.0003	115.7895	2200	1.7413	{'precision': 0.8364640883977901, 'recall': 0.9265605875152999, 'f1': 0.8792102206736353, 'number': 817}	{'precision': 0.6336633663366337, 'recall': 0.5378151260504201, 'f1': 0.5818181818181819, 'number': 119}	{'precision': 0.8966789667896679, 'recall': 0.9025069637883009, 'f1': 0.8995835261453031, 'number': 1077}	0.8579	0.8907	0.8740	0.8021
0.0002	126.3158	2400	1.7415	{'precision': 0.8502857142857143, 'recall': 0.9106487148102815, 'f1': 0.8794326241134752, 'number': 817}	{'precision': 0.63, 'recall': 0.5294117647058824, 'f1': 0.5753424657534247, 'number': 119}	{'precision': 0.8847884788478848, 'recall': 0.9127205199628597, 'f1': 0.8985374771480805, 'number': 1077}	0.8581	0.8892	0.8734	0.8030

Framework versions

• Transformers 4.49.0
• Pytorch 2.5.1+cu124
• Datasets 3.3.1
• Tokenizers 0.21.0