SriDoc/LiLT-RE

checkpoints

This model is a fine-tuned version of kavg/LiLT-RE-IT on the xfun dataset. It achieves the following results on the evaluation set:

• Precision: 0.4898
• Recall: 0.6641
• F1: 0.5638
• Loss: 0.6049

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-05
• train_batch_size: 8
• eval_batch_size: 2
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_ratio: 0.1
• training_steps: 10000

Training results

Training Loss	Epoch	Step	F1	Validation Loss	Precision	Recall
0.061	41.67	500	0.4764	0.2580	0.4142	0.5606
0.0332	83.33	1000	0.4906	0.3439	0.4181	0.5934
0.0264	125.0	1500	0.5194	0.3892	0.4436	0.6263
0.0104	166.67	2000	0.5250	0.4165	0.4468	0.6364
0.0064	208.33	2500	0.5245	0.4479	0.4460	0.6364
0.0013	250.0	3000	0.5204	0.4655	0.4532	0.6111
0.0019	291.67	3500	0.5342	0.4859	0.4630	0.6313
0.0009	333.33	4000	0.5420	0.5162	0.4640	0.6515
0.0006	375.0	4500	0.5515	0.5724	0.4795	0.6490
0.0039	416.67	5000	0.5470	0.5687	0.4662	0.6616
0.0012	458.33	5500	0.5595	0.5582	0.4860	0.6591
0.0001	500.0	6000	0.5730	0.5709	0.4981	0.6742
0.0022	541.67	6500	0.5578	0.5795	0.4877	0.6515
0.0012	583.33	7000	0.5674	0.5710	0.4953	0.6641
0.0009	625.0	7500	0.5607	0.5994	0.4879	0.6591
0.0002	666.67	8000	0.5616	0.5865	0.4879	0.6616
0.0016	708.33	8500	0.4972	0.6717	0.5714	0.5878
0.0	750.0	9000	0.4898	0.6641	0.5638	0.6049
0.0002	791.67	9500	0.4826	0.6641	0.5590	0.6223
0.0014	833.33	10000	0.4890	0.6742	0.5669	0.6318

Framework versions

• Transformers 4.38.2
• Pytorch 2.1.0+cu121
• Datasets 2.18.0
• Tokenizers 0.15.1