output_LiLT_test_05

This model is a fine-tuned version of nielsr/lilt-xlm-roberta-base on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.1210
• Precision: 0.8130
• Recall: 0.8086
• F1: 0.8108
• Accuracy: 0.9647

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 300
• num_epochs: 60

Training results

Training Loss	Epoch	Step	Validation Loss	Precision	Recall	F1	Accuracy
No log	0.0808	100	0.6084	0.0053	0.0014	0.0023	0.8727
No log	0.1617	200	0.3137	0.2946	0.3010	0.2977	0.9036
No log	0.2425	300	0.2373	0.5717	0.6368	0.6025	0.9289
No log	0.3234	400	0.1952	0.7224	0.6518	0.6853	0.9473
0.4203	0.4042	500	0.1695	0.7066	0.6936	0.7000	0.9491
0.4203	0.4850	600	0.1826	0.6140	0.7420	0.6720	0.9371
0.4203	0.5659	700	0.1806	0.6236	0.7818	0.6938	0.9407
0.4203	0.6467	800	0.1706	0.6900	0.8201	0.7495	0.9511
0.4203	0.7276	900	0.1551	0.7852	0.7567	0.7707	0.9588
0.1009	0.8084	1000	0.1361	0.7655	0.7780	0.7717	0.9622
0.1009	0.8892	1100	0.1439	0.7876	0.7697	0.7785	0.9601
0.1009	0.9701	1200	0.1441	0.7775	0.7694	0.7734	0.9612
0.1009	1.0509	1300	0.1282	0.7813	0.8158	0.7982	0.9643
0.1009	1.1318	1400	0.1421	0.7794	0.8054	0.7922	0.9621
0.0791	1.2126	1500	0.1557	0.7163	0.7780	0.7459	0.9528
0.0791	1.2935	1600	0.1460	0.7489	0.7985	0.7729	0.9589
0.0791	1.3743	1700	0.1343	0.7846	0.8031	0.7937	0.9621
0.0791	1.4551	1800	0.1394	0.7944	0.7875	0.7910	0.9611
0.0791	1.5360	1900	0.1504	0.7803	0.8066	0.7932	0.9631
0.0652	1.6168	2000	0.1292	0.7615	0.8247	0.7919	0.9599
0.0652	1.6977	2100	0.1298	0.7993	0.8115	0.8053	0.9644
0.0652	1.7785	2200	0.1498	0.7394	0.8351	0.7844	0.9577
0.0652	1.8593	2300	0.1319	0.7800	0.8086	0.7941	0.9618
0.0652	1.9402	2400	0.1383	0.8145	0.7824	0.7981	0.9648
0.0618	2.0210	2500	0.1210	0.8130	0.8086	0.8108	0.9647

Framework versions

• Transformers 4.44.2
• Pytorch 2.4.1+cu121
• Datasets 2.21.0
• Tokenizers 0.19.1