jesusromerodev-NLP-model-jesus-romero

This model is a fine-tuned version of distilroberta-base on the datasetX dataset. It achieves the following results on the evaluation set:

• Loss: 0.6336
• Accuracy: 0.8627
• F1: 0.9031

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: Use OptimizerNames.ADAMW_TORCH_FUSED with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• num_epochs: 4

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy	F1
No log	0.1089	50	0.5659	0.7108	0.8228
No log	0.2179	100	0.4883	0.7181	0.8212
No log	0.3268	150	0.4253	0.7966	0.8576
No log	0.4357	200	0.5380	0.7525	0.8444
No log	0.5447	250	0.4402	0.8284	0.8801
No log	0.6536	300	0.5078	0.8088	0.875
No log	0.7625	350	0.3496	0.8407	0.8816
No log	0.8715	400	0.4220	0.8358	0.8870
No log	0.9804	450	0.3651	0.8186	0.8555
0.4970	1.0893	500	0.3958	0.8529	0.8969
0.4970	1.1983	550	0.5836	0.7990	0.8686
0.4970	1.3072	600	0.3959	0.8284	0.8776
0.4970	1.4161	650	0.4410	0.8358	0.8797
0.4970	1.5251	700	0.5136	0.8358	0.8835
0.4970	1.6340	750	0.4864	0.8407	0.8866
0.4970	1.7429	800	0.7316	0.8137	0.8770
0.4970	1.8519	850	0.4211	0.8456	0.8893
0.4970	1.9608	900	0.4480	0.8505	0.8935
0.4970	2.0697	950	0.4524	0.8603	0.8984
0.3404	2.1786	1000	0.5361	0.8407	0.8803
0.3404	2.2876	1050	0.5880	0.8480	0.8869
0.3404	2.3965	1100	0.5207	0.8480	0.8893
0.3404	2.5054	1150	0.5578	0.8480	0.8908
0.3404	2.6144	1200	0.6977	0.8407	0.8908
0.3404	2.7233	1250	0.5877	0.8407	0.8870
0.3404	2.8322	1300	0.5399	0.8529	0.8932
0.3404	2.9412	1350	0.5704	0.8505	0.8975
0.3404	3.0501	1400	0.5532	0.8578	0.8997
0.3404	3.1590	1450	0.5824	0.8603	0.9009
0.2465	3.2680	1500	0.6351	0.8505	0.8935
0.2465	3.3769	1550	0.6178	0.8431	0.8861
0.2465	3.4858	1600	0.7223	0.8407	0.8908
0.2465	3.5948	1650	0.6361	0.8554	0.8967
0.2465	3.7037	1700	0.6336	0.8627	0.9031
0.2465	3.8126	1750	0.6348	0.8578	0.9
0.2465	3.9216	1800	0.6198	0.8529	0.8947
0.2465	4.0	1836	0.6227	0.8554	0.8967

Framework versions

• Transformers 5.16.1
• Pytorch 2.11.0+cu128
• Datasets 4.0.0
• Tokenizers 0.23.1