ft_rugec_A_c1

This model was trained from scratch on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.2239

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 3e-05
• train_batch_size: 64
• eval_batch_size: 64
• seed: 42
• gradient_accumulation_steps: 2
• total_train_batch_size: 128
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 3
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss
0.3843	0.2088	100	0.2549
0.194	0.4175	200	0.2173
0.1732	0.6263	300	0.2121
0.1696	0.8351	400	0.2007
0.1495	1.0438	500	0.2043
0.1387	1.2526	600	0.2050
0.1362	1.4614	700	0.2051
0.1357	1.6701	800	0.2009
0.128	1.8789	900	0.2027
0.1307	2.0877	1000	0.1997
0.1213	2.2965	1100	0.2024
0.1169	2.5052	1200	0.2034
0.1194	2.7140	1300	0.2034
0.125	2.9228	1400	0.2031

Framework versions

• Transformers 4.48.1
• Pytorch 2.5.1+cu124
• Datasets 3.0.1
• Tokenizers 0.21.0