long_first_noditransitive_seed-21_1e-3

This model was trained from scratch on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 3.3998
• Accuracy: 0.3906

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.001
• train_batch_size: 32
• eval_batch_size: 64
• seed: 21
• gradient_accumulation_steps: 8
• total_train_batch_size: 256
• optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 32000
• num_epochs: 20.0
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy
6.1608	0.9998	1495	4.6451	0.2730
4.5932	1.9997	2990	4.1582	0.3151
3.9765	2.9995	4485	3.8948	0.3378
3.8036	4.0	5981	3.7310	0.3536
3.5617	4.9998	7476	3.6269	0.3641
3.4838	5.9997	8971	3.5590	0.3713
3.363	6.9995	10466	3.5149	0.3759
3.3222	8.0	11962	3.4847	0.3794
3.2491	8.9998	13457	3.4645	0.3820
3.2295	9.9997	14952	3.4486	0.3839
3.1788	10.9995	16447	3.4360	0.3856
3.1649	12.0	17943	3.4273	0.3868
3.131	12.9998	19438	3.4221	0.3876
3.1219	13.9997	20933	3.4127	0.3885
3.0977	14.9995	22428	3.4118	0.3891
3.0941	16.0	23924	3.4057	0.3899
3.0749	16.9998	25419	3.4053	0.3900
3.0712	17.9997	26914	3.4017	0.3905
3.0602	18.9995	28409	3.3990	0.3907
3.0586	19.9967	29900	3.3998	0.3906

Framework versions

• Transformers 4.46.2
• Pytorch 2.5.1+cu124
• Datasets 3.2.0
• Tokenizers 0.20.0