lugman/Droplet-1M-Base

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0006
• train_batch_size: 256
• eval_batch_size: 64
• seed: 42
• optimizer: Use OptimizerNames.ADAMW_TORCH_FUSED with betas=(0.9,0.95) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• num_epochs: 1

Training results

Training Loss	Epoch	Step	Validation Loss
3.6861	0.0525	1000	3.6969
3.4465	0.1050	2000	3.4650
3.3588	0.1574	3000	3.3802
3.3058	0.2099	4000	3.3293
3.2706	0.2624	5000	3.2954
3.2440	0.3149	6000	3.2658
3.2016	0.3674	7000	3.2214
3.1543	0.4198	8000	3.1778
3.1084	0.4723	9000	3.1303
3.0711	0.5248	10000	3.0894
3.0436	0.5773	11000	3.0646
3.0254	0.6298	12000	3.0446
3.0104	0.6822	13000	3.0276
2.9920	0.7347	14000	3.0130
2.9847	0.7872	15000	3.0042
2.9580	0.8397	16000	2.9773
2.9439	0.8922	17000	2.9628
2.9322	0.9446	18000	2.9523
2.9237	0.9971	19000	2.9448
2.9283	1.0	19055	2.9446

Framework versions

• Transformers 5.14.1
• Pytorch 2.13.0+cu130
• Datasets 5.0.1
• Tokenizers 0.22.2