alcyone-v1

This model is a fine-tuned version of laskar-ks/alcyone-base-100m on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.9838

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0001
• train_batch_size: 32
• eval_batch_size: 32
• seed: 42
• optimizer: Use OptimizerNames.ADAMW_TORCH_FUSED with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_steps: 268
• num_epochs: 2

Training results

Training Loss	Epoch	Step	Validation Loss
1.1118	0.1116	500	1.0552
1.0948	0.2232	1000	1.0465
1.0857	0.3347	1500	1.0413
1.0879	0.4463	2000	1.0338
1.0822	0.5579	2500	1.0271
1.0628	0.6695	3000	1.0194
1.0542	0.7811	3500	1.0150
1.0550	0.8927	4000	1.0089
1.0333	1.0042	4500	1.0032
0.9644	1.1158	5000	1.0014
0.9668	1.2274	5500	0.9973
0.9558	1.3390	6000	0.9936
0.9507	1.4506	6500	0.9904
0.9630	1.5622	7000	0.9876
0.9601	1.6737	7500	0.9855
0.9536	1.7853	8000	0.9844
0.9652	1.8969	8500	0.9839
0.9596	2.0	8962	0.9838

Framework versions

• Transformers 5.19.0
• Pytorch 2.11.0+cu130
• Datasets 5.1.0
• Tokenizers 0.23.2