A-mlp-s10-46L

This model is a fine-tuned version of on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 2.0840

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.001
• train_batch_size: 256
• eval_batch_size: 256
• seed: 42
• gradient_accumulation_steps: 2
• total_train_batch_size: 512
• optimizer: Use OptimizerNames.ADAMW_TORCH_FUSED with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: constant
• training_steps: 1000

Training results

Training Loss	Epoch	Step	Validation Loss
12.1543	0.0270	50	5.9332
10.9227	0.0539	100	5.3042
9.8488	0.0809	150	4.6722
8.4669	0.1078	200	4.1294
7.8304	0.1348	250	3.7507
7.0656	0.1618	300	3.4784
6.6266	0.1887	350	3.2288
6.1030	0.2157	400	3.0229
5.8233	0.2427	450	2.8442
5.4586	0.2696	500	2.7096
5.2787	0.2966	550	2.6019
5.0347	0.3235	600	2.4971
4.8938	0.3505	650	2.4249
4.7188	0.3775	700	2.3604
4.6209	0.4044	750	2.2850
4.4705	0.4314	800	2.2386
4.4134	0.4583	850	2.1917
4.2924	0.4853	900	2.1409
4.2525	0.5123	950	2.1151
4.1808	0.5392	1000	2.0840

Framework versions

• Transformers 5.15.0.dev0
• Pytorch 2.6.0+cu124
• Datasets 5.0.1
• Tokenizers 0.22.2