mlm

This model is a fine-tuned version of on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 4.6448

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0001
• train_batch_size: 64
• eval_batch_size: 64
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 200

Training results

Training Loss	Epoch	Step	Validation Loss
2.3365	2.1739	100	4.2522
2.2685	4.3478	200	4.4035
2.2671	6.5217	300	4.6664
2.2098	8.6957	400	4.3654
2.1098	10.8696	500	4.5015
2.0928	13.0435	600	4.6692
1.9925	15.2174	700	4.5303
1.9345	17.3913	800	4.5046
1.895	19.5652	900	4.4449
1.8184	21.7391	1000	4.6315
1.7848	23.9130	1100	4.3992
1.6882	26.0870	1200	4.5161
1.6321	28.2609	1300	4.4619
1.5764	30.4348	1400	4.3926
1.5599	32.6087	1500	4.5671
1.5019	34.7826	1600	4.6053
1.4319	36.9565	1700	4.6624
1.3862	39.1304	1800	4.5250
1.3343	41.3043	1900	4.4262
1.3174	43.4783	2000	4.6491
1.2386	45.6522	2100	4.3157
1.2019	47.8261	2200	4.6977
1.1498	50.0	2300	4.3964
1.1099	52.1739	2400	4.6960
1.1246	54.3478	2500	4.7397
1.0449	56.5217	2600	4.3909
1.014	58.6957	2700	4.5703
0.9593	60.8696	2800	4.7871
0.9302	63.0435	2900	4.5844
0.9194	65.2174	3000	4.7409
0.8594	67.3913	3100	4.7251
0.831	69.5652	3200	4.5278
0.8152	71.7391	3300	4.6695
0.8082	73.9130	3400	4.5366
0.7407	76.0870	3500	4.7395
0.724	78.2609	3600	4.8780
0.6892	80.4348	3700	4.6850
0.6709	82.6087	3800	4.6448

Framework versions

• Transformers 4.41.2
• Pytorch 2.3.0+cu121
• Datasets 2.20.0
• Tokenizers 0.19.1