mlm

This model is a fine-tuned version of on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 4.7316

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0001
• train_batch_size: 64
• eval_batch_size: 64
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 200

Training results

Training Loss	Epoch	Step	Validation Loss
No log	1.0	46	6.3460
No log	2.0	92	6.0795
No log	3.0	138	5.9936
No log	4.0	184	5.9349
No log	5.0	230	5.8038
No log	6.0	276	5.8418
No log	7.0	322	5.6895
No log	8.0	368	5.6775
No log	9.0	414	5.5350
No log	10.0	460	5.6979
5.77	11.0	506	5.4408
5.77	12.0	552	5.4329
5.77	13.0	598	5.3481
5.77	14.0	644	5.4081
5.77	15.0	690	5.3823
5.77	16.0	736	5.2333
5.77	17.0	782	5.1979
5.77	18.0	828	5.1963
5.77	19.0	874	5.1478
5.77	20.0	920	5.1180
5.77	21.0	966	5.1331
4.8737	22.0	1012	5.0965
4.8737	23.0	1058	4.9988
4.8737	24.0	1104	4.8939
4.8737	25.0	1150	4.9542
4.8737	26.0	1196	4.9090
4.8737	27.0	1242	4.8418
4.8737	28.0	1288	4.8449
4.8737	29.0	1334	4.7790
4.8737	30.0	1380	4.7865
4.8737	31.0	1426	4.6720
4.8737	32.0	1472	4.6885
4.1936	33.0	1518	4.7643
4.1936	34.0	1564	4.5922
4.1936	35.0	1610	4.8800
4.1936	36.0	1656	4.6132
4.1936	37.0	1702	4.6581
4.1936	38.0	1748	4.6109
4.1936	39.0	1794	4.7316

Framework versions

• Transformers 4.35.2
• Pytorch 2.1.0+cu121
• Datasets 2.15.0
• Tokenizers 0.15.0