results

This model is a fine-tuned version of gpt2 on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.7976

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 2

Training results

Training Loss	Epoch	Step	Validation Loss
1.0437	0.1052	500	0.8308
0.8542	0.2103	1000	0.8337
0.8404	0.3155	1500	0.8218
0.8336	0.4207	2000	0.8246
0.8305	0.5259	2500	0.8148
0.8413	0.6310	3000	0.8129
0.8191	0.7362	3500	0.8074
0.8327	0.8414	4000	0.8047
0.8167	0.9466	4500	0.8099
0.8159	1.0517	5000	0.8080
0.8116	1.1569	5500	0.8041
0.8087	1.2621	6000	0.8019
0.7994	1.3673	6500	0.8037
0.8221	1.4724	7000	0.7998
0.8012	1.5776	7500	0.8007
0.8018	1.6828	8000	0.7988
0.8061	1.7880	8500	0.7985
0.7976	1.8931	9000	0.7980
0.8089	1.9983	9500	0.7976

Framework versions

• Transformers 4.41.1
• Pytorch 2.3.0+cu121
• Datasets 2.19.1
• Tokenizers 0.19.1