model-v3

This model is a fine-tuned version of distilgpt2 on the None dataset. It achieves the following results on the evaluation set:

• Loss: 5.0643

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0001372
• train_batch_size: 8
• eval_batch_size: 8
• seed: 1268669541
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss
2.7294	1.0	39	3.5148
2.6148	2.0	78	3.5284
2.157	3.0	117	3.6368
2.1294	4.0	156	3.6644
1.9443	5.0	195	3.7962
1.8812	6.0	234	3.8349
1.6243	7.0	273	4.0135
1.5363	8.0	312	4.1426
1.3983	9.0	351	4.1834
1.3106	10.0	390	4.1304
0.9305	11.0	429	4.3527
1.0721	12.0	468	4.4012
0.8783	13.0	507	4.5971
0.8152	14.0	546	4.6419
0.6485	15.0	585	4.7665
0.6553	16.0	624	4.9744
0.5309	17.0	663	4.8834
0.5271	18.0	702	4.9749
0.4727	19.0	741	5.0540
0.4828	20.0	780	5.0643

Framework versions

• Transformers 4.26.0
• Pytorch 1.13.1
• Datasets 2.9.0
• Tokenizers 0.13.2