gpt2-maptask-GF

This model is a fine-tuned version of gpt2 on the None dataset. It achieves the following results on the evaluation set:

• Loss: 2.7116

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-06
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 100
• num_epochs: 30

Training results

Training Loss	Epoch	Step	Validation Loss
3.0139	0.84	1000	2.9045
2.7923	1.67	2000	2.7825
2.6799	2.51	3000	2.7264
2.607	3.34	4000	2.6917
2.55	4.18	5000	2.6708
2.4988	5.01	6000	2.6570
2.4697	5.85	7000	2.6480
2.426	6.68	8000	2.6452
2.4031	7.52	9000	2.6404
2.3654	8.35	10000	2.6416
2.3471	9.19	11000	2.6418
2.3195	10.03	12000	2.6444
2.2969	10.86	13000	2.6455
2.2767	11.7	14000	2.6489
2.2608	12.53	15000	2.6525
2.2381	13.37	16000	2.6563
2.2228	14.2	17000	2.6602
2.2037	15.04	18000	2.6641
2.1911	15.87	19000	2.6684
2.1742	16.71	20000	2.6739
2.1626	17.54	21000	2.6776
2.1504	18.38	22000	2.6800
2.143	19.21	23000	2.6832
2.1277	20.05	24000	2.6892
2.1178	20.89	25000	2.6924
2.1128	21.72	26000	2.6952
2.1009	22.56	27000	2.6978
2.0957	23.39	28000	2.7006
2.0885	24.23	29000	2.7024
2.0849	25.06	30000	2.7065
2.0794	25.9	31000	2.7075
2.0783	26.73	32000	2.7090
2.0698	27.57	33000	2.7106
2.0718	28.4	34000	2.7109
2.069	29.24	35000	2.7116

Framework versions

• Transformers 4.18.0
• Pytorch 1.11.0+cu113
• Datasets 2.1.0
• Tokenizers 0.12.1