gpt_neo_poem_generation

This model is a fine-tuned version of duydatnguyen/gpt_npv_neo_poem_generation on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.2706

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 30
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss
0.2096	1.17	500	0.2102
0.1815	2.33	1000	0.2102
0.2004	3.5	1500	0.2123
0.172	4.66	2000	0.2149
0.1716	5.83	2500	0.2187
0.1622	6.99	3000	0.2211
0.1471	8.16	3500	0.2272
0.1433	9.32	4000	0.2313
0.1293	10.49	4500	0.2360
0.1245	11.66	5000	0.2395
0.1157	12.82	5500	0.2408
0.1069	13.99	6000	0.2432
0.0994	15.15	6500	0.2486
0.0948	16.32	7000	0.2507
0.0899	17.48	7500	0.2552
0.088	18.65	8000	0.2557
0.0866	19.81	8500	0.2548
0.084	20.98	9000	0.2562
0.0807	22.14	9500	0.2583
0.0804	23.31	10000	0.2614
0.0783	24.48	10500	0.2636
0.0786	25.64	11000	0.2645
0.0767	26.81	11500	0.2658
0.0762	27.97	12000	0.2646
0.0748	29.14	12500	0.2706

Framework versions

• Transformers 4.35.2
• Pytorch 2.1.0+cu121
• Datasets 2.16.1
• Tokenizers 0.15.2