gen-qm-17-small

This model is a fine-tuned version of t5-small on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.0339

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 32
• eval_batch_size: 32
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 60

Training results

Training Loss	Epoch	Step	Validation Loss
1.2939	1.0	468	0.2461
0.3542	2.0	936	0.1119
0.21	3.0	1404	0.0772
0.1602	4.0	1872	0.0656
0.1325	5.0	2340	0.0579
0.116	6.0	2808	0.0527
0.1068	7.0	3276	0.0520
0.0986	8.0	3744	0.0497
0.0911	9.0	4212	0.0469
0.0849	10.0	4680	0.0449
0.0808	11.0	5148	0.0442
0.0783	12.0	5616	0.0433
0.0761	13.0	6084	0.0414
0.074	14.0	6552	0.0410
0.0715	15.0	7020	0.0408
0.068	16.0	7488	0.0402
0.0674	17.0	7956	0.0393
0.0661	18.0	8424	0.0399
0.0645	19.0	8892	0.0392
0.063	20.0	9360	0.0397
0.0619	21.0	9828	0.0382
0.0623	22.0	10296	0.0373
0.0609	23.0	10764	0.0372
0.0593	24.0	11232	0.0372
0.0584	25.0	11700	0.0369
0.0579	26.0	12168	0.0364
0.0572	27.0	12636	0.0365
0.0573	28.0	13104	0.0364
0.0556	29.0	13572	0.0363
0.0562	30.0	14040	0.0354
0.0544	31.0	14508	0.0358
0.0547	32.0	14976	0.0359
0.0529	33.0	15444	0.0353
0.0534	34.0	15912	0.0355
0.0524	35.0	16380	0.0353
0.052	36.0	16848	0.0354
0.0525	37.0	17316	0.0354
0.0533	38.0	17784	0.0351
0.0507	39.0	18252	0.0347
0.0504	40.0	18720	0.0347
0.0509	41.0	19188	0.0344
0.051	42.0	19656	0.0346
0.0507	43.0	20124	0.0342
0.0505	44.0	20592	0.0342
0.0497	45.0	21060	0.0344
0.0491	46.0	21528	0.0341
0.0489	47.0	21996	0.0340
0.0488	48.0	22464	0.0342
0.0481	49.0	22932	0.0338
0.0488	50.0	23400	0.0339
0.048	51.0	23868	0.0340
0.0481	52.0	24336	0.0341
0.0483	53.0	24804	0.0339
0.0492	54.0	25272	0.0339
0.0475	55.0	25740	0.0339
0.0477	56.0	26208	0.0339
0.0481	57.0	26676	0.0339
0.0478	58.0	27144	0.0338
0.0485	59.0	27612	0.0339
0.0479	60.0	28080	0.0339

Framework versions

• Transformers 4.27.3
• Pytorch 1.13.1
• Datasets 2.10.1
• Tokenizers 0.13.2