t5-base-finetuned-question-to-answer

This model is a fine-tuned version of t5-base on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.4006
• Bleu: 54.0167
• Gen Len: 28.902

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 16
• eval_batch_size: 4
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 25
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Bleu	Gen Len
1.3089	1.0	516	0.8868	35.5598	34.108
1.2622	2.0	1032	0.8313	37.1928	34.906
1.2093	3.0	1548	0.7822	40.5334	31.082
1.1607	4.0	2064	0.7350	41.6835	32.294
1.1269	5.0	2580	0.6991	41.3956	31.084
1.0765	6.0	3096	0.6644	43.152	31.324
1.0551	7.0	3612	0.6305	45.2289	30.064
1.0326	8.0	4128	0.5984	44.9963	30.856
0.9974	9.0	4644	0.5723	45.8182	30.08
0.9847	10.0	5160	0.5474	46.6307	28.812
0.9553	11.0	5676	0.5245	47.3503	30.256
0.9363	12.0	6192	0.5059	48.8164	29.258
0.9218	13.0	6708	0.4872	49.1785	30.37
0.9096	14.0	7224	0.4743	49.7033	29.48
0.8852	15.0	7740	0.4551	50.9333	30.21
0.886	16.0	8256	0.4456	51.7962	28.472
0.8694	17.0	8772	0.4351	51.9603	29.89
0.8785	18.0	9288	0.4250	52.3147	29.17
0.8606	19.0	9804	0.4158	52.5438	28.96
0.8632	20.0	10320	0.4082	53.7264	28.85
0.8549	21.0	10836	0.4037	53.6781	28.446
0.8608	22.0	11352	0.4017	53.8526	29.088
0.8644	23.0	11868	0.3999	53.8358	28.47
0.8589	24.0	12384	0.3987	53.949	28.792
0.8699	25.0	12900	0.4006	54.0167	28.902

Framework versions

• Transformers 4.36.2
• Pytorch 2.0.0
• Datasets 2.1.0
• Tokenizers 0.15.0