controlable-question-generation-Flan-T5

This model is a fine-tuned version of google/flan-t5-base on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 2.0228
• Pred Len: 43.2148
• Rougel: 0.4980

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 8e-05
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• gradient_accumulation_steps: 4
• total_train_batch_size: 32
• optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• lr_scheduler_warmup_ratio: 0.15
• num_epochs: 50
• mixed_precision_training: Native AMP
• label_smoothing_factor: 0.1

Training results

Training Loss	Epoch	Step	Validation Loss	Pred Len	Rougel
4.8504	1.0	64	4.7072	17.1297	0.0830
4.7059	2.0	128	4.4417	16.8117	0.0859
4.5518	3.0	192	3.8291	38.3640	0.1143
3.8532	4.0	256	3.1857	62.1883	0.1473
3.3613	5.0	320	2.7606	41.0293	0.3314
3.1056	6.0	384	2.5275	45.6192	0.3595
2.8393	7.0	448	2.4018	51.2678	0.3904
2.6368	8.0	512	2.3054	45.7908	0.4350
2.5695	9.0	576	2.2495	46.6485	0.4393
2.4539	10.0	640	2.2019	45.1213	0.4568
2.434	11.0	704	2.1706	46.0335	0.4733
2.4007	12.0	768	2.1521	45.2971	0.4758
2.3712	13.0	832	2.1348	46.5983	0.4879
2.3723	14.0	896	2.1119	50.4603	0.4830
2.3336	15.0	960	2.0996	45.1632	0.4947
2.316	16.0	1024	2.0851	44.6862	0.4960
2.3414	17.0	1088	2.0845	46.1841	0.4935
2.3121	18.0	1152	2.0667	46.7950	0.4906
2.2742	19.0	1216	2.0678	44.2218	0.5000
2.2243	20.0	1280	2.0508	46.6862	0.4979
2.2661	21.0	1344	2.0445	48.3640	0.4941
2.243	22.0	1408	2.0371	45.1632	0.5057
2.2318	23.0	1472	2.0322	43.8494	0.5026
2.2299	24.0	1536	2.0291	43.2092	0.5081
2.2273	25.0	1600	2.0259	43.8828	0.5031
2.2208	26.0	1664	2.0195	43.9163	0.5023
2.2146	27.0	1728	2.0142	44.1213	0.5066
2.1849	28.0	1792	2.0123	42.6360	0.5026
2.2227	29.0	1856	2.0109	44.1632	0.5024
2.1619	30.0	1920	2.0067	44.0460	0.4995
2.2069	31.0	1984	2.0014	43.5941	0.5068
2.1533	32.0	2048	2.0048	44.1674	0.5017
2.2226	33.0	2112	2.0037	43.6318	0.5064
2.1624	34.0	2176	2.0022	44.4812	0.5008
2.1393	35.0	2240	2.0021	43.9833	0.5011
2.1729	36.0	2304	1.9956	43.9791	0.5035
2.1466	37.0	2368	1.9945	44.5188	0.5035
2.1663	38.0	2432	1.9953	44.5690	0.5034
2.1711	39.0	2496	1.9921	44.3515	0.5029
2.1786	40.0	2560	1.9915	44.6569	0.5006
2.1759	41.0	2624	1.9896	45.0	0.4993
2.1273	42.0	2688	1.9884	44.6192	0.5002
2.178	43.0	2752	1.9882	44.7824	0.5014
2.1441	44.0	2816	1.9889	44.3556	0.5024
2.1384	45.0	2880	1.9869	44.6151	0.4999
2.1347	46.0	2944	1.9874	44.4226	0.5021
2.165	47.0	3008	1.9861	44.7699	0.4976
2.1309	48.0	3072	1.9863	44.7406	0.4994
2.1564	49.0	3136	1.9861	44.8619	0.4989
2.1428	50.0	3200	1.9866	44.8619	0.4989

Framework versions

• PEFT 0.16.0
• Transformers 4.53.3
• Pytorch 2.6.0+cu124
• Datasets 4.1.1
• Tokenizers 0.21.2