T5-based-Masked-keywords-to-Sentence-Epoch-100

This model is a fine-tuned version of Ziyi98/T5-based-Masked-keywords-to-Sentence-Epoch-10 on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 4.3610
• Bleu: 0.0608
• Gen Len: 16.6969

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 128
• eval_batch_size: 128
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 90
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Bleu	Gen Len
2.1627	1.0	527	2.3051	6.4693	13.3479
2.1432	2.0	1054	2.3051	6.411	13.334
2.1311	3.0	1581	2.3100	6.4619	13.3673
2.112	4.0	2108	2.3178	6.4873	13.3452
2.1025	5.0	2635	2.3190	6.4202	13.4955
2.0878	6.0	3162	2.3256	6.384	13.602
2.0761	7.0	3689	2.3266	6.4555	13.4283
2.0612	8.0	4216	2.3350	6.3692	13.5062
2.0522	9.0	4743	2.3411	6.1491	13.4014
2.0387	10.0	5270	2.3451	6.3275	13.5269
2.0295	11.0	5797	2.3464	6.334	13.5535
2.0157	12.0	6324	2.3460	6.3827	13.6229
2.0075	13.0	6851	2.3502	6.2845	13.4709
1.9982	14.0	7378	2.3546	6.2879	13.5585
1.988	15.0	7905	2.3568	6.2152	13.7016
1.9789	16.0	8432	2.3683	6.2631	13.6195
1.9705	17.0	8959	2.3671	6.2774	13.7046
1.9595	18.0	9486	2.3720	6.3235	13.6249
1.942	19.0	10013	2.3750	6.2509	13.664
1.9344	20.0	10540	2.3775	6.1464	13.7529
1.9256	21.0	11067	2.3831	6.1836	13.7041
1.9187	22.0	11594	2.3855	6.2706	13.732
1.9083	23.0	12121	2.3922	6.2403	13.6844
1.9042	24.0	12648	2.3880	6.2265	13.6294
1.8977	25.0	13175	2.3920	6.2174	13.7556
1.89	26.0	13702	2.3971	6.1378	13.7636
1.8829	27.0	14229	2.4059	6.1993	13.8609
1.878	28.0	14756	2.4021	6.2305	13.7735
1.8682	29.0	15283	2.4078	6.214	13.718
1.8633	30.0	15810	2.4144	6.1878	13.6239
1.8579	31.0	16337	2.4134	6.2359	13.767
1.8507	32.0	16864	2.4209	6.0733	13.7668
1.8476	33.0	17391	2.4219	6.1941	13.8415
1.8379	34.0	17918	2.4229	6.1035	13.8166
1.8344	35.0	18445	2.4191	6.006	13.8071
1.8288	36.0	18972	2.3862	6.1256	13.8425
1.8288	37.0	19499	2.3582	6.1404	13.6742
1.8957	38.0	20026	2.3621	6.1799	13.5436
1.9333	39.0	20553	2.3643	6.1766	13.3626
1.9881	40.0	21080	2.4340	6.0297	13.2076
2.2203	41.0	21607	2.6165	5.3393	12.3985
2.6689	42.0	22134	3.0787	3.7714	11.4059
3.3581	43.0	22661	3.3609	2.7532	10.8581
3.5599	44.0	23188	3.3780	2.7111	11.4898
3.5843	45.0	23715	3.4272	2.3855	12.0565
3.6436	46.0	24242	3.5025	2.1241	12.9236
3.8568	47.0	24769	4.8767	0.2474	12.2402
5.1045	48.0	25296	4.9483	0.1739	11.8688
5.3064	49.0	25823	4.8630	0.2754	11.5488
5.2256	50.0	26350	4.8141	0.31	11.4764
5.184	51.0	26877	4.7820	0.2611	11.6575
5.1601	52.0	27404	4.7407	0.2535	11.8586
5.1313	53.0	27931	4.7061	0.2523	12.1366
5.1018	54.0	28458	4.6740	0.243	12.4433
5.0831	55.0	28985	4.6411	0.2083	12.781
5.038	56.0	29512	4.6110	0.1449	13.1707
5.0216	57.0	30039	4.5820	0.2085	13.4438
5.0055	58.0	30566	4.5551	0.1939	13.8537
4.9907	59.0	31093	4.5303	0.1925	14.2357
4.9833	60.0	31620	4.5065	0.1827	14.6075
4.9686	61.0	32147	4.4851	0.1329	14.9664
4.9593	62.0	32674	4.4666	0.1243	15.4012
4.9555	63.0	33201	4.4485	0.0884	15.6971
4.9478	64.0	33728	4.4310	0.0706	15.9808
4.9386	65.0	34255	4.4103	0.0702	16.0742
4.9356	66.0	34782	4.3895	0.0709	16.2364
4.9207	67.0	35309	4.3738	0.0676	16.4515
4.9125	68.0	35836	4.3620	0.0636	16.6541
4.908	69.0	36363	4.3604	0.0625	16.6969
4.9133	70.0	36890	4.3605	0.0625	16.6829
4.9143	71.0	37417	4.3603	0.0625	16.6941
4.9108	72.0	37944	4.3602	0.0627	16.7096
4.9132	73.0	38471	4.3607	0.0629	16.6869
4.9128	74.0	38998	4.3607	0.0602	16.7173
4.9133	75.0	39525	4.3608	0.0602	16.7148
4.9114	76.0	40052	4.3609	0.0603	16.712
4.9101	77.0	40579	4.3610	0.0608	16.6969
4.9128	78.0	41106	4.3610	0.0608	16.6969
4.9106	79.0	41633	4.3610	0.0608	16.6969
4.9139	80.0	42160	4.3610	0.0608	16.6969
4.9145	81.0	42687	4.3610	0.0608	16.6969
4.9117	82.0	43214	4.3610	0.0608	16.6969
4.9153	83.0	43741	4.3610	0.0608	16.6969
4.9107	84.0	44268	4.3610	0.0608	16.6969
4.9139	85.0	44795	4.3610	0.0608	16.6969
4.9111	86.0	45322	4.3610	0.0608	16.6969
4.9101	87.0	45849	4.3610	0.0608	16.6969
4.9137	88.0	46376	4.3610	0.0608	16.6969
4.9142	89.0	46903	4.3610	0.0608	16.6969
4.913	90.0	47430	4.3610	0.0608	16.6969

Framework versions

• Transformers 4.37.2
• Pytorch 2.2.2+cu118
• Datasets 2.18.0
• Tokenizers 0.15.1