t5-fine-tuned-with-25-yake-keywords

This model is a fine-tuned version of t5-base on the None dataset. It achieves the following results on the evaluation set:

• Loss: 1.7255
• Rouge1: 25.5531
• Rouge2: 11.1657
• Rougel: 20.7513
• Rougelsum: 24.054
• Gen Len: 19.0

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 2
• eval_batch_size: 2
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 25

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum	Gen Len
1.3097	1.0	604	1.3789	25.5146	11.2077	20.8249	23.9199	19.0
1.1951	2.0	1208	1.3779	25.3347	11.2485	20.6781	23.7106	19.0
1.1081	3.0	1812	1.3903	26.1109	11.8345	21.2205	24.551	18.994
1.0272	4.0	2416	1.4042	26.027	11.5618	21.1159	24.3576	18.992
0.919	5.0	3020	1.4225	25.8294	11.5972	21.0053	24.3003	18.992
0.8643	6.0	3624	1.4410	25.9719	11.6151	21.0454	24.4411	18.99
0.8215	7.0	4228	1.4599	25.68	11.2692	20.9075	24.2681	19.0
0.7931	8.0	4832	1.4926	25.0808	10.9178	20.4053	23.6258	19.0
0.7664	9.0	5436	1.5090	25.458	10.9978	20.6381	23.9113	19.0
0.7053	10.0	6040	1.5259	25.4787	10.8938	20.5842	23.9459	18.998
0.6725	11.0	6644	1.5481	25.2993	10.7172	20.5288	23.8319	19.0
0.6462	12.0	7248	1.5710	25.6251	11.0816	20.7758	24.082	19.0
0.6275	13.0	7852	1.5884	25.8573	11.0737	20.988	24.294	19.0
0.5838	14.0	8456	1.6131	26.1096	11.3973	21.3659	24.6114	19.0
0.5682	15.0	9060	1.6259	25.7213	11.1484	20.8604	24.1114	19.0
0.5629	16.0	9664	1.6473	25.6197	11.2045	20.8956	24.1237	19.0
0.5446	17.0	10268	1.6645	25.4284	10.7362	20.4946	23.9147	19.0
0.5108	18.0	10872	1.6716	25.6986	11.2317	20.8851	24.272	19.0
0.5358	19.0	11476	1.6882	25.8002	11.2396	21.0001	24.2643	19.0
0.4959	20.0	12080	1.7027	25.636	11.2417	20.8785	24.1355	18.992
0.4942	21.0	12684	1.7131	25.6154	11.1795	20.7925	24.1343	19.0
0.4833	22.0	13288	1.7178	25.7708	11.2434	20.9096	24.1974	19.0
0.4702	23.0	13892	1.7227	25.6977	11.2352	20.9147	24.1121	19.0
0.4747	24.0	14496	1.7241	25.6248	11.2042	20.8192	24.1186	19.0
0.4691	25.0	15100	1.7255	25.5531	11.1657	20.7513	24.054	19.0

Framework versions

• Transformers 4.34.0
• Pytorch 2.0.1+cu118
• Datasets 2.14.5
• Tokenizers 0.14.1