text_shortening_model_v26

This model is a fine-tuned version of t5-small on the None dataset. It achieves the following results on the evaluation set:

• Loss: 2.2306
• Rouge1: 0.5085
• Rouge2: 0.2908
• Rougel: 0.4563
• Rougelsum: 0.456
• Bert precision: 0.88
• Bert recall: 0.8755
• Average word count: 8.5646
• Max word count: 17
• Min word count: 3
• Average token count: 13.2012
• % shortened texts with length > 12: 8.7087

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0001
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 100

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum	Bert precision	Bert recall	Average word count	Max word count	Min word count	Average token count	% shortened texts with length > 12
2.1038	1.0	145	1.6481	0.4984	0.2848	0.4508	0.4519	0.8723	0.8719	9.1502	18	3	13.7117	15.9159
1.7274	2.0	290	1.5436	0.5177	0.3156	0.4706	0.4714	0.8771	0.8775	9.1141	18	4	13.6637	14.7147
1.561	3.0	435	1.4685	0.5264	0.3157	0.4671	0.469	0.8773	0.8793	9.2823	17	4	13.955	14.1141
1.4244	4.0	580	1.4429	0.5213	0.3136	0.4674	0.4689	0.8772	0.8774	9.0811	17	4	13.8288	12.9129
1.3375	5.0	725	1.4171	0.5326	0.3172	0.4768	0.4778	0.8785	0.8807	9.3063	18	5	13.964	14.4144
1.2462	6.0	870	1.3989	0.5259	0.3126	0.4707	0.4714	0.8807	0.8768	8.6577	17	4	13.1441	9.6096
1.1822	7.0	1015	1.3797	0.5321	0.3147	0.4687	0.4699	0.8798	0.8792	9.009	17	4	13.6877	12.9129
1.1001	8.0	1160	1.3735	0.5387	0.325	0.481	0.4814	0.8805	0.8835	9.3213	17	4	14.0601	14.7147
1.0329	9.0	1305	1.3813	0.53	0.3122	0.4694	0.4706	0.8799	0.8811	9.024	17	4	13.7057	11.1111
0.9891	10.0	1450	1.3734	0.5334	0.3191	0.4715	0.4726	0.8793	0.8829	9.3243	17	4	14.1291	13.8138
0.9205	11.0	1595	1.3687	0.5279	0.3111	0.4663	0.4676	0.8793	0.8802	9.03	16	4	13.6577	11.4114
0.8857	12.0	1740	1.3986	0.5219	0.3098	0.4694	0.4703	0.8811	0.879	8.8018	15	3	13.3934	11.4114
0.8444	13.0	1885	1.4143	0.5291	0.3084	0.4707	0.4718	0.8802	0.8796	9.03	17	4	13.6727	13.5135
0.8039	14.0	2030	1.4352	0.5216	0.2989	0.4631	0.464	0.8812	0.878	8.7958	16	4	13.4805	9.3093
0.7653	15.0	2175	1.4509	0.525	0.3076	0.4743	0.4751	0.8834	0.8783	8.5526	16	4	13.2162	8.7087
0.7256	16.0	2320	1.4541	0.5153	0.2952	0.4566	0.4579	0.8779	0.8768	8.8739	16	4	13.5405	12.012
0.7018	17.0	2465	1.4859	0.5312	0.306	0.4722	0.4727	0.8812	0.8823	9.0841	17	4	13.6967	14.4144
0.6784	18.0	2610	1.4977	0.5215	0.3068	0.4674	0.4684	0.8817	0.877	8.5766	16	4	13.2072	10.2102
0.6483	19.0	2755	1.5040	0.5297	0.3192	0.4757	0.4756	0.8817	0.8818	9.021	16	4	13.7327	12.012
0.6166	20.0	2900	1.5376	0.526	0.3119	0.4768	0.4774	0.8835	0.8808	8.8138	16	4	13.3634	10.2102
0.5955	21.0	3045	1.5198	0.528	0.3129	0.4795	0.4805	0.8829	0.8807	8.8769	16	4	13.5075	9.9099
0.5678	22.0	3190	1.5499	0.518	0.2988	0.4636	0.464	0.8802	0.8785	8.9249	17	4	13.6006	12.6126
0.5599	23.0	3335	1.5487	0.519	0.3057	0.4691	0.4698	0.8812	0.8773	8.6607	18	4	13.2192	9.3093
0.535	24.0	3480	1.5912	0.5243	0.3054	0.4708	0.4717	0.8828	0.8779	8.6456	16	4	13.1532	9.9099
0.5189	25.0	3625	1.5995	0.5314	0.3106	0.4735	0.474	0.8827	0.8815	8.9099	18	4	13.6126	12.6126
0.4981	26.0	3770	1.6036	0.5222	0.3037	0.4675	0.4676	0.8824	0.8788	8.7658	15	4	13.3784	9.9099
0.4729	27.0	3915	1.6360	0.5114	0.2927	0.46	0.4604	0.8807	0.875	8.5676	15	4	13.1592	9.009
0.462	28.0	4060	1.6648	0.5145	0.2945	0.4586	0.459	0.8812	0.8754	8.5435	17	3	13.0841	9.009
0.4467	29.0	4205	1.6749	0.5076	0.2828	0.4527	0.4533	0.8794	0.8746	8.6697	16	3	13.1772	9.6096
0.4298	30.0	4350	1.6873	0.5215	0.2976	0.4683	0.4679	0.8822	0.8774	8.5766	16	3	13.1682	7.8078
0.4186	31.0	4495	1.7008	0.5129	0.2915	0.4614	0.4614	0.8814	0.8763	8.5736	16	4	13.1892	8.7087
0.4043	32.0	4640	1.7077	0.5121	0.2859	0.4572	0.457	0.8796	0.8765	8.7387	16	3	13.4114	10.2102
0.3835	33.0	4785	1.7421	0.5106	0.2831	0.4579	0.4577	0.8785	0.8763	8.7988	17	3	13.4865	10.8108
0.377	34.0	4930	1.7763	0.5135	0.2907	0.4585	0.4586	0.8808	0.8768	8.6787	15	3	13.4084	10.8108
0.3672	35.0	5075	1.7642	0.5243	0.3018	0.4701	0.4694	0.8826	0.8777	8.5616	15	3	13.1892	9.6096
0.3499	36.0	5220	1.7840	0.5175	0.2965	0.466	0.4656	0.8815	0.8772	8.5796	17	3	13.2252	9.9099
0.3417	37.0	5365	1.8032	0.5163	0.2964	0.4638	0.4636	0.8801	0.8785	8.8348	16	3	13.6156	11.4114
0.3364	38.0	5510	1.8112	0.5096	0.2832	0.4532	0.4536	0.8783	0.8763	8.8829	17	4	13.4925	10.5105
0.315	39.0	5655	1.8360	0.5208	0.3034	0.4692	0.4694	0.8836	0.8797	8.7177	17	4	13.3213	11.4114
0.3117	40.0	5800	1.8419	0.5069	0.285	0.4555	0.4558	0.879	0.8746	8.7117	17	3	13.3634	9.009
0.3195	41.0	5945	1.8435	0.5214	0.2984	0.4686	0.4691	0.8817	0.8779	8.7297	17	3	13.3303	11.4114
0.3062	42.0	6090	1.8574	0.5174	0.2941	0.4672	0.4676	0.8827	0.8779	8.6907	17	3	13.3604	9.6096
0.2892	43.0	6235	1.8839	0.5083	0.2939	0.4603	0.4603	0.8789	0.8763	8.7147	17	4	13.5045	10.8108
0.283	44.0	6380	1.8838	0.5078	0.2873	0.4546	0.4552	0.879	0.8757	8.7327	17	4	13.5135	10.8108
0.2813	45.0	6525	1.8947	0.5126	0.2919	0.4603	0.4608	0.8803	0.8762	8.7027	16	3	13.4505	10.8108
0.2716	46.0	6670	1.9045	0.5163	0.3	0.4687	0.4686	0.8813	0.8771	8.6126	17	4	13.3303	9.3093
0.2604	47.0	6815	1.9097	0.5106	0.2928	0.4617	0.4621	0.8796	0.8761	8.7477	17	3	13.5135	9.009
0.2514	48.0	6960	1.9477	0.5156	0.2959	0.463	0.4633	0.8813	0.876	8.6006	17	3	13.3453	8.4084
0.2444	49.0	7105	1.9599	0.5107	0.2903	0.4581	0.4586	0.8796	0.875	8.6607	16	4	13.3994	8.4084
0.2428	50.0	7250	1.9775	0.5082	0.2903	0.4587	0.4587	0.88	0.8748	8.5435	16	3	13.2823	8.1081
0.2395	51.0	7395	1.9783	0.5154	0.2948	0.4647	0.4647	0.8809	0.8768	8.6817	17	3	13.3303	9.6096
0.2317	52.0	7540	1.9881	0.5092	0.2895	0.4545	0.4546	0.8807	0.8766	8.6126	17	3	13.3964	7.8078
0.224	53.0	7685	2.0001	0.5165	0.3017	0.4622	0.4627	0.8802	0.8777	8.7598	17	3	13.4895	9.3093
0.2161	54.0	7830	2.0140	0.5176	0.2974	0.465	0.4652	0.881	0.878	8.7327	17	3	13.4384	9.9099
0.2201	55.0	7975	2.0317	0.5102	0.2904	0.4554	0.4553	0.8802	0.8765	8.6306	16	3	13.3754	10.8108
0.2153	56.0	8120	2.0427	0.5172	0.2983	0.4632	0.4632	0.8808	0.8771	8.7297	17	3	13.4114	11.1111
0.211	57.0	8265	2.0432	0.5165	0.2983	0.4652	0.4652	0.8815	0.8765	8.5976	17	3	13.2432	9.9099
0.1995	58.0	8410	2.0720	0.5062	0.2913	0.4528	0.4528	0.8781	0.8739	8.6006	17	3	13.2763	8.7087
0.2072	59.0	8555	2.0574	0.5099	0.2902	0.4554	0.4563	0.8803	0.8751	8.5435	17	3	13.1411	9.009
0.1989	60.0	8700	2.0722	0.5127	0.2943	0.459	0.4585	0.8807	0.8767	8.6967	17	4	13.3213	11.1111
0.1911	61.0	8845	2.0669	0.5125	0.2922	0.459	0.4581	0.8806	0.875	8.5556	16	3	13.1622	9.009
0.1902	62.0	8990	2.0912	0.5063	0.2892	0.4498	0.45	0.8795	0.8739	8.5105	17	4	13.0751	9.9099
0.1905	63.0	9135	2.0875	0.5029	0.2845	0.4492	0.4492	0.878	0.8745	8.6727	16	4	13.3423	10.5105
0.1895	64.0	9280	2.0787	0.5094	0.2941	0.4551	0.4557	0.8791	0.8751	8.7117	17	4	13.2973	9.9099
0.1813	65.0	9425	2.0960	0.5168	0.2998	0.462	0.4619	0.8812	0.8773	8.7177	17	4	13.3634	10.8108
0.1856	66.0	9570	2.0888	0.5053	0.2921	0.4549	0.4552	0.8793	0.8746	8.5676	17	3	13.1772	8.7087
0.1669	67.0	9715	2.1158	0.5184	0.3018	0.4623	0.4624	0.8814	0.8772	8.6517	17	4	13.2462	12.012
0.1676	68.0	9860	2.1246	0.5195	0.2977	0.4642	0.4638	0.8814	0.8778	8.7207	17	4	13.3243	11.4114
0.1682	69.0	10005	2.1325	0.5112	0.2963	0.4572	0.4579	0.8805	0.8759	8.5916	17	4	13.1742	9.9099
0.1664	70.0	10150	2.1442	0.5048	0.2828	0.4505	0.4506	0.8786	0.8743	8.6366	17	4	13.2883	8.7087
0.1655	71.0	10295	2.1339	0.5132	0.295	0.4603	0.4603	0.8802	0.8754	8.7087	17	4	13.3273	10.8108
0.1621	72.0	10440	2.1391	0.5036	0.2858	0.4527	0.4526	0.8786	0.8722	8.4715	17	4	13.0901	9.009
0.1624	73.0	10585	2.1438	0.5055	0.2865	0.4558	0.4557	0.8786	0.8737	8.5255	17	4	13.1832	9.009
0.1486	74.0	10730	2.1623	0.5073	0.2871	0.4554	0.4551	0.8794	0.8745	8.5375	17	4	13.2372	8.4084
0.1593	75.0	10875	2.1699	0.5054	0.2873	0.4527	0.4526	0.8782	0.874	8.6126	17	4	13.2913	10.2102
0.16	76.0	11020	2.1652	0.5062	0.284	0.4557	0.4556	0.8788	0.8748	8.6937	17	4	13.2733	9.9099
0.1464	77.0	11165	2.1777	0.5073	0.2876	0.4556	0.4553	0.8786	0.8749	8.6787	17	4	13.3453	10.8108
0.1492	78.0	11310	2.1705	0.5027	0.2854	0.4498	0.45	0.8774	0.8738	8.6937	17	4	13.3724	10.5105
0.1565	79.0	11455	2.1738	0.4946	0.2768	0.4432	0.4431	0.8757	0.8718	8.5916	17	4	13.3303	10.2102
0.1429	80.0	11600	2.1968	0.5021	0.2878	0.4523	0.452	0.8781	0.8737	8.5375	17	4	13.2583	9.009
0.1424	81.0	11745	2.1810	0.509	0.2909	0.4562	0.4558	0.8785	0.8752	8.6186	17	3	13.2703	10.8108
0.1447	82.0	11890	2.1790	0.5042	0.283	0.4504	0.4507	0.8782	0.874	8.5616	15	4	13.2162	10.5105
0.1399	83.0	12035	2.1908	0.5018	0.2801	0.4489	0.4488	0.8772	0.8733	8.5796	17	3	13.2042	10.2102
0.1417	84.0	12180	2.1985	0.504	0.2812	0.4534	0.4527	0.8782	0.8739	8.5375	17	3	13.0751	9.6096
0.1375	85.0	12325	2.1914	0.5061	0.2844	0.4557	0.4549	0.8791	0.8749	8.5435	17	4	13.1441	9.9099
0.1354	86.0	12470	2.2087	0.5084	0.2889	0.4592	0.4589	0.8798	0.8755	8.5315	17	4	13.1321	10.2102
0.1381	87.0	12615	2.2014	0.5068	0.2857	0.4555	0.4551	0.8792	0.8754	8.5345	17	4	13.1802	10.2102
0.137	88.0	12760	2.2022	0.5077	0.2894	0.4561	0.4552	0.8793	0.8753	8.5495	17	4	13.1682	10.2102
0.1301	89.0	12905	2.2055	0.5096	0.2905	0.4581	0.4581	0.8795	0.8758	8.6186	17	4	13.1802	10.2102
0.1374	90.0	13050	2.2118	0.507	0.2865	0.4544	0.4544	0.8793	0.8751	8.5766	17	4	13.1532	9.9099
0.1338	91.0	13195	2.2074	0.5048	0.2863	0.453	0.4529	0.8791	0.8747	8.5135	17	4	13.0661	8.7087
0.1308	92.0	13340	2.2144	0.5053	0.2886	0.4542	0.4545	0.8789	0.8742	8.5195	17	3	13.0961	8.4084
0.1254	93.0	13485	2.2208	0.5118	0.294	0.4611	0.4612	0.8805	0.8763	8.5225	17	3	13.1141	8.4084
0.1311	94.0	13630	2.2254	0.5084	0.2909	0.4573	0.4573	0.8798	0.8752	8.5165	17	3	13.0751	7.8078
0.1272	95.0	13775	2.2274	0.5056	0.2872	0.454	0.4538	0.8792	0.8745	8.5766	17	3	13.1982	8.4084
0.1304	96.0	13920	2.2313	0.5053	0.2879	0.4526	0.4526	0.8794	0.8747	8.5435	17	3	13.1652	8.7087
0.1303	97.0	14065	2.2304	0.5061	0.2871	0.4532	0.4532	0.8793	0.8748	8.5586	17	3	13.2012	8.7087
0.1306	98.0	14210	2.2303	0.5081	0.2889	0.4556	0.4552	0.8796	0.8753	8.5766	17	3	13.2102	8.7087
0.1387	99.0	14355	2.2304	0.5088	0.2903	0.4563	0.4561	0.8799	0.8754	8.5766	17	3	13.2042	9.009
0.1339	100.0	14500	2.2306	0.5085	0.2908	0.4563	0.456	0.88	0.8755	8.5646	17	3	13.2012	8.7087

Framework versions

• Transformers 4.33.1
• Pytorch 2.0.1+cu118
• Datasets 2.14.5
• Tokenizers 0.13.3